Generative AI for Software Development: Run a Reviewable Change Pilot

Scope a generated-change pilot with versioned context, independent expected results, restricted execution and candidate-bound testing and release evidence.

Use generative AI to propose a bounded software change whose intended behavior can be checked independently. Give it relevant, permitted context; constrain what its execution environment can do; and compare the complete accepted-change effort with the existing method. A passing demonstration or an assistant's approval message is insufficient evidence to release its output.

This guide helps a technical lead run the first generated-change pilot. It concentrates on choosing the task and preparing its context and verification, rather than replacing a full engineering release policy. Code, tests and documentation can all be generated, but each remains a proposal until the responsible team checks it against the actual requirement.

Select a change small enough to understand completely

A useful pilot might add a pure data transformation, draft tests for a documented interface, or update an adapter against a stable contract. Prefer work where reviewers can inspect the whole changed behavior and produce meaningful negative cases. Small file count alone is not a risk boundary: a one-line authorization edit can affect every customer.

Write the included input forms, expected outputs and excluded operations before asking for code. Keep a deterministic implementation or the existing process as a comparator. If a requirement is disputed, resolve it with its owner rather than asking a model to choose the business rule.

For a legacy-code task, identify which existing behavior must remain compatible and which behavior is deliberately changing. Tests copied from the current implementation may preserve its defects. Tests invented after reading a candidate may preserve the candidate's misunderstanding. Separate the requirement discussion from both sources of implementation behavior.

Assemble a versioned context packet

Include the base revision, relevant functions, interface contracts, runtime and dependency versions, formatting conventions and representative permitted fixtures. Explain why each file belongs in the packet. Conflicting documentation should be marked unresolved or ranked by an accountable owner; supplying both without explanation can generate a plausible but unsupported compromise.

Repository retrieval can help find relevant material, but retrieved content may be stale, incomplete or adversarial. OWASP's prompt-injection guidance identifies instructions arriving through external material as an indirect attack path. Treat comments, tickets and documentation as data for the task, not authority to upload secrets or change protected settings.

Retain enough context identity to explain a result later without storing every sensitive prompt. Record the source revision, selected context references, tool configuration and output revision. If retrieval changes between attempts, record that change rather than attributing all differences to the model. A larger context window does not resolve contradictory contracts.

Work through a fictional decimal parser proposal

Suppose an internal reporting service needs a parser for nonnegative amounts expressed as strings with exactly two decimal places. The agreed teaching contract accepts ASCII digits, a decimal point and two fractional digits, with no sign, whitespace, exponent or thousands separator. Leading zeros are allowed. Outputs are integer minor units, bounded to the application's agreed range. This example changes no payment account and executes no real financial transaction.

The owner supplies these independent expected results before implementation:

A candidate that converts everything through a permissive numeric parser may accept exponent notation. A candidate that multiplies a binary floating-point value and truncates can mishandle a decimal boundary. Require exact parsing and explicit bounds rather than accepting familiar-looking code. This is a proposed review exercise, not a claim that a particular assistant produced either defect.

Now change the business requirement to permit negative adjustments. The previously accepted parser is no longer the whole answer. Revisit the sign rule, range, expected cases and callers before broadening the implementation. A successful earlier fixture suite cannot approve a changed contract.

Challenge the tests as well as the patch

Ask a reviewer to explain why each expected result follows from the contract. Include invalid forms, boundary values and callers that previously relied on a different representation. Check that the test reaches the actual public function rather than a helper or mocked replacement that bypasses the relevant conversion.

GitHub's inline-suggestion application card warns that generated code can be inaccurate and generated tests may omit scenarios. Apply that limitation to the evidence: generated tests can be useful candidates, but their presence does not establish independent verification.

Deliberately alter an inert candidate to accept exponent notation or omit the range check. The proposed suite should reject it. That negative control tests the verifier's sensitivity under the selected contract. It does not prove the verifier detects every defect. Preserve unrun and inconclusive cases instead of reporting only successful assertions.

Choose execution and provider handling explicitly

Let the assistant edit the scoped workspace and run approved local checks without inheriting production credentials. Review package installation, generated scripts and external connections before execution. A parser pilot does not need customer exports, a cloud-administration token or permission to publish a release.

Managed inference also needs a precise data review. Amazon Bedrock's data-protection documentation describes service-operated model deployment accounts, not a promise that inference runs inside the customer's own account. PrivateLink endpoints provide a private network path for supported service APIs; they do not by themselves determine retention or every downstream copy.

Check the selected model, API, Region, retention settings, logging and any external tools. AWS's retention documentation describes configuration and model-dependent handling. Do not generalize one feature's behavior to every Bedrock route. Application logs, repository integrations and copied debugging artifacts need their own decisions, even when the inference connection is private.

Bind the evidence to a candidate and a release decision

Record the final diff, base and candidate revisions, resolved dependencies, executed checks and remaining gaps. If the assistant modifies code after review, determine which evidence must be rerun. A successful result on an earlier candidate does not automatically cover a later correction or changed runtime configuration.

Keep proposal generation, acceptance and deployment authority separate. The pilot can end with a useful unmerged patch. Releasing it additionally needs the team's established review, artifact, target-environment and recovery controls. The AI-assisted code accountability paper owns those deeper release-policy decisions; the generated-code acceptance playbook provides the execution procedure.

Recovery follows the changed state. A pure parser can be reverted only if callers and persisted values remain compatible. If the change also rewrites historical records, the recovery plan must address those records. Do not turn a narrow pilot into a migration without revisiting permissions, verification and correction ownership.

Make an expansion decision from the whole pilot

Use a short reusable record: task and contract; permitted context; execution boundary; baseline method; candidate identity; independent fixtures; observed failures; review and correction effort; residual gaps; and accepting owner. Separate tool charges from engineering labor, and record both within the chosen comparison horizon. No fixed test count or prompt score establishes success.

Expand when retained observations support a specific task class and the team can absorb its review and operating work. Narrow the task when context preparation outweighs the benefit. Stop when the tool needs unapproved data, the expected behavior cannot be established, or the candidate changes its own acceptance mechanism without independent scrutiny.

The broader AI-in-development operating model helps choose where this pilot belongs in the lifecycle. Start with one agreed contract and an inert rejection fixture, then establish that the verifier rejects a deliberately wrong candidate before interpreting the first green run. Sources checked on 19 October 2026; all parser cases are fictional teaching inputs, not observed product results.