Generating code is no longer the part that interests me most.
The harder problem starts one step later:
Who gets to decide that the generated software is actually correct?
I have been working on an open-source project called Derivative, and its software-building pipeline, Forge, is based on a deliberately uncomfortable rule:
The component that writes the software does not have the authority to declare that software correct.
That sounds simple.
Implementing it turned into a much deeper problem involving requirement compilation, isolated execution, independent validation, evidence integrity, adversarial checks, immutable blind benchmarks, and — eventually — validators that also had to be treated as potentially wrong.
This article is an attempt to explain that system without requiring you to understand the entire Derivative architecture first.
Repository:
https://github.com/Daniele-Cangi/Derivative
The 30-second version
You give Forge a software requirement:
python forge.py \
"Build a Python CLI that reads a CSV of contracts,
extracts expiration dates,
flags contracts expiring in less than 90 days,
writes a summary CSV,
and includes tests."
The important part is what happens next.
natural-language requirement
↓
structured contract
↓
grounded plan
↓
candidate software
↓
isolated execution
↓
independent validation
↓
verified package
OR
explicit failure evidence
Forge does not simply generate some files, run the tests generated alongside them, and call the result successful.
The generated software has to survive checks that it does not control.
A run can end in three build outcomes:
verified
validation_failed
infeasible_proven
verified allows packaging.
validation_failed means software may exist, but the available evidence is insufficient to certify it.
infeasible_proven means the stated constraints were shown to be mutually incompatible.
That distinction is the core of the project.
First principle: preserve the requirement before writing code
A natural-language requirement is dangerous because it is easy to quietly lose pieces of it.
Consider:
Read a CSV file.
If the file is empty, print exactly "EMPTY" and exit with status 2.
Otherwise process every valid row and write a summary file.
A coding model can understand that.
It can also accidentally implement:
if not rows:
print("No rows found")
return
That implementation looks reasonable.
It is also wrong.
The output is wrong.
The exit status is wrong.
And if those details disappear while the requirement is being converted into a plan, no amount of testing against the weakened plan will recover them.
Forge therefore does not treat the prompt merely as context for a model.
It compiles it.
A requirement becomes typed data describing functional goals, requirement atoms, conditional obligations, acceptance criteria, quality constraints, public interfaces, ambiguity flags and verification expectations.
The idea is not that natural language suddenly becomes mathematically perfect.
The idea is that information loss becomes visible.
Technical deep dive: how the requirement becomes a BuildSpec
The real `RequirementCompiler` currently starts by extracting requirement atoms and normalizing conditional obligations:
```python
requirement_atoms = self._extract_requirement_atoms(normalized)
conditional_normalization = (
self.conditional_normalizer.normalize(requirement_atoms)
)
functional_goals = [
atom.text
for atom in requirement_atoms
if atom.category in {"functional", "validation"}
]
```
It then derives acceptance, obligation and quality contracts and produces a `BuildSpec`:
```python
return BuildSpec(
build_id=self._build_id(normalized),
raw_requirement=requirement,
normalized_requirement=normalized,
functional_goals=functional_goals,
non_functional_constraints=non_functional_constraints,
requirement_atoms=requirement_atoms,
conditional_obligations=conditional_normalization.obligations,
coverage_directives=conditional_normalization.coverage_directives,
conditional_normalization_issues=conditional_normalization.issues,
acceptance_contract=acceptance_contract,
obligation_contract=obligation_contract,
quality_contract=quality_contract,
target_artifact_type=target_artifact_type,
public_module=public_module,
public_import_contract=public_import_contract,
risk_hints=...,
ambiguity_flags=ambiguity_flags,
assumptions=...,
)
```
The contracts themselves are typed dataclasses.
For example, a conditional obligation records more than a piece of prose:
```python
@dataclass
class ConditionalObligation:
obligation_id: str
parent_requirement_id: str
trigger: str
precondition: Dict[str, Any]
observable_channel: str
comparison_relation: str
expected_value: Any
polarity: str
observation_fidelity: str
verification_method: str
source_fragment: str
witness_class: str = ""
```
That means a statement such as:
```text
If input is invalid, print "ERROR" and exit with status 2.
```
can eventually become separate observable obligations instead of surviving only as prose inside an LLM context window.
This matters later because validation can ask for evidence tied back to those obligations.
Source:
`core/forge/requirement_compiler.py`
`core/forge/contracts.py`
The coder is not the judge
This is where Forge differs from the simplest AI coding loop.
A common architecture is roughly:
prompt
↓
model
↓
code + tests
↓
tests pass
↓
done
There is a circularity hidden inside that structure.
The same generation process can create both the implementation and the assumptions used to test it.
If both contain the same misunderstanding, everything can be green.
Forge separates responsibilities.
The planner can propose a plan.
The coder can produce an implementation.
Neither can mark the implementation as verified.
That authority belongs to the validation path.
The current validator uses three layers:
Layer 1
syntax / import / execution
Layer 2
requirements / obligations / acceptance
Layer 3
adversarial validation
All three must pass.
A candidate that merely runs is not enough.
A candidate whose own tests pass is not enough.
A candidate that satisfies only the easy interpretation of the requirement is not enough.
Technical deep dive: the actual three-layer validator
The current `ValidatorStage` creates the three validation layers independently:
```python
self.runtime_layer = RuntimeValidationLayer(
self.executor,
timeout_seconds,
)
self.obligation_layer = ObligationValidationLayer(
self.executor,
timeout_seconds,
quality_checker,
)
self.adversarial_layer = AdversarialValidationLayer()
```
During validation:
```python
layer1 = self.runtime_layer.validate(
code_artifact,
plan,
build_spec,
materialized,
workspace,
)
layer2 = self.obligation_layer.validate(
code_artifact,
plan,
build_spec,
materialized,
workspace,
)
layer3 = self.adversarial_layer.validate(
code_artifact,
plan,
build_spec,
materialized,
workspace,
)
```
The final decision is deliberately boring:
```python
passed = (
layer1.passed
and layer2.passed
and layer3.passed
)
```
But there is another important detail.
Forge preserves failure signatures:
```python
for layer in (layer1, layer2, layer3):
failures.extend(layer.failures)
for signature in layer.evidence.get(
"failure_signatures", []
):
self._append_unique(
signatures,
str(signature),
)
```
Those signatures can later drive **bounded repair**.
The repair process therefore has evidence about an observed failure instead of simply receiving:
```text
Try again.
```
The validator also records seals for the behavioral contract and the exact artifact being validated.
Source:
`core/forge/validator_stage.py`
Generated code runs in a sandbox
Another boundary is execution.
Production verification uses an isolated Docker backend.
The generated code does not inherit host credentials.
The sandbox is configured with restrictions including no network access, a read-only root filesystem, resource limits and an environment allowlist.
This is important for security, but also for epistemology.
A program that succeeds because it accidentally reads something from the host machine is not evidence that the generated artifact itself satisfies its contract.
And Forge can refuse validation entirely when isolation is required but unavailable.
Technical deep dive: refusing to silently downgrade isolation
The validator contains an explicit check:
```python
if (
self.require_isolation
and not self.executor.policy.isolated
):
return self._isolation_refusal(contract_seal)
```
That refusal becomes structured validation evidence with the signature:
```text
sandbox_policy_violation
```
All three layers are then marked as not executed.
This matters because silently switching from:
```text
isolated verification
```
to:
```text
run it locally and hope for the best
```
would change the meaning of `verified`.
The orchestration layer creates the production validator with isolation required:
```python
validator = ValidatorStage(
executor=process_executor,
require_isolation=True,
)
```
Source:
`forge.py`
`core/forge/validator_stage.py`
Packaging is a privilege, not the next line of the script
One design decision I particularly like is that validation does not merely return a boolean used by convention.
Packaging itself checks that the evidence matches the exact plan and artifact it is about to package.
So even after validation succeeds, the system still checks identity and integrity boundaries.
Technical deep dive: why a passing boolean is not enough
The packaging stage immediately refuses a failed validation artifact:
```python
if not validation.passed:
raise PackagingRefusedError(
"Packaging requires a passed ValidationArtifact."
)
```
But that is only the beginning.
It recomputes the expected behavioral contract seal:
```python
expected_contract_seal = behavioral_contract_seal(plan)
declared_contract_seal = validation.evidence.get(
"behavioral_contract_seal"
)
if declared_contract_seal != expected_contract_seal:
raise PackagingRefusedError(
"Packaging requires the validated "
"behavioral contract seal."
)
```
Then it checks that validation refers to the exact code artifact:
```python
expected_artifact_seal = artifact_validation_seal(
code_artifact
)
declared_artifact_seal = validation.evidence.get(
"validated_artifact_seal"
)
if declared_artifact_seal != expected_artifact_seal:
raise PackagingRefusedError(
"Packaging requires the matching "
"validated artifact seal."
)
```
And the validation artifact itself has an integrity seal:
```python
expected_validation_artifact_seal = (
validation_artifact_seal(validation)
)
if (
validation.integrity_seal
!= expected_validation_artifact_seal
):
raise PackagingRefusedError(
"Packaging requires the matching "
"validation artifact seal."
)
```
The point is not cryptographic magic.
The point is preventing an architectural mistake:
```text
validate A
modify A into B
package B using A's validation result
```
The package must correspond to the artifact and behavioral contract that actually passed.
Source:
`core/forge/packaging_stage.py`
Then I started testing the verifier
At this point the obvious temptation would be to demonstrate a few successful builds and write:
Forge generates verified software.
I don't think that would be a useful claim.
A system designed to verify generated software also needs to be tested against requirements it did not optimize itself around.
So Derivative now has a sequence of blind benchmark bundles.
The basic rule is:
create requirements
create independent acceptance oracles
freeze them
hash them
run Forge
preserve the result
After Forge has seen the bundle, it becomes a known regression corpus.
Fixing the implementation does not allow me to rewrite the old score.
A later execution is recorded as a post-fix replay.
Not as a new blind result.
This distinction has become one of the most important parts of the project.
A good failure is more useful than a fake 100%
Blind V10 produced an interesting result.
There were 12 cases:
6 expected verified
3 expected validation_failed
3 expected infeasible_proven
On the immutable baseline, seven of the twelve terminal statuses matched.
Four of the six positive cases reached verified and also passed their independent external oracles.
All three expected validation_failed cases were correctly rejected.
The infeasibility path exposed additional problems, including two execution exceptions.
That was progress.
It was not the interesting part.
V11 was.
The V11 frozen bundle again contained:
6 verified
3 validation_failed
3 infeasible
The immutable baseline scored:
6 / 12 terminal statuses correct
But the composition of that score matters much more than the number.
All three expected validation failures were rejected correctly.
All three infeasible cases were identified correctly.
And:
0 / 6 expected-valid programs reached verified
Every positive case ended as validation_failed.
No external oracle even ran for them.
That looked terrible.
It was also exactly the kind of failure I wanted the benchmark to expose.
The system had become too strict for the wrong reason
The six failed positive cases shared a failure signature:
uncompiled_hard_conditional
The requirements contained conditions with multiple consequences.
Conceptually:
if the input is invalid:
print an exact error message
exit with status 2
The conditional normalizer understood parts of these statements, but it did not correctly preserve the combined printed-output and exit-status obligations.
The validator then did the safe thing.
It refused to certify the candidate.
So Forge was fail-closed.
But the failure was still a bug.
The verifier was rejecting valid work because its own representation of the requirement was incomplete.
This is the point where the project changed for me.
Independent validation is not enough.
The validator itself becomes part of the system that must be challenged.
Technical postmortem: what V11 actually exposed
The six expected-valid V11 cases all included one or more invalid-input branches with multiple observable consequences.
Examples had the general form:
```text
If condition X occurs,
print exactly Y
and exit with status 2.
```
The conditional parser lacked sufficient normalization for the combined consequences.
That produced:
```text
uncompiled_hard_conditional
```
in all six positive baseline cases.
The important evidence rule here is that the V11 baseline remains unchanged.
The parser was subsequently corrected.
Known-case regression testing demonstrated that the original failure signature could be removed.
But those later results cannot be retroactively inserted into the blind baseline.
V11 remains what happened when the frozen benchmark first met that frozen version of Forge.
That distinction is essential:
```text
blind baseline
!=
post-fix replay
```
Otherwise a benchmark slowly becomes a collection of problems the implementation has already learned.
The complete evidence ledger is available here:
https://github.com/Daniele-Cangi/Derivative/blob/main/docs/BENCHMARK_EVIDENCE.md
The external oracle can be wrong too
There was another uncomfortable discovery.
I originally thought an independent external oracle would provide a relatively clean final authority:
Forge says verified
↓
external black-box tests
↓
accepted / rejected
Then some frozen oracles turned out to contain their own assumptions and defects.
One oracle supplied an extra program-name placeholder to an importable main(argv) interface.
Another used a fake stdin object that supported line iteration but not the standard read() method.
Older benchmark cases exposed fixture contradictions and harness assumptions that were not actually present in their requirements.
So Forge eventually gained an oracle preflight layer too.
Not because a failed external test should be ignored.
The opposite.
Because a failing test is evidence only if the test itself implements the contract it claims to evaluate.
Technical deep dive: verification now includes verification of the verifier
The benchmark pipeline distinguishes several different facts:
```text
internal verified
external oracle accepted
external oracle rejected
oracle invalid
```
These are not collapsed into one status.
For example, an internally verified program rejected by a valid external oracle is evidence of **false verification**.
That is serious and is preserved.
But an oracle with a demonstrably incoherent harness is classified separately as:
```text
oracle_invalid
```
The current repository also performs static preflight checks over frozen oracle sources.
A recent offline audit covered the historical frozen oracle corpus without executing Forge or making model calls.
It found several classes of harness problem while leaving all immutable benchmark inputs and original scores untouched.
The deeper principle is the same one used elsewhere in Forge:
```text
a claim does not become true
because it came from the component
we called "validator"
```
Evidence itself needs provenance and boundaries.
verified does not mean "correct forever"
This distinction is important.
Derivative is not claiming formal verification of arbitrary software.
verified currently means something narrower:
At that revision, the artifact satisfied the compiled requirement, execution, quality and adversarial contracts that Forge knew how to check.
That statement has boundaries.
A missing contract can still hide a defect.
A faulty requirement compiler can reject good software.
A validator can contain a bug.
An external benchmark can contain a bad oracle.
A future input can expose behavior none of the checks covered.
The system is designed around making those boundaries more visible rather than hiding them behind model confidence.
Where Derivative fits
There are actually two connected layers in the repository.
Derivative is the broader computational reasoning substrate.
It contains deterministic solvers, computational lenses, obligation handling, contradiction witnesses, audit mechanisms and other reasoning capabilities.
Forge is the software-building pipeline discussed in this article.
It uses that substrate for planning and reasoning, but adds the software-specific authority structure:
RequirementCompiler
↓
PlannerStage
↓
CoderStage
↓
ValidatorStage
↓
PackagingStage
The important architectural rule is:
planner ≠ authority on correctness
coder ≠ authority on correctness
validator ≠ authority to redesign the build
packager ≠ allowed to bypass validation
The boundaries are intentional.
The current scope is deliberately narrow
Forge currently focuses on greenfield Python artifacts such as:
- CLI applications
- REST services
- data pipelines
- libraries
It does not currently claim:
- arbitrary existing-repository editing
- arbitrary programming-language support
- frontend generation
- formal verification of arbitrary programs
- universal semantic coverage
I prefer those limitations to remain explicit.
The project already has enough difficult problems inside its current boundary.
The part I find most interesting
Code generation is getting cheap.
Generating another implementation is often easier than establishing that the implementation deserves to be trusted.
Derivative started as a reasoning project.
Forge pushed it into a more specific problem:
How do you build software
when no single component
is allowed to simply declare success?
The blind benchmarks pushed it one level further.
The coder can be wrong.
The validator can be wrong.
The external oracle can be wrong.
The benchmark producer can be wrong.
And a green CI run is evidence of exactly what that CI tested — not evidence of everything we wish it meant.
So the architecture is slowly becoming less about producing confident answers and more about separating claims from the evidence that is allowed to support them.
That is also why some of the most useful Derivative runs are failures.
A validation_failed result with precise evidence can tell me more about the system than an easy demo returning verified.
The generated code does not get to declare itself correct.
It turns out that neither does the verifier.
Derivative is open source:
https://github.com/Daniele-Cangi/Derivative
The technical architecture is documented here:
https://github.com/Daniele-Cangi/Derivative/blob/main/docs/FORGE_TECHNICAL_REFERENCE.md
The full benchmark ledger — including the ugly results — is here:
https://github.com/Daniele-Cangi/Derivative/blob/main/docs/BENCHMARK_EVIDENCE.md