Skip to content

Deterministic with AI on top

A generator is good at what follows a rule: file names, type names, function signatures, wiring, one test per specification. A coding agent is good at what needs judgment: the fields of an event, the business rule in a function, the data a test needs. Deterministic with AI on top splits a build between them. The generator writes the predictable part from the order and marks every place that needs judgment. The agent fills in only those places. Your checks decide whether the result is committed.

The result is code with your structure every time, and the agent’s effort spent where it counts. For how this compares with the other ways to build, see Build a builder.

It is one agent build, run by cratis build with a build kit. The kit carries the generator, and:

  • permits the generator as a command, so the agent may run it;
  • tells the agent, in its prompt, to run the generator first and then fill in only the marked places. The prompt passes the generator the path of the order with the {order} placeholder;
  • verifies with the generator itself, in a check mode that fails while a file, a test or a marked place is missing, followed by the type check and the tests.

The generator never overwrites a file, so running it again on a later order adds what is new and leaves the agent’s work alone.

Sample: a generator and an agent for TypeScript

Section titled “Sample: a generator and an agent for TypeScript”

This sample builds state change slices in plain TypeScript on Node.js, tested with Vitest, like the kit in Build with AI. Here the generator writes each slice’s file and test file, decides every name from the order, and leaves TODO(agent) where the agent works. Make a folder ts-vitest-generated with these files:

ts-vitest-generated/
├── build-kit.yaml # the manifest
├── AGENTS.md # the stack instructions
├── build-slice.md # the prompt for one build order
├── generator/
│ └── generate.mts # the slice generator
└── scaffold/ # the files of a new project, as in Build with AI
├── .gitignore
├── package.json
└── tsconfig.json

The scaffold folder holds the same three files as in the Build with AI sample.

The generator reads the order and, for every state change slice, writes src/<feature>/<slice>.ts and src/<feature>/<slice>.test.ts when they are not there yet. It takes the names from the specifications: event Invoice Issued becomes the type InvoiceIssued, and command Pay Invoice becomes the type PayInvoice and the function payInvoice. With --check, it changes nothing and fails while any file, test or TODO(agent) is missing. Node.js 22.18 or later runs it as it is.

ts-vitest-generated/generator/generate.mts
// The slice generator: writes the predictable part of every state change slice in a build order,
// with TODO(agent) where judgment is needed. With --check, it fails while any of that is missing.
// node generate.mts <order.json> write what is missing, never overwrite
// node generate.mts --check <order.json> exit 1 while a file, a test or a filled-in place is missing
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
type Slice = { id: string; name: string; type: string };
type Specification = { sliceId: string; name: string; given: string[]; when: string; then: string[] };
type Order = { scope: { feature: { name: string }; slices: Slice[] }; specifications: Specification[] };
const check = process.argv.includes('--check');
const orderFile = process.argv.slice(2).find(argument => argument !== '--check') ?? process.env.STUDIO_BUILD_ORDER;
if (!orderFile) throw new Error('Name the build order: node generate.mts <order.json>');
const order: Order = JSON.parse(readFileSync(orderFile, 'utf8'));
const header = 'Generated from the build order by the slice generator.';
const todo = 'TODO(agent)';
const safe = (name: string) => /^[A-Za-z][A-Za-z0-9]*$/.test(name);
// "event Invoice Issued" -> "InvoiceIssued"; "command Pay Invoice" -> "PayInvoice".
const typeName = (element: string) => element.replace(/^(event|command)\s+/, '').split(/\s+/).map(word => word[0].toUpperCase() + word.slice(1)).join('');
const camel = (name: string) => name[0].toLowerCase() + name.slice(1);
const isEvent = (element: string) => element.startsWith('event ');
function source(slice: Slice, command: string, events: string[]): string {
const lines = [`// ${header}`, `// ${slice.name}: a state change in ${order.scope.feature.name}. Keep the names; fill in only the marked places.`, ''];
for (const event of events) {
lines.push(`export type ${event} = {`, ` type: '${event}';`, ` // ${todo}: the fields of ${event}.`, '};', '');
}
lines.push(
`export type Event = ${events.length > 0 ? events.join(' | ') : 'never'};`, '',
`export type ${command} = {`, ` // ${todo}: the fields of ${command}.`, '};', '',
`export function ${camel(command)}(history: Event[], command: ${command}): Event[] {`,
` // ${todo}: decide from the history which events ${command} appends, or throw an Error to reject it.`,
` throw new Error('${command} is not built yet.');`,
'}');
return lines.join('\n') + '\n';
}
function test(slice: Slice, command: string, specifications: Specification[]): string {
const lines = [
`// ${header}`,
`// One test per specification of ${slice.name}, named like it. Fill in only the marked places.`,
`import { describe, expect, it } from 'vitest';`,
'import {', ` ${camel(command)}, type Event, type ${command}`, `} from './${slice.name}';`, '',
`describe('${slice.name}', () => {`];
for (const [index, specification] of specifications.entries()) {
if (index > 0) lines.push('');
lines.push(
` // Given ${specification.given.join(', ') || 'nothing'}; when ${specification.when}; then ${specification.then.join(', ')}.`,
` it('${specification.name.replace(/'/g, "\\'")}', () => {`,
` const history: Event[] = [${specification.given.length > 0 ? `/* ${todo}: ${specification.given.map(typeName).join(', ')} */` : ''}];`,
` const command: ${command} = { /* ${todo} */ };`,
specification.then.every(isEvent)
? ` expect(${camel(command)}(history, command)).toEqual([/* ${todo}: ${specification.then.map(typeName).join(', ')} */]);`
: ` expect(() => ${camel(command)}(history, command)).toThrow(); // ${todo}: ${specification.then.join(', ')}`,
' });');
}
lines.push('});');
return lines.join('\n') + '\n';
}
const feature = order.scope.feature.name;
if (!safe(feature)) throw new Error(`The feature name '${feature}' cannot be used as a folder name.`);
const problems: string[] = [];
for (const slice of order.scope.slices) {
if (slice.type !== 'stateChange') throw new Error(`The generator only writes state change slices, not ${slice.name}.`);
if (!safe(slice.name)) throw new Error(`The slice name '${slice.name}' cannot be used as a file name.`);
const specifications = order.specifications.filter(specification => specification.sliceId === slice.id);
const elements = specifications.flatMap(specification => [...specification.given, ...specification.then]);
const events = [...new Set(elements.filter(isEvent).map(typeName))];
const command = specifications[0] ? typeName(specifications[0].when) : slice.name;
const files = {
[`src/${feature}/${slice.name}.ts`]: source(slice, command, events),
[`src/${feature}/${slice.name}.test.ts`]: test(slice, command, specifications)
};
for (const [file, content] of Object.entries(files)) {
const existing = existsSync(file) ? readFileSync(file, 'utf8') : undefined;
if (check) {
if (existing === undefined) problems.push(`${file} is missing.`);
else if (!existing.includes(header)) problems.push(`${file} lost its generated header.`);
else if (existing.includes(todo)) problems.push(`${file} still has places marked ${todo}.`);
} else if (existing === undefined) {
mkdirSync(`src/${feature}`, { recursive: true });
writeFileSync(file, content);
console.log(`Wrote ${file}`);
} else {
console.log(`Kept ${file}, which is already there.`);
}
}
const tests = existsSync(`src/${feature}/${slice.name}.test.ts`) ? readFileSync(`src/${feature}/${slice.name}.test.ts`, 'utf8') : '';
for (const specification of specifications.filter(specification => !tests.includes(`it('${specification.name.replace(/'/g, "\\'")}'`))) {
const message = `src/${feature}/${slice.name}.test.ts has no test named '${specification.name}'.`;
if (check) problems.push(message); else console.log(`Add a test: ${message}`);
}
}
if (problems.length > 0) {
console.error(problems.join('\n'));
process.exit(1);
}

For the example backend order, it writes this source file. It already compiles; only the marked places are left:

src/Invoicing/PayInvoice.ts
// Generated from the build order by the slice generator.
// PayInvoice: a state change in Invoicing. Keep the names; fill in only the marked places.
export type InvoiceIssued = {
type: 'InvoiceIssued';
// TODO(agent): the fields of InvoiceIssued.
};
export type InvoicePaid = {
type: 'InvoicePaid';
// TODO(agent): the fields of InvoicePaid.
};
export type Event = InvoiceIssued | InvoicePaid;
export type PayInvoice = {
// TODO(agent): the fields of PayInvoice.
};
export function payInvoice(history: Event[], command: PayInvoice): Event[] {
// TODO(agent): decide from the history which events PayInvoice appends, or throw an Error to reject it.
throw new Error('PayInvoice is not built yet.');
}

and this test file, with one test per specification, named like it:

src/Invoicing/PayInvoice.test.ts
// Generated from the build order by the slice generator.
// One test per specification of PayInvoice, named like it. Fill in only the marked places.
import { describe, expect, it } from 'vitest';
import {
payInvoice, type Event, type PayInvoice
} from './PayInvoice';
describe('PayInvoice', () => {
// Given event Invoice Issued; when command Pay Invoice; then event Invoice Paid.
it('Paying an open invoice', () => {
const history: Event[] = [/* TODO(agent): InvoiceIssued */];
const command: PayInvoice = { /* TODO(agent) */ };
expect(payInvoice(history, command)).toEqual([/* TODO(agent): InvoicePaid */]);
});
});
ts-vitest-generated/build-kit.yaml
kitVersion: "1"
name: ts-vitest-generated
version: 1.0.0
description: TypeScript slices on Node.js whose structure a generator writes and a coding agent completes.
sliceTypes: [stateChange]
scaffold:
path: ./scaffold
commands:
- npm install
instructions: ./AGENTS.md
prompt: ./build-slice.md
verify:
- node .cratis/build/kit/generator/generate.mts --check {order}
- npx tsc --noEmit
- npx vitest run src/{feature}
git:
commit: "Build {slices}"
permissions:
paths: ["src/**"]
commands:
- node .cratis/build/kit/generator/generate.mts
- npx tsc --noEmit
- npx vitest run
requires:
- command: node
version: ">=22.18"
- command: npm

What differs from the Build with AI kit:

Lines What they do
sliceTypes Only state changes: the generator writes nothing else. The CLI refuses other orders before the agent starts
verify First the generator’s check, with the order’s path filled in for {order}: every file, every test and no TODO(agent) left. Then the type check and the tests
permissions.commands The generator, so the agent can run it, with or without --check. A permitted command is a prefix, so the arguments after it are allowed
requires Node.js 22.18 or later, which runs the generator’s TypeScript directly

The generator lives in the kit, so cratis build init installs it into the project at .cratis/build/kit/generator/generate.mts, and the commands name it there. Install the kit with init rather than passing --kit <folder> to a build, or the path does not exist.

ts-vitest-generated/AGENTS.md
# TypeScript slices, generated and completed
This project is a set of event-sourced slices in plain TypeScript on Node.js. A generator writes the structure of every
slice; you complete it.
- The slice generator, `node .cratis/build/kit/generator/generate.mts <order>`, writes `src/<feature>/<slice>.ts` and
`src/<feature>/<slice>.test.ts` for every slice in a build order. It decides the file names, the type and function
names, and one test per specification. It never overwrites a file.
- Every place that needs judgment is marked `TODO(agent)`: the fields of each event and command, the body of the
function that decides which events a command appends, and the data and expectations of each test.
- Fill in those places and remove the marker. Change nothing else: keep the generated header comments, the names, the
test names, and the files where they are. Do not add files.
- An event has a `type` field with its name. The decide function returns the events the command appends, or throws an
`Error` when the history rejects it. It does no I/O.
- Strict TypeScript, no `any`, no new packages.
ts-vitest-generated/build-slice.md
# Build {slices}
Build {slices} in the feature `{feature}` of `{application}`. The build order is at `{order}`; this is attempt {attempt}.
1. Run the slice generator first: `node .cratis/build/kit/generator/generate.mts {order}`. It prints the files it wrote or kept, and any test that is missing.
2. Fill in every place marked `TODO(agent)` in those files, as the instructions describe. Add a test the generator says is missing, named exactly like its specification.
3. The order's `specifications` are the acceptance criteria. `context.hints` holds each slice's description, the summary of what it is supposed to do; follow it. `context.feedback` comes from people on the board, and `context.previousFailure` says why the last attempt failed. Where the order does not say something, such as a field, decide it sensibly and list it as an assumption.
4. Run `npx tsc --noEmit`, then `npx vitest run src/{feature}`, then `node .cratis/build/kit/generator/generate.mts --check {order}`, until all three pass. Run one command at a time: chained or piped commands are refused.
5. Report the result of each specification as `passed` or `failed`, from its test. Write the outcome with your file editing tool.
  1. Check the kit:

    Terminal window
    cratis build kit validate ./ts-vitest-generated
  2. Run the generator by hand on the example backend order, saved as order.json, in an empty folder with the scaffold. It writes the two files above, and its check fails until the marked places are filled in:

    Terminal window
    node ../ts-vitest-generated/generator/generate.mts ../order.json
    node ../ts-vitest-generated/generator/generate.mts --check ../order.json
  3. Create a project and install the kit in it:

    Terminal window
    mkdir shop && cd shop && git init
    git commit --allow-empty -m "Start the shop"
    cratis build init --kit ../ts-vitest-generated --agent claude
    git add . && git commit -m "Install the ts-vitest-generated kit"
  4. Build the order:

    Terminal window
    cratis build once --order ../order.json --agent claude --outcome ../outcome.json
  5. Look at the commit with git show. The generated header comments, names and test names are as the generator wrote them; the fields, the rule and the test data are the agent’s.

  • The structure never drifts. Every slice has the same files, names and test layout, however the agent phrases its code, because the agent does not decide them. Reviews get easier: a reviewer reads the marked places, not the boilerplate.
  • The agent works on less. It reads two short files with clear gaps instead of designing a slice, so it finishes sooner, uses fewer tokens and goes wrong less often.
  • The checks know the structure. The generator’s own check knows exactly what must exist, so it catches a missing test or a gap the agent skipped, which a type check and a test run cannot see: an empty test passes.
  • It grows with you. Every time you notice the agent writing the same thing on every slice, move it into the generator and mark less.
  • Permit fewer paths. permissions.paths can name the slice files’ folder only. The CLI commits only files inside it, whatever the agent changed.
  • Check more in verify. Add a linter or an architecture check that the agent cannot argue with. Make every command one the agent may run too, so it can see a failure before the CLI does.
  • Generate more. Wiring, registration and shared types are deterministic: generate them, and leave the agent the rules and the test data.
  • Write conventions down. Put naming and style rules in the instructions, and the ones a program can check in verify.

The inverse: AI writes, deterministic checks decide

Section titled “The inverse: AI writes, deterministic checks decide”

You can split the work the other way round: the agent writes all the code, as in Build with AI, and deterministic tools hold it to your conventions. Put them in the kit’s verify, after the type check and the tests:

verify:
- npx tsc --noEmit
- npx vitest run src/{feature}
- npx prettier --check src
- npx eslint src
permissions:
commands:
- npx tsc --noEmit
- npx vitest run
- npx prettier
- npx eslint

Permit the same tools, so the agent runs them and fixes what they find before it reports. A command in verify that rewrites files, such as npx prettier --write src, runs before the commit, so its changes to the files the agent reported are committed with them. Add the tools to the scaffold’s package.json, with their configuration, so they are installed in every new project.