Testing

Every test tier in this repo, the command that runs it, and the isolation policy that decides which tier a new test belongs in. For contributors adding or debugging tests.

Test authoring rules — how to pick a seam, bound a conductor fixture, keep time deterministic — live in .agents/skills/write-tests/SKILL.md, which this repo mandates for anyone touching tests. This page describes the suite; that skill describes the craft.

Commands

Run everything from src/conductor unless stated otherwise.

Task	Command
Install dependencies	`cd src/conductor && npm ci`
Full suite (what CI runs)	`cd src/conductor && npm test`
One file while authoring	`cd src/conductor && npm test -- test/<path>.test.ts`
Watch mode	`cd src/conductor && npm run test:watch`
Type check (`src/` only)	`cd src/conductor && npm run typecheck`
Type check including `test/`	`cd src/conductor && npm run typecheck:test`
Lint TypeScript	`cd src/conductor && npm run lint` (`npm run lint:fix` to autofix)
Type check a plugin	`cd plugins/<name> && npm ci && npm run typecheck`
Lint a plugin	`cd plugins/<name> && ../../src/conductor/node_modules/.bin/eslint --max-warnings=0 "*/.ts"`
Lint shell scripts	`bash test/lint_shell.sh` (from the repo root)
Check documentation links	`lychee --config lychee.toml docs README.md AGENT_INSTRUCTIONS.md src/conductor/README.md`
Build the engine	`cd src/conductor && npm run build`
Structural integrity of the repo	`bash test/test_harness_integrity.sh` (from the repo root)

npm test expands to:

node scripts/run-vitest.mjs run --reporter=dot --silent --slowTestThreshold=1800000 && echo 'AGGREGATE_TEST_SUITE_PASS'

AGGREGATE_TEST_SUITE_PASS is a human-readable shell success indicator. The pre-SHIP test_suite gate classifies the aggregate command’s exit code and records its evidence; it does not inspect this sentinel. The package script is also the containment boundary that creates the run-scoped temp root before Vitest loads, so use npm test -- <selectors> rather than invoking vitest run directly. The runner launches the package-local Vitest binary when available, so direct node scripts/run-vitest.mjs invocations need no PATH changes.

Changed-only BUILD laps, one full run before SHIP

This repository sets test_suite.verification.mode: changed in .ai-conductor/config.yml. Each BUILD lap’s test_suite gate runs only the tests Vitest relates to files changed since the merge-base with origin/main (--changed <merge-base>), plus the integrity suite. The FINISH validation fence requires full-suite evidence, so the aggregate npm test runs once before the PR opens. Any change to dependencies, config, test setup/helpers, a Vitest config, or scripts/ makes the lap run the full suite instead. The repository sets test_suite.verification.full_suite: once, so that full run happens once per feature: after the first full-suite PASS, later laps’ changed-only passes satisfy publication and the full suite is not re-run. See configuration.

The engine-dist guard

Thirteen test files spawn the real bin/ai-conductor, which exits 1 when src/conductor/dist is missing or its symlink dangles. dist is gitignored, so it does not exist in a fresh clone or git worktree after npm ci, and there is no pretest hook — nothing built it before the tests ran. It appeared only partway through a run, whenever some test happened to publish an engine, and every real-binary test scheduled before that point failed on exit 1.

test/global-setup.ts now calls ensureEngineDist (test/engine-dist-guard.ts) before the first test: it builds the engine when dist does not resolve and is a no-op otherwise, so a warm checkout pays nothing. When it builds, it prints engine-dist-guard: built the engine before the run. You do not need to run npm run build by hand before npm test.

This presents as flakiness — a cold worktree fails a handful of real-binary tests, and every re-run afterwards is green with no code change. If you see that pattern, check whether dist resolves before assuming a race in the code under test.

Deterministic daemon end-to-end fixture

test/engine/daemon-e2e-fixture.e2e.test.ts drives a committed fixture from test/fixtures/daemon-e2e/ through the real daemon claim, Conductor build, evidence, completion-gate, and local finish path. A scripted provider fake is the only external boundary; it makes real local Git commits while the internal pipeline remains production code. The negative case proves missing task evidence halts instead of completing.

This test belongs to the end-to-end tier described below.

End-to-end tier

Files named *.e2e.test.ts use real tmux sockets or drive the whole daemon. They are the slowest and most load-sensitive tests, so vitest.config.ts excludes them from the default suite (npm test, and therefore the test_suite gate). vitest.e2e.config.ts runs them with the ordinary runtime guards:

npm run test:e2e

CI runs the tier in its own conductor-e2e job, which ci-gate requires, so every PR still executes it. test/structural/e2e-tier.test.ts keeps the exclusion, the config, the script, and the CI job in step. Name a new test *.e2e.test.ts only when it needs a real tmux server or a whole-daemon drive; mock the process boundary otherwise. vitest.config.ts includes test/**/*.test.ts and excludes only smoke paths and *.smoke.test.ts names, so the existing conductor job’s npm test invocation runs it and reports through ci-gate.

Live-provider daemon E2E smoke

test/engine/daemon-e2e-live-claude.smoke.test.ts and test/engine/daemon-e2e-live-codex.smoke.test.ts are the opt-in real-provider legs over that deterministic fixture. They select provider descriptors and share one run body in test/fixtures/live-e2e-run-body.ts. Before dispatch, that body provisions an isolated provider home from test/fixtures/live-provider-home.ts (skills copied from the checkout under test, never operator state) and preflights every registry-rendered step command via test/fixtures/step-command-preflight.ts. An unresolved skill fails before any provider dispatch or token spend. Each leg asserts the same successful terminal state (DONE, with no HALT or park marker), fixture commit, and Task: 1 trailer, and reports its observed spend under the shared DAEMON_E2E_LIVE_TOKEN_CAP (default 300000). On failure, both use the shared dumpPipelineDiagnostics helper to print the daemon log, halt reason, task status, task evidence, and park markers.

CI sources that cap from the repository Actions variable named DAEMON_E2E_LIVE_TOKEN_CAP, falling back to 300000 when the variable is unset. Change the repository variable to recalibrate the release gate without editing the workflow; the same value continues to govern both provider legs.

Each leg needs its matching binary and credential: claude with CLAUDE_CODE_OAUTH_TOKEN, or codex with CODEX_API_KEY. Set SMOKE_FORCE_SKIP=capability:credentialed:claude or SMOKE_FORCE_SKIP=capability:credentialed:codex to skip one local provider leg. An absent credential is a named non-gating skip; a present credential makes that provider leg gate-enforced. Run either leg directly from src/conductor:

npm run smoke -- test/engine/daemon-e2e-live-claude.smoke.test.ts

The reusable Live daemon E2E workflow runs one matrix leg per provider, selecting only that provider’s smoke file. Each leg records its result independently, and the release gate requires at least one provider leg to pass; a failure from one provider remains visible but does not block publication when the other provider succeeds. The non-provider smoke tier remains independently mandatory. The separate require-live-provider-credential job requires at least one provider credential overall. To add a provider, add its descriptor, its provider-specific smoke file, and its matrix entry in the same change; do not create a parallel live-E2E workflow.

Linters

Three linters run in CI, each scoped to what tsc and bash -n cannot tell you.

Errors only — there is no advisory tier. Every enabled rule fails the build. Nothing is configured to warn: a rule too noisy to run at error is turned off outright and the reason recorded, because a warning nobody reads only teaches people to ignore the tool. npm run lint runs with --max-warnings=0; ShellCheck runs at --severity=error and never prints info/style findings. A clean run prints almost nothing.

Linter	Config	Scope	Threshold
ESLint (typescript-eslint, type-aware)	`src/conductor/eslint.config.mjs`	`src/*/.ts` and `test/*/.ts`	`no-floating-promises`, `await-thenable`, `no-misused-promises` (with `checksVoidReturn.arguments` off)
ShellCheck	`test/lint_shell.sh`	`bin/` (by shell shebang), `hooks//.sh`, `test/.sh`, `.github/scripts/*.sh`	`--severity=error`
lychee	`lychee.toml`	`docs/`, `README.md`, `AGENT_INSTRUCTIONS.md`, `src/conductor/README.md`	internal links only (offline)

The ESLint rule set is deliberately tiny. strict: true already covers the ground a stock preset would, so the only rules enabled are ones tsc structurally cannot provide: promises created and then dropped. This is an async daemon on execa/chokidar, where a dropped promise does not throw — it presents as a silent stall.

No formatter is configured, deliberately: Prettier or Biome across ~85k lines would produce a diff that buries every real change.

no-misused-promises runs with checksVoidReturn.arguments disabled. With it on it fires 90 times, every one of them an async callback passed to a void-return API (process.on('SIGINT', handler), commander .action()), where the only available fix is a void wrapper that changes nothing at runtime. require-await is not enabled for the same reason: 40 hits, dominated by async functions that conform to an awaited interface without needing await themselves.

ShellCheck’s error floor is the bar the tree passes today, chosen so the gate enforces from the day it lands instead of being advisory. Deferred: 91 findings at warning, 171 at info, 191 at style. Raising it is not mechanical — 45 of the 91 warnings are SC2319 against the deliberate assert "desc" "$(cmd; echo $?)" idiom used throughout the bash suite.

CHANGELOG.md is excluded from link checking on purpose: an entry correctly names the document that existed when it was written, so entries pointing at since-deleted pages are history, not rot.

Tests are linted on the same terms as engine source. ESLint resolves types through tsconfig.test.json, not tsconfig.json — the latter excludes test/, which would make the project service fail to resolve every test file and report 599 parse errors instead of linting them.

no-floating-promises matters more in a test than in engine code: an unawaited promise in a test does not fail the test, it leaks async work into whichever test runs next, which is how a suite starts failing in groups but passing in isolation.

Test tiers

1205 *.test.ts files live under src/conductor/test/. Vitest includes test/**/*.test.ts and excludes test/smoke/**, **/*.smoke.test.ts and **/*.e2e.test.ts (src/conductor/vitest.config.ts:17-20), so a bare npm test discovers 1185 files and excludes the 16 opt-in smoke files and 4 e2e files.

Directory	Files	Covers	Run just this tier
`test/engine/`	494	Mirrors `src/engine/`, including subdirectories for `engineer/`, `engineer/intake/`, `self-host/`, `otel/`, `halt-issues/`, `owner-gate/`.	`npm test -- test/engine`
`test/acceptance/`	175	Observable story and gate behavior across the minimum real internal path, with third-party boundaries faked.	`npm test -- test/acceptance`
`test/` (top level)	71	Cross-cutting suites not owned by one layer: `wiring-`, `build-progress-`, `backlog-priority`, `config-validation`, and tests of the leak guards themselves.	`npm test -- 'test/*.test.ts'`
`test/integration/`	41	Real collaboration between internal components; real temp files or local git only where git semantics are the subject.	`npm test -- test/integration`
`test/ui/`	14	Renderers, subscribers, dashboard snapshot and text, live region, prompt host.	`npm test -- test/ui`
`test/execution/`	25	Provider adapters, the `LLMProvider` contract, token usage, rate-limit parsing, sessions.	`npm test -- test/execution`
`test/smoke/`	4	Real binaries and real third parties. Excluded by default; seven additional `*.smoke.test.ts` files live beside the subsystem they exercise.	See Smoke tests.
`test/cli/`	6	CLI entry-point and argument behavior.	`npm test -- test/cli`
`test/structural/`	8	Meta-tests that parse the suite itself. See Structural meta-tests.	`npm test -- test/structural`
`test/types/`	3	Type-level contracts.	`npm test -- test/types`
`test/fixtures/`	5	Fixture helpers and their executable contract tests.	`npm test -- test/fixtures`

Runner shape (src/conductor/vitest.config.ts): pool: 'forks' with top-level maxWorkers: 2, testTimeout: 20000, hookTimeout: 30000, environment: 'node'. No reporter is configured in the file — it comes from the command line. Vitest 4 removed poolOptions and minWorkers; isolated generated smoke fixtures set maxWorkers: 1.

npm run typecheck covers src/ only — src/conductor/tsconfig.json sets "exclude": ["node_modules", "dist", "test"]. npm run typecheck:test (tsconfig.test.json) covers src/ and test/, and CI runs both. Use it to check the test you just wrote; Vitest transpiles without type-checking, so it will happily run a test that does not compile.

Beyond strict: true, both conductor configs and each plugin’s tsconfig.json enable noImplicitOverride, noUnusedLocals, and noUnusedParameters. Prefix a parameter that must stay in a signature with _. noUncheckedIndexedAccess and exactOptionalPropertyTypes are tracked separately in #2792.

Isolation policy

The rule, in one line: a test may reach a third party only if it is an explicitly named smoke test.

Level	May use	Must fake
Unit	The function, class, transition, or adapter contract under test.	Every process, network, LLM, GitHub, and filesystem boundary not under test — injected as a mocked adapter.
Integration	Real collaboration between internal components; real temp files and local git when git semantics are the subject.	Every third party.
Acceptance	The real application entry point, real internal wiring, locally controlled infrastructure.	Every third-party boundary, replaced with a faithful fake through the production adapter seam.
Smoke	The real binary or service.	Nothing — that is the point. Must live in `test/smoke/` or be named `*.smoke.test.ts`, and is excluded from the default command and CI.

“Third party” means LLM providers, hosted APIs, GitHub, email and payment services, webhooks, package registries, and other network services. The policy text is HARNESS.md:303-310, restated at repo level in AGENT_INSTRUCTIONS.md:60-64.

Project teardown acceptance coverage

test/acceptance/project-teardown-hook.acceptance.test.ts exercises the project-supplied bin/teardown hook through real local Git worktrees and executable scripts. It covers post-ship reaping, operator reclaim, and parked reconciliation while faking only GitHub and shipped-record boundaries. The hook’s environment, ordering before removal, contained non-zero exits, retained worktree skip, and configured timeout are observable assertions. Run it with:

cd src/conductor && npm test -- test/acceptance/project-teardown-hook.acceptance.test.ts

Worktree-removal classification guard

test/structural/worktree-removal-coverage.test.ts parses every production source module and detects real git worktree remove calls. When adding or moving a removal path, run the guard and classify the calling module before landing it:

Route a path that removes a provisioned feature worktree through runProjectTeardown before its removal call.
Otherwise add the module to that test’s WORKTREE_REMOVAL_EXEMPTIONS registry with a specific, non-empty reason. An exemption must still contain a real removal call; stale entries fail.
Do not add teardown to the shared worktree-shared.removeWorktree primitive. Its callers have different scope, so the primitive is deliberately an exempt pass-through and each caller is classified separately.

An unclassified path, a routed path with no teardown invitation, or a stale/empty exemption fails the standard suite. This is the enforcement described by .docs/decisions/adr-2026-08-07-worktree-removal-coverage-guard.md. Run it directly with:

cd src/conductor && npm test -- test/structural/worktree-removal-coverage.test.ts

Global guards

Four files run automatically and exist because each one prevented a real incident.

vitest.config.ts — the run-scoped `TMPDIR`

scripts/run-vitest.mjs creates one ai-conductor-vitest-run-* root beneath the ignored, checkout-local src/conductor/.vitest-tmp/ directory and points TMPDIR at it before loading Vitest. The config module then idempotently reuses that root. To select another writable filesystem, set AI_CONDUCTOR_TEST_TMP_BASE to an absolute path before running the suite:

cd src/conductor
AI_CONDUCTOR_TEST_TMP_BASE=/var/tmp/ai-conductor-tests npm test

An explicitly blank, relative, or NUL-containing override stops startup; the runner does not fall back to the system temporary directory.

os.tmpdir() reads TMPDIR on every call, so all ~1,426 mkdtemp(join(tmpdir(), '<prefix>-')) call sites across the suite — including ones written later — land inside that root with no test-file changes, and global-setup.ts deletes the root wholesale at teardown. Before this, the tests that never cleaned up left tens of thousands of directories in the operator’s real /tmp; on a tmpfs that exhausted inodes and broke unrelated production processes with ENOSPC.

The package runner must install it before Vitest loads because Vitest 4 creates its root tmpDir before evaluating the config; globalSetup is later still. A later redirect leaves Vitest’s own random-named SSR cache in the real tmpdir every run. test/tmpdir-redirect-propagation.test.ts runs inside a forked worker and asserts os.tmpdir() resolves to the run root, so the env propagation this all depends on is proven rather than assumed.

None of this excuses a fixture from cleaning up after itself — it bounds the damage when one does not.

Before it takes the original-temporary-directory baseline, global-setup.ts also reaps stale ai-conductor-vitest-run-* roots left by an interrupted earlier run from both the original temporary directory and the selected storage location. Each live root has an owner marker refreshed every minute; marked roots are eligible after three hours, while legacy unmarked roots wait 24 hours. The sweep retains its own root, live roots, unreadable markers, and every non-directory prefixed entry (including symlinks), and reports but does not fail the new run when it cannot remove a candidate. This keeps abandoned test artifacts from accumulating without risking a live run or a symlink target.

setup.ts

src/conductor/test/setup.ts runs before every test file (setupFiles) and sets three process-wide kill-switches:

NO_AUTOLAUNCH_ENV=1 — the engineer handoff’s default launch path becomes a no-op, so no test spawns a real tmux new-session -d 'ai-conductor daemon --continuous' that outlives its tmpdir.
AI_CONDUCTOR_NO_REAL_EXEC=1 — makeProductionGh and makeProductionGit refuse to exec. A test once added a needs-remediation label and a boom comment to a live PR; this is the guard against that.
AI_CONDUCTOR_ENGINEER_DIR — redirected to a fresh mkdtempSync directory unless a test already set it, so nothing writes into the operator’s real ~/.ai-conductor/engineer/.

global-setup.ts

src/conductor/test/global-setup.ts snapshots state before the run and diffs it after:

.pipeline under the test cwd — any added or modified file throws `.pipeline leak into <cwd> during test run: …`.
Daemon tmux sessions — leaked cc-daemon-* sessions are reaped; a killed session fails the run, an indeterminate one is logged non-fatally.
The real engineer signals store — a test-project-tagged line that leaked into it throws.
The original temporary directory’s top-level entries — anything that appeared during the run and is neither Vitest’s own direct project-temp entry nor known concurrent-tooling noise (self-host-*, claude-*, …) throws tmpdir-leak-guard: N temp entry/entries leaked into the REAL tmpdir …. That is a temp dir the TMPDIR redirect did not contain: a hardcoded /tmp, an os.tmpdir() value cached before the redirect, or a subprocess spawned without the inherited env. The Vitest exemption is exact: it applies only to the project directory created directly under the real temporary directory, not to its contents or any other entry. Fix the call site; widening IGNORED_TMPDIR_PREFIXES is only for a genuine false positive from a new concurrent tool.

The real temporary directory is shared with every concurrent suite on the host (other worktrees, daemon builds), so two shapes are attributed rather than blamed on this run. A bare 21-character nanoid is another Vitest process’s project directory — one started without the package scripts’ redirect — and is ignored. A fixture that must live in the real temporary directory (outside the run root’s GIT_CEILING_DIRECTORIES, or a short tmux socket path) takes its mkdtemp prefix from externalFixturePrefix(label) in test/tmpdir-leak-guard.ts: the acx-<run id>- name lets each run fail on its own leftovers and ignore another run’s in-flight fixtures.

The parked-marker leak guard (#1251) runs last of all, after the tmpdir check, so any more specific guard failure still throws first. It resolves the real repository’s .daemon/parked directory (via git rev-parse --git-common-dir, so it finds the main checkout’s ledger from any worktree), snapshots the marker files there before the run, and diffs them after. Any slug added, removed, or modified in the real ledger throws `park-leak-guard: parked marker ledger changed during test run: …` — a fixture that resolved the real ledger instead of a redirected one and parked or unparked a slug in it. It never repairs the ledger it detects a change in. An unexpected error while checking it is fail-safe, not fail-closed: it is logged as park-leak-guard: NOT enforced (fail-safe): … rather than failing the run, because the guard’s own resolution logic (a git subprocess) can legitimately fail outside the ledger itself.

It also sweeps stale tmpdir-rooted daemon sessions before the run and installs a best-effort SIGINT and SIGTERM reap, because Vitest’s global teardown only fires on a normal exit.

Leak guards

test/pipeline-leak-guard.ts, test/signals-leak-guard.ts, test/tmux-leak-guard.ts, test/tmpdir-leak-guard.ts, and test/park-leak-guard.ts hold the snapshot and diff logic. The tmux guard is fail-closed by design: killing a session requires both that the baseline snapshot succeeded and that the pane cwd resolves and is tmpdir-rooted. Missing either signal leaves the session running and logs tmux-leak-guard: NOT killed (fail-closed): ….

test/test-conductor.ts is the shared Conductor test double. It extends the production Conductor with a passing full-suite verifier so ordinary fixtures do not trip the native aggregate gate.

Structural meta-tests

test/structural/ enforces the isolation policy mechanically. Read it before adding a test that touches a process.

test-execution-policy.test.ts parses every .ts under test/ with the TypeScript compiler, excluding itself, smoke/, and *.smoke.test.ts, and fails on forbidden process calls. It watches exec, execFile, execFileSync, execSync, spawn, spawnSync, execa, and execaCommand, and rejects:

claude, codex, curl, wget as the executable;
npm install or npm ci; npx claude|codex; npm exec claude|codex;
any gh invocation carrying a network subcommand (api, auth, cache, gist, issue, label, pr, project, release, repo, run, search, secret, variable, workflow);
bin/setup in any form, including join('bin', 'setup').

The same test re-reads vitest.config.ts and reports vitest.config.ts: default run includes smoke tests if either exclusion glob has been removed.

fixture-portability.test.ts requires git init -b <branch> in all four exec shapes unless the call is --bare, commented out, or annotated // portability-ok: <reason>. It also flags .unref() under src/engine/** and hardcoded absolute /tmp/... string literals — use os.tmpdir().

module-header-caller-claims.test.ts scans leading comment blocks in src/engine/** for explicit no-caller claims (nothing imports, no callers/no importers, inert-module claims, and nothing calling, using, or invoking a backticked identifier). It fails only when a relative import or symbol reference contradicts a claim; truthful claims and matching prose below the leading comment block pass.

Smoke tests

Smoke tests are excluded from npm test by the two globs in vitest.config.ts. Run the complete, glob-discovered smoke tier from src/conductor:

npm run smoke

The smoke config includes test/smoke/** and every *.smoke.test.ts file. It currently discovers sixteen files. Each declares exactly one required capability beside the test:

Capability	Current files	Requirement
`hermetic`	`finish-record`, `surgical-finish-retry`	No external binary or credential.
`toolchain`	`publish-interrupted`, `backlog-priority`, `gh-version-floor`, `github-bot-credential`, `codex-provider`, `daemon-tmux`	A local toolchain or network-backed setup.
`credentialed:claude`	`claude-provider`, `claude-subagent-stream`, `build-token-auth`, `daemon-e2e-live-claude`, `git-guard-claude`	`CLAUDE_CODE_OAUTH_TOKEN` and the `claude` binary.
`credentialed:codex`	`daemon-e2e-live-codex`, `git-guard-codex`	`CODEX_API_KEY` and the `codex` binary.
`credentialed:pi`	`daemon-e2e-live-pi`	`PI_API_KEY` and the `pi` binary.

publish-interrupted.smoke.test.ts is toolchain, not hermetic: it creates a worktree and runs the real bin/setup, which may install dependencies. Select one production smoke file with npm run smoke -- <smoke_file>. npm run smoke runs in advisory mode by default (SMOKE_MODE unset or anything but gate): a file whose capability is unmet — no toolchain binary, or that provider’s credential or binary — is skipped, not failed, and a run that executed zero smoke assertions for a file still fails that file. Set SMOKE_MODE=gate for the fail-closed release variant: a complete-tier run fails when any required capability is unmet, and it requires at least one executed credentialed:claude or credentialed:codex file — an all-skipped credentialed tier can never pass. A selected credentialed file whose credential is absent is instead a passing non-gating skip; when its credential is present, that selected leg must execute and remains gate-enforced. SMOKE_FORCE_SKIP (comma- separated capability:<name> or file:<path> entries) forces an operator override in either mode; in gate mode a forced skip still counts as a failure. Every run ends with one smoke ledger: line per file naming its capability and outcome (ran, skipped (unmet: …), or failed (evidence: …)).

Keep the normal vitest.config.ts exclusions intact: ordinary unit and integration runs must never execute smoke tests.

Bash test scripts

40 .sh files live under test/. Only six ever execute:

test/test_harness_integrity.sh, run by CI and by the BUILD test_suite gate. See validation.
test/test_ci_detect_docs_only.sh and test/test_provider_skill_contracts.sh, executed by the integrity suite as checks 13 and 14.
test/test_docs_navigation.sh and test/test_docs_pages_smoke.sh, executed by the integrity suite as check 17. test_docs_navigation.sh in turn shells out to test/check_docs_navigation.sh, the offline contract checker it validates against fixture and real-tree cases.

ripgrep is optional, and skipping is silent

rg is not required to run the suite, but three scripts scope their own coverage on it and say so only in their output: test/test_bin_migrate_approval.sh and test/test_bin_migrate_multi_version_jump.sh print SKIP and exit 0, and integrity check 12b records a pass while skipping test/test_release_pr_workflow.sh. On a checkout without ripgrep the suite therefore runs smaller than CI does — CI installs it (.github/workflows/ci.yml) — while still reporting green. bin/install --check warns when rg is missing; nothing the harness ships at runtime needs it.

test/docs_pages.smoke.test.sh is a real, opt-in Pages probe — run it by hand after a default-branch deployment; it is never invoked from integrity or CI. test/run_browsable_documentation_site_acceptance.sh runs test_docs_navigation.sh and test_docs_pages_smoke.sh together as the deterministic acceptance suite for the hosted documentation site story; nothing invokes it automatically.

Known limitation. The rest — test_bin_update.sh, test_conduct_worktree.sh, the five test_install_*.sh, the ten test_examples_*.sh, test_skill_pipeline_contract.sh, test_release_unreleased_state.sh, docs_pages.smoke.test.sh, and run_browsable_documentation_site_acceptance.sh — are statically checked only: bash -n by integrity check 1 and ShellCheck by check 1b. Nothing executes them, in CI or locally, and no documented command runs them as a suite. Static analysis raises the floor but does not make them tests: a behavioral regression in bin/install, bin/update, or bin/setup is still caught by no automated gate. Run the relevant script by hand (bash test/test_bin_update.sh) when you change those surfaces. Tracked in #1021.

examples/ holds runnable end-to-end scenarios (interactive.sh, daemon.sh, engineer.sh, intake-loop.sh, each taking a tier s|m|l). Each creates a throwaway sandbox via sandbox_up and tears it down on exit. They invoke real flows, are not run by CI, and are not a scored regression suite.

The aggregate gate

test_suite is a real step in the linear sequence, not just a command. The engine reads its contract from .ai-conductor/config.yml:

test_suite:
  commands:
    - command: npm test
      working_directory: src/conductor
    - command: test/test_harness_integrity.sh
      working_directory: .
  timeout_seconds: 1800

The entries run in order, so the integrity suite runs only after the conductor suite passes. The --slowTestThreshold=1800000 in the npm script matches that 1800-second budget, suppressing slow-test warnings that would otherwise fire on every long run.

For where test_suite sits in the flow and what happens when it fails, see steps and gates.

CI

.github/workflows/ci.yml runs on pull requests targeting main:

changes — computes docs_only by piping git diff --name-only BASE HEAD through .github/scripts/ci-detect-docs-only.sh.
integrity — skipped when docs_only is true; installs shellcheck, then runs bash test/test_harness_integrity.sh.
shellcheck — skipped when docs_only is true; runs bash test/lint_shell.sh, the same script integrity check 1b calls.
lint — skipped when docs_only is true; npm ci then npm run lint in src/conductor, then each plugins/*/ is installed and linted with its own eslint.config.mjs, which shares the engine’s rule set.
typecheck — npm ci, npm run typecheck, and npm run typecheck:test in src/conductor, then npm ci and npm run typecheck in each plugins/*/.
conductor — npm ci, npm run build, npm test in src/conductor.
links — never skipped. Checks documentation links via lycheeverse/lychee-action.
ci-gate — if: always(); fails when any of the above is failure or cancelled. This is the required-status aggregator.

links is deliberately the one job with no docs_only gate. docs_only is true only when every changed path is under .docs/ (the internal spec-artifact tree) — see .github/scripts/ci-detect-docs-only.sh — and such a pull request skips every other job here. A link checker carrying the same gate would inherit that hole. Leaving it ungated also means the guarantee survives any future widening of the predicate, and it costs about six seconds with no npm install and no network.

Node comes from src/conductor/.tool-versions (nodejs 26.7.0) and the npm cache keys on src/conductor/package-lock.json.