Quality gates
Knossos uses one versioned quality profile locally, in Git hooks, and in CI. The recommended command is container-backed, so host tool versions do not affect results:
tools/quality-container fast
tools/quality-container fulltools/quality-container builds the quality image, mounts the checkout read-only at /workspace so the gate lane can reach the history, mounts coverage/ back into the checkout, and runs tools/quality inside.
tools/quality also runs on the host, when the tools are installed there. As root, Composer refuses to run its scripts until you say so, and a run that hits that aborts at composer lint:
COMPOSER_ALLOW_SUPERUSER=1 tools/quality fastcomposer.json raises Composer's process-timeout to 1200 seconds, because the default 300 would kill composer test partway through the suite on a slow machine.
Profiles and lanes
fast is what the commit hook runs. full is what the push hook and CI run. The script is one linear gate, and locally it runs as one. CI runs it as jobs that run at the same time, because most of the work does not depend on the rest of it: five lanes in a matrix, and the coverage lane as several shards and a merge (see how CI runs it). Both commands take an optional second argument naming a lane, so a lane that failed in CI can be reproduced here:
tools/quality-container full release
tools/quality full static| lane | profiles | holds |
|---|---|---|
static | both | Composer and npm validation, linters, PHP-CS-Fixer, PHPStan, ESLint, typecheck:mod, Prettier, markdownlint, Ruff, mypy, the documentation and repository checks, pre-commit config, ShellCheck, Hadolint, docker compose config |
tests | both | the PHPUnit suite (in CI on pushes to main only, see below) and the hook shell tests, the TypeScript worker's checks, pytest, scanner conformance, the mod's vitest suite and plugin checks |
rust | both | cargo fmt, clippy and test, and conformance of the Rust worker, which are slow and self-contained |
release | full | Composer and npm audits, the external link check, the MCP Inspector smoke, the runtime image build and doctor, the release lifecycle, supply chain, the benchmark |
coverage | full | the pcov run and the coverage floors; in CI, split into shards and a merge |
gate | full | Knossos scanned by Knossos, held to the budgets in knossos.json |
Omitting the argument runs every lane the profile has, which is the local default.
Checks that do not run
A check whose tool is missing is not skipped in silence. It prints NOT run: <check> (<reason>) where it would have run, and the run ends with a list of every check that did not run, whether it passed or failed:
tools/quality: 2 check(s) NOT run:
- Hadolint (hadolint not found)
- the external link check (KNOSSOS_EXTERNAL_LINKS=0)On your machine that is a warning: install the tool, or use tools/quality-container, to run the check. In the quality container (KNOSSOS_QUALITY_CONTAINER=1) the image is built to hold every tool, so a missing one fails the run at once. That covers cargo and the Rust worker's conformance, cargo-audit, pre-commit, ShellCheck, Hadolint, docker compose, the Claude CLI, the staged register.tsx type-check and the lint list freshness check. A check you turn off on purpose, such as the external links with KNOSSOS_EXTERNAL_LINKS=0 or PHPUnit in the tests lane with KNOSSOS_PHPUNIT_IN_COVERAGE=1, is listed the same way and never fails the run. KNOSSOS_PHPUNIT_IN_COVERAGE=1 is refused with exit 64 under fast, which has no coverage lane to run PHPUnit instead.
The lint lists
php -l, PHP-CS-Fixer, PHPStan, ESLint and ShellCheck read their files from tools/lint-files.txt, which php tools/lint-files.php --write derives from git ls-files: every tracked file of its kind, by extension or by its #! line, except the scanner input under tests/Fixtures/. The quality image carries no .git, so the list is committed, and the static lane derives it again and refuses a committed copy that differs. When you add, rename or remove a script, a PHP file or a JavaScript or TypeScript module, run:
php tools/lint-files.php --writeand commit the result. ESLint runs with --max-warnings 0, so a warning fails the lane. The container lanes need the checkout for this: tools/quality-container and CI mount it read-only at /workspace and name it with KNOSSOS_GATE_REPO.
The split follows independence. Coverage shows why it cannot follow language: a single PHPUnit run under pcov produces the PHP figure and, from the worker subprocesses that run drives, the JavaScript and Python figures. Splitting it per language would measure three suites that never exercise the workers and report floors nothing earns.
Mutation testing is in no profile, see adversarial testing.
The layering it holds
knossos.json makes each namespace under src/ a boundary and puts the namespaces in four tiers. A namespace may depend on its own tier and the tiers below it, never on one above:
| Tier | Namespaces |
|---|---|
| foundation | Cancellation, Filesystem, Result, Scanner, Bundle, Classification, Store, Roots |
| analysis | Discovery, Boundary, Configuration, Git, Reconciliation, Enrichment, Scan |
| services | Query, Maintenance, Runtime, Watch |
| adapters | Mcp, Cli (and Application above them) |
Each language worker also may not depend on core or on another worker. A dependency that breaks either rule is a boundary_violations finding, so the gate lane fails on it. When a new dependency is the right design, move the namespace to the tier it belongs in and say why in the change, rather than denying less.
What the gate lane enforces
The lane scans two trees, the commit the change is measured against and the change itself, then holds the second scan to the budgets and policies in knossos.json. All six budgets are live. new_cycles and hub_degree_growth compare the two scans. boundary_violations, error_diagnostics, warning_diagnostics and unreferenced_candidates are read off the second one.
The baseline is what CI works out from the event, a pull request's target commit or the tip a push replaced. KNOSSOS_GATE_BASE_REF names it explicitly. Failing that the lane takes the point the branch left origin/main, or else main. When that point is HEAD itself, HEAD is a commit of the default branch, and only then does the lane step back to HEAD^. A branch that shares no history with either is refused rather than measured against its last commit, which would let everything earlier on the branch through. If nothing resolves, the lane stops. Scanning one tree twice would zero both delta budgets and report a pass that checked nothing. tools/gate-baseline makes that choice, and tests/shell/gate-baseline.sh holds it to each of these cases.
When the gate fails, the lane prints every budget's value against its limit and names the ones that failed (tools/gate-report.php), so the CI log says which budget broke without a local rerun.
Both trees come from git archive and are scanned at the same path. The path matters: a project's identity is its root, so a baseline scanned elsewhere would be a different project with nothing to compare against. Extracting both sides the same way matters too, since reading the active side off the container's baked source would diff the change against .dockerignore along with it. One consequence is that git archive HEAD is the committed tree, so uncommitted work is not gated. That is what CI measures anyway.
Budgets and policies are read from the tree under test, not the baseline, so a change that needs a limit raised is reviewed as the diff that raises it.
The image bakes this source without .git, so the lane is handed the checkout on a read-only mount and given the baseline commit by name. The CI lane checks out with fetch-depth: 0, because a shallow clone cannot reach its own baseline.
How CI runs it
The quality image is built in two parts. The dependency stages of the Dockerfile (system packages, PHP extensions, Composer and npm dependencies, the Rust toolchain and the compiled Rust worker) are published to the repository's registry under a key: a hash of their inputs, printed by tools/quality-deps-key (--inputs lists them), plus the UTC date and a stamp of Debian's security archive. Each job pulls that image and adds the commit's source on top in a build of a few seconds. The key changes, and the run first rebuilds and publishes the dependency images, when the Dockerfile changes in anything but comments and the release version, when a Composer or npm manifest or lockfile changes, when anything under workers/rust changes, on the first run of each UTC day, so Debian's fixes reach the image, and when Debian publishes a security release, so a fix that lands during the day does not leave the release lane's vulnerability scan red until midnight. That run takes several minutes longer. A scheduled run shortly after midnight UTC does the daily rebuild, so the day's first push usually finds the images published. A pull request from a fork, or from Dependabot, cannot write to the registry and builds the whole image in the run instead.
Coverage runs as several coverage-shard jobs and one coverage job after them. Each shard runs tools/coverage run --shard=I/N, the test files tools/phpunit-shard assigns it, and uploads its raw data. The shard count is the length of the matrix in quality.yml. The files are balanced by the times in tests/phpunit-shard-weights.json; a file missing there weighs the median. To refresh the weights, download the coverage-shard-* artifacts of a run and pass their junit.xml files to tools/phpunit-shard --weigh. The coverage job runs tools/coverage merge and then tools/coverage report, which enforces every floor exactly as an unsharded run does. The merge refuses unless every shard of the same N finished, left PHP coverage data, and every test file ran in exactly one shard, the one it was assigned to, so a shard that lost files cannot pass with a lower figure. To reproduce one shard locally, run tools/coverage run --shard=2/4 in the quality image.
An aggregating job named quality fails unless every lane, every shard and the merge succeeded, which is the check branch protection requires: a job that is skipped or cancelled when it should have run fails it just as a red one does. The rule is tools/ci/quality-verdict, tested by tests/shell/quality-verdict.sh. A second push to a pull request cancels the run in progress.
CI runs PHPUnit once, on pull requests and on pushes to main alike. The tests lane gets KNOSSOS_PHPUNIT_IN_COVERAGE=1 and runs only the shell tests from composer test, because the suite already runs under pcov in the coverage shards. Locally the variable is unset and tools/quality runs the whole suite.
A release-please pull request runs static alone and skips the coverage jobs. Its diff is version files, a manifest and a changelog entry, so every other lane would re-verify code identical to the main it was cut from, which had just passed. The full matrix runs again on the push to main after it merges, so nothing reaches a tag unchecked.
What counts as a release-please pull request is decided by its content, with tools/ci/release-pr, not by its author or its branch name: release-please opens it with a personal token, and anyone can push a branch of that shape. It must come from this repository, carry the autorelease: pending label, change only the manifest, CHANGELOG.md, version.txt and the extra-files in release-please-config.json, and change nothing in those extra-files but the version, on lines release-please's markers allow: a line that carries x-release-please-version itself, or one between x-release-please-start-version and x-release-please-end, with the start marker within the ten lines of context the workflows ask git for. Anything else runs every lane. The diff it judges is the merge commit the lanes test against its base, and on a pull request both tools/ci/release-pr and tools/ci/quality-verdict run from the base commit, so a pull request cannot pass with a broken copy of either.
Pull request titles are checked by the pr-title workflow with tools/check-commit-style.sh, the same script the commit-msg hook uses, because the squash-merged title is what release-please reads. The workflow checks the branch's commits too, and passes --ci, which drops the local exemptions for Merge, Revert ", fixup!, squash! and amend! subjects: a title can be typed to look like one, and a rebase merge lands them on main as they are. The release subject chore(main): release <version> passes only when tools/ci/release-pr says the pull request is release-please's own.
The documentation checks
php tools/documentation-check.php runs in the static lane. It reads README.md, CONTRIBUTING.md and every Markdown file under docs/, skills/ and plugins/, and fails on:
- a link or an image whose target file does not exist;
- a
#anchorthat matches no heading in the file it points at (#L12and#L12-L20line anchors are left to GitHub); - a
tools/orbin/command in ash,shellorbashblock that does not exist; - a link scheme other than
https://.
Fenced code is not prose and is skipped, following the CommonMark fence rules. php tools/generate-reference.php --check fails when docs/reference/cli.md, mcp-tools.md or api.md differ from what the code generates, and php tools/api-documentation-check.php and php tools/docstring-report.php hold the docblock coverage. The API check also fails when it finds no interface method to check at all, and npm run lint:md fails when it finds no Markdown to lint. Each of these checks takes another tree (--docs=DIR for the reference, --root=DIR for the other two), which is how the tests plant a stale or undocumented file without touching the repository.
The constants every worker keeps its own copy of (the launch interpreters, the built-in ignore rules, the symlink hop limit, the invalid-limits refusal and the scan/input_hashes part size) come from schemas/worker-constants.json. Edit that file and run php tools/worker-constants.php, which rewrites each copy between its BEGIN GENERATED worker-constants:<key> and END GENERATED comments. Only the core's IgnoreMatcher reads the JSON itself; every worker, the PHP one included, holds a literal, so a worker copied without the rest of the installation still runs. php tools/worker-constants.php --check runs in the static lane and fails on a copy edited by hand, naming the file.
External links are fetched only by php tools/documentation-check.php --external, in the release lane. A dead link there fails the lane. tools/quality runs the check whenever KNOSSOS_EXTERNAL_LINKS is unset, under set -e, so it also fails the quality check on main and a local tools/quality full, which is the pre-push hook, and blocks the push.
A dead link means a 404, a 410, any other error status, a DNS failure or a TLS failure. A rate limit is not a dead link: GitHub answers a shared CI runner with HTTP 429 on pages that are fine. The checker retries a 429, and a 503 that sends Retry-After, at most twice, waiting as long as Retry-After says and never more than 30 seconds in total for one URL. If the limit persists, it prints warning: external link rate-limited and not checked: <url> and the check still passes. I chose retry-then-warn over adding github.com to the skipped hosts, because most GitHub links are answered normally and a skip would stop catching the ones that really die. The tests replace curl with the KNOSSOS_CURL variable.
Pull requests set KNOSSOS_EXTERNAL_LINKS=0, because whether a web server is answering right now is not a property of the change under review. Set it to 0 yourself to skip the fetch locally. The badge hosts img.shields.io and mcpobservatory.com are counted but never fetched.
What the release lane holds its reports to
Four steps of the release lane used to pass on an exit status that said nothing about the result. Three now hand their output to a check of its own:
- The MCP Inspector's
tools/listanswer must name exactly the tools indocs/reference/mcp-tools.md(tools/inspector-tools-check.php). - The lifecycle rollback must leave the database holding the backup's snapshots, with the same one active, after an upgrade scan that added one (
tools/lifecycle-restore-check.php). - The quality image's Trivy scan keeps
--exit-code 0, because that image is the gate's own toolchain and is never shipped, but its fixed HIGH and CRITICAL findings are held totools/quality-image-vulnerabilities.txt(tools/trivy-allowlist.php). On a pull request a finding not on that list is printed underNEW FINDINGS (report only on PRs)and does not fail, because it arrives with a database update, not with the change. A push tomain, the release and a local run fail on it. CI chooses withKNOSSOS_QUALITY_IMAGE_FINDINGS(reportorfail, the default). A finding that is no longer found is printed, so the list can shrink.
The Cosign signature check is a smoke test and says so: the key pair is made on the spot and thrown away, so it shows that signing and verifying work, not who built the image.
The Claude Code mod
The mod has three checks of its own, and how the mod is built explains each of them. Here is where they run:
npm run typecheck:modruns instatic.npm run test:modruns intests: the vitest specs underhooks/lib,hooks/mod,tools/andtools/capture.claude plugin validateandclaude plugin testrun intests, against a staging copy of the plugin. The quality image pins the Claude CLI (@anthropic-ai/claude-code@2.1.314). A machine without theclaudecommand reports them as not run.- The staged
register.tsxtype-check runs intestsagainst the engine's own declarations, committed intools/claude-code-api/, or theindex.d.tsKNOSSOS_CLAUDE_CODE_TYPESnames. Where they are missing the check is reported as not run, which fails the quality container.
Regenerating the Claude Code declarations
The CLI does not ship its API declarations. It builds them at run time and writes them beside a mod when a session loads it, so they are generated and committed:
tools/claude-code-typesIt installs the Claude Code version the Dockerfile pins into a temporary prefix, stages the plugin, and starts one interactive session under script with the plugin loaded, its own config directory and an invalid API key. A headless claude -p run no longer writes the declarations as of 2.1.303. The session writes them as soon as it loads the mod and is stopped there, so nothing reaches your account or your own configuration. The script copies claude-code/ and claude-code-tools/ into tools/claude-code-api/ and records the version in tools/claude-code-api/VERSION. It needs npm, node, script from util-linux and the network.
The static lane runs php tools/claude-code-types-check.php, which fails when the Dockerfile's pin, the VERSION stamp and the version the declarations name in their first line disagree. When you move the pin, run the script and commit what it writes in the same change.
Tool inventory
| Surface | Enforced tools |
|---|---|
| PHP | PHP syntax, PHP-CS-Fixer 3.95.27, PHPStan 2.3.1 (level 5, over the tests too) |
| JavaScript/TypeScript worker | ESLint 10.13.0, Prettier 3.9.10, TypeScript compiler checks, vitest suite |
| Claude Code mod | ESLint, npm run typecheck:mod, vitest, claude plugin validate/test |
| Python worker | Ruff 0.17.1, mypy 2.4.0, pytest, isolated compile/runtime tests |
| Rust worker | cargo fmt, Clippy with -D warnings, cargo test, cargo audit |
| Markdown | Prettier, markdownlint 0.42.0 |
| JSON/JSONC/YAML | Prettier, strict JSON decode, pre-commit JSON/YAML checks |
| Shell | ShellCheck 0.9.0 |
| Dockerfile | Hadolint 2.15.1, digest-pinned base images, clean build/doctor |
| Dependencies | Composer validation/audit, npm lock integrity/audit |
| Repository hygiene | private-key/access-key patterns, 2 MB file cap, conflict markers, line endings |
| MCP contract | test suite plus Inspector 2.9.0 tools/list held to the tool catalogue |
| Adversarial testing | fixed-seed properties/fuzz, differential scans, semantic mutation score |
| Mutation testing | Infection 0.35 over src, scheduled workflow (not a profile gate) |
| Performance | mixed-language cold/incremental/query/RSS/SQLite budgets |
| Documentation | generated CLI/MCP contracts, internal links, anchors and images, external links |
| Supply chain | CycloneDX SBOMs, Trivy gates, quality image allowlist, provenance, Cosign smoke |
| Maintainability | per-file size/decision metrics and normalized cross-file duplication report |
The PHP test suite
tests/phpunit/ is the single PHP suite, run by composer test, which also runs the hook shell tests in tests/shell/ (composer test:shell runs those alone). The suite drives every language through one runner that spawns the TypeScript and Python workers as subprocesses, so worker behaviour is proven end-to-end from PHP. The TypeScript worker additionally has a focused vitest suite (workers/typescript/src/__tests__/, run via npm --prefix workers/typescript run check) for scanner internals that are awkward to reach through the protocol. Groups mirror the architecture areas and are selectable with vendor/bin/phpunit --group=store.
Inside the quality container the suite runs as the unprivileged knossos user, because the permission-error tests skip themselves as root, and a branch that never runs would read as uncovered.
Because the suite is a real PHPUnit suite, Infection can mutate all of src, and Chaos-MCP's audit_code_resilience works against this repository unmodified. Mutation testing is not part of any tools/quality profile, since a full-src run takes hours, so it runs through .github/workflows/mutation.yml. See adversarial testing.
There are no standalone YAML configuration files beyond the workflows and hook configuration, and no release packaging manifest. Those categories are validated by the generic YAML and repository checks. Actionlint is not bundled because the workflows deliberately contain no expressions or custom action inputs beyond what YAML parsing and running the same container command already prove.
Hadolint rule DL3008 is narrowly disabled: Debian package revisions come from the digest-pinned base image, and exact apt revision pins would prevent base image security rebuilds. DL3002 applies only to the development quality stage, which needs the mounted Docker socket, and the shipped runtime stays unprivileged. DL3059 preserves separate Composer/npm cache layers. SC2086 applies only to the intentional package-list expansion of the base image's PHPIZE_DEPS. No source-analysis baseline or blanket file exclusion is used.
The development-only quality stage requests Docker API 1.44 when it performs its nested clean-image verification. This keeps the pinned Docker CLI compatible with current daemons whose minimum supported API is 1.44, and does not affect the shipped runtime image. The image also builds pcov from a checksum-pinned tarball and enables pcntl, which the signal and watcher tests need in order to run instead of skipping.
Hooks
Install pinned pre-commit 4.6.0, then run:
tools/install-hooksThe commit hook runs the hygiene hooks and tools/quality fast. The pre-push hook runs tools/quality full. Developers without native tools can run the container-backed commands before committing, and CI always uses the quality image.
The script installs the pre-commit, pre-push and commit-msg hooks. The commit-msg hook checks each subject with tools/check-commit-style.sh.
Re-shooting the docs images
The images in the docs and the README GIF come from tools/capture, a dev-only tool that sits outside every gate and ships with nothing. It drives a real Claude Code session in a terminal, takes the screen and renders it to PNG, and builds the hero GIF from the frames.
node tools/capture/shoot.mjs
node tools/capture/shoot.mjs --only=overview,hero
node tools/capture/shoot.mjs --out=/tmp/shotsWith no options it shoots every shot into docs/images/claude-code/. --only takes a comma-separated list of shot names from tools/capture/shots.mjs, and --out changes the directory. You need npm ci for puppeteer-core, Chrome at /usr/bin/google-chrome, ffmpeg, and a logged-in claude CLI.
The sessions never touch your own setup. Each runs under a temporary CLAUDE_CONFIG_DIR that holds a copy of your ~/.claude/.credentials.json and nothing else, against a staged copy of the graph. Because the hero's model turn edits a file, the sessions run in a throwaway git worktree of HEAD, never in your checkout. The stage, the worktree and the sessions are removed however the run ends, and the tool refuses to start a session whose data or config directory is your real one. CAPTURE_TRACE=1 prints each step. The specs for the tool run with npm run test:mod.
Maintainability ratchets
The static lane publishes coverage/quality/maintainability.json and fails when any checked-in budget in maintainability-budgets.json regresses. The report covers all first-party PHP, TypeScript/JavaScript, and Python runtime sources. The Rust worker is the one first-party runtime outside it: tools/quality runs formatting, Clippy with -D warnings, and tests only when cargo is available; without Cargo they are reported as not run outside the quality container, while the quality container treats its absence as a failure.
The budgets in maintainability-budgets.json are:
- no more than 4 normalized cross-file duplicate blocks;
- PHP function cyclomatic complexity no higher than 60 and function length no higher than 205 lines;
- direct per-file dependency fanout no higher than 15;
- docstring coverage of 100% for
src,workers-php,toolsand overall, and type-docblock coverage of 100%.
php tools/documentation-check.php compares each number above with the file, so the list cannot drift from what the lane enforces. Each of these must be in the file as a JSON number, and every area the docstring report measures needs its own floor. A missing or quoted limit stops the lane with exit 1 and names the key, rather than being read as 0 or skipped. php tools/maintainability-report.php and php tools/docstring-report.php both take --root=DIR and --budgets=FILE, which is how their tests plant a violation without touching the checked-in file. The maintainability report also takes --report=FILE, so its tests write their JSON outside the checkout.
The TypeScript/JavaScript and Python limits sit in the linters' own configuration and fail there: ESLint holds function complexity to 29 and function length to 101 non-comment lines, and Ruff holds McCabe complexity to 20, with its branch and statement limits.
These maxima are ratchets, not design targets. New or changed functions should stay substantially below them. When a decomposition lowers the real maximum, lower the checked-in limit with it. Raising a limit requires before/after evidence and an explicit review. PHPStan, ESLint, Ruff/mypy, Clippy, and compiler syntax gates run before the report, so unused variables, unreachable Python paths, and invalid typed dependencies cannot be hidden by the metric report.
Generated dependencies, fixtures, and tests are excluded from product-code metrics. Duplicate blocks shorter than eight logical lines or 160 normalized characters are omitted to avoid treating ordinary language structure as a refactoring target.
Maintenance
Upgrade one ecosystem at a time, regenerate its lockfile, rebuild the quality image, and run the full profile. Tool suppressions require a specific rationale in this document. Generated dependency directories and coverage output are not committed. Security audits depend on current registry advisory data, so a newly published advisory can fail the release lane on a change that touched nothing related. Prefer bumping the direct dependency over adding an override.