The design, noun by noun

What Simplon is is the short answer. This is the long one, for a reader who wants to know why the design is shaped this way before adopting it.

Simplon is the link between the CI/CD process and the technologies, and it brings structure and reusability.

That sentence is the design, and it is a blueprint rather than a figure of speech. Every noun in it names something you can point at: the process is a fixed set of groups, the technologies are what a task body shells out to, the link is a resolution step in the loader with a name, and structure and reusability are each a short list of things the loader refuses to do. This chapter takes it apart noun by noun, and only then gets to the question a hand-written delivery script answers by default - why not write the five verbs yourself.

The three nouns

The CI/CD process is a closed vocabulary

Every component that ships has the same five verbs behind it: build the artefacts, verify them, publish them, put them into an environment, watch what is running there. Simplon declares those five once, plus a sixth for the machine itself, and they are the only groups there are:

groups:
  build:   { help: "Produce the artefacts." }
  test:    { help: "Verify them." }
  release: { help: "Publish them." }
  deploy:  { help: "Put them into an environment.", env_first: true }
  monitor: { help: "Watch what is running there.", env_first: true }
  support: { help: "Host preflight, environment introspection and host tooling." }

A product fills those groups. It cannot invent a seventh, and the refusal is not a rule layered over the merge - it is the merge. A product’s tree is merged onto the platform’s, so a path the platform does not already declare has nowhere to land, and the loader says so in as many words - the message is quoted in full under Modular, where it is this site’s only copy of it.

There is one tree, so there is no second way to bring a group into existence. That is what makes the vocabulary hold: test means verify in every product that uses the kernel, and deploy prod up reads the same to somebody who has never seen this particular product before.

The technologies are behind a task, never in the vocabulary

Nothing above mentions pytest, Hugo, Docker, Allure, mypy, git or a package registry. Those live one layer down, each behind a task: a body plus a coordinate that names it.

docs:site:
  impl: simplon.tasks.site:build
  help: "Build the product's documentation website with Hugo, in Docker (HTML only)."
test:gate:
  impl: simplon.tasks.testrun:gate
  help: "Run a declared pytest suite against the running lab."
vcs:commit:
  impl: simplon.tasks.vcs:commit
  help: "git add -A + git commit -m."

A coordinate is deliberately not a module path. docs:site is a name in the catalogue’s coordinate space, and the body it points at can move inside the kernel without breaking a single product manifest. The technology is an implementation detail of the body, which is why swapping one is a kernel change and not a change to six manifests.

The link is a named resolution step

Between the vocabulary and the technologies sits the manifest, and the linking is not a metaphor for “gluing things together” - it is one function. treeform.resolve walks every command in the tree and replaces its task: with the impl: the template names. On one side of that step there is a group path and a command name; on the other there is a module:function. Nothing in between is inferred:

One resolution rule, two sources, and the two name spaces cannot intersect - so nothing shadows anything and there is no precedence to remember. A name that resolves in neither is a load error that lists what is available, rather than a command that exists in the help and dies on invocation.

That is the whole of the link, and it is why the sentence at the top is a construction plan. You can read the seam, test it, and put a breakpoint in it.

What “structure” means here

Structure is not layout. It is a short list of things the loader refuses to do, and each refusal removes one way for a command line to lie about itself.

• A group you name is a promise. Declare a group in your own tree and leave its whole subtree without a single command, and the manifest fails to load, naming the group. A group only the catalogue offers, which your tree never mentions, is dropped from the assembled CLI instead - a menu entry that leads nowhere is worse than no entry.
• A command is an instance, never a body. A command that writes impl: is refused with “declare the body once under tasks: and point this command at it with task:”. There is one place a body is declared, so there is one place to change it.
• A collision breaks loudly. Two different bodies under one command name is a load error that prints both by their real module:function, not a silent win for whichever dictionary was merged last.
• The group’s shape is the platform’s. A product may add commands and sub-groups to a group it inherits; it may not rewrite that group’s help: or its env_first:. Turning env_first off quietly would ungate every command underneath it.
• A declaration that renders nowhere fails. A task key nobody reads, a with: pinning a parameter the body does not take, a parameter both pinned and given a params: entry it can no longer show - all of them are errors rather than silent no-ops.

The help text is generated from the same declaration the dispatch uses, so the two cannot disagree. That is the practical shape of “structure”: --help is not a document about the CLI, it is a rendering of it.

What “reusability” means here

The body lives once, in the kernel, under a coordinate. A command is a placement of it: the name, the group, and the values pinned for this particular instance. So the same body can be placed several times, in one product or in twenty:

groups:
  test:
    commands:
      unit:   { task: "test:gate", with: { name: "unit" },   help: "Run the unit suite." }
      system: { task: "test:gate", with: { name: "system" }, help: "Run the system suite." }

Two commands, one body, and the difference between them is data. What a product supplies is exactly that: the pinned image tag, where its sources are, which suite level to run. The mechanism is the kernel’s; the values are the product’s; the manifest section where they meet is the seam that makes a catalogue task a promise to several products rather than a convenience for one.

The consequence that matters is the one you feel on a bad day: the fix to a body reaches every product on the next version bump, and there is no second copy to forget. The test for whether a body belongs in the kernel at all is written down in Writing a task - can it be written without naming this product’s directories, images or services?

The same five verbs over very different work

The vocabulary is fixed and the technologies are not. What makes that trade worth making is not visible from either half on its own - it becomes visible when you ask what kind of work the tasks are.

Delivering a component is not one activity. Version control, testing, documentation, packaging, the machine everything runs on: these are separate kinds of work, done with separate tools, frequently by separate people, and no one of those toolchains has anything to say to the others. pytest knows nothing about Hugo. oras knows nothing about mypy. A vocabulary that described them technically would have to be six vocabularies, and nobody would be able to read across them.

The catalogue is where this can be checked rather than taken on trust. Every coordinate in it, grouped by the kind of work instead of by the phase it runs in:

Seven kinds of work, and the middle column barely intersects. Every one of them is reached through build, test, release, deploy, monitor, support and nothing else. That is the claim the fixed verb list is worth making - not that five verbs are elegant, but that they are the only thing six unrelated toolchains can all be addressed by at once. Take the list apart per team and you have not gained expressiveness; you have lost the one word that meant the same thing in all six places.

The last row was called the manifest itself until si#102 put two generators in it, and widening the name is the honest half of that change. build:cmake-files and build:dotnet-solution read a source tree rather than a manifest, so the old name would have been wrong about them - but the work is the same work as the four beside them. Each takes something the product has already declared - a manifest, a command tree, a directory of sources - and renders a file the product then commits, reaching no external tool at all to do it. That is why the row’s middle column can say nothing external and be held to it by the suite rather than believed.

The testing row’s middle column names one thing that is not a tool, and that is the honest half of si#205’s change. test:walk shells out to nothing: it reads the same .feature files the documentation row’s docs:acceptance reads and asks a person to answer each step. It is testing work all the same, because it produces the one thing the four beside it produce - a verdict, red or green, addressed to the same <feature file>:<scenario> key the archive uses - and the row would be lying about what verifies this product if it left the only mode a customer can drive out of the list.

The two ways of grouping cut the same coordinates differently, and the difference is the point rather than an untidiness. release:tag is version-control work that happens in the release phase; test:release-notes is version-control work that happens in the test phase - it reads tags and merge subjects and shells out to nothing else, so filing it under testing would have put git in that row’s middle column and cost this page the one claim it is making; docs:site is documentation work that simplon files under build and another product could file under release. Which half of a coordinate decides which is the placement-or-family rule, in The five phases.

One word does double duty across the two, and it is worth naming rather than leaving for a reader to trip over: test is a phase, and testing is a kind of work. They are not the same thing said twice - docs:reference is documentation work that runs in the build phase, and a product that renders its site as part of shipping puts documentation work in release. Where this page says phase it means one of the six groups; where it says kind of work it means a row of the table above.

Running things was the row that used to be missing, and half of it still is. si#235 put deploy:up and deploy:down in the table: they say which version goes out and prove it exists. si#5 added the third, deploy:carrier, which makes the machine the other two deploy onto - and it is the first place the kernel drives a tool it does not ship, in a container, the way docs:render does. Watching things is what the catalogue still carries nothing for - and so is the provider half of running them, the part that talks to a Portainer or a Proxmox. Not because it is not real work, but because those tools belong to the shared environment rather than to the product, and who should own them is still open. deploy has since gained the two tasks that say WHICH version goes out, which is not that half; monitor still carries nothing. See The five phases.

Why the technology travels in a container

A task body shells out to a tool, which raises the question the section above walked past: where does the tool come from? There are two answers, and the kernel does not give the same one everywhere.

The first is to provision the machine - install Hugo, install docToolchain, write the versions down somewhere, and hope the next machine gets the same ones. The second is to bring the tool with the task: name an image, run the tool inside it, and let the build depend on that image instead of on whatever happens to be installed where it runs. The second is what the two documentation renders do, and it is what those two catalogue entries mean when they say in Docker:

Build the product’s documentation website with Hugo, in Docker (HTML only).

Render the project docs via docToolchain (generateHTML + generatePDF) in Docker.

It collapses the tool question into one question. Hextra, the theme this site is built with, is a Hugo module, and building a Hugo module needs Hugo Extended, Go and Git together - three installations on a developer’s machine, three ways to be the wrong version, three unrelated error messages to learn to recognise. In an image it is none of them. “Which of the three is missing, and in which version?” becomes “is docker there?”, and that second question has an answer a script can act on.

Which is why the image reference is refused at load unless it names a version. A container that brings whatever a moving tag points at today has relocated the problem rather than solved it - the machine no longer decides what the build uses, but the calendar does:

simplon.yaml: ‘site’: ‘image’ must pin a version (’:’), got ‘hugomods/hugo’ - an untagged image means ‘:latest’, which moves under the build; pin it as ‘:’ (e.g. ‘hugomods/hugo:exts-0.148.2’), or by digest

The theme module is refused on the same grounds - @latest, @master and a bare branch name are all queries that resolve differently tomorrow. Both declarations, and the rest of the site: section, are in The manifest.

What it costs

A page that only listed the upside would be recommending something it had not used.

• The pin fixes the version; a cache is what makes it repeatable. docs:site runs hugo mod get <module>@<version> before every build, and hugo resolves the module a second time while building - so with nothing kept on the host, both went to the network on every run and a build with nothing changed since the last one was impossible without one. Reproducible without being repeatable, and those are not the same property. Hugo’s cache now lives in build/hugo-cache/ and outlives the container: measured on this repository, ./simplon.sh build docs with every container cut off from the network publishes the site, where the same run against the previous kernel died at git ls-remote. The remaining cost is real rather than none - a fresh checkout, a clean, and any move of the pin each need the network once, and the cache is a directory the product’s build/ rule has to cover. The measurements are on simplon#27.
• A green Hugo proves nothing about the pictures, so the build checks them separately. Hugo emits a Mermaid block into the HTML whether or not the Mermaid parses, because the diagram is drawn in the reader’s browser and not during the build. Measured: a one-character error in the source gives exit code 0, a full page count, and the broken text verbatim in the published page. docs:site therefore renders every Mermaid block in the site source with a pinned mermaid-cli after Hugo has run, and is red with the page and the line when one of them does not draw. Two costs come with that: a second image is pulled on any build that has a diagram in it, and the version it checks with is not necessarily the version the reader’s browser draws with - the gate says the source parses, not that it looks the same everywhere. A build with no diagram at all is green and says there was nothing to check, because “nothing to do” and “checked, and fine” are different statements.
• A container that writes into your tree writes as somebody. The uid the image happens to run as is the uid that ends up owning the output, and both directions of getting that wrong have been measured here. The rule and both measurements are in A container that writes into a mount runs as --user.
• A container that is not there is not the same as one that failed - and which of the two a task chooses is the task’s decision, not the container’s. For docs:site, no docker on the host is a hint and a zero exit code, because the machine that runs the loop is not always the one that publishes the page; docker present and no site produced is red. docs:render decides the other way and dies on a missing docker, so the two renders quoted above are a pair in how they get their tool and not in what they do when it is absent. The distinction is a rule of its own: A missing tool is not a failed tool.

Where a container is the wrong answer

Not every tool belongs in one, and the catalogue says so out loud. The type gate’s entry ends the other way round:

Type-check the product’s Python sources with mypy, in the host venv (no Docker, no lab).

The reason is in what that tool has to see. mypy resolves the product’s installed dependencies, and without them every third-party import degrades into a blanket exception and the gate stops meaning anything - so the interpreter it runs under has to be the one the product actually ships against. An image would bring a Python of its own, which is precisely the wrong Python. The rule that comes out of the pair: bring the tool in a container when the tool has no opinion about the product’s installed set, and use the product’s own environment when it does.

Two more shapes are worth not confusing with either. build:image and release:image use docker because the artefact is a container image - docker is the subject there, not the delivery vehicle - and their verdict on a missing docker is the opposite of the site build’s: they call the gate that dies, because producing the image is the only thing they were asked to do. And test:report prefers a local allure on the PATH, falls back to an image when there is none, and degrades to a hint when there is neither: a container is one answer to “where does the tool come from”, not the only one.

Why not just write the five verbs yourself

Almost every component does write them itself, and almost every one writes them a little differently. That is not a tidiness complaint; it costs three concrete things.

A command line nobody maintains. The delivery script starts as build.sh, grows a case statement, then a second script for tests, then a Makefile someone added because the case statement got long. By the time it has options with defaults and a help text, it is an argument parser written by hand - and an argument parser written by hand is a thing whose --help is a separate document from its behaviour. The day they disagree, the help text is what a human read. Under Simplon there is no argument parser to maintain, because there is no argument parser: there is a declaration, and the sub-applications, the option types, the defaults and the help are all built from it.

A capability that lives in one repo. Somebody solves rendering the architecture docs, or pushing a stack to Portainer, or provisioning the host’s tooling. It works. It is four hundred lines in one product’s scripts/ directory, and the next product that needs it either copies those four hundred lines or does without. Copies drift; the second copy is the one that never gets the fix. A capability that is a catalogue task has no second copy.

Nowhere to put “and be careful about X”. The uid a container writes as, the difference between a tool that is missing and a tool that failed, the reason a step must use the interpreter that is running rather than whatever python resolves to - these are learned once, expensively, and then live in a person’s head or a comment in one file. They are not the kind of knowledge a README carries, because they only become interesting at the moment something is being built. Here they are rules with the failure that forced each one, and the mechanism that enforces them is in the kernel rather than in a person’s memory.

The kernel is its own first product

simplon.yaml in the kernel’s own repository declares its own build, test and support commands, and the CLI that runs them is assembled by the very kernel that repository builds. That is not a demonstration. It is a guard, and it exists because of a specific class of defect: the code path that only a consumer walks is the one no test in the repository reaches.

The concrete shape of it is worth keeping in mind whenever a package reads a file of its own. Resolving a data file by walking a fixed number of directory levels up to a repository root is exactly correct while the package is vendored inside that repository, and meaningless once it is installed - from site-packages, five levels up is somewhere else entirely, and the read raises FileNotFoundError. Every test that reaches the source tree through a sys.path adjustment passes anyway, because none of them is in the arrangement where the defect exists.

So the catalogue travels inside the package, and the kernel builds and tests itself with itself: when a commit breaks the assembly, it fails in the same round instead of after a release. A tool never used in its own house rots unseen.