The manifest

The file, at a glance

The whole top level of a manifest, and where each part is explained:

product: myctl            # the product's name
tasks:                    # the BODIES, each a module:function, declared once
groups:                   # the COMMAND TREE - build, test, release, deploy, monitor, support
default: dev              # the environment used when no env token is given
environments:             # the environment matrix
site:                     # ... and any other product data section a catalogue task reads

Two rules hold everywhere below, and most load errors are one of them: a command never writes impl: - it names a task: and the body is declared once - and the platform owns which groups exist, so a product adds to a group and never rewrites its help: or its env_first:.

One YAML file per product. It declares what commands exist, what they run, what they are called and which of them take an environment - and the CLI is assembled from it, so there is no second place where any of that is also true.

The command tree

Groups are slots in the CI/CD loop; commands are the members of a group. The kernel’s catalogue owns which groups exist - build, test, release, deploy, monitor, support - and a product fills them:

product: myctl

groups:
  build:
    commands:
      wheel: { task: "pkg:wheel", help: "Build the wheel." }
  support:
    groups:
      git:
        commands:
          commit: { task: "vcs:commit" }

A group node has exactly four keys: help, env_first, groups and commands. Members live under commands: rather than directly on the node, and that is not tidiness - without it help: would be a group attribute in one place and a command called “help” in another.

The platform owns the group shape. A product may add commands and sub-groups to a group it inherits. It may not rewrite that group’s help: or its env_first:. Both are statements about what the group is, and env_first in particular is load-bearing: a product quietly turning it off would ungate every command underneath it. The group lock is not a check layered over the merge - it is the merge. There is one tree, so there is no second way to bring a group into existence.

A group you name is a promise. Declare a group in your own tree and leave it without a single command anywhere in its subtree, and the manifest fails to load, naming the group. A group only the catalogue offers, that your tree never mentions, is simply dropped from the assembled CLI - see the rule and why both halves are needed.

task:, and what the command adds

Every command is an instance of a task: the task carries the body, the command carries the name, the group and the values pinned for this placement. That model, its five refusals and the two name spaces the colon tells apart are a chapter of their own; what follows is only what the manifest file looks like once you have it.

tasks:
  wheel:
    impl: "orchestrator.cli:build_wheel"
    help: "Build the wheel."

groups:
  build:
    commands:
      wheel: { task: "wheel" }

A command never writes impl: itself. A bare task: value names a task this manifest declares; one with a colon names a catalogue coordinate, whose body lives in the kernel.

Pinning values with with:

The same body, placed twice, with different data:

groups:
  test:
    commands:
      unit:   { task: "test:gate", with: { name: "unit" },   help: "Run the unit suite." }
      system: { task: "test:gate", with: { name: "system" }, help: "Run the system suite." }

A pinned parameter is removed from the generated signature and supplied at call time, so it is absent from the command line entirely: myctl test unit --name system is not a command.

params: is the other half, and it is strictly about presentation: help text, the short flag, the metavar, the order the declarations render in. The signature - name, type, default - is read off the body. Declaring the type in YAML as well would state it twice and let the two drift, which is the exact failure impl: on a command already has.

prune-branches:
  task: "vcs:prune-branches"
  params:
    dry_run: { help: "preview only", short: "-n" }
    remote:  { help: "also delete merged branches on origin" }

Refining a platform command, and replacing one

A command the catalogue already places can be refined in the product’s own tree: change its help:, add params:, pin a with:. Point it at a different task: and the loader stops, naming both bodies by their real module:function and offering override: true as the explicit yes. That refusal has its own rule, and the merge behaviour behind it is in Task and command.

Aggregates: depends_on

A command with no body, only a plan:

all:
  help: "Build then deploy, end to end."
  depends_on: [build, up]
  stop_on_failure: false

impl and depends_on are mutually exclusive. A command is either a leaf with a body or an aggregate that plans other commands - never both. The reason is mechanical: plan steps execute as subprocesses, so an impl-bearing command that also carried dependencies would re-expand them in the child and break the run-each-once guarantee.

The plan is a post-order depth-first walk over depends_on, deduplicated by name, so a command reached along several paths appears exactly once. List order is execution order among siblings, and that is how one step is made to run before another:

docs:
  help: "Write the command reference, then build the website from it."
  depends_on: [reference, site]

reference writes the page that site reads. Nothing in the list says so except the order, and that is deliberate: the same fact written twice - once as an order, once as an edge - is a second source that can drift.

parallel: the one key that suspends list order

Because position is the only statement of order, a command whose dependencies genuinely do not need each other has to say so:

images:
  help: "The five device images."
  depends_on: [sidecar, frr, vyos, ios, radius]
  parallel: true

Those five subtrees then run at the same time, and whatever follows images in its own parent’s list starts only when every one of them is done. That is the join, and it needs no new key: depends_on already means “after”, and parallel is the one place it stops meaning it.

It is declared and not derived, and that was measured. A dependency graph already states what must come after what, so deriving parallelism from it - everything unconnected runs at once - looks like the better answer. Over the six manifests simplon can reach, 541 pairs of planned steps have no edge between them and 466 of those would break if they ran together: the orders those manifests rely on are written as list position, not as edges. Deriving would have run a website build before the page it publishes, a packaging step before the thing it packages, and a gradle build inside an image that did not exist yet.

Two things it does not change. A branch is still a chain - parallel applies to the dependencies of the command that declares it, not to what is inside them - and stop_on_failure still decides what a failure skips. A step already running when a sibling fails is left to finish rather than killed, because a killed subprocess exits non-zero and that number cannot be told from the step having failed on its own; the members of one fan are never skipped for each other, since the declaration says none of them needs another; and the work after the join is skipped in the ordinary way.

How MANY run at once is the machine’s answer rather than the manifest’s, because the manifest travels between machines and the number does not: SIMPLON_MAX_PARALLEL sets it, and the default is four or the CPU count, whichever is smaller.

WHICH RUNNER draws the plan is the machine’s answer too, and asking for the plain one is --no-tui. That belongs with what a run looks like rather than with what a manifest declares: Running a command.

Every depends_on entry must name a known, unambiguous command, and the graph must be acyclic. Both are checked at load, not at run: a dependency naming a command that does not exist is a manifest error, and it should not wait until the eleventh minute of a pipeline to say so.

hidden: true keeps a command out of every --help listing while leaving it fully invocable - which is what a plan step named in a depends_on needs, since it must be a real command but need not clutter a menu meant for a human.

Environments

env_groups: [deploy, monitor]

default: dev
environments:
  dev:  { backend: local,    description: "Local development environment." }
  prod: { backend: exoscale, description: "Production." }

A group’s env gate is normally the catalogue’s statement, not yours: deploy and monitor carry env_first: true on their nodes there, so every product gates them the same way and no manifest has to say it twice. env_groups: is the flat spelling of the same statement, and it is held to the same rule: listing a group the catalogue already gates is a harmless restatement, listing one the catalogue declares not env-first is refused exactly as env_first: true on that group’s node is. It is the manifest’s own statement only for a top-level group no catalogue owns. Everything not gated refuses the token. See Environments for what the dispatch does with it.

Product data sections

Beyond the command tree, a manifest carries the data its tasks read. This is the seam that makes a catalogue task a promise to three products instead of a convenience for one: the mechanism is the kernel’s, the values are the product’s, and the section is where they meet.

site:
  image: "hugomods/hugo:exts-0.148.2"
  output: "build/website"
  base_url: "https://example.github.io/myctl/"
  theme: "github.com/imfing/hextra@v0.12.3"
  # source: "docs/site"   # omitted: that is the default

image and output are required to be declared rather than defaulted. A kernel that named the image would be choosing a documentation generator on every product’s behalf, and one that guessed output would hand a recursive delete a directory nobody typed. A missing section fails at load, naming the key, instead of as a container run against a directory that is not there.

source defaults to docs/site (si#183). docs/ is the documentation root - architecture, specs, plans and the site all belong in it - so a product that says nothing lands on the convention, and one that names a path still gets exactly what it named. Omitting the key is the recommended way to write it. Note the line the default does not cross: source is only ever read, output is handed to shutil.rmtree before every build, and that is the whole reason one of them has a default and the other never will.

Two of those values are refused unless they pin a version. image: "hugomods/hugo" means :latest, which moves under the build; theme: "...@latest" fetches whatever is newest on the day it runs. A build whose output depends on when it ran is not a build, and a documentation site is committed-to prose - a generator change rewrites it wholesale. source and output must be plain relative paths under the product root, for a blunter reason: output is handed to a recursive delete, and output: /var/tmp/x would delete /var/tmp/x.

It is a default and not a rule, and that was measured rather than preferred. A source: outside docs/ REFUSED would turn away a product that has done nothing wrong. Measured over the same population si#159 and si#172 used - every manifest this kernel can reach: its own, the five in simplon.surface.CONSUMERS, and secure-windows-images - three of those seven declare a site: section and the three do not agree: cleon at site, biz-cockpit at docs/website, simplon at docs/site. Two of those three would fail on their first run with a kernel that insisted - an expression rule with no measured cause, which this platform declines to write (the rules chapter says why).

By that same count the default serves none of the three, because all three name the key. It is for the fourth: the day your product gets a website, it writes four keys instead of five and lands on the convention without having had to read this page. Simplon itself is the first consumer - its own manifest leaves source: out, so the default is exercised by every build docs in the kernel’s own repository rather than only by a test.

The names the kernel has claimed

A top-level key the kernel does not read is the product’s own, and nothing here rules on it. That is the design rather than a gap: the section is where a catalogue task’s mechanism meets a product’s values, and a product that could not invent one would have to ask the kernel for permission to have data. Measured over every manifest this kernel can reach, its own and the six in other repositories, eleven of the seventy top-level keys are exactly that, and every one of them is read by a body in the product’s own repository. Adding a key nobody here has heard of is a supported thing to do.

What was missing is the other half: which names are already taken. The product si#159 was reported from builds three Windows Server releases, and it learned that releases: already means {page, from, complete_from} to test:release-notes by reading src/simplon/tasks/releasenotes.py. A reserved name that can only be found in the source is a trap with a delay on it, so the twenty are published here and tests/test_manifest_top_level.py holds this table to the kernel in both directions.

A key that is mistyped is therefore not the silent no-op it looks like. Seventeen of the twenty are named by the reader that wanted them, the moment that reader runs: sietv: instead of site: answers the 'site' section is missing or is not a mapping, and every other reader refuses the same way, naming the key it looked for. The three that say nothing say nothing on purpose:

• build: - an absent section is the normal case. build cmake-files and build dotnet-solution render the build files from the SOURCES, and build: targets: only adds the edges the directories cannot show. A build: section that exists for some other reason and carries no targets: says the same thing, on purpose - two live manifests have one, and the next section is about them.
• env_var: - a product that selects its environment by token and default: alone has no such variable, and a listing must still work on a manifest that has not adopted the key.
• layout: - an absent section is what every product in this family has. It arrived with si#250 and its defaults are exactly what the kernel assumed before it existed, so a manifest that says nothing runs the line it ran yesterday. The cost is stated rather than left to be found: layotu: is silent too, and only a key mistyped inside a correctly spelled layout: is refused.

Both are driven in the test module above, so the silence is measured rather than assumed.

Reading a section, yours or the kernel’s

A product’s own task body reads its own section through simplon.context.section, the same walk the kernel’s ten readers use:

from simplon import context

ctx = context.current()
packer, blame, got = context.section(ctx.manifest_data(), "build", "packer")
if blame:
    raise ValueError(f"{ctx.manifest_path}: `build: packer:` is not declared")

It walks a path, because build: is shared and a product that wants its own data under a phase has to nest (the section above). It stops at the first step that is not a mapping and names that step: buidl: packer: blames build, so a typo in the outer key never reports the inner one as absent. got is what the blamed step held - None exactly when it is not declared at all, which is how a reader tells “the product said nothing” from “the product said the wrong thing”. A section declared and empty is found, not blamed.

It never raises, and that is deliberate. Each of the nineteen readers above says something specific when its section is missing, and si#159 measured that fourteen of the sixteen it measured already refuse by name and quote the key. An accessor that raised would replace those sentences with one. So it answers the question and the reader keeps its own refusal - which is also what keeps every refusal countable where it is written, in the rules chapter’s census.

build: is shared, and build: targets: is the kernel’s

build is the one name that is a group and a top-level data section at the same time: the catalogue declares a build group, and build:cmake-files and build:dotnet-solution read their targets: out of a top-level build: section. The kernel’s own source used to carry a comment saying the two could never meet. They already do (si#172).

Measured over the same seven manifests si#159 read, two products carry a top-level build: section of their own and no targets: in it: cleon’s holds bundle:, ant: and site:, and secure-windows-images’ holds packer: and templates:. Neither is doing anything wrong. The kernel picked a name that was already taken, and one of those manifests had already written a comment to its own authors explaining the collision and telling them not to add a targets: key.

The rule, so nobody has to read the source for it again:

• build: is yours. Declare it, put what you like in it, and nothing here refuses it. Refusing it is what si#159’s measurement rules out - eleven of the seventy top-level keys across those manifests are read by product task bodies in repositories this kernel cannot see, so a rule over the top-level namespace refuses live products on its first run.
• The kernel reads exactly one key out of it, targets:, and rules on nothing else in there.
• An absent targets: means what an absent build: means: the source tree is the whole declaration. That is not a fallback, it is si#102’s design - the directories say what the targets are, and targets: only adds the dependency edges they cannot show. Driven: a product carrying either of those two real sections and placing build:cmake-files writes exactly the files a product with no build: section writes.
• Inside build:, the word targets is the kernel’s. If your own build vocabulary has targets - cleon’s Ant block is one rename away, with generate_targets:, compile_targets: and package_targets: - call yours something else. A targets: of the wrong shape is refused by name and tells you to rename; it is not silent.

images: - the container image

build:image and release:image read one entry of this section, pinned per command with with: { name: ... }, because a product may build more than one:

images:
  app:
    registry: ghcr.io/example
    repository: myctl
    dockerfile: Dockerfile
    context: .
    tag: latest                 # optional; `--tag` overrides it
    build_args:                 # optional; yours, on top of the two the kernel derives
      PYTHON_VERSION: "3.12"

The first four are required, registry included even if you only ever build locally: the reference the build tags is the reference the release pushes, and an unqualified name is one docker resolves against Docker Hub. dockerfile and context must stay under the product root - context: / would stream your whole filesystem to the daemon as a build context.

VERSION and REVISION are passed on every build, and you do not declare them. The kernel derives them from your checkout - git describe --tags --always --dirty and git rev-parse HEAD - so the image built on a laptop carries its provenance exactly as the one built in Actions does. Declare either name in build_args: and yours wins; a Dockerfile that declares neither ARG ignores both at no cost. The two are missing, with a warning, only when there is no checkout to ask - an exported tarball - and your Dockerfile’s own ARG VERSION=dev then stands rather than being overwritten with a placeholder.

release:image asks the registry whether the tag arrived before it reports success. A push whose result nobody reads is the same defect as a report nobody reads, and this project has shipped that twice. The question goes through oras - the tool the kernel provisions anyway - and not through a docker client, because docker manifest inspect answers out of ~/.docker/manifests/ and will confirm a tag that only ever existed on your machine.

If registry: does not name ghcr.io, no GitHub token is minted for it. The push then uses whatever credential your own docker login <host> stored, and a rejection says so instead of pointing at gh auth refresh, which would mean nothing on someone else’s registry.

releases: - where the notes live, and from when they are complete

test:release-notes reads this one. It is the smallest product-data section in the kernel, and that is the point of it: the rule it feeds needs no product knowledge at all, so what a product owes it is three values and nothing else.

releases:
  page: "docs/site/content/when/releases.md"  # where the notes live, under the product root
  from: "0.4.0"                            # the first release that must have a section at all
  complete_from: "0.5.0"                   # the first section that must name every ticket in its range

The gate then answers four questions out of git tag, git log and the page’s own ## X.Y.Z headings: is every release at or above from written up at all; does the page promise a version nobody can install; does every merge in a documented range name a ticket in its subject; and does every section from complete_from on name each of them.

Nothing reads the prose. What a change meant is knowledge no tool has. The whole claim is that a number appears somewhere in the section, which is deliberately the weakest claim that catches the failure this exists for - a release section that describes the pull request it was written in rather than the release it names.

The two floors, and why the second is a floor rather than a list. from is where notes begin: earlier releases have their tags and their commits, and writing them up now would mean reconstructing a record from memory. complete_from is where completeness begins, and the gap between the two is the excused set. An exemption list would be the thing that grows quietly, one entry at a time, each looking justified on its own; a second floor can only ever be raised in public. A product that excuses nothing writes the same number twice, and a complete_from below from is refused, because it would demand completeness of a section the same manifest says may be absent.

The spelling is the page’s, not git’s. 0.4.0, three parts and no v. The tag carries the v and the heading does not, so from: "v0.4.0" is refused rather than helpfully stripped - it would otherwise be a floor that matches no section at all.

It is offered, not placed. A product with no release notes never sees the command, and one that declares this section places it in one line under test. A checkout with no tags is diagnosed as the checkout - actions/checkout fetches no tags unless you ask - rather than blamed on the page, and a run that ruled on nothing is red rather than green.

workflows: - the CI files, generated from this same manifest

The manifest has always been meant to have two outputs. It assembles the command line, and - with support:workflows - it writes the GitHub workflows that call it.

workflows:
  ci:
    note: |
      Every push and every pull request, and it is the SAME command a developer types.
    on: [push, pull_request]
    jobs:
      self-build:
        runs-on: ubuntu-latest
        python: "3.12"
        steps:
          - command: test all
          - command: build wheel

The line that matters is command: test all. It is not a string that happens to look like a command - it is resolved against the command tree above before anything is written, and what lands in the file is ./myctl.sh test all. Rename that command and generation fails, loudly, in the same commit. Today, in a hand-written workflow, a renamed command leaves a run: line calling something that is gone, and the first anybody hears of it is a red runner.

Everything the kernel cannot know stays yours and is carried through untouched, at the level GitHub puts it:

The levels are not interchangeable, and getting them the wrong way round is the mistake this table exists to prevent: environment: and needs: are a job’s, defaults: and run-name: are a workflow’s. Neither list is enumerated in the kernel - the keys are simply passed through, because a kernel that policed GitHub’s schema would be wrong the week GitHub extends it.

A step is the exception, and deliberately: it has exactly one body, so a key that is neither a modifier nor a body is refused rather than carried. Two bodies - command: beside uses: or run: - are refused for the same reason.

flow: on a workflow rolls a release out to your environments, in order. The kernel writes one job per stage:

environments:
  test: { backend: portainer, carrier: haus, stack: app-test, deploys: latest }
  prod: { backend: portainer, carrier: haus, stack: app-prod, deploys: "1.4.0" }

workflows:
  rollout:
    on: { release: { types: [published] } }
    runner: { kind: github-ubuntu }
    flow:
      - to: test
      - to: prod
        approval: true

Two declarations, because they answer two questions. The environment says what it receives, with deploys: — latest for the newest published version, or a pinned tag. The flow says in which order environments are reached and which stage a person releases. A staging instance wants the newest publication and a production one wants a chosen tag, and that is true however the rollout is triggered; putting the version in the flow as well would give one truth two homes.

What comes out is deploy-test and deploy-prod, the second with needs: deploy-test — chained rather than merely ordered in the file, because GitHub runs jobs in parallel unless told otherwise and a rollout whose order lived only in the emitted order would reach production and staging at once while looking correct.

approval: true gates nothing here: it gives the job GitHub’s own environment:, which is where an account configures required reviewers, wait timers and the secrets that stage may see. Who may approve is an account setting, and it is the only half that can be changed without a commit.

Your own jobs stand beside it. A workflow may declare flow: and jobs: together — the generated names are stable, so a notification job hangs needs: deploy-prod on one. And a step may now hand a command a parameter:

        - command: deploy up
          params: { version: "1.4.0" }

The parameter name is checked against what the command declares, which is si#40’s join one level in: a renamed parameter breaks generation instead of becoming a run: line that fails on a runner three minutes into a job.

derive: on a step writes the pipeline the command tree already implies (si#267). A step that declares it is not one step but a placeholder for however many the named groups carry:

    jobs:
      self-build:
        runner: { kind: self-hosted-debian, labels: ghr-8 }
        steps:
          - derive: [build, test]

Every leaf of each named group that may run with nobody watching becomes a step, in the order the manifest declares it. Leaves rather than aggregates, because a GitHub job stops at its first red step - emitting test all would collapse four verdicts into one and lose the order its members were written in. An aggregate is skipped without comment, because nothing is missing from the file; a leaf that is skipped is named in the file, with the reason, since a reader wondering where test walk went is standing in the workflow and not in the kernel’s source.

Which leaves may run is the unattended: flag from task and command, declared in the platform catalogue once for every product that imports the coordinate. A leaf that says nothing is refused by name rather than guessed at.

A derived step stands beside your own, and that is the reason it is a step rather than a key on the job. A product that wants the derived pipeline plus a Check Run reporter writes both and decides where the seam falls:

        steps:
          - uses: actions/setup-node@v4
          - derive: [build, test]
          - command: release image

A job-level derive: would have had to refuse steps: beside it — and that refusal would have been an expression rule, forbidding something a product legitimately wants and sending it back to writing the whole file by hand, which is the work this feature exists to remove.

runner: names a KIND of machine, and the kernel supplies what follows from it (si#267). This is the answer to a day that was spent twice: moving one repository’s CI to a self-hosted runner meant learning that actions/setup-python never installs Python, that a container job cannot be the thing that provisions docker, and that DELIVERY_DOCKER_BOOTSTRAP=1 exists — all of it already written down, in a comment in another product’s manifest.

    jobs:
      self-build:
        runner: { kind: self-hosted-debian, labels: ghr-8 }

The label stays yours: ghr-8 names one machine and no kernel can guess it, and [self-hosted, Linux, X64] is not an answer — it describes every self-hosted Linux machine the account will ever register, so the day a second one joins, the jobs are shared out with nothing saying so. A kind that carries no label of its own therefore refuses until the job names the machine.

What the kind buys is the rest. A python: pin on a kind that cannot honour it is refused at generation, because a pin that cannot be honoured reads as a guarantee and is a wish; the kind’s own environment is merged into the job’s, and a variable the two set differently is refused rather than resolved. The kind’s one-line reason is written into the generated file, so a reader who finds runs-on: ghr-8 beside DELIVERY_DOCKER_BOOTSTRAP: '1' does not have to find the kernel to learn why they belong together.

runner: and runs-on: answer the same question, so declaring both is refused. Neither is deprecated: runs-on: is the whole of what a product needs when its machine has nothing to teach anybody, and a new kind is added to the kernel’s table once, for everybody, rather than described again in each manifest.

runs-on: may be a label, a list of labels, or GitHub’s group:/labels: mapping - all three of GitHub’s shapes, carried through as written. A list is the documented way to reach a self-hosted runner, because one label is rarely enough once an account has more than one machine:

    jobs:
      template:
        runs-on: [self-hosted, windows, vmware]

Until 0.16.0 the value was coerced with str(), so that list came out as the single label "['self-hosted', 'windows', 'vmware']" - valid YAML, valid GitHub syntax, and a runner that cannot exist, so the job queued for ever with no error anywhere (si#266). A value GitHub has no reading for at all - a number, a bool, a list with something other than a label in it - is refused naming the line.

Whether a tag publishes is your statement, so a workflow that declares no on: is refused rather than given a default. A step the kernel has no business modelling - pypa/gh-action-pypi-publish, an upload, a shell script that reports something - is written out verbatim beside the resolved ones.

A command step keeps its modifiers, which is what lets it stay a command step:

steps:
  - name: The system gate
    command: test system
  - name: Publish the report
    if: always()
    command: test report

Without that, every step carrying a name: or an if: would have had to be written as a verbatim run: line - the hand-typed, unchecked string this section exists to abolish. Measured across six real products: 34 of 41 command-invoking steps carry one of name:, if: or env:.

A command step that declares no name: shows its run: line in the Actions UI, and that line is ./myctl.sh test all - the same string you type in a checkout. The kernel does not invent a name for it; say name: if you want different words.

Two things are the kernel’s, and both are settings you should not have to remember. The checkout is emitted with fetch-depth: 0, because actions/checkout defaults to a shallow clone that carries no tags - so anything deriving a version from one gets a wrong answer silently rather than an error; and python: becomes a setup-python step, or nothing at all if you declare none. A job that wants neither says checkout: false and writes its own.

Your comments survive. note: may sit on the workflow, on a job or on a step, and is written out as a comment in that position. That is a requirement rather than a nicety - a real workflow carries measured values and the reasoning for absences, and a generator that dropped them would make the file worse than the one it replaced.

--check, and the file nobody owns

myctl support workflows --check reports drift and returns 1, so one command is both a pre-commit hook and a CI step. It also returns 1 for a file in .github/workflows/ that no entry names. That second case is the reason the section exists: a workflow nothing generates and nothing declares looks, from the directory, exactly like one somebody maintains - and it keeps running long after it stopped meaning anything.

There are two ways to answer it, and the second is a real answer rather than an escape hatch:

workflows:
  release:
    handwritten: >-
      two-thirds prose and two multi-line shell scripts; declaring it would move that work
      rather than remove it

A declined workflow is never written to, and it is named with its reason on every run - because “this one is hand-written” is easy to keep believing after it has stopped being true.

The declaration has to keep being true, too: if the file it names is not there, --check returns 1 and says so. A name with nothing behind it fails exactly the way a file nobody names does - by looking accounted for.

Other sections work the same way: suites: is the test-level taxonomy a product’s own test tree defines, environments: the deployment matrix, nexus: and claude: the data their respective tasks read. A task that needs a section it does not find fails on its first line, which is why such tasks stay tasks in the catalogue rather than being placed as commands for everybody.

layout: - where your tree sits

The kernel generates against two facts about your directories, and before si#250 it guessed both.

layout:
  build_root: kernel      # default: "" - the product root
  tests: tests            # default: tests

build_root: is the question workdir: looks like it answers and does not. workdir: is the path your tree is mounted at inside the container - toolchain:run writes -v <product root>:<workdir> -w <workdir> - so naming a subdirectory there relocates the whole tree rather than descending into it. build_root: descends, and the mount stays the product root, which a build that walks up to find its fixtures needs:

without layout:            -v /home/dev/firn:/work -w /work        gradle assemble
build_root: kernel         -v /home/dev/firn:/work -w /work/kernel gradle assemble

That trap cost a real adoption: the Java profile scaffolds gradle assemble, the product’s Gradle root was one directory down, and the scaffolded command was silently wrong.

tests: is the directory above the acceptance line - where test report writes its merged report and where docs acceptance reads scenarios. The spelling the kernel writes is tests, decided once because both were in live use and nothing had chosen. A product whose root directory is test/ says so here in one line, instead of working around it in every command that touches a path.

Nothing here is prescribed, and that is a measurement rather than a preference. Across this family, six products have six layouts - one is an Eclipse plugin tree, one has no src/ at all. A kernel that stated one layout and held products to it would refuse five of the six. So this section lets a product say what the kernel was otherwise guessing; it never rules on a tree.

An absent section is the normal case, and its defaults are exactly what the kernel assumed before it existed - so a manifest that says nothing runs the line it ran yesterday. The cost is stated rather than left to be found: a section name mistyped as layotu: is silent, and only a key mistyped inside a correctly spelled layout: is refused.

carriers: - what an environment is realised on

An environment says where it deploys with backend:. It says onto what by naming a carrier - and a carrier is named once and pointed at many times, because one Portainer serves several applications and, on the same instance, several environments.

Both halves of a carrier are optional, and each absence is a statement. proxmox: is the machine the kernel makes; portainer: is what it installs on it. A carrier with only proxmox: is a machine created and deliberately not configured (si#258). A carrier with only portainer: is a Portainer somebody else built - it has been running for months, the kernel did not make it and will not, and all this manifest needs is where it answers (si#280):

carriers:
  theirs:
    portainer:
      url_from: PORTAINER
      insecure: true

deploy carrier refuses that one by name, because there is no machine for it to make, and it says so in terms of the alternative rather than of a fault. A carrier declaring neither half is refused at load: each absence says something, both at once say nothing that any command can act on.

carriers:
  hausportainer:
    proxmox:
      endpoint: https://10.0.0.6:8006/
      token_from: PROXMOX       # a PREFIX -> PROXMOX_API_TOKEN; never the token itself
      insecure: true            # only for a self-signed certificate; the default verifies
      node: pve1
      kind: lxc                 # or vm - the choice is per environment
      template: local:vztmpl/debian-13-standard_amd64.tar.zst
      storage: local-lvm
      cores: 2                  # these three may be left out
      memory: 2048
      disk: 4
      ssh_key: "ssh-ed25519 AAAAC3Nz... you@host"
    portainer:
      url_from: PORTAINER       # a PREFIX, never a value
      endpoint: 1               # Portainer's own id; 1 is what a single-host install has
      insecure: true            # Portainer makes its own certificate on first start

environments:
  test:
    backend: portainer
    carrier: hausportainer
    stack: myctl-test
    repository:
      url: github.com/you/myctl
      compose: deploy/docker-compose.yml   # default: docker-compose.yml at the root
  prod:
    backend: portainer
    carrier: hausportainer      # the SAME carrier
    stack: myctl-prod
    repository:
      url: github.com/you/myctl
      ref: release/2.x          # default: main; `refs/tags/v2.1.0` deploys a tag
      compose: deploy/docker-compose.yml
      credential_from: GIT      # a PREFIX -> GIT_USER, GIT_PASSWORD; omit it for a public repository
    required:                   # must have a value, or the deployment stops
      - COCKPIT_DATA_DIR
      - BACKUP_DIR
      - HTTP_BIND
    optional:                   # travels when set, absent when not
      - SMALLINVOICE_CLIENT_SECRET
      - APP_TITLE

The sizes you leave out are the ones the community Proxmox helper script uses - 2 cores, 2048 MB, 4 GB - read off ct/docker.sh rather than invented, so a carrier that says nothing gets what that tool would have given it.

Two secrets are read from the environment and neither has a field here. token_from: PROXMOX means PROXMOX_API_TOKEN; url_from: PORTAINER means PORTAINER_URL, PORTAINER_TOKEN and - because Portainer needs an admin account at start or it locks itself after five minutes - PORTAINER_PASSWORD, which must be at least twelve characters.

ssh_key: is a PUBLIC key and belongs in the file. It is not a secret, and putting it here is what makes the carrier fully described by the manifest. The private half never appears.

Two things the kernel sets and you cannot get wrong: an unprivileged LXC will not start a Docker daemon without nesting and keyctl, so they are not fields - they are what a carrier is, set on every container the kernel creates. Whoever has set them by hand once has also forgotten them once.

insecure: must be true or false, never a string. Every non-empty string is truthy, so insecure: "no" would mean the opposite of what it says. It is the one value here that is refused rather than read leniently.

A lower-case prefix is accepted. token_from: proxmox reading proxmox_API_TOKEN is legal on every platform the kernel runs on, and a rule against it was written and struck: it would have refused a manifest that works.

Why a section rather than more keys on each environment. Written into the environments, the carrier above would stand in the file twice - and the two could drift apart with nothing comparing them. Stated once, they cannot.

carrier: is not a second backend:. They answer different questions and both are needed: backend: says who deploys, carrier: says onto what. The backend stays what it has always been - the one axis a product extends by registering an implementation.

repository: is where Portainer pulls the compose document from, itself. Which means Portainer needs read access to it; the orchestrator does not. It is a block rather than a bare URL because the first real consumer’s compose document is not at the repository root, and compose: is the only place that can say so. credential_from: names a PREFIX like every other credential here, and the credential is sent with every deployment rather than stored in Portainer - so a stack somebody opens in the Portainer UI carries no usable read access to your source.

backend: portainer needs no product code. It is the one backend the kernel ships, because the whole chain under it is already the kernel’s: deploy carrier builds the Portainer, carriers: describes it, and this section points at it. A product that registers its own implementation under that name still wins.

portainer: insecure: is the same value as the Proxmox one and is there for a measured reason. Portainer generates its own certificate on first start, so a carrier the kernel has just built answers only with verification off. Give it a real certificate and leave this out.

required: and optional: name the variables a deployment is given. The values come from the environment the deploy command runs in, on every deployment - not from a copy maintained by hand in Portainer. Portainer stores them either way; the question is only whether its copy is an image something refreshes or an original nobody does, and two masters of one set of values drift.

Two lists, because “travels” and “may not be empty” are two statements. A name in required: must have a value when the deployment runs. A name in optional: travels when it has one and is left out when it has not - and left out, not sent as an empty string, because to a document that writes ${X:-} those are the same today and the day they differ the kernel would have decided for it. A name in neither list does not reach the deployment at all, so the two lists are the whole answer to “what goes over”.

It was one list first, and the second one is not symmetry - it is a case one list could not say. A real product writes ${SMALLINVOICE_CLIENT_SECRET:-} where empty means “not connected”: the integration is deliberately optional. In required: the product becomes uninstallable without a Smallinvoice account; in neither list the secret never arrives and the integration is not optional but impossible. A name in both lists is refused rather than ranked.

An empty value fails the deployment. It is the one place a consuming product asked for more strictness than was offered, and the case is worth the refusal:

# in the compose document
- "${COCKPIT_DATA_DIR:-${HOME}/.biz-cockpit}:/data"

A missing value does not make compose fail - it falls back, and the bind mount lands in the carrier’s /root/.biz-cockpit instead of /srv/biz-cockpit/prod. The container writes happily, the deployment looks green, and the database sits outside everything the product’s own backup knows about. An instance that runs and is not backed up is this project’s recurring defect exactly: green because nobody looks. So every missing value is named, all of them at once - repairing eight variables one run at a time is the kernel’s work handed to an operator.

A name that is not in the list does not reach the deployment, which is what keeps the manifest readable as the answer to “what goes over”. And a manifest names variables here, never holds one: TOGGL_API_TOKEN=4c2b9f is refused rather than read as a name with an equals sign in it.

The version reaches the stack as SIMPLON_VERSION. A compose document writes image: ghcr.io/acme/app:${SIMPLON_VERSION}, so deploy up --version 1.4.0 deploys 1.4.0 rather than whatever the document happened to name. It is the kernel’s variable and required: may not claim it - two answers to one question, and nothing would print which had been used. local is refused here: Portainer clones, and there is nothing on your machine for it to clone.

An absent section is not a refusal. A product that deploys nowhere yet has no carrier to describe, and every field above defaults to empty. The section becomes required the moment an environment points at one - and then a carrier: naming nothing declared is refused, and told which carriers exist.

portainer: is optional, and leaving it out is a statement. A carrier that declares only proxmox: is a machine the kernel creates and does not configure:

carriers:
  lab:
    proxmox:
      endpoint: https://10.0.0.6:8006/
      token_from: PROXMOX
      node: pve-2
      kind: vm
      template: local:vztmpl/debian-13-standard_amd64.tar.zst
      storage: local-zfs
      ssh_key: "ssh-ed25519 AAAAC3Nz... you@host"

deploy carrier then stops after the machine exists and says so - “exists on pve-2 and nothing was installed on it”. What runs there is your own to deploy, through a backend you register. Until si#258 the portainer: block was required, which made carrier and Portainer host the same word: an apt -> Docker -> Portainer playbook ran at the end of every run with no branch in it, so a product that wanted a machine got a workload it never asked for - and on a machine without apt, a red run.

backend: portainer pointed at such a carrier is refused by name, because that backend has nowhere to put its stack. The refusal names the carrier and both ways out.

deploy: - which version a deployment is deploying

A deployment names its version, and it has three kinds of answer:

./myctl.sh prod deploy up --version 1.4.0     this exact published version
./myctl.sh prod deploy up --version latest    the newest published version
./myctl.sh prod deploy up --version local     the current code base, built now

Three values for three meanings, rather than one value that sometimes means a lookup and sometimes a build. local touches no registry at all - that is the point of it - while the other two resolve against one, and a version the registry does not serve is refused before anything starts:

ghcr.io/acme/demo-app does not serve ‘9.9.9’, so nothing was deployed. Published versions are what latest resolves against; local deploys the current code base without needing one

Asking for nothing is refused too. “Whatever is newest” is a choice somebody makes, and a deployment that made it silently would be exactly the accident this section exists to remove.

The section points at an entry you already have, rather than naming a registry a second time:

images:
  app: { registry: ghcr.io/acme, repository: demo-app, dockerfile: Dockerfile }

deploy:
  source: { images: app }        # or { artifacts: <name> }

Everything about where comes from that entry, so there is one place a registry is written down. An entry that cannot serve as a deployment source says so by name - one that is a bare string rather than a mapping, or that carries no registry: - and the message lists the keys it does carry, because the next question is what to add.

generated: - the files that must be what their command produces

A generated file you commit carries one fact twice: the source it comes from, and the bytes in the tree. They drift, and the drift is visible only to somebody who regenerates - which is exactly the person who was not going to.

generated:
  completion: { path: deploy/completions/myctl.bash, by: support completion }
  reference:  { path: docs/site/content/with-what/commands.md, by: docs reference }

./myctl.sh test generated runs each by: and fails if the file it produced is not the file that was committed, naming which one and what to run. by: is the command a person would type, without the launcher.

A file git does not track is a failure, not a pass. The obvious implementation of this gate is git diff --exit-code, and over an untracked path that exits 0 - so the artefact that was never committed at all, which is this mistake in its most complete form, would come back green. Each entry is checked for being tracked before anything is regenerated.

And a regeneration that fails is not freshness either. A command that could not run leaves the file exactly as it was, so the diff is empty. The gate reads the return code, because “regenerated, and it matched” and “did not regenerate, and nothing changed” are two different things.

An API contract is one of these, and that is the whole of the answer (si#240). A product that exports its OpenAPI document and commits it has a generated file like any other - so the export is a build command the spec entry of the python and java toolchain profiles scaffolds, and the checking is this section:

generated:
  contract: { path: api/rest/openapi.json, by: build spec }

Nothing about that is REST-specific, and nothing about it is a language’s. The question had been asked twice in this family before it was answered once: a Python product wrote git diff --exit-code over its contract and called it its CI staleness gate, and this kernel wrote a suite for its own shell completion. Both are the same six lines - and the first of them is the version that passes green over a contract nobody ever committed.

tracker: - where a refused step becomes a ticket

test:walk puts a person through the product’s .feature files one step at a time. When they refuse a step, that refusal is a bug report somebody has to write - so the kernel writes it, and this section is what it needs to know:

tracker:
  kind: github                 # the only one implemented; the default
  repo: ""                     # "owner/name", or "" for the checkout's own remote
  labels: ["bug"]
  title: "Acceptance refused: {scenario} ({version})"
  preamble: >
    A person walked the acceptance scenarios and refused this step. The verdict is a
    human judgement rather than a measurement.

title: is the one key with no default, on purpose: a kernel-invented title is the first thing a maintainer reads and the last thing the kernel should be guessing. It may name product, version, revision, address, feature, scenario, step, step_number and step_total; a field that is not on that list is answered with the list.

preamble: is the product’s own wording, put above the evidence the kernel writes underneath - the step, the version, the revision, who walked it, when, and what they said.

A second walk that refuses the same step again does not open a second ticket. Each one carries a simplon-walk-id: line in visible prose, and the search for it is what makes the walk idempotent - the line is prose rather than an HTML comment so the person reading the ticket can see why, too.

The flat form, and how to leave it

You may meet an older spelling, in a manifest that predates the tree: a body written straight onto a command, and catalogue tasks placed through an import: section plus a coordinate-keyed tasks: entry.

groups:
  build:
    wheel: { impl: "orchestrator.cli:build_wheel", help: "Build the wheel." }

import:
  delivery: [docs]

tasks:
  docs:reference:
    group: build

That form was abolished in 0.4.0. A manifest written that way no longer loads, and the loader says so by name rather than letting it die further down as a missing key on some node:

this manifest is written in the flat command form, which no longer loads: the form was abolished in
0.4.0 and this kernel is past it.

What says so here: group(s) 'build' name commands directly, with `impl:` on them; task(s)
'docs:reference' are keyed by a platform coordinate; an `import:` section makes catalogue coordinates
available.

What it becomes:
  - a command is an INSTANCE of a task: declare the body once under `tasks:` and let the command point
    at it with `task:`, under `groups: <group>: commands:`
  - a catalogue task keeps its body in the kernel - the command names the coordinate
    (`task: "<namespace>:<name>"`) and copies nothing
  - the catalogue's own commands arrive by merging its tree, so `import:` has nothing left to do -
    delete the section

This migration is documented at
  https://marcozwyssig.github.io/simplon/how/manifest/#the-flat-form-and-how-to-leave-it
and the shape it leads to at
  https://marcozwyssig.github.io/simplon/how/manifest/

Nothing here rewrites the file for you: the sections have to be edited by hand, which is also the only
way your comments survive the move.

The manifest above becomes:

tasks:
  wheel: { impl: "orchestrator.cli:build_wheel", help: "Build the wheel." }

groups:
  build:
    commands:
      wheel: { task: "wheel" }
      reference: { task: "docs:reference" }

(Until 0.11.0 the refusal also printed that block for you, rendered from your own manifest. The renderer went in si#85: no manifest that installs this kernel had been on the flat form since 0.4.0, so it was some 250 lines describing the manifest’s shape a second time, where nobody reading was left to notice it drifting - and si#56 had just found a property of it that was mis-documented from the day it was written. This page is the source that is maintained.)

Seven things to know while you convert:

• An aggregate crosses unchanged. A command with depends_on: and no impl: was never a body, so there is nothing to move out of it.
• A shared body becomes one task. Two commands that spelled out the same impl: are one template with two placements - which is the point of the form. Do not carry the duplication across.
• A name collision needs two task names. build build and deploy build are two different bodies under one command name. That is legal in the tree and impossible in one flat tasks: block, so one of the two bodies has to be declared under a different task name; the commands keep the names they had.
• A name the catalogue places needs override: true. support install is your body under a name the platform already uses, so the merge demands you say you mean it. Where your placement names the platform’s own body, it is a refinement and needs no key.
• A tasks: entry with group: keeps its body and loses the key. Its body is already where the tree form wants it; what makes it flat is that it places its own command. Delete group:, and add the command under groups: <group>: commands: pointing at the task with task:. Leaving the key behind is the one case where a manifest fixed by eye is refused a second time.
• A coordinate-shaped name with its own impl: is your body, not the platform’s. docs:site: with no impl: names the kernel’s task and copies nothing; docs:site: carrying an impl: was never a placement of the platform’s body at all - it is yours under a name that looks like theirs. Keep the impl:, rename the task to a bare name, and point the command at that.
• Your comments are yours to carry. Nothing rewrites the file for you any more, which is the one respect in which this is easier than it was: a generated block was correct YAML carrying none of the reasoning the file it replaced carried, and a consumer whose manifest held forty lines of it kept them only because somebody looked.

What no conversion can fix is an old manifest that says something the tree form does not allow at all - a group the platform’s tree does not declare, a coordinate placed outside the group its namespace names, a name that is an aggregate for you and a task-backed command in the catalogue. Those are refusals of your manifest rather than of its spelling, and each names its own way out.

What you lose is the import: section, and you lose nothing with it: the catalogue’s own commands arrive by merging its tree, and any other coordinate is named directly by the command that wants it.