This page lists 9 acceptance scenarios across 2 feature files, read off tests/acceptance/ in simplon’s own tree - so it lists what is verified, not what somebody remembered writing a test for.

What this page does and does not say. It says what the scenarios ARE. Whether a run executed them, and how each step went, is the run’s own archive - and that archive only carries a step at all when the product’s suite requirements name allure-pytest-bdd. With plain allure-pytest a result carries the mangled pytest function name and an empty step list, measured; with this plugin it carries the scenario title verbatim, one entry per step with its own status, and a failing step named with its assertion. A person walking these scenarios produces per-step evidence either way, so without the plugin the two modes cannot be compared at all.

Each scenario is addressed by its feature file and its title, separated by a colon. That is the string the run’s archive records as a result’s fullName, byte for byte, so a walked verdict and a machine one line up on one key. The file is named the way the RUNNER names it - the base directory’s own name plus the path under it - so the address is not where the file sits in the repository; each feature section below says that too.

Scenario	Feature	Steps	Address
The command reference names every command the assembled CLI offers	The generated documents cannot go stale	6	acceptance/generated-documents.feature:The command reference names every command the assembled CLI offers
The acceptance document names every scenario the feature files carry	The generated documents cannot go stale	6	acceptance/generated-documents.feature:The acceptance document names every scenario the feature files carry
A new scenario reaches the page without anybody editing the page	The generated documents cannot go stale	8	acceptance/generated-documents.feature:A new scenario reaches the page without anybody editing the page
Both pages reach the published website	The generated documents cannot go stale	5	acceptance/generated-documents.feature:Both pages reach the published website
The local gate runs every test the pipeline runs	One command reaches one verdict	4	acceptance/one-verdict.feature:The local gate runs every test the pipeline runs
A failing suite turns the aggregate red	One command reaches one verdict	6	acceptance/one-verdict.feature:A failing suite turns the aggregate red
A failing typecheck-python turns the aggregate red	One command reaches one verdict	6	acceptance/one-verdict.feature:A failing typecheck-python turns the aggregate red
A failing release-notes turns the aggregate red	One command reaches one verdict	6	acceptance/one-verdict.feature:A failing release-notes turns the aggregate red
A gate that found nothing to run is not green	One command reaches one verdict	5	acceptance/one-verdict.feature:A gate that found nothing to run is not green

The generated documents cannot go stale

Source: tests/acceptance/generated-documents.feature, which the runner knows as acceptance/generated-documents.feature.

Tagged @documents.

Simplon publishes two pages it never writes by hand: the command reference, read off the assembled command line, and this acceptance document, read off these very files. Both exist because a hand-maintained list goes stale silently - a command is added, a scenario is added, and nothing anywhere goes red. These scenarios are how somebody confirms that from outside the code.

Every scenario below opens with the same 2 background steps, which is why they are repeated in each walk rather than stated once: a person performs them once per scenario, and so does the runner.

The command reference names every command the assembled CLI offers

acceptance/generated-documents.feature:The command reference names every command the assembled CLI offers

Tagged @reference.

1. Given a clean checkout of simplon (background)
2. And docker is available to the caller (background)
3. When I run “./simplon.sh build reference”
4. Then “docs/site/content/with-what/commands.md” exists
5. And every command “./simplon.sh –help” lists has a section on that page
6. And the page’s own lead sentence states the number of commands it documents

The acceptance document names every scenario the feature files carry

acceptance/generated-documents.feature:The acceptance document names every scenario the feature files carry

Tagged @acceptance.

1. Given a clean checkout of simplon (background)
2. And docker is available to the caller (background)
3. When I run “./simplon.sh build acceptance”
4. Then “docs/site/content/with-what/acceptance.md” exists
5. And every scenario title in “tests/acceptance” appears on that page
6. And each one is addressed by its feature file and its title, separated by a colon

A new scenario reaches the page without anybody editing the page

acceptance/generated-documents.feature:A new scenario reaches the page without anybody editing the page

Tagged @acceptance @growth.

1. Given a clean checkout of simplon (background)
2. And docker is available to the caller (background)
3. Given the acceptance document has been generated once
4. When I add a scenario to any file under “tests/acceptance”
5. And I run “./simplon.sh build acceptance” again
6. Then the page carries the new scenario
7. And the count in its lead sentence has grown by one
8. And no file under “docs/site/content” was edited by hand

Both pages reach the published website

acceptance/generated-documents.feature:Both pages reach the published website

Tagged @site.

1. Given a clean checkout of simplon (background)
2. And docker is available to the caller (background)
3. When I run “./simplon.sh build docs”
4. Then “build/website/with-what/commands/index.html” exists
5. And “build/website/with-what/acceptance/index.html” exists

One command reaches one verdict

Source: tests/acceptance/one-verdict.feature, which the runner knows as acceptance/one-verdict.feature.

Tagged @gate.

The local gate and the pipeline have to be the same command, or the two drift and the next divergence is found by whoever has the least context. These scenarios walk that claim from outside: what a developer types and what CI types reach the same body, and a step that found nothing is never green.

Every scenario below opens with the same 1 background step, which is why they are repeated in each walk rather than stated once: a person performs them once per scenario, and so does the runner.

The local gate runs every test the pipeline runs

acceptance/one-verdict.feature:The local gate runs every test the pipeline runs

1. Given a clean checkout of simplon (background)
2. When I run “./simplon.sh test all”
3. Then the run reports the pytest suite, the type gate and the release-notes guard
4. And every leaf named in “.github/workflows/ci.yml” was one of them

A failing suite turns the aggregate red

acceptance/one-verdict.feature:A failing suite turns the aggregate red

Row 1 of the outline A failing <leaf> turns the aggregate red. The outline itself is not a scenario and never runs: each of its rows does, under the title above.

1. Given a clean checkout of simplon (background)
2. Given the suite step is made to fail
3. When I run “./simplon.sh test all”
4. Then the run exits non-zero
5. And the summary names suite as the step that failed
6. And the other steps still ran and still printed their own verdict

A failing typecheck-python turns the aggregate red

acceptance/one-verdict.feature:A failing typecheck-python turns the aggregate red

Row 2 of the outline A failing <leaf> turns the aggregate red. The outline itself is not a scenario and never runs: each of its rows does, under the title above.

1. Given a clean checkout of simplon (background)
2. Given the typecheck-python step is made to fail
3. When I run “./simplon.sh test all”
4. Then the run exits non-zero
5. And the summary names typecheck-python as the step that failed
6. And the other steps still ran and still printed their own verdict

A failing release-notes turns the aggregate red

acceptance/one-verdict.feature:A failing release-notes turns the aggregate red

Row 3 of the outline A failing <leaf> turns the aggregate red. The outline itself is not a scenario and never runs: each of its rows does, under the title above.

1. Given a clean checkout of simplon (background)
2. Given the release-notes step is made to fail
3. When I run “./simplon.sh test all”
4. Then the run exits non-zero
5. And the summary names release-notes as the step that failed
6. And the other steps still ran and still printed their own verdict

A gate that found nothing to run is not green

acceptance/one-verdict.feature:A gate that found nothing to run is not green

1. Given a clean checkout of simplon (background)
2. Given a suite directory holding no test at all
3. When the gate for it runs
4. Then the run does not report success
5. And the message says that nothing was collected rather than that nothing failed