Commits: 17
Expand the roadmap
index 420e7e4..78a7ecb 100644
--- a/README.md
+++ b/README.md
@@ -61,9 +61,11 @@ In summary:
= - [ ] Usage
= - [ ] Installation
= - [ ] Rationale
+ - [ ] Protocol
= - [ ] Bag of tricks
= - [ ] Mocking email delivery through logs
= - [ ] Assert general success (like exit code) last
+- [ ] Demo
=- [x] Proof of concept
= - [x] Interpretter in a different language (Python)
= - [x] Report why steps fail
@@ -76,8 +78,10 @@ In summary:
= - [ ] Tagging
= - [x] Filtering
= - [x] Large steps
+ - [ ] Serialization (how steps are serialized)
=- [ ] Timeout for steps
=- [ ] Recursion guard
+- [ ] Protocol versioning (`ready(0)`)
=- [ ] Complete examples
= - [ ] Reverse polish notation calculator:
= - [ ] CLI (Python)
@@ -99,6 +103,7 @@ In summary:
= - [ ] Collapse ommited steps (`□□□□□□ following 6 steps skipped`)
= - [ ] Collapse filtered out suites
= - [ ] Collapse filtered out scenarios
+ - [ ] Mark scenarios without steps
=- [ ] Pass more step data to interpreters
= - [x] Code blocks
= - [x] TablesSeparate the roadmap document
Also warn about breaking changes and make other small improvements to the readme.
index 78a7ecb..0db1bf3 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,5 @@
+> This software is currently in early stage of development. Breaking changes may be introduced without a warning. If you try it, please let me know and I'll try to provide best support I can, but keep your expectations in check ;-) See [The Roadmap](./roadmap.md) for an overview of features planned and implemented.
+
=# TBB: Tad Better Behavior
=
=A BDD test runner inspired by Gauge, but better.
@@ -9,7 +11,7 @@ A BDD test runner inspired by Gauge, but better.
=
=## No magic
=
-Test scenarios are evaluated by interpreters, which are normal programs that read JSON on stdin and write JSON to stdout. User can implement them in any language or framework they like, using any tools they like (editors, lanuage servers, libraries).
+Test scenarios are evaluated by interpreters, which are normal programs that read JSON on `stdin` and write JSON to `stdout`. User can implement them in any language or framework they like, using any tools they are comfortable with (editors, language servers, libraries, etc).
=
=
=## Flexibility
@@ -54,88 +56,3 @@ In summary:
= - A scenario is run
= - A step is executed
=
-
-## Roadmap
-
-- [ ] A proper readme
- - [ ] Usage
- - [ ] Installation
- - [ ] Rationale
- - [ ] Protocol
- - [ ] Bag of tricks
- - [ ] Mocking email delivery through logs
- - [ ] Assert general success (like exit code) last
-- [ ] Demo
-- [x] Proof of concept
- - [x] Interpretter in a different language (Python)
- - [x] Report why steps fail
-- [ ] Self-check
- - [x] Proof of concept
- - [ ] Comprehensive
- - [x] Help
- - [x] List
- - [x] Evaluate
- - [ ] Tagging
- - [x] Filtering
- - [x] Large steps
- - [ ] Serialization (how steps are serialized)
-- [ ] Timeout for steps
-- [ ] Recursion guard
-- [ ] Protocol versioning (`ready(0)`)
-- [ ] Complete examples
- - [ ] Reverse polish notation calculator:
- - [ ] CLI (Python)
- - [ ] HTTP API (Python)
- - [ ] Web UI (HTML+ JS)
-- [x] Tags to filtering suites and scenarios
- - [x] Per document (in front-matter)
- - [x] Per suite (in a `yaml tbb` codeblock under `h1`)
- - [x] Per scenario (in a `yaml tbb` codeblock under `h2`)
- - [x] `--exclude` CLI option (logical *or*)
- - [x] `--only` CLI option (logical *and*)
- - [x] prefixes (like `suite:foo`, `scenario:ready`)
-- [ ] More readable report
- - [x] The emojis are misaligned and lack color (at least in my terminal)
- - [x] A summary at the bottom (esp. list of errors)
- - [x] Use colors
- - [ ] More instructive error messages
- - [x] Indent multiline error messages
- - [ ] Collapse ommited steps (`□□□□□□ following 6 steps skipped`)
- - [ ] Collapse filtered out suites
- - [ ] Collapse filtered out scenarios
- - [ ] Mark scenarios without steps
-- [ ] Pass more step data to interpreters
- - [x] Code blocks
- - [x] Tables
- - [x] Lists
- - [ ] Definition lists
- - [ ] Original markdown fragment
- - [ ] Block quotes
-- [x] Nix package (from Flake)
-- [x] Use for evaluating Jewiet's Form to Mail specification
-- [ ] Helper libraries
- - [x] for Python
- - [x] Proof-of-concept
- - [x] `@step` decorator
- - [x] Automatic arguments conversion (casting)
- - [x] Insightful assertion errors
- - [x] Split library code from the basic interpreter
- - [ ] Run unit tests autmatically
- - [ ] for Clojure
- - [ ] For POSIX shells
- - [ ] For NuShell
-- [ ] Built-in interpreter (`tbb automation`)
- - [ ] HTTP client
- - [ ] Web Driver client
- - [ ] E-Mail client
- - [ ] Recursive calls (call `tbb evaluate` and such)
-- [ ] Capture more data in reports
- - [ ] Attachments (screenshots, videos, datasets, etc.)
- - [ ] Performance data (interpreters' startup times, steps' durations)
- - [ ] Annotations from interpreters
-- [ ] Better reporters
- - [ ] TUI
- - [ ] Web
-- [ ] WASM target
-
-new file mode 100644
index 0000000..c117e8b
--- /dev/null
+++ b/roadmap.md
@@ -0,0 +1,89 @@
+# The Roadmap for Tad Better Behavior
+
+We use this document mostly to keep track of our progress and a scratchpad for ideas. Not everything from here will be implemented, and not necessarily in the order it's presented.
+
+- [ ] A proper readme
+ - [ ] Usage
+ - [ ] Installation
+ - [ ] Rationale
+ - [ ] Protocol
+ - [ ] Bag of tricks
+ - [ ] Mocking email delivery through logs
+ - [ ] Assert general success (like exit code) last
+ - [ ] Compare with other solutions
+ - [ ] Gauge
+ - [ ] Cucumber
+ - [ ] Playwright
+- [ ] Demo
+- [x] Proof of concept
+ - [x] Interpretter in a different language (Python)
+ - [x] Report why steps fail
+- [ ] Self-check
+ - [x] Proof of concept
+ - [ ] Comprehensive
+ - [x] Help
+ - [x] List
+ - [x] Evaluate
+ - [ ] Tagging
+ - [x] Filtering
+ - [x] Large steps
+ - [ ] Serialization (how steps are serialized)
+- [ ] Timeout for steps
+- [ ] Recursion guard
+- [ ] Protocol versioning (`ready(0)`)
+- [ ] Complete examples
+ - [ ] Reverse polish notation calculator:
+ - [ ] CLI (Python)
+ - [ ] HTTP API (Python)
+ - [ ] Web UI (HTML+ JS)
+- [x] Tags to filtering suites and scenarios
+ - [x] Per document (in front-matter)
+ - [x] Per suite (in a `yaml tbb` codeblock under `h1`)
+ - [x] Per scenario (in a `yaml tbb` codeblock under `h2`)
+ - [x] `--exclude` CLI option (logical *or*)
+ - [x] `--only` CLI option (logical *and*)
+ - [x] prefixes (like `suite:foo`, `scenario:ready`)
+- [ ] More readable report
+ - [x] The emojis are misaligned and lack color (at least in my terminal)
+ - [x] A summary at the bottom (esp. list of errors)
+ - [x] Use colors
+ - [ ] More instructive error messages
+ - [x] Indent multiline error messages
+ - [ ] Collapse ommited steps (`□□□□□□ following 6 steps skipped`)
+ - [ ] Collapse filtered out suites
+ - [ ] Collapse filtered out scenarios
+ - [ ] Mark scenarios without steps
+- [ ] Pass more step data to interpreters
+ - [x] Code blocks
+ - [x] Tables
+ - [x] Lists
+ - [ ] Definition lists
+ - [ ] Original markdown fragment
+ - [ ] Block quotes
+- [x] Nix package (from Flake)
+- [x] Use for evaluating Jewiet's Form to Mail specification
+- [ ] Helper libraries
+ - [x] for Python
+ - [x] Proof-of-concept
+ - [x] `@step` decorator
+ - [x] Automatic arguments conversion (casting)
+ - [x] Insightful assertion errors
+ - [x] Split library code from the basic interpreter
+ - [ ] Run unit tests autmatically
+ - [ ] for Clojure
+ - [ ] For POSIX shells
+ - [ ] For NuShell
+- [ ] Built-in interpreter (`tbb automation`)
+ - [ ] HTTP client
+ - [ ] Web Driver client
+ - [ ] E-Mail client
+ - [ ] Recursive calls (call `tbb evaluate` and such)
+- [ ] Capture more data in reports
+ - [ ] Attachments (screenshots, videos, datasets, etc.)
+ - [ ] Performance data (interpreters' startup times, steps' durations)
+ - [ ] Annotations from interpreters
+- [ ] Better reporters
+ - [ ] TUI
+ - [ ] Web
+- [ ] WASM target
+Write how steps to verify "no suites" behavior
index 961c36c..6ea477d 100644
--- a/spec/filtering.md
+++ b/spec/filtering.md
@@ -122,6 +122,10 @@ If as a result of filtering there are no scenarios to evaluate, the program shou
=
=Same as above.
=
+ * Run the program with `evaluate --only suite:no-such-tag samples/` command line arguments
+ * The standard error will contain `\[.+ WARN +tbb\] Nothing to evaluate`
+ * The output will be empty
+ * The exit code should be `1`
=
=## Passing multiple comma delimited tags
=Fix a regular expression in self-check interpreter
index bf735fc..94a8422 100644
--- a/spec/self-check.py
+++ b/spec/self-check.py
@@ -90,8 +90,8 @@ def step_implementation_04(label: str, **kwargs):
=
= # Trim all blank lines and trailing whitespece.
= # Without it the assertions are very brittle.
- needle = re.sub("\s+$", "", block)
- heystack = re.sub("\s+$", "", output)
+ needle = re.sub("\\s+$", "", block)
+ heystack = re.sub("\\s+$", "", output)
= # tester.assertIn gives unreadable output
=
=Implement a step checking that there is no output
index 94a8422..3467b97 100644
--- a/spec/self-check.py
+++ b/spec/self-check.py
@@ -160,4 +160,9 @@ def step_implementation_09(
= subtree = get_at (state, [big_list_name, "items", item_index, "subtrees", subtree_index])
= state[small_list_name] = subtree
=
+@step("The output will be empty")
+def step_implementation_10(**kwargs):
+ global completed
+ tester.assertEqual(completed.stdout.strip().decode("utf-8"), "")
+
=tbb.ready()Implement the "no suites to evaluate" spec
index 959b88a..4cb68f7 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -176,6 +176,10 @@ fn evaluate(
=
= match EvaluationSummary::from(report) {
= EvaluationSummary::AllOk => Ok(()),
+ EvaluationSummary::NothingToEvaluate => {
+ log::warn!("Nothing to evaluate");
+ process::exit(1)
+ }
= EvaluationSummary::Failed { failed_steps } => {
= for (suite, scenario, step) in failed_steps.iter() {
= log::error!(index 23d7054..29c4018 100644
--- a/src/report.rs
+++ b/src/report.rs
@@ -404,6 +404,7 @@ pub enum ControlMessage {
=
=pub enum EvaluationSummary {
= AllOk,
+ NothingToEvaluate,
= Failed {
= failed_steps: Vec<(Suite, Scenario, Step)>,
= },
@@ -412,25 +413,32 @@ pub enum EvaluationSummary {
=impl<'a> From<EvaluationReport<'a>> for EvaluationSummary {
= fn from(report: EvaluationReport) -> Self {
= let mut failed_steps: Vec<(Suite, Scenario, Step)> = Vec::default();
+ let mut anything_evaluated = false;
=
= for suite in report.suites.iter() {
= for scenario in suite.scenarios.iter() {
= for step in scenario.steps.iter() {
- if let StepStatus::Failed { .. } = step.status {
- failed_steps.push((
- suite.suite.clone(),
- scenario.scenario.clone(),
- step.step.clone(),
- ));
+ match step.status {
+ StepStatus::Failed { .. } => {
+ failed_steps.push((
+ suite.suite.clone(),
+ scenario.scenario.clone(),
+ step.step.clone(),
+ ));
+ }
+ StepStatus::Ok => anything_evaluated = true,
+ StepStatus::NotEvaluated => {}
= }
= }
= }
= }
=
- if failed_steps.is_empty() {
+ if failed_steps.is_empty().not() {
+ Self::Failed { failed_steps }
+ } else if anything_evaluated {
= Self::AllOk
= } else {
- Self::Failed { failed_steps }
+ Self::NothingToEvaluate
= }
= }
=}Write verification for "no scenarios" behavior
It already passes.
index 6ea477d..867a2d7 100644
--- a/spec/filtering.md
+++ b/spec/filtering.md
@@ -118,9 +118,18 @@ If filtering results in some suites having no scenarios to evaluate, they should
=If as a result of filtering there are no scenarios to evaluate, the program should print a warning and exit with an error code. This is to prevent false positives when running unsupervised (e.g. in a CI/CD system).
=
=
+ * Run the program with `evaluate --only scenario:no-such-tag samples/` command line arguments
+ * The standard error will contain `\[.+ WARN +tbb\] Nothing to evaluate`
+ * The output will contain `Basic BDD suite`
+ * The output will contain `A little suite that could`
+
+ And others.
+
+ * The exit code should be `1`
+
=## No suites to evaluate at all
=
-Same as above.
+Same as above, but there should be no output.
=
= * Run the program with `evaluate --only suite:no-such-tag samples/` command line arguments
= * The standard error will contain `\[.+ WARN +tbb\] Nothing to evaluate`Escalate the "nothing to evaluate" warning to an error
The program exists with an error status, so the message should indicate it.
index 867a2d7..e01000d 100644
--- a/spec/filtering.md
+++ b/spec/filtering.md
@@ -115,11 +115,11 @@ If filtering results in some suites having no scenarios to evaluate, they should
=
=## No scenarios to evaluate at all
=
-If as a result of filtering there are no scenarios to evaluate, the program should print a warning and exit with an error code. This is to prevent false positives when running unsupervised (e.g. in a CI/CD system).
+If as a result of filtering there are no scenarios to evaluate, the program should print an error and exit with an error code. This is to prevent false positives when running unsupervised (e.g. in a CI/CD system).
=
=
= * Run the program with `evaluate --only scenario:no-such-tag samples/` command line arguments
- * The standard error will contain `\[.+ WARN +tbb\] Nothing to evaluate`
+ * The standard error will contain `\[.+ ERROR +tbb\] Nothing to evaluate`
= * The output will contain `Basic BDD suite`
= * The output will contain `A little suite that could`
=
@@ -132,7 +132,7 @@ If as a result of filtering there are no scenarios to evaluate, the program shou
=Same as above, but there should be no output.
=
= * Run the program with `evaluate --only suite:no-such-tag samples/` command line arguments
- * The standard error will contain `\[.+ WARN +tbb\] Nothing to evaluate`
+ * The standard error will contain `\[.+ ERROR +tbb\] Nothing to evaluate`
= * The output will be empty
= * The exit code should be `1`
=index 4cb68f7..0025eb6 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -177,7 +177,7 @@ fn evaluate(
= match EvaluationSummary::from(report) {
= EvaluationSummary::AllOk => Ok(()),
= EvaluationSummary::NothingToEvaluate => {
- log::warn!("Nothing to evaluate");
+ log::error!("Nothing to evaluate");
= process::exit(1)
= }
= EvaluationSummary::Failed { failed_steps } => {Bump the minor version to 0.7
There's a breaking change. When there is nothing to evaluate, the program will exit with an error now.
index 2c22711..7fcd666 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -400,7 +400,7 @@ dependencies = [
=
=[[package]]
=name = "tad-better-behavior"
-version = "0.6.0"
+version = "0.7.0"
=dependencies = [
= "anyhow",
= "clap",index c8b6859..afce511 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,6 @@
=[package]
=name = "tad-better-behavior"
-version = "0.6.0"
+version = "0.7.0"
=edition = "2024"
=
=[dependencies]Merge big-steps interpreter into self-check
There is really no reason to keep it separate.
index 9a6c6f2..847d607 100644
--- a/spec/big-steps.md
+++ b/spec/big-steps.md
@@ -1,5 +1,5 @@
=---
-interpreter: "python -m spec.big-steps"
+interpreter: "python -m spec.self-check"
=---
=
=# Big stepsdeleted file mode 100644
index 1f7a261..0000000
--- a/spec/big-steps.py
+++ /dev/null
@@ -1,24 +0,0 @@
-#!/usr/bin/env python3
-
-import spec.tbb as tbb
-from spec.tbb import log, step
-import unittest
-
-state = {} # Store dynamic variables here
-tester = unittest.TestCase()
-
-@step("Load the following table as {0}")
-def step_implementation_00(name: str, **kwargs):
- state[name] = kwargs["tables"][0]
-
-@step("There should be {0} rows in {1}")
-def step_implementation_01(row_count: int, table_name: str, **kwargs):
- table = state[table_name]
- tester.assertEqual(len(table), row_count)
-
-@step("Row {0} column {1} of {2} should contain {3}")
-def step_implementation_02(row: int, column: int, table_name: str, expected_value: str, **kwargs):
- table = state[table_name]
- tester.assertEqual(table[row][column], expected_value)
-
-tbb.ready()index 3467b97..9b1af63 100644
--- a/spec/self-check.py
+++ b/spec/self-check.py
@@ -165,4 +165,18 @@ def step_implementation_10(**kwargs):
= global completed
= tester.assertEqual(completed.stdout.strip().decode("utf-8"), "")
=
+@step("Load the following table as {0}")
+def step_implementation_11(name: str, **kwargs):
+ state[name] = kwargs["tables"][0]
+
+@step("There should be {0} rows in {1}")
+def step_implementation_12(row_count: int, table_name: str, **kwargs):
+ table = state[table_name]
+ tester.assertEqual(len(table), row_count)
+
+@step("Row {0} column {1} of {2} should contain {3}")
+def step_implementation_13(row: int, column: int, table_name: str, expected_value: str, **kwargs):
+ table = state[table_name]
+ tester.assertEqual(table[row][column], expected_value)
+
=tbb.ready()Require that a failing interpreter leads to error
There is currently a bug - if some scenarios completely fail (e.g. interpreter is invalid), but some other pass (i.e. there are no failed steps, just some whole scenarios), the exit status is 0 and there is no error at the bottom.
similarity index 96%
rename from samples/invalid.md
rename to samples/some-invalid/invalid.md
index 1e99cc7..99ec0b1 100644
--- a/samples/invalid.md
+++ b/samples/some-invalid/invalid.md
@@ -14,7 +14,7 @@ It should be possible to define multiple suites in each document.
=tags: [ seriously-underbaked ]
=```
=
-## Scenario 1.1
+## Hopeless scenario
=
=``` yaml tbb
=tags: [ work-in-progress, very-important ]similarity index 100%
rename from samples/passing.md
rename to samples/some-invalid/valid.mdnew file mode 100644
index 0000000..274adcc
--- /dev/null
+++ b/spec/failing-interpreters.md
@@ -0,0 +1,43 @@
+---
+interpreter: python -m spec.self-check
+---
+
+# Failing interpreters
+
+This suite describes what should happen when the interpreter process misbehaves in various ways.
+
+
+## Some won't start
+
+If some interpreters' commands are invalid (e.g. refers to a non-existing program), but others run fine, then `tbb` should report it as an error and exit with error status code.
+
+ * Run the program with `evaluate samples/some-invalid/` command line arguments
+
+ There are two documents in this directory: `valid.md` and `invalid.md`.
+
+ * The output will contain `the valid suite header` block
+
+ ```text
+ A little suite that could (python -m samples.basic)
+ tagged: basic passing
+ ```
+
+ * The output will contain `the invalid suite header` block
+
+ ```text
+ Suite 1 from the invalid document (invalid interpreter)
+ tagged: not-implemented seriously-underbaked
+ ```
+
+ * The output will contain `the failing scenario` block
+
+ ```text
+ x Hopeless scenario
+ tagged: even more tags very-important work-in-progress
+ ```
+
+ Notice the failing sigil in front.
+
+ * The standard error will contain `\[.+ ERROR +tbb\] Scenario failed: Suite 1 from the invalid document ❯ Hopeless scenario`
+
+ * The exit code should be `1`index e01000d..e462842 100644
--- a/spec/filtering.md
+++ b/spec/filtering.md
@@ -73,7 +73,7 @@ With the `--exclude suite:<tag>` any suites that have the given tag should be ex
=
=With the `--exclude scenario:<tag>` any scenarios that have the given tag should be excluded from the report. What's not in the report, should not be evaluated (laziness).
=
- * Run the program with `evaluate --exclude scenario:interesting samples/passing.md` command line arguments
+ * Run the program with `evaluate --exclude scenario:interesting samples/some-invalid/valid.md` command line arguments
= * The output will contain `the expected suite header` block
=
= ```textImplement the "Failing interpreters" spec
index 0025eb6..b37e0a9 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -180,14 +180,24 @@ fn evaluate(
= log::error!("Nothing to evaluate");
= process::exit(1)
= }
- EvaluationSummary::Failed { failed_steps } => {
+ EvaluationSummary::Failed {
+ failed_scenarios: failed_steps,
+ } => {
= for (suite, scenario, step) in failed_steps.iter() {
- log::error!(
- "Step failed: {suite} ❯ {scenario} ❯ {step}",
- suite = suite.title,
- scenario = scenario.title,
- step = step.description
- )
+ if let Some(step) = step {
+ log::error!(
+ "Step failed: {suite} ❯ {scenario} ❯ {step}",
+ suite = suite.title,
+ scenario = scenario.title,
+ step = step.description
+ )
+ } else {
+ log::error!(
+ "Scenario failed: {suite} ❯ {scenario}",
+ suite = suite.title,
+ scenario = scenario.title,
+ )
+ }
= }
= // TODO: Print errors from failing steps
= process::exit(1)index 29c4018..64868c2 100644
--- a/src/report.rs
+++ b/src/report.rs
@@ -406,24 +406,27 @@ pub enum EvaluationSummary {
= AllOk,
= NothingToEvaluate,
= Failed {
- failed_steps: Vec<(Suite, Scenario, Step)>,
+ failed_scenarios: Vec<(Suite, Scenario, Option<Step>)>,
= },
=}
=
=impl<'a> From<EvaluationReport<'a>> for EvaluationSummary {
= fn from(report: EvaluationReport) -> Self {
- let mut failed_steps: Vec<(Suite, Scenario, Step)> = Vec::default();
+ let mut failed_scenarios: Vec<(Suite, Scenario, Option<Step>)> = Vec::default();
= let mut anything_evaluated = false;
=
= for suite in report.suites.iter() {
= for scenario in suite.scenarios.iter() {
+ if let ScenarioStatus::FailedToRun { .. } = scenario.status {
+ failed_scenarios.push((suite.suite.clone(), scenario.scenario.clone(), None))
+ }
= for step in scenario.steps.iter() {
= match step.status {
= StepStatus::Failed { .. } => {
- failed_steps.push((
+ failed_scenarios.push((
= suite.suite.clone(),
= scenario.scenario.clone(),
- step.step.clone(),
+ step.step.clone().into(),
= ));
= }
= StepStatus::Ok => anything_evaluated = true,
@@ -433,8 +436,8 @@ impl<'a> From<EvaluationReport<'a>> for EvaluationSummary {
= }
= }
=
- if failed_steps.is_empty().not() {
- Self::Failed { failed_steps }
+ if failed_scenarios.is_empty().not() {
+ Self::Failed { failed_scenarios }
= } else if anything_evaluated {
= Self::AllOk
= } else {Demand that scenarios without steps are marked with ?
index 7c1be74..4160666 100644
--- a/samples/basic.md
+++ b/samples/basic.md
@@ -88,3 +88,11 @@ This
= This step is intentionally wrong to allow demonstrate that TBB will not proceed with following steps.
=
= * There are `2` `o`s in the word `boost`
+
+## Geometry
+
+``` yaml tbb
+tags: [math]
+```
+
+There are no steps in this scenario. It's ok. Sometimes it's easy to start by describing the behavior in prose, and write formal steps to verify it later. A scenario like this will be listed, but won't be evaluated.index 91c0590..56caed6 100644
--- a/spec/basic-usage.md
+++ b/spec/basic-usage.md
@@ -174,6 +174,13 @@ Notice it's similar to the output of `tbb list`, but now contains unicode symbol
= + AIC
= ```
=
+ * The output will contain `geometry scenario` block
+
+ ```text
+ ? Geometry
+ tagged: math
+ ```
+
= * The standard error will contain `\[.+ ERROR +tbb\] Step failed: Basic BDD suite ❯ Text ❯ The reverse of CIA is KGB`
=
= * The exit code should be `1`Do not mark scenarios without steps as done
Keep them pending, which results in a question mark sigil in the report.
index 64868c2..f54c4ee 100644
--- a/src/report.rs
+++ b/src/report.rs
@@ -88,7 +88,7 @@ impl<'a> SuiteReport<'a> {
= .context(format!("running scenario: {}", scenario.scenario.title));
= if let Err(error) = result {
= scenario.status = ScenarioStatus::FailedToRun { error }
- } else {
+ } else if scenario.steps.is_empty().not() {
= scenario.status = ScenarioStatus::Done
= }
= }Fix a typo
index 56caed6..e258f55 100644
--- a/spec/basic-usage.md
+++ b/spec/basic-usage.md
@@ -192,7 +192,7 @@ Running `tbb` without a subcommand will print the help message (to stderr) and e
=
=## A whole scenario failure
=
-Sometimes the failur is not in any particular step, but in a whole scenario, e.g. when an interpreter misbehaves. The status code and the summary should reflect it.
+Sometimes the failure is not in any particular step, but in a whole scenario, e.g. when an interpreter misbehaves. The status code and the summary should reflect it.
=
=
=## Multiple scenarios failureReport scenarios without steps more explicitly
Now there is a text explaining that a scenario have no steps.
index e258f55..411d5d7 100644
--- a/spec/basic-usage.md
+++ b/spec/basic-usage.md
@@ -179,6 +179,8 @@ Notice it's similar to the output of `tbb list`, but now contains unicode symbol
= ```text
= ? Geometry
= tagged: math
+
+ There are no steps to execute in this scenario.
= ```
=
= * The standard error will contain `\[.+ ERROR +tbb\] Step failed: Basic BDD suite ❯ Text ❯ The reverse of CIA is KGB`index f54c4ee..546bbf9 100644
--- a/src/report.rs
+++ b/src/report.rs
@@ -332,6 +332,16 @@ impl Display for EvaluationReport<'_> {
= writeln!(f, "\n")?;
= }
=
+ if scenario.steps.is_empty() {
+ writeln!(
+ f,
+ "{}",
+ "There are no steps to execute in this scenario."
+ .indent(2)
+ .dimmed()
+ )?
+ }
+
= for StepReport { step, status } in steps.iter() {
= let sigil = match status {
= StepStatus::Ok => "⊞".to_string().bold().green(),Bump version to 0.8.0
To celebrate error status on failing scenarios and explicit warning about scenarios without any steps.
index 7fcd666..09ff789 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -400,7 +400,7 @@ dependencies = [
=
=[[package]]
=name = "tad-better-behavior"
-version = "0.7.0"
+version = "0.8.0"
=dependencies = [
= "anyhow",
= "clap",index afce511..768e846 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,6 @@
=[package]
=name = "tad-better-behavior"
-version = "0.7.0"
+version = "0.8.0"
=edition = "2024"
=
=[dependencies]