Commits: 4
Fix a regression in display, update the spec
index d579577..0a1cd93 100644
--- a/spec/basic-usage.md
+++ b/spec/basic-usage.md
@@ -72,7 +72,7 @@ We would also need a step like `Change working directory to ...` to execute befo
=
=## Showing a spec from a different directory
=
-Whe a directory is given as the last argument, load all documents inside (recursively).
+When a directory is given as the last argument, load all documents inside (recursively).
=
=
=## Showing suites and scenarios from a single document
@@ -110,7 +110,7 @@ Whe a directory is given as the last argument, load all documents inside (recurs
= ``` text
= * Text
= tagged: strings work-in-progress
- source: samples/basic.md:35-91
+ source: samples/basic.md:35-100
=
= 00. The word blocks has 6 characters
= arguments: ["blocks", "6"]
@@ -121,18 +121,18 @@ Whe a directory is given as the last argument, load all documents inside (recurs
= 02. There are 3 rs in the word strawberry
= arguments: ["3", "r", "strawberry"]
= source: samples/basic.md:58-58
- 03. The following table maps words to their lengths
- code blocks: 1, tables: 1
- source: samples/basic.md:59-84
+ 03. The following tables map words to their lengths
+ code blocks: 1, tables: 2
+ source: samples/basic.md:59-93
= 04. The reverse of abc is cba
= arguments: ["abc", "cba"]
- source: samples/basic.md:85-85
+ source: samples/basic.md:94-94
= 05. The reverse of CIA is KGB
= arguments: ["CIA", "KGB"]
- source: samples/basic.md:86-89
+ source: samples/basic.md:95-98
= 06. There are 2 os in the word boost
= arguments: ["2", "o", "boost"]
- source: samples/basic.md:90-91
+ source: samples/basic.md:99-100
= ```
=
= * The exit code should be `0`
@@ -163,7 +163,7 @@ Whe a directory is given as the last argument, load all documents inside (recurs
= ⬑ all 3 steps successful.
= ```
=
- Notice it's similar to the output of `tbb show`, but now contains unicode symbols to indicate the results of each step. Each completed scenario has a check mark `✓`. The number of steps is indicated by the `⊞` sigils and a message on the following line. This is called "condensed output".
+ Notice it's similar to the output of `tbb show`, but now contains Unicode symbols to indicate the results of each step. Each completed scenario has a check mark `✓`. The number of steps is indicated by the `⊞` sigils and a message on the following line. This is called "condensed output".
=
= TODO: Create a separate "condensed output" suite that focuses on this aspect.
=
@@ -172,53 +172,60 @@ Whe a directory is given as the last argument, load all documents inside (recurs
= ```text
= x Text
= tagged: strings work-in-progress
- source: samples/basic.md:35-91
+ source: samples/basic.md:35-100
+ ```
=
+ * The output will contain `an expanded successful step` block
=
+ ``` text
= ⊞ The word blocks has 6 characters
= arguments: ["blocks", "6"]
= source: samples/basic.md:45-45
=
- ⊞ There are 3 properties in the following JSON
- arguments: ["3"], code blocks: 1
- source: samples/basic.md:46-57
-
- ⊞ There are 3 rs in the word strawberry
- arguments: ["3", "r", "strawberry"]
- source: samples/basic.md:58-58
-
- ⊞ The following table maps words to their lengths
- code blocks: 1, tables: 1
- source: samples/basic.md:59-84
+ > Is 'blocks' of length 6?
+ ```
=
- ⊞ The reverse of abc is cba
- arguments: ["abc", "cba"]
- source: samples/basic.md:85-85
+
+ If some steps fail, the output will be expanded. Successful steps have a squared plus `⊞` sigil. If there were any observations reported by the interpreter, they will be displayed.
+
+
+ * The output will contain `the failing step` block
=
+ ``` text
= ⊠ The reverse of CIA is KGB
= arguments: ["CIA", "KGB"]
- source: samples/basic.md:86-89
+ source: samples/basic.md:95-98
+
+ > Is 'KGB' the reverse of 'CIA'? I think the reverse is 'AIC'.
+
=
= 'KGB' != 'AIC'
=
= - KGB
= + AIC
+ ```
=
+ The failing step is marked with a squared times symbol `⊠`. There is an observation (the block quote) and the error message from the interpreter.
+
+ * The output will contain `the skipped step` block
+
+ ``` text
= □ There are 2 os in the word boost
= arguments: ["2", "o", "boost"]
- source: samples/basic.md:90-91
+ source: samples/basic.md:99-100
= ```
-
- If some steps fail, the output will be expanded. Successful steps have a squared plus `⊞` sigil . The failing step is markd with a squared times symbol `⊠`. Once a step fails, the subsequent steps in a scenario are not exercised. In the report they are marked with a white square symbol `□`.
=
- To aid in debugging, each step is now expanded, showing it's source position and arguments.
+
+ Once a step fails, the subsequent steps in a scenario are not exercised. In the report they are marked with a white square symbol `□`.
+
+ To aid in debugging, each step is now expanded, showing its source position and arguments.
=
= * The output will contain `geometry scenario` block
=
= ```text
= ? Geometry
= tagged: math
- source: samples/basic.md:92-99
+ source: samples/basic.md:101-108
=
= There are no steps to execute in this scenario.
= ```
@@ -234,4 +241,4 @@ Whe a directory is given as the last argument, load all documents inside (recurs
=
=## Running without a subcommand
=
-Running `tbb` without a subcommand will print the help message (to stderr) and exit with code 2.
+Running `tbb` without a subcommand will print the help message (to `stderr`) and exit with code 2.index 3ae8227..6acb94c 100644
--- a/src/report.rs
+++ b/src/report.rs
@@ -387,7 +387,7 @@ impl Display for EvaluationReport<'_> {
= if scenario.steps.is_empty() {
= writeln!(
= f,
- "{}",
+ "\n{}",
= "There are no steps to execute in this scenario."
= .indent(4)
= .yellow()Implement the Link observation
index 75f73ef..96dc34a 100644
--- a/samples/basic.py
+++ b/samples/basic.py
@@ -44,6 +44,9 @@ def step_implementation_04(a: float, b: float, expected: float, **kwargs):
=@step("There are {0} {1}s in the word {2}")
=def step_implementation_05(count: int, letter: str, word: str, **kwargs):
= actual_count = word.count(letter)
+
+ tbb.send_link(f"https://www.dictionary.com/browse/{word}", f"Check the spelling of `{word}`")
+
= tester.assertEqual(count, actual_count)
=
=@step("The reverse of {0} is {1}")index 74e1b01..cf3096f 100644
--- a/spec/tbb.py
+++ b/spec/tbb.py
@@ -204,6 +204,15 @@ def send_snippet(language, content, meta = ""):
= })
=
=
+def send_link(url, label=None):
+ """ Send a link with an optional label to be displayed in a TBB report
+ """
+ send({
+ "type": "Link",
+ "url": url,
+ "label": label
+ })
+
=# TODO: Docstring and test cases for `get_at`. See `self-check` for example use.
=def get_at(collection, path: [str]):
= value = collectionindex 6acb94c..57da103 100644
--- a/src/report.rs
+++ b/src/report.rs
@@ -184,6 +184,9 @@ impl<'a> ScenarioReport<'a> {
= content,
= });
= }
+ InterpreterMessage::Link { url, label } => {
+ observations.push(Observation::Link { url, label });
+ }
= InterpreterMessage::Success => {
= log::debug!("Step executed successfully: {step:#?}");
= *status = StepStatus::Ok;
@@ -294,7 +297,10 @@ pub enum Observation {
= // Audio,
= // Video,
= // Attachment,
- // Link,
+ Link {
+ url: String,
+ label: Option<String>,
+ },
=}
=
=impl Display for Observation {
@@ -312,6 +318,13 @@ impl Display for Observation {
= Observation::Text { content } => {
= writeln!(f, "{}", content.prefix_with("> "))
= }
+ Observation::Link { url, label } => {
+ if let Some(label) = label {
+ writeln!(f, "[{label}]({url})")
+ } else {
+ writeln!(f, "<{url}>")
+ }
+ }
= }
= }
=}
@@ -496,6 +509,10 @@ pub enum InterpreterMessage {
= meta: String,
= content: String,
= },
+ Link {
+ url: String,
+ label: Option<String>,
+ },
= Success,
= Failure {
= reason: String,Update the roadmap
index d4be474..185958d 100644
--- a/roadmap.md
+++ b/roadmap.md
@@ -14,10 +14,13 @@ We use this document mostly to keep track of our progress and a scratchpad for i
= - [ ] Gauge
= - [ ] Cucumber
= - [ ] Playwright
+- [ ] An icon (slingshot?)
+- [ ] Release branch and tagging
+- [ ] HTML reports
=- [ ] Demo
-- [ ] Show and report output formats
- - [ ] JSON
- - [ ] Markdown
+- [x] Show and evaluate output formats
+ - [x] JSON
+ - [x] Markdown
=- [x] Proof of concept
= - [x] Interpretter in a different language (Python)
= - [x] Report why steps fail
@@ -52,7 +55,7 @@ We use this document mostly to keep track of our progress and a scratchpad for i
= - [x] Use colors
= - [ ] More instructive error messages
= - [x] Indent multiline error messages
- - [ ] Collapse ommited steps (`□□□□□□ following 6 steps skipped`)
+ - [x] Collapse ommited steps (`□□□□□□ following 6 steps skipped`)
= - [x] Collapse filtered out suites
= - [x] Collapse filtered out scenarios
= - [x] Mark scenarios without steps
@@ -88,5 +91,4 @@ We use this document mostly to keep track of our progress and a scratchpad for i
=- [ ] Better reporters
= - [ ] TUI
= - [ ] Web
-- [ ] WASM target
=Write assertions about snippet observations
My Emacs doesn't handle nested code blocks well, so I implemented a little hack, where it's possible to escape a backtick with a backslash (thus making it meaningless to Markdown). The step implementation un-escapes them before comparison.
index 0a1cd93..86d94af 100644
--- a/spec/basic-usage.md
+++ b/spec/basic-usage.md
@@ -175,7 +175,7 @@ When a directory is given as the last argument, load all documents inside (recur
= source: samples/basic.md:35-100
= ```
=
- * The output will contain `an expanded successful step` block
+ * The output will contain `an expanded successful step with a text observation` block
=
= ``` text
= ⊞ The word blocks has 6 characters
@@ -186,9 +186,110 @@ When a directory is given as the last argument, load all documents inside (recur
= ```
=
=
- If some steps fail, the output will be expanded. Successful steps have a squared plus `⊞` sigil. If there were any observations reported by the interpreter, they will be displayed.
+ If some steps fail, the output will be expanded. Successful steps have a squared plus `⊞` sigil. If there were any observations reported by the interpreter, they will be displayed. Here is an example of a text observation, in a block quote.
=
=
+ * The output will contain `an expanded successful step with a link observation` block
+
+ ``` text
+ ⊞ There are 3 rs in the word strawberry
+ arguments: ["3", "r", "strawberry"]
+ source: samples/basic.md:58-58
+
+ [Check the spelling of `strawberry`](https://www.dictionary.com/browse/strawberry)
+ ```
+
+ This step also succeeded, producing a Link observation. These observations can be useful, because sometimes a root cause of a failure is in a proceeding steps, not the one that actually failed.
+
+ * The output will contain `an expanded successful step with a various observations` block
+
+ ``` text
+ ⊞ The following tables map words to their lengths
+ code blocks: 1, tables: 2
+ source: samples/basic.md:59-93
+
+ > Received a table with 6 x 2 cells
+
+ \``` json
+ [
+ [
+ "word",
+ "length"
+ ],
+ [
+ "cat",
+ "3"
+ ],
+ [
+ "stork",
+ "5"
+ ],
+ [
+ "rabbit",
+ "6"
+ ],
+ [
+ "snake",
+ "5"
+ ],
+ [
+ "minx",
+ "4"
+ ]
+ ]
+ \```
+
+ > Is 'cat' 3 characters long? I think it's 3.
+
+
+ > Is 'stork' 5 characters long? I think it's 5.
+
+
+ > Is 'rabbit' 6 characters long? I think it's 6.
+
+
+ > Is 'snake' 5 characters long? I think it's 5.
+
+
+ > Is 'minx' 4 characters long? I think it's 4.
+
+
+ > Received a table with 4 x 2 cells
+
+
+ \``` json
+ [
+ [
+ "word",
+ "length"
+ ],
+ [
+ "rust",
+ "4"
+ ],
+ [
+ "clojure",
+ "7"
+ ],
+ [
+ "elm",
+ "3"
+ ]
+ ]
+ \```
+
+
+ > Is 'rust' 4 characters long? I think it's 4.
+
+
+ > Is 'clojure' 7 characters long? I think it's 7.
+
+
+ > Is 'elm' 3 characters long? I think it's 3.
+ ```
+
+ This long block presents other observations, like snippets.
+
= * The output will contain `the failing step` block
=
= ``` textindex aebf88c..94e628f 100644
--- a/spec/self-check.py
+++ b/spec/self-check.py
@@ -88,9 +88,10 @@ def step_implementation_04(label: str, **kwargs):
= block = get_at(kwargs, ['code_blocks', 0, 'value'])
= output = completed.stdout.decode("utf-8")
=
- # Trim all blank lines and trailing whitespece.
- # Without it the assertions are very brittle.
- needle = re.sub("\\s+\n", "\n", block)
+ # Trim all blank lines and trailing whitespece. Without it the assertions
+ # are very brittle. Also un-escape back-ticks in the block. This is useful
+ # for nested code-blocks in the spec documents.
+ needle = re.sub("\\s+\n", "\n", block).replace("\`", "`")
= haystack = re.sub("\\s+\n", "\n", output)
= # tester.assertIn gives unreadable output
=