kevgo / text-runner Goto Github PK

https://github.com/sindresorhus/cli-cursor

tut-run docs/*.md

As a tutorial writer
I want the tests for my tutorial to run in a separate directory from my tutorial
So that I don't clutter up my tutorial source code with temporary files creating by running it.

https://github.com/wooorm/remark/tree/master/packages/remark-parse

http://www.appveyor.com

and see if that feels better than anchor tags

As a non-technical user
I want to be able to download a binary file for Tutorial Runner
So that I can simply use it without having to install and configure Node.JS first.

• each release contains binary files for Windows, Mac, and Linux

https://github.com/sindresorhus/meow

use YARN on CI

If they don't take a callback parameter, don't wait for them.

Use Tertestrial to check the Tertestrial readme

tut-run should empty the tmp directory before running to make sure it only tests things in the actual markup

As a Node.JS library writer
I want to be able to use Tutorial Runner programmatically via its JS API
So that I can fully embed it into my own test suites and control it easily.

• extract helpers into a CliWorld
• add an ApiWorld

With embedded screenshots against which it diffs

It doesn't seem to be there

As a developer writing a tutorial for a software tool or library I have built
I want to be able to implement actions in the language my tool is written in
So that my actions can call my library programmatically similar to how a developer would
And I can code custom actions in the programming language I am familiar with.

Notes:

• extract the part that runs actions from the main runtime into its own process called "action runner"
• the main runtime and the action runner coordinate via an RPC mechanism like (web)socket, Thrift, or whatever
• the user is responsible for creating the action runner
• we provide helper libraries for common languages that make creating an action runner as simple as defining the actions is today, by spinning up the action runner process and IPC listener, finding the actions the user has defined, and calling them on demand

Communication Protocol:

• the main runtime parses the documentation and sends a "runAction" command via IPC to the action runtime
• the action runtime replies with formatter messages and test outcomes, which are rendered by the main runtime to the user
• the wire protocol over the IPC channel is JSON

Example:
wire format communication during testing of a library that exposes a "hello" method that users can call:

1. main runtime sends:

{
  "command": "runAction",
  "name": "callHello",
  "argument": "world"
}

1. action runner sends:

{
  "event": "action.start",
  "activity": "calling 'hello' with 'world'"
}

1. action runner calls "hello" with parameter "world"

2. action runner sends:

{
  "event": "action.success"
}

Or figure out how to test against multiple Node versions

• makes sure URLs point to something useful

for example https://github.com/wooorm/remark-lint

When building a text suite
I want that missing actions are scaffolded for me on demand
So that I don't waste time copy-and-pasting boilerplate code.

• when TextRunner finds an unknown action, it asks to create one
• when the user agrees, it creates the files for the action

They are too complicated right now

• extract superclass
• provide default implementations for tags
• provide a standardized AST to the step, to let steps pick out the data themselves in one swoop?

Right now the readme mentions the globals key for all commands, but it really only applies to runConsole actions.

In order to use custom steps:

• add a directory "tut-run"
• create files with the name of the activity, e.g. "tut-run/write-file.ls"
• the file exports the runner
• so, to make this work, the CLI just has to load steps in the built-in directory as well as in "#{process.cwd!}/tut-run"

right now it only tests files right in that directory

https://www.npmjs.com/package/update-notifier

• the example script uses that

TutorialRunner should verify that all links outside of actionable blocks point to useful URLs.
Only print warnings if they don't since external sites could be down.

With embedded screenshots against which it diffs

Optional separate implementation of the API and CLI version of scenarios.

• some of the current feature specs are not particularly expressive. Reading them doesn't give one a good idea how to use the product via the CLI or API, because defining Gherkin in a way that works for both is too abstract
• CLI and API are two different ways of consuming the product. Both should be specified in feature specs.

Example:

Feature: running a single MarkDown file

Scenario:
  Given I am in a directory that contains the "multiple-files" example
  When running Tutorial Runner with only the file 1.md
  Then it runs only the tests in 1.md

This scenario doesn't tell the reader much, because all the interesting parts are buried in the step definitions. It's nice to know that I can run it with only one file, but how exactly am I doing this via the CLI or API?

Feature: running a single MarkDown file

  Background:
    Given I am in a directory that contains the "multiple-files" example

  @clionly
  Scenario Outline: testing a single file via the CLI
    When running "tut-run 1.md"
    Then it runs only the tests in "1.md"

  @clionly
  Scenario: testing a non-existing file via the CLI
    When trying to run "tut-run zonk.md"
    Then the test fails with:
      | ERROR MESSAGE | unknown command: zonk.md |
      | EXIT CODE     | 1                        |


  @apionly
  Scenario Outline: testing a single file via the API
    When executing "tutorialRunner.execute 'run', '1.md', <callback>"
    Then it runs only the tests in "1.md"

  Scenario: testing a non-existing file via the API
    When executing "tutorialRunner.execute 'run', 'zonk.md', <callback>"
    Then it returns the error "unknown command: zonk.md"

https://github.com/SamVerschueren/listr

• displays the currently running test in yellow, with output below
• if the test passes, redraws the currently running test in green
• if the test fails, redraws the currently running test in red, with output of it below
• optionally show yellow arrow next to the currently running task, similar to https://github.com/SamVerschueren/listr
• use https://github.com/sindresorhus/log-update for the terminal output
• use https://github.com/sindresorhus/figures for symbols