dakusui / commandunit Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 0.0 472 KB

A testing framework for command line tools.

Home Page: https://dakusui.github.io/commandunit/

License: MIT License

Dockerfile 1.29% Shell 84.78% Makefile 3.47% Ruby 0.75% Logos 9.70%

bash test testtools

commandunit's People

Contributors

Stargazers

commandunit's Issues

Make it possible to reference environment variables from `environmentVariables` section.

Make it possible to reference environment variables from environmentVariables section.

That is, what we want to do is the following, but the SCRIPTS_DIR environment variable cannot be defined based on other environment variables.

---
"$extends":
  - base/normal.json
when:
  environmentVariables:
    - SCRIPTS_DIR: "${COMMANDUNIT_HOSTFSROOT_MOUNTPOINT}/${COMMANDUNIT_PWD}/src/main/scripts"
  source:
    - ${COMMANDUNIT_DEPENDENCIES_ROOT}/bud/lib/core.rc
    - ${COMMANDUNIT_HOSTFSROOT_MOUNTPOINT}/${COMMANDUNIT_PWD}/src/main/scripts/target_lib.rc
  cmd: cat
  args:
    - ${SCRIPTS_DIR}/hello.txt
then:
  exitCode:
    - EQUAL
    - 0
  stdout:
    present:
      - REGEX:Hello world!
  stderr:
    absent:
      - REGEX:.+
      - ```

Improve documentation

Improve documentation.
Perhaps, it's a good idea to make the commandunit-example project a part of this project rather than keeping it separated.

https://github.com/dakusui/commandunit-example

Change default values of --xyzdir options

Change default values of --xyzdir options because the behavior is not intuitive.

That is, I always use the following setting and it would be easier to understand the behavior

commandunit --test-srcdir=./src/test/scripts/ --test-workdir=./commandunit-out/work  --test-reportdir=./commandunit-out/report

commandunit requires COMMANDUNIT_PWD to be set

commandunit fails with the following output.

$ commandunit --native
/home/hiroshi/Documents/github/commandunit-example/src/dependencies/commandunit/src/main/scripts/bin/commandunit: line 77: COMMANDUNIT_PWD: unbound variable
ERROR:(exit code:1, pipe status: 1): 
  at src/dependencies/commandunit/src/main/scripts/dependencies/bud/lib/core.rc:54 (abort)
  at src/dependencies/commandunit/src/main/scripts/bin/commandunit:77 (main)
  at src/dependencies/commandunit/src/main/scripts/bin/commandunit:193 (main)

Command unit should set an appropriate for the environment variable no to bother uses to set it.

Installation without docker is not explicitly explained

Although installation with docker is explained in the document, another way without docker is not explained.

Consider having capability to install another command that is not natively installed

While I was adding tests using commandunit, one of them requires curl command to be installed.
However, the docker image used for commandunit (You can see it here, which is originated in the image in jq-front).

I also got below error when I was trying to install curl command in the docker image that is launched by commandunit --debug-shell command.

I have no name!@8e93788c6fc1:/$ apt-get install curl E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied) E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?

If I add sudo to the beginning, I got this.

I have no name!@8e93788c6fc1:/$ sudo apt-get install curl bash: sudo: command not found

I'm not sure whether curl should be natively installed, but at least I'd like to have a capability to install another command required for the test.

Remove unnecessary link in the test report

Currently, the generated report (testreport.adoc) contains a link (inclusion) shown below:

  [%collapsible]  
 ====
 [source, json]
 ----
 include::./core/test-cat.json[]
 ----

But this is not necessary and doesn't render with the following error:

This element should go away.

Unresolved directive in testreport.adoc - include::./core/test-cat.json[]

How to $extend a custom file?

---
"$extends":
  - base.json

This definition doesn't do $extends but silently ignores the inheritance.

Introduce "clean" subcommand

Introduce "clean" subcommand, which removes the report and work directories.

Implement clean subcommand

Implement clean subcommand.
It should cleans standard workdir (target) at least.

Improve error message(s)

Error message for bailout is not informative

Error message for bailout is not informative.
Since commandunit is designed to bailout when a malformed test script is given, this is very painful.

Where (how) can I find an execution log of command unit?

Where (how) can I find an execution log of command unit? Not the test related ones but framework side's.

commandunit fails on macOS in non-native mode

commandunit fails with the following output:

INFO: Local repository path that has SSH Git URL '/Users/scott.tiger/Desktop/somewhere'
mkdir: cannot create directory '/tmp/bud': No space left on device
ERROR:(exit code:1, pipe status: 1): 
  at /app/dependencies/bud/lib/core.rc:54 (abort)

Some other users can successfully use commandunit (non-native) on macOS.
The only known difference is

If TMPDIR is not set -> This failure is observed.
If TMPDIR is set (Something like /var/folders/c9/ndzj29354pn6h9hcjqz6gy15j6nqy4/T/) -> This failure is not observed.

We don't have knowledge about what component is (should be) responsible for defining TMPDIR environment variable in macOS.
(Help needed)

Rename commandunit script to commandunit-main

Rename commandunit script to commandunit-main since a user facing script (wrapper script) should be named commandunit and it will be confusing if two files have the same name.

Implement none subcommand.

Implement none subcommand, which doesn't do anything.

Sourcing a file inside the sourced file is not working

I have a script file(scriptA.sh) which sources scriptB.sh;
scriptA.sh

#!/usr/bin/env bash
  source scriptB.sh

when I create commandunit script such as below and source scriptA.sh, commandunit cannot find scriptB.sh file and giving ".../scriptA.sh: line 6: /dev/scriptB.sh: No such file or directory" error message.
test.yaml

"$extends":
  - some.json
when:
  source: 
   -scriptA.sh
  cmd:
then:
  exitCode:
    - EQUAL
    - 0

Update commandunit function

Update commandunit function.
In the commandunit-example project, there is an improved version of commandunit function and it should be imported to the main project side.

Directory for test target cannot be created as there is no permission to do it

How to reproduce

Install commandunit in the former procedure written in https://github.com/dakusui/commandunit#installation.
Try to run tests by commandunit run command in the directory where commandunit tests are defined.
The following error was logged and the execution failed

$ commandunit run mkdir: cannot create directory '/var/lib/commandunit/Users/<user_name>/Desktop/dev/gspci-shared-resources/gspci/src/test/target': Permission denied ERROR:(exit code:1, pipe status: 1): at /app/dependencies/bud/lib/core.rc:54 (abort) at /app/bin/commandunit:138 (main) at /app/bin/commandunit:193 (main)

Improve help message

commandunit print help message as follows:

Usage: commandunit [OPTION]... [SUBCOMMAND]...

Runs tests.

Sub-commands:
  preprocess:
    Preprocesses test definition files (yaml++, yaml, and json++) and convert them into executable JSON test files
  run:
    Runs tests under a directory specified by --test-workdir and writes a report file: testreport.json under a 
    directory specified by --test-reportdir.
  report:
    Reads a file testreport.json under directory specified by --test-reportdir and renders a report file (testreport.adoc)
    under the same directory.

 -h, --help            show this help
 -p, --parallel        execute the tests in parallel
 -f, --filter          filter tests with the specified regular expression(default:'.*')
     --commandunit-dir directory to store config and data files (default: current {directory/.commandunit})
     --project-name    set project name of the test execution. used as the report's title(default:'unknown')
     --test-srcdir     write test reports under the specified directory (default: current directory)
     --test-workdir    write test reports under the specified directory (default: value for --test-srcdir + .commandunit/work)
     --test-reportdir  write test reports under the specified directory (default: value for --test-workdir)
     --ignore-mtime    ignore mtime and forcibly compile tests
     --clean           clean working directory
     --tapview         show test progress with 'tapview'

Examples:
  commandunit                 Run tests found under current directory in sequential mode.
  commandunit --srcdir=DIR    Run tests found under DIR in sequential mode.
  commandunit --srcdir=DIR -p Run tests found under DIR in parallel mode.

- documentation: <https://dakusui.github.io/commandunit/>
- github project: <https://github.com/dakusui/bud>

This will be better.

Usage: commandunit [OPTION]... [SUBCOMMAND]...

Runs tests.

Sub-commands:
  preprocess:
    Preprocesses test definition files (yaml++, yaml, and json++) and convert them into executable JSON test files
  run:
    Runs tests under a directory specified by --test-workdir and writes a report file: testreport.json under a 
    directory specified by --test-reportdir.
  report:
    Reads a file testreport.json under directory specified by --test-reportdir and renders a report file (testreport.adoc)
    under the same directory.

 -h, --help            show this help
 -p, --parallel        execute the tests in parallel
 -f, --filter          filter tests with the specified regular expression(default:'.*')
     --commandunit-dir directory to store config and data files (default: current {directory/.commandunit})
     --project-name    set project name of the test execution. used as the report's title(default:'unknown')
     --test-srcdir     execute tests under the specified directory (default: current directory)
     --test-workdir    set a working directory to the specified directory (default: value for --test-srcdir + .commandunit/work)
     --test-reportdir  write test reports under the specified directory (default: value for --test-workdir)
     --ignore-mtime    ignore mtime and forcibly compile tests
     --clean           clean working directory
     --tapview         show test progress with 'tapview'

Examples:
  commandunit                 Run tests found under current directory in sequential mode.
  commandunit --srcdir=DIR    Run tests found under DIR in sequential mode.
  commandunit --srcdir=DIR -p Run tests found under DIR in parallel mode.

- documentation: <https://dakusui.github.io/commandunit/>
- github project: <https://github.com/dakusui/bud>

dakusui / commandunit Goto Github PK

commandunit's People

Contributors

Stargazers

commandunit's Issues

Recommend Projects

Recommend Topics

Recommend Org