mniak / bench Goto Github PK

It's more meaningful

Related to #23

A complex cache invalidation logic must be implemented, because toolchains can change in the same machine
with the installation and uninstallation of tools and even the change of environment variables

Topics to address

• how to install - go install / gobinaries
• purpose - competitive programming
• name of the tool. comes from a workbench
• features

When trying to run test examples from a folder, the program is not built and then it complains that the program was not found.

Example

Files

my-program/
├─main.cpp
└─examples/
    test01.input.txt
    test01.output.txt

main.cpp

#include <iostream>
#include <string>

using namespace std;
void main()
{
    string name;
    getline(cin, name);
    cout << "Hello, " << name << "!" << endl;
}

examples/test01.input.txt

John Doe

examples/test01.output.txt

Hello, John Doe!

Command

bench test examples my-program/

Output

Test test01 running...
program not found

• Try to find binary
  • found:
    • Try to find source file
      • found: return source file path
      • not found: return binary path
  • not found:
    • Try to find source file
      • found: return source file path
      • not found: return "program not found" error

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update all non-major dependencies (github.com/brianvoe/gofakeit/v6, github.com/oxyno-zeta/gomock-extra-matcher, github.com/spf13/cobra, github.com/stretchr/testify)
• Update actions/checkout action to v4
• Update actions/setup-go action to v5
• Update module github.com/brianvoe/gofakeit/v6 to v7
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

It should detect the program trying the following file names

• main (PR #14)
• the same name of the folder (PR #14)

with the following file extensions

• .exe on windows (PR #14)
• (none) on non-windows (PR #14)
• .py
• .sh on non-windows
• .bat on windows
• .cmd on windows
• .ps1 on windows

Create an option that would allow enabling the verbose mode.
If not specified, should not show in console

The application is taking much time to run the first line of the main method.

I did some research and found that the MSCV CPP toolchain is taking around 2s to initialize due to running the script vcvars.bat.

• Require arguments to the program, if any, to be provided after the double dash (--).

bench test app.cpp -- [<args>]
bench test -- app.cpp [<args>]

bench test --arg1 app.cpp -- [<args>]
bench test --arg1 -- app.cpp [<args>]

bench test app.cpp --arg01 #should raise an error like "argument --arg01 is not valid"
bench test app.cpp arg01   #should raise an error like "argument arg01 is not valid"

Related to #23

The toolchains are needed in order to compute which are the input and output extensions, influencing in the detection of source files when running the commands.

I could find the toolchain in one step and initializing it in another.
Using the MSVC example, i could find the file vcvars.bat and then use it to consider the MVSC toolchain as usable and thus exposing it on the DefaultToolchains slice.
Then, when the toolchain is chosen, a factory method would be called in order to initialize the toolchain.
For the MSVC toolchain, it would represent calling the vcvars.bat and collecting the resulting environment variables for usage in the build step.

Examples of use

Test

bench test .
bench test app.cpp
bench test app.exe
bench test app     #for linux executables

bench test dir01/
bench test dir01/app.cpp
bench test dir01/app.exe
bench test dir01/app

• Remove --dir option (PR #9)

• Allow both executable/interpretable and compilable files

  • When a compilable source file is provided, it must always also compile the program even if the respective compiled version already exists.
    • Add a flag to bypass this like --no-build

bench test examples dir01/app.cpp
bench test examples --no-build dir01/app.cpp
bench test examples dir01/app.exe

• For now, dont accept any arguments for the tested program (PR #9)
  • Accept later (#8)

Test Examples

bench test examples .
bench test examples /absolute/path
bench test examples dir01/
bench test examples dir01/app.cpp
bench test examples dir01/app.exe
bench test examples dir01/app

• Allow specification of examples directory

bench test examples --examples test_cases/ dir01/app.cpp

Desired features:

• Configurable timeout --timeout=5 in seconds
• Default timeout (something like 10 seconds)

Idea 1

Maybe I should rename the project to desk

Pros

• Bench brings to mind some idea of benchmarking
• A workbench is were you work. A desk is where you study and solves exercises
• Desk seems like a more unique name

Cons

• Desk could seem to reference desktop in the GUI sense

Everytime I run my tests, some files main.exe and main.obj are created in the folder containing the source.
This bothers me, because I need to add a .gitignore to avoid these files.

When running test in a file

• If executable file, test directly
  • py
  • sh
  • exe
  • any other extension except the known source-file extensions (the source-file extension are those for which a toolchain can be found)
  • without extension with flag x
• If not executable
  • if nobuild, raise error
  • if not nobuild, build file

Do not keep build logic inside Test/Test Example.

It's ok to keep it on the command but not on the library.
The reason is that if you run Test Examples it will build many times
Also, there is no way for the user to control if he wants to skip building and just test, for example with an older version of the binary.

Either

• move the library package to a subfolder and hurt the import path, or
• move the executable to a folder cmd/bench and hurt the install path