@cmccandless I was just wondering, if you have time, would you like to refactor the processor script in this repo with the changes you made here: exercism/c-test-runner#8? If not, would it be okay if I made the changes?

In line with our new org-wide policy, the master branch of this repo will be renamed to main. All open PRs will be automatically repointed.

GitHub will show you a notification about this when you look at this repo after renaming:

In case it doesn't, this is the command it suggests:

git branch -m master main
git fetch origin
git branch -u origin/main main

You may like to update the primary branch on your forks too, which you can do under Settings->Branches and clicking the pencil icon on the right-hand-side under Default Branch:

We will post a comment below when this is done. We expect it to happen within the next 12 hours.

The results.json files produced by the test runner don't include the version key, which means that the website will interpret it as being version 1. Looking at the produced results.json files, it looks like this version of the test runner is somewhere between version 1 and version 2, in that it does nicely list the individual tests and their status, but doesn't include the test_code. That last bit is especially useful for Concept Exercises when being run in the editor, as the test file is not displayed.

The test runner should be modified to output the version key (see the docs). Theoretically, the version should be 1 due to missing the test_code field, but I would nonetheless recommend outputting version 2 as that will allow the individual test results to be displayed.

You can find more information here: https://github.com/exercism/docs/blob/main/building/tooling/test-runners/interface.md

See exercism/v3-beta#124 for the relevant bug report and what the current output looks like without specifying the version key.

We've recently updated the test runner interface specification to allow for two types of test runners, identified via a new version property in the results.json file (see the spec for the version property):

1. version: 1 test runners are quite basic, and only detect if either all tests passed or not. These test runners capture the console output of the test runner and output that. They do not include details on individual tests passing/failing
2. version: 2 test runners contain detailed information on individual tests passing/failing. We've recently added a required field named test_code, which contains the code the test ran to verify the behavior (see the spec for the test_code property).

The test runner's output is currently what I refer to as version 1.5: it contains the individual test information, but does not yet contain the test_code key. If possible, the test runner should be updated to also include the test code, in which case the version property should also be added with 2 as its value.

You are completely free on how you want to extract the test code, either via the AST or via finding the test code in the source code text via string manipulation.

Let me know if there are any questions.