Each test case in the results.json file has the option of having an output field. This field is used when students are using the in-browser editor to help them with "debugging" their code (as they won't have an actual debugger available in the browser).
See the spec for more information.

You may either capture content that is output through normal means (e.g. puts in Ruby, print in Python or Debug.WriteLine in C#), or you may provide a method that the user may use (e.g. the Ruby Test Runner provides a user with a globally available debug method that they can use, which has the same characteristics as the standard puts method.

There is existing code to write the field (albeit with a default value) in https://github.com/exercism/scala-test-runner/blob/main/src/main/scala/Application.scala#L62

The Scala test runner is functional right now, but it is quite slow. In fact, it is so slow that the test runner times out before being able to return the results. We should speedup the test runner to not have it timeout.

Some possibly useful references:

• https://www.scala-lang.org/old/sites/default/files/linuxsoft_archives/docu/files/tools/fsc.html
• https://scalacenter.github.io/bloop/setup#sbt

In line with our new org-wide policy, the master branch of this repo will be renamed to main. All open PRs will be automatically repointed.

GitHub will show you a notification about this when you look at this repo after renaming:

In case it doesn't, this is the command it suggests:

git branch -m master main
git fetch origin
git branch -u origin/main main

You may like to update the primary branch on your forks too, which you can do under Settings->Branches and clicking the pencil icon on the right-hand-side under Default Branch:

We will post a comment below when this is done. We expect it to happen within the next 12 hours.

We've recently updated the test runner interface specification to allow for two types of test runners, identified via a new version property in the results.json file (see the spec for the version property):

1. version: 1 test runners are quite basic, and only detect if either all tests passed or not. These test runners capture the console output of the test runner and output that. They do not include details on individual tests passing/failing
2. version: 2 test runners contain detailed information on individual tests passing/failing. We've recently added a required field named test_code, which contains the code the test ran to verify the behavior (see the spec for the test_code property).

The test runner's output is currently what I refer to as version 1.5: it contains the individual test information, but does not yet contain the test_code key. If possible, the test runner should be updated to also include the test code, in which case the version property should also be added with 2 as its value.

You are completely free on how you want to extract the test code, either via the AST or via finding the test code in the source code text via string manipulation.

Let me know if there are any questions.

An error occurred while running your tests. This might mean that there was an issue in our infrastructure, or it might mean that you have something in your code that's causing our systems to break.

Please check your code, and if nothing seems to be wrong, try running the tests again.

Example solution:

object Twofer {
  def twofer(name: String = "you"): String = s"One for $name, one for me."
}

All tests pass with sbt test

Other exercises seem to be OK.

We've recently updated the test runner interface specification to allow for two types of test runners, identified via a new version property in the results.json file (see the spec for the version property):

1. version: 1 test runners are quite basic, and only detect if either all tests passed or not. These test runners capture the console output of the test runner and output that. They do not include details on individual tests passing/failing
2. version: 2 test runners contain detailed information on individual tests passing/failing. We've recently added a required field named test_code, which contains the code the test ran to verify the behavior (see the spec for the test_code property).

The test runner's output is currently what I refer to as version 1.5: it contains the individual test information, but does not yet contain the test_code key. If possible, the test runner should be updated to also include the test code, in which case the version property should also be added with 2 as its value.

You are completely free on how you want to extract the test code, either via the AST or via finding the test code in the source code text via string manipulation.

Let me know if there are any questions.

With the latest image upgrade, we're now running out of memory:

two-fer: testing...\n[0.020s][warning][os,thread] Failed to start thread \"GC Thread#0\" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.\n#\n# There is insufficient memory for the Java Runtime Environment to continue.\n# Cannot create worker GC thread. Out of system resources.\n# An error report file with more information is saved as:\n# /opt/test-runner/hs_err_pid54.log\ntwo-fer: done\n

I seem to recall us having a similar issue for the Java, Kotlin or Groovy test runner, so I'll look into it.

CC @artamonovkirill

When running in Docker, test runners should support being run on a read-only filesystem. The only directories that are writeable are:

• The /opt/test-runner directory
• The /tmp directory
• The input directory
• The output directory

To simulate this behavior, the --read-only flag has to be passed to the two docker run calls at the following locations:

• https://github.com/exercism/scala-test-runner/blob/main/bin/run-in-docker.sh#L36
• https://github.com/exercism/scala-test-runner/blob/main/bin/run-tests-in-docker.sh#L19

I've briefly looked into this, and the main problem seems to be that SBT is trying to write a lock file in the ~/.sbt directory.

Trying to build the Docker image using docker build -t exercism/scala-test-runner . fails with the following error:

------
 > [ 7/12] COPY dependencies/ivy2.tar.gz /opt/scala-test-runner:
------
------
 > [ 8/12] COPY dependencies/sbt.tar.gz /opt/scala-test-runner:
------
failed to compute cache key: "/dependencies/sbt.tar.gz" not found: not found

• config.json supports multiple test files
• 1 Scala test file can contain multiple test classes

Scala test runner only allows passing 1 test XML report (=1 test class in 1 test file).

Tests pass on local fail on Exercism.

object Twofer {
def twofer(name: String = "you"): String = s"One for ${name}, one for me.";
}

PS C:\Users\Trent\Exercism\scala\two-fer> scala --version
Scala code runner version 3.1.3 -- Copyright 2002-2022, LAMP/EPFL
PS C:\Users\Trent\Exercism\scala\two-fer> scalac --version
Scala compiler version 3.1.3 -- Copyright 2002-2022, LAMP/EPFL

We've recently updated the test runner interface specification to allow for two types of test runners, identified via a new version property in the results.json file (see the spec for the version property):

1. version: 1 test runners are quite basic, and only detect if either all tests passed or not. These test runners capture the console output of the test runner and output that. They do not include details on individual tests passing/failing
2. version: 2 test runners contain detailed information on individual tests passing/failing. We've recently added a required field named test_code, which contains the code the test ran to verify the behavior (see the spec for the test_code property).

The test runner's output is currently what I refer to as version 1.5: it contains the individual test information, but does not yet contain the test_code key. If possible, the test runner should be updated to also include the test code.

There is an existing TODO in the code where the test code is written: https://github.com/exercism/scala-test-runner/blob/main/src/main/scala/Application.scala#L63 You are completely free on how you want to extract the test code, either via the AST or via finding the test code in the source code text via string manipulation. Whatever is most convenient.

Let me know if there are any questions.