The aims of this workshop is to demonstrate how you can get your tests to run automatically for you 'in the clouds' using Docker and the GitHub Actions continous integration (CI) framework. This means that every time you make a change and push this to the main
branch of your repository, your tests will run, and the results will become visible in the GitHub web interface without any further actions from you.
If you're not used to using GitHub to host your APL code, worry not -- we'll work it through together. Even if you prefer another source code revisioning system, or another CI framework (like circle or jenkins), the broad strokes of the approach we'll cover here should hopefully still be relevant to you.
GitHub Actions is free to use for public repositories, and private repositories get a limited, but generous, allocation of free minutes of runner time.
To make the most of this workshop, check that
- you have Docker desktop installed
- you have Dyalog APL v18.2 installed
- you have a GitHub account you can create repositories in
- you either have git installed on your computer, or
- ..if you're a Windows user, consider installing GitHub Desktop
If you're on a Mac, install the follwing shell alias to make it easier to run Dyalog from the command line:
alias dyalog="/Applications/Dyalog-18.2.app/Contents/Resources/Dyalog/mapl"
Place this in your shell's expected config file (~/.bashrc
or ~/.bash_profile
if you use bash
or ~/.zshrc
if you use zsh
) and run source {your-shell-config-file}
.
These are the minimum requirements.
Optionally, there are plenty of auxiliary tools that make working with git
and GitHub
more streamlined if you're not a fan of the git
command line. Some examples are:
- GitHub CLI
- GitHub Desktop.
- Mac users may want to consider installing GitUp
Many popular IDEs and code editors have excellent git
and GitHub integration, e.g. Visual Studio Code.
If you do intend to use the GitHub CLI (note: optional), it might be a good idea to get that installed and set up before the day. Same with using git
over SSH (a good idea, but also optional) -- ensure you have working SSH keys installed prior to the workshop.
Now let's get this repository into your own account, and then onto your computer.
-
Fork this repository into your own GitHub account, either using
- the GitHub web interface, or
- the GitHub CLI:
gh repo fork dyalog-training/2023-TP1b
-
Clone your fork to your local machine, using one of the following methods, noting the
--recursive
switch to ensure we also clone submodules:- Git SSH:
git clone --recursive [email protected]:{YOUR-GITHUB-ACCOUNT}/2023-TP1b.git
- Git HTTPS:
git clone --recursive https://github.com/{YOUR-GITHUB-ACCOUNT}/2023-TP1b.git
- GitHub Desktop
- Git SSH:
ℹ️ Note: it's NOT sufficient to clone the workshop repository directly, it must be forked first. The reason for this is that in order to be able to trigger the GitHub Action, you need to be able to push to the remote repository, and if you clone it directly, you won't have sufficient credentials to push.
This repository contains:
- A test runner, run.aplf
- A single function under test (for demo purposes), mysum.aplf
- A directory containing two demo unit tests, written using ]dtest, tests
- A Dockerfile, capable of executing the unit tests
- A yaml file test-runner.yml, which will use the Dockerfile to run the unit tests as a GitHub Actions workflow.
Any push to the main branch of this repository should trigger the unit tests to run in the workflow.
Start Dyalog, and execute the following to link the source code into the session:
]link.create # {/path/to}/2023-TP1b/src
Replace {/path/to}
with the actual path to the 2023-TP1b
you cloned.
In the session, use the ]dtest
user command to run the tests. Note that the repository contains the latest version of DTest -- if you didn't install this earlier, copy the file {/path/to}/2023-TP1b/DBuildTest/DyalogBuild.dyalog
in your MyUCMDs
folder.
Run the command:
]dtest {/path/to}/2023-TP1b/tests
The source directory contains a test runner that enables us to trigger the tests from the command line:
dyalog -b -s LOAD=src
That command exits with a 0 if all went OK, and 11 otherwise.
Try changing the file src/mysum.aplf
to
mysum ← {
⍺-⍵
}
and show that there is now a test failure. If all tests are still succeeding, this can be a symptom of not using the latest version of ]dtest
-- something in the fork+clone process went awry. Remove the DBuildTest
directory, and re-clone the sub-module:
rm -rf DBuildTest
git submodule update --init --recursive
docker build -t dytest .
Mac/Linux:
docker run --rm \
-v "$(pwd)/DBuildTest/DyalogBuild.dyalog:/home/dyalog/MyUCMDs/DyalogBuild.dyalog" \
-v "$(pwd)/src:/src" \
-v "$(pwd)/tests:/tests" \
dytest
In Windows PowerShell:
docker run --rm `
-v "${PWD}/DBuildTest/DyalogBuild.dyalog:/home/dyalog/MyUCMDs/DyalogBuild.dyalog" `
-v "${PWD}/src:/src" `
-v "${PWD}/tests:/tests" `
dytest
Using the Windows Command Prompt:
docker run --rm -v "%cd%\DBuildTest\DyalogBuild.dyalog:/home/dyalog/MyUCMDs/DyalogBuild.dyalog" -v "%cd%\src:/src" -v "%cd%\tests:/tests" dytest
If you see an error of the type:
/src does not define the function #.Run
then you most likely ran the container from the wrong directory: it must be run from the root of the checked-out repository.
If you're new to git
, it can seem daunting. However, it's worth the time investment to learn it, given its ubiquity. Here are some common incantations to use on the command line. For a more in-depth introduction, see GitHub's git guide.
Clone a repository
git clone [--recursive] [[email protected]:{ACCT}/{REPO}.git | https://github.com/{ACCT}/{REPO}.git]
Stage, and commit modified (and deleted) files
git commit -a -m 'Your well-formed commit message here'
Push branch to remote
git push origin {branch-name}
Make a new local branch, and switch to it
git checkout -b {my-new-branch-name}
Switch to other local branch
git switch {branch-name}
Pull all remote changes in main branch
git pull origin main [--ff-only]
If you experience problems using the Docker container, you can request shell access and examine it manually. Debugging a Docker container can sometimes feel like a dark art. With this container, you can try the following:
In the root of the repository, run the following command to bypass the set entry point to gain shell access (Mac/Linux):
docker run --rm --entrypoint /bin/sh \
-v "$(pwd)/DBuildTest/DyalogBuild.dyalog:/home/dyalog/MyUCMDs/DyalogBuild.dyalog" \
-v "$(pwd)/src:/src" \
-v "$(pwd)/tests:/tests" \
-it dytest
In Windows PowerShell:
docker run --rm --entrypoint /bin/sh `
-v "${PWD}/DBuildTest/DyalogBuild.dyalog:/home/dyalog/MyUCMDs/DyalogBuild.dyalog" `
-v "${PWD}/src:/src" `
-v "${PWD}/tests:/tests" `
-it dytest
Start with verifying that the mapped files look as they should. Look in the /src
and /tests
folders and check that their contents match those of the repository's matching folders. Check that /home/dyalog/MyUCMDs
contains the file DBuildTest.dyalog
. Check that the latter is indeed a file containing APL code, and not a directory (a sign that the submodule wasn't correctly cloned).
If that all looks good, let's ensure we can run dyalog manually in interactive mode. For this we need to adjust a couple of environment variables. Execute the following statements:
unset LOAD
export DYALOG=/opt/mdyalog/18.2/64/unicode/
export LD_LIBRARY_PATH="${DYALOG}:${LD_LIBRARY_PATH}"
export WSPATH=$WSPATH:${DYALOG}/ws
export TERM=xterm-256color
export APL_TEXTINAPLCORE=${APL_TEXTINAPLCORE-1}
export TRACE_ON_ERROR=0
export SESSION_FILE="${SESSION_FILE-$DYALOG/default.dse}"
Now see what happens if you try to run the tests from the command-line (inside the container):
dyalog -b -s LOAD=/src
If that does not yield the behaviour you expect, start Dyalog interactively:
dyalog
Link the source code:
]link.create # /src
and try running ]dtest
:
]dtest /tests