OpenActive Test Suite 
To join the conversation, we're on the OpenActive Slack at #openactive-test-suite.
The general aim of this project is to allow end to end testing of the various flows and failure states of the Open Booking API.
Running npm start in the root will run the OpenActive Test Suite, which is actually comprised of these packages:
- Integration Tests: this performs automated tests against the API.
- Broker Microservice: this sits in between the test suite and the target Open Booking API implementation. This allows the integration tests to watch for changes to the various RPDE feeds.
- OpenID Test Client: this connects to the target Open Booking API's OpenID Provider. This allows the Broker and Integration tests to authorize with the implementation
- Test Interface Criteria: this allows test suite to tailor specific opportunities to specific tests by implementing the OpenActive Test Interface Criteria.
Usage
Running npm start will orchestrate running the Broker Microservice and the Integration Tests in order to test your Open Booking API implementation.
Note that the implementation under test will need to implement the OpenActive Test Interface to run in controlled mode, and for selected tests.
Quick start
You can check that the test suite works in your local environment by running it against the hosted OpenActive Reference Implementation, simply by using the default configuration:
git clone [email protected]:openactive/openactive-test-suite.git
cd openactive-test-suite
npm install
npm start -- coreNote that the above command only runs the "core" tests within the test suite, which should take around 60 seconds to complete.
The hosted OpenActive Reference Implementation is running on a basic developer tier Azure instance with a burst quota, so it will not handle the load of a test suite run for all tests (hence npm start -- core); if the hosted application shuts down, simply wait 5 minutes and try again.
Configuration
In order to run the test suite against your own implementation, configure the test suite by creating a copy of config/default.json named config/{NODE_ENV}.json (where {NODE_ENV} is the value of your NODE_ENV environment variable), including the following properties:
The test suite uses the file config/{NODE_ENV}.json to override the settings in default.json. For development and deployment create a new file instead of making changes to the default.json file, so that any new required settings that are added in future versions can be automatically updated in default.json.
For more information about this use of NODE_ENV see this documentation.
By convention, much of the documentation assumes you to have created a config/dev.json file, which Test Suite will use when env var NODE_ENV=dev. But you can use any name you like, and have multiple configuration files for different environments.
Configuration for sellers within ./config/{NODE_ENV}
The primary Seller is used for all tests, and random opportunities used when "useRandomOpportunities": true are selected from this Seller. The secondary Seller is used only for multiple-sellers tests.
An example, using OpenID Connect Authentication:
"sellers": {
"primary": {
"@type": "Organization",
"@id": "https://reference-implementation.openactive.io/api/identifiers/sellers/0",
"authentication": {
"loginCredentials": {
"username": "test1",
"password": "test1"
}
},
"taxMode": "https://openactive.io/TaxGross",
"paymentReconciliationDetails": {
"name": "AcmeBroker Points",
"accountId": "SN1593",
"paymentProviderId": "STRIPE"
}
},
"secondary": {
"@type": "Person",
"@id": "https://reference-implementation.openactive.io/api/identifiers/sellers/1",
"authentication": {
"loginCredentials": {
"username": "test2",
"password": "test2"
}
},
"taxMode": "https://openactive.io/TaxNet"
}
}Description of each field:
-
authentication: Check out the Configuration for Seller Authentication section. -
taxMode: Which Tax Mode is used for this Seller.Note: If testing both Tax Modes, make sure that there is at least one Seller with each. Alternatively, if not supporting multiple Sellers, you can run the Test Suite once with
"taxMode": "https://openactive.io/TaxNet"and once with"taxMode": "https://openactive.io/TaxGross". However, it is not currently possible to generate a certificate that covers both configurations unless multiple Sellers are supported. -
paymentReconciliationDetails: If testing Payment Reconciliation Detail Validation, include the required payment reconciliation details here.
Configuration for Seller Authentication
In order to make bookings for a specific Seller's Opportunity data, some kind of authentication is required to ensure that the caller is authorized to make bookings for that Seller.
Test Suite allows for a few different options for Seller Authentication. This determines the data to put in the authentication field for each Seller:
OpenID Connect
You'll need the username/password that the Seller can use to log in to your OpenID Connect Provider.
Example:
"sellers": {
"primary": {
// ...
"authentication": {
"loginCredentials": {
"username": "test1",
"password": "test1"
}
}
},Request Headers
Just a set of request HTTP headers which will be used to make booking requests. There are no restrictions on the requestHeaders that can be specified.
Example:
"sellers": {
"primary": {
// ...
"authentication": {
"loginCredentials": null,
"requestHeaders": {
"X-OpenActive-Test-Client-Id": "booking-partner-1",
"X-OpenActive-Test-Seller-Id": "https://localhost:5001/api/identifiers/sellers/1"
}
}
},Client Credentials
OAuth Client Credentials are used to make booking requests.
Example:
"sellers": {
"primary": {
// ...
"authentication": {
"loginCredentials": null,
"clientCredentials": {
"clientId": "clientid_XXX",
"clientSecret": "example"
}
}
},This is different from the behaviour in the Client Credentials sub-section mentioned within the OpenID Connect Booking Partner Authentication for Multiple Seller Systems section in the spec as, in this case, Client Credentials are used to make booking requests for this Seller, rather than just to view the Booking Partner's Orders Feed.
Installation
Node.js version 14 or above is required.
npm installThis will install the dependencies needed for all packages in the test suite.
For developers that are customising the installation, for use in e.g. Docker, the directories ./packages/test-interface-criteria and ./packages/openactive-openid-test-client are dependencies, and so must be present during npm install.
Running
Where dev.json is the name of your {NODE_ENV}.json configuration file:
export NODE_ENV=dev
npm startThis will start the broker microservice (openactive-broker-microservice) and run all integration tests (openactive-integration-tests) according to the feature configuration. It will then kill the broker microservice upon test completion. The console output includes both openactive-broker-microservice and openactive-integration-tests. This is perfect for CI, or simple test runs.
Alternatively the Broker microservice and Integration tests may be run separately, for example in two different console windows. This is more useful for debugging.
Running specific tests
Any extra command line arguments will be passed to jest in openactive-integration-tests. For example:
export NODE_ENV=dev
npm start -- --runInBand test/features/core/availability-check/It is also possible to use a category identifier or feature identifier as short-hand:
export NODE_ENV=dev
npm start -- coreexport NODE_ENV=dev
npm start -- availability-checkRead about Jest's command line arguments in their CLI docs.
Environment variables
NODE_CONFIG
The configuration of the test suite can be overridden with the environment variable NODE_CONFIG, where any specified configuration will override values in both config/default.json. More detail can be found in the node-config docs. For example:
NODE_CONFIG='{ "waitForHarvestCompletion": true, "datasetSiteUrl": "https://localhost:5001/openactive", "sellers": { "primary": { "@type": "Organization", "@id": "https://localhost:5001/api/identifiers/sellers/0", "requestHeaders": { "X-OpenActive-Test-Client-Id": "test", "X-OpenActive-Test-Seller-Id": "https://localhost:5001/api/identifiers/sellers/0" } }, "secondary": { "@type": "Person", "@id": "https://localhost:5001/api/identifiers/sellers/1" } }, "useRandomOpportunities": true, "generateConformanceCertificate": true, "conformanceCertificateId": "https://openactive.io/openactive-test-suite/example-output/random/certification/" }' npm startPORT
Defaults to 3000.
Set PORT to override the default port that the openactive-broker-microservice will expose endpoints on for the openactive-integration-tests. This is useful in the case that you already have a service using port 3000.
FORCE_COLOR
E.g. FORCE_COLOR=1
Set this to force the OpenActive Test Suite to output in colour. The OpenActive Test Suite uses chalk, which attempts to auto-detect the color support of the terminal. For CI environments this detection is often inaccurate, and FORCE_COLOR=1 should be set manually.
Continuous Integration
Assuming configuration is set using the NODE_CONFIG environment variable as described above, the test suite can be run within a continuous integration environment, as shown below:
#!/bin/bash
set -e # exit with nonzero exit code if anything fails
# Get the latest OpenActive Test Suite
git clone [email protected]:openactive/openactive-test-suite.git
cd openactive-test-suite
# Install dependencies
npm install
# Start broker microservice and run tests
npm start"ci": true must be included in the supplied NODE_CONFIG to ensure correct console logging output within a CI environment.
Note that running npm start in the root openactive-test-suite directory will override waitForHarvestCompletion to true in default.json, so that the openactive-integration-tests will wait for the openactive-broker-microservice to be ready before it begins the test run.
Test Data Requirements
In order to run the tests in random mode, the target Open Booking API implementation will need to have some Opportunity data pre-loaded. Use Test Data Generator to find out how much data is needed and in what configuration.
Contributing
Concepts
Booking Partner Authentication Strategy
The method by which a Booking Partner authenticates with the Open Booking API implementation. There are a number of supported strategies, including OpenID Connect, HTTP Header, etc.
Your impementation will need to support at least one Authentication Strategy for each of Orders Feed Authentication and Booking Authentication.
Orders Feed Authentication
How a Booking Partner accesses the Orders Feed containing updates to Orders that they have created.
For Test Suite, the selected Orders Feed Authentication Strategy is configured with the broker.bookingPartners configuration property and documentation on the supported strategies can be found there.
Booking Authentication
How a Booking Partner accesses the booking endpoints (C1, C2, B, etc) for a specific Seller's data. This differs from Orders Feed Authentication as it can be specified at the per-Seller level for Multiple Seller Systems (relevant feature: multiple-sellers).
For Test Suite, the selected Booking Authentication Strategy is configured with the sellers configuration property and documentation on the supported strategies can be found there.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent