Comments (9)
Additionally the test suite currently takes around 60 seconds to startup, before it runs any tests.
It outputs jest
and then appears to wait for 30 seconds, before outputting Found 7 test suites
and then waiting for a further 30 seconds.
Can we improve performance here? What are we doing to slow things down?
from openactive-test-suite.
Also we should make parallel execution of tests configurable, as this can have the effect of slowing down testing for a larger number of tests against a database
from openactive-test-suite.
This is already configurable, just needs documenting. Defaults are in the package.json, and can also be changed via commandline args.
npm run test -- --maxWorkers 1
from openactive-test-suite.
Running a subset of tests can be done just by specifying the test path, this can be a directory (run everything within).
npm run test -- test/flows/book-and-cancel
(run all book and cancel flows).
all the way down to specifying an individual test file:
npm run test -- test/flows/book-and-cancel/book-and-customer-cancel-success/book-free-test.js
from openactive-test-suite.
Ok great! Can we also use wildcards?
The usecase I'm thinking of is where we have (following discussions this morning and notes in #34):
test/flows/book-and-cancel/book-and-customer-cancel-success/book-free-test-scheduledsession.js
(booking aSessionSeries
)test/flows/book-and-cancel/book-and-customer-cancel-success/book-free-test-slot.js
(booking aSlot
)test/flows/book-and-cancel/book-and-customer-cancel-success/book-free-test-slot-and-scheduledsession.js
(booking multiple OrderItems - aSessionSeries
and aSlot
together - as per #34 (comment))
and we perhaps only want to run those that only involve "ScheduledSession" (because they haven't implemented Slot
s in their system)
Unless there's another approach we can take here?
from openactive-test-suite.
Need to allow a subset of the tests to be run for only SessionSeries
, or FacilityUse
, or both or more depending on what feeds have been implemented by the booking system
from openactive-test-suite.
Trying to figure out whether we can implement some sort of filtering system - being able to define them much like rspec would be nice: https://relishapp.com/rspec/rspec-core/docs/hooks/filters
from openactive-test-suite.
@ylt to scope this issue down to something achievable within a reasonable timebox, could we just cover the following two in a config file:
Configuration spec
The test suite can be configured to test "optional" features, by indicating whether they are either:
implemented
- the tests will run to confirm proper implementation of the featurenot-implemented
- the tests will run to confirm the feature is correctly advertised as "not implemented"disable-tests
- disable all tests for this feature (only recommended during development)
Additionally, bookable opportunity types can be configured, to indicate which types the implementation is expected to support:
sessions
(dual feeds ofSessionSeries
andScheduledSession
)facilities
(dual feeds ofFacilityUse
andSlot
)events
(feeds ofEvent
)headline-events
(feeds ofHeadlineEvent
with embeddedEvent
)courses
(feeds ofCourseInstance
)
Configuration example
An example config file could look like this:
{
"features": {
"opportunity-feed": "implemented",
"dataset-site": "implemented",
"availability-check": "not-implemented",
"simple-book-free-opportunities": "not-implemented",
"simple-book-with-payment": "not-implemented",
"payment-reconciliation-detail-validation": "not-implemented",
"booking-window": "not-implemented",
"customer-requested-cancellation": "not-implemented",
"seller-requested-cancellation": "not-implemented",
"seller-requested-cancellation-message": "disable-tests",
"cancellation-window": "implemented",
"seller-requested-replacement": "implemented",
"named-leasing": "implemented",
"anonymous-leasing": "implemented"
},
"opportunity-types": {
"sessions": true
"facilities": true
"events": false
"headline-events": false
"courses": false
}
}
Logic example
An example of how this should work:
If "simple-book-with-payment": "implemented"
, the following test should be run:
- within
openactive-integration-tests/test/features/payment/simple-book-with-payment/implemented
If "simple-book-with-payment": "not-implemented"
, the following test should be run:
- within
openactive-integration-tests/test/features/payment/simple-book-with-payment/not-implemented
If "simple-book-with-payment": "disable-tests"
- neither tests should be run
Depending on the boolean values of opportunity-types
, the array in the following should be populated:
openactive-integration-tests/test/features/payment/simple-book-with-payment/implemented/book-random-test.js
from openactive-test-suite.
All the issues mentioned here have been resolved / implemented
from openactive-test-suite.
Related Issues (20)
- Check `position` of `orderedItem` at B
- Running npm run update-test-suite resulting in unsupported engine warning
- leasing - anonymous/named-leasing - Capacity goes back up after lease expires (without completed booking)
- Add release process around test suite
- opportunity-attendance-updates : Setting opportunities as AttendeeAttended/AttendeeAbsent before the opportunity has started
- Test Suite cannot use Client Credentials to authorise a Seller
- Single/Multiple Seller config tests HOT 1
- Content-Type/Accept headers not used consistently
- Add support for deprecated "Small provider" feed combinations HOT 1
- Add tests to the openactive-broker-microservice
- Consider preserving test output after each run
- Consider adding the runtime to the top of summary.md
- Document `npm run update-test-suite`
- It is not possible to pass Test Suite unless Seller Requested Cancellation is enabled HOT 1
- Issue with checking remainingUses = 1 for leased slots HOT 1
- Write unit tests for broker-microservice untested functionality
- Unable to test booking window in Random Mode HOT 1
- Test Suite not restarting parent feed on test runs
- Broker Microservice blocking updates from the RPDE feed
- Specifically test the cancellation of both free and non-free bookings
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openactive-test-suite.