This repository shows how Pact, PactFlow and Cypress could work together to provide increased confidence and reliability for web applications that rely on backend API communication.
The end-to-end project is based off the PactFlow CI/CD workshop at https://docs.pactflow.io/docs/workshops/ci-cd/.
It is using a public tenant on PactFlow, which you can access here using the credentials dXfltyFMgNOFZAxr8io9wJ37iUpY42M/O5AIZWxelWbLvqMd8PkAVycBJh2Psyg1. The latest version of the Example Consumer/Example Provider pact is published here.
NOTE: this repository took inspiration from the great work over at https://github.com/YOU54F/cypress-pact.
The following is an over simplified view of how this would work in a full end-to-end workflow:
- Cypress tests the React website running at
http://localhost:3000. - Pact tests within the Cypress suite mock out network calls, generating a contract file that captures the interactions between the two systems. The contract is stored in
pacts/pactflow-example-consumer-cypress-pactflow-example-provider.jsonif test run was successful. - The contract is then published to a publicly available PactFlow account at https://test.pactflow.io/pacts/provider/pactflow-example-provider/consumer/pactflow-example-consumer-cypress/latest (login with username:
dXfltyFMgNOFZAxr8io9wJ37iUpY42M/ password:O5AIZWxelWbLvqMd8PkAVycBJh2Psyg1, example build https://github.com/pactflow/example-consumer-cypress/workflows) - Provider build is triggered by a webhook to validate the contract that was just published (e.g. https://github.com/pactflow/example-provider/workflows).
- Run
can-i-deployto see if the Web App is compatible with the Product API and if it is safe to release to production.
- Run
npm ito install cypress and related dependencies - Start the react app:
make mocked(this uses a stubbed backend provided by PactFlow) - Run Cypress in GUI mode:
npm cypress:open:stubbed - Run Cypress in CLI mode:
npm cypress:run:stubbed
There is also a Makefile to run via the CLI which is used by CI (GitHub Actions).
Running UI tests can suffer from a number of issues:
- Flakiness - UI tests can be notoriously flakey if they are run against a real provider, due to the need to manage test data and mutations.
- Managing test environments and test data itself can be a huge burden on a team, resulting in reducing the size of the UI test suite to assist with maintenance.
- Reliability - Often times to address (1) and (2), test authors will stub out endpoints to make the tests faster and more reliable
In order to combat some of the issues from above, teams may choose to fake out a backend.
The Cypress documentation contains an excellent guide on the tradeoffs of stubbing vs e2e tests:
Real Server (End to End tests issuing real network requests)
Real Server:
| Pros | Cons |
|---|---|
| More likely to work in production | Requires seeding data |
| Test coverage around server endpoints | Much slower |
| Great for traditional server-side HTML rendering | Harder to test edge cases |
Stubbing:
| Pros | Cons |
|---|---|
| Control of response bodies, status, and headers | No guarantee your stubbed responses match the actual data the server sends |
| Can force responses to take longer to simulate network delay | No test coverage on some server endpoints |
| No code changes to your server or client code | Not as useful if you’re using traditional server side HTML rendering |
| Fast, < 20ms response times | - |
Pact tests are generally authored at a unit test layer, and we (currently) discourage the use of it at the UI layer for a number of reasons. Teams using both tools could reduce the overlapping test areas and corresponding maintenance if the two tools were better integrated.
Currently, Pact tests are written as "white box" style tests, meaning they are generally authored by developers who maintain the code base. The Cypress experience is suited to a wider range of test authors who may benefit from writing and contributing to Pact tests.
Whilst we currently recommend avoiding doing Pact testing through a UI test, sometimes old code bases are a bit harder to pick apart and test at the "right layer", making unit testing much more difficult. So difficult that you just don't do it. You know the kinds of problems I'm talking about. Having a Pact test run "from the outside" has significant advantages for these use cases.
By integrating Pact with Cypress, there is an opportunity to reduce some of the downsides of stubbing in Cypress, namely:
- Providing guarantees that request/responses will be supported by the provider
- Coverage of all server endpoints and interactions required by the (web) application
Additionally, we can bring in new test authors to the Pact ecosystem, and reduce duplication for teams already using both tools, thereby saving time to value and reducing maintenance costs.
- Create a
cypress-pactplugin that natively maps over thecy.serverandcy.requestinterfaces to create a seamless Cypress experience - Support "compressing" of interactions in PactFlow, to reduce the problems created by having too many examples in each contract
A separate solution is proposed, that makes use of the more advanced route2 functionality, which is able to proxy all network requests. This would be useful for a broader application when a "provider driven" workflow is implemented in PactFlow, removing the drawbacks of this kind of contract test.
![mend-for-github-com[bot] avatar](https://avatars.githubusercontent.com/in/30796?v=4 "mend-for-github-com[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent