Checkout UI Test Engine
Description
The repository for mapped and implemented purchase end-to-end testing scenarios for the Checkout UI.
The test engine is composed by four main utilities:
- Cypress - the tool for developing and running end-to-end tests
- Github Actions - where the actual tests are run
- Horus Files - AWS Lambda function used to send test result videos to s3
- Horus Proxy - AWS Lambda function to send status to monitoring
This repo has two Github Actions (BETA and STABLE) with a cron schedule to run all tests using Cypress, then sending videos to s3, evidences and the run result to monitoring.
Manual usage
This repository uses a GitHub Workflow Action to trigger the end-to-end tests execution on our Cypress Dashboard project.
To trigger this workflow, follow those steps:
- Access the End-to-End tests GitHub Workflow
- Click in the
Run workflow
button and fill the form - After triggering the workflow, wait for about one minute and check the Checkout UI Cypress Dashboard page to find your run.
ℹ️ Running the complete suite with 30 containers take about ~11 minutes to end.
⚠️ Also, each complete suite run cost about $1, so be mindful about how many executions are being made. To better understand how this is billed, check the Cypress pricing page.
You can also trigger this action via the GitHub CLI using gh workflow run
.
Developing
To implement a test or just run a test individually run the following command and the Cypress ui should open:
$ yarn cypress
Optional parameters can be set such as environment
and vtex io workspace
as follows:
$ yarn cypress --env VTEX_ENV={{local|beta|stable|io|beta-io}}
$ yarn cypress --env VTEX_WORKSPACE={{example_workspace}}
Multiple parameters should be separated by comma, e.g.:
$ yarn cypress --env VTEX_ENV=beta,VTEX_WORKSPACE=myWorkspace
Optionally, you can also set them as regular environment variables in your shell, just prepend them with
CYPRESS_
, for exampleCYPRESS_VTEX_ENV
To run ALL tests without sending to monitoring run:
$ yarn cypress:run
To run the suite locally actually sending the results to monitoring run:
disclaimer - you should have HORUS_PROXY_KEY
and HORUS_FILES_KEY
API keys in a .env
file for the results and videos to be sent to monitoring.
$ yarn test # for tests using STABLE environment
$ yarn test:beta # for tests using BETA environment
Environments
We currently support 4 different environments values (which are passed as VTEX_ENV
), and this sections explains each one.
stable
: This is our usual environment for testing. It opens the tests using thevtexcommercestable.com.br
domain, and uses Portal V2 as the rendering backend. It is expected that all tests pass when using this environment.beta
: This is the environment we use for testing prerelease versions. It uses thevtexcommercebeta.com.br
domain, and also uses Portal V2 as the rendering backend. This environment will use all beta versions of the services (both UI and API), and is not expected that all tests pass.io
: This is our newest environment, and what we hope to replacestable
with. This opens our testing environment using themyvtex.com
domain, and uses the new Portal IO rendering backend. It requires two environment variables to be able to access myvtex without authentication:APP_KEY
andAPP_TOKEN
. It is also expected that all tests pass when using this environment.beta-io
: This is an older environment which we used for running the stable tests on a prerelease version ofomnishipping
orshipping-preview
. It uses thevtexcommercestable.com.br
domain, but with the workspacebeta
. We used to link (or install) the version we wanted to test in this workspace of our tests accounts and wait for the result on the healthcheck page. This was mainly used for running tests on a development version, so the tests were not expected to pass.
Actions
In the utils
folder you have at your disposal a series of implemented actions for each step of the checkout process, making it easier to implement a new scenario.
Profile
fillEmail
- Fills a given email at theorderForm
step of the checkoutfillProfile
- Fills the profile form with a user datagetRandomEmail
- Returns a random email for the first purchase scenariogetSecondPurchaseEmail
- Returns the[email protected]
email for recurring purchasesconfirmSecondPurchase
- After identifying with the second purchase email confirms the identification modallogin
- simulates the login flow
Shipping
unavailableDeliveryGoToPickup
- Goes to pickup if an item doesn't have delivery optionsfillShippingInformation
- Fills shipping information such as postal code and the remaining address formfillRemainingInfo
- Clicks in the remaining info button after a pickup is selected in split (delivery/pickup) scenariosgoToPayment
- Goes to payment stepchooseDelivery
- Selects Delivery tab at shipping stepchoosePickup
- Selects Pickup tab at shipping stepfillShippingPreviewDelivery
- Fills delivery information in shipping previewchoosePickupShippingPreview
- Selects Delivery tab at cart stepchooseDeliveryShippingPreview
- Selects Pickup tab at cart stepchooseDeliveryDate
- Selects a date for scheduled optionsfillPickupAddress
- Searches for an address in the pickup points modalgoToInvoiceAddress
- Selects go to invoice address to open this stepfillInvoiceAddress
- Fills the invoice address information
Payment
payWithBoleto
- Selects Boleto payment methodpayWithCreditCard
- Selects Credit card payment methodtypeCVV
- Types CVV for recurring purchasescompletePurchase
- Clicks finish purchase buttonfillCreditCardAndSelectInstallmentWithInterest
- Fills credit card with Elo flag and select an installment with interestfillGiftCard
- Opens Gift card input and fills with a voucherselectWHGooglePay
- Selects Google Pay payment method
Summary
goBackToCart
- Turns back from the checkout to cart
Contributing
The main purpose of this repository build a more effective e2e test suite for the Checkout UI. Read below to learn how you can contribute.
Contributing Guide
Read the contributing guide to learn about the development process and how to build a new test scenario.
Old removed tests
In the end of 2021 we realized that some all-time-broken tests were impacting our confiability, so we decided to fix them. Unfortunately some of them represented real bugs with Checkout, but after finding out how rare and hard to fix those scenarios were, some investigation, we opted to just discard those tests.
Those tests were:
- [Shipping] Delivery + Scheduled Delivery and Delivery - Credit card (vtexgame1, clean, geo, invoice, nolean)
- [Shipping Preview] Delivery + Scheduled Delivery + Pickup (vtexgame1nolean)
- [Shipping Preview] Delivery + Scheduled Delivery + Scheduled Pickup (vtexgame1nolean)
- [Shipping Preview] Delivery Only + Delivery Pickup (vtexgame1nolean)
You can check them using version control (last commit before removal) but additional information can be found here.