Describe the bug
Canarytrace save the same collection from navigate to next navigate in the different test steps.

Expected behavior
Clear performance entries directly in a browser after download it.

Screenshots

Environment information:

• Canarytrace: Professional
• Resources: [e.g. CPU, RAM]
• Browser version
• Version of Canarytrace 3.0.5

https://canarytrace.atlassian.net/l/c/n7EEA310

IDEA

Canarytrace Listener will be automatically check the rules of quality a web application.

Sources:

• https://httparchive.org
• https://www.smashingmagazine.com/2021/01/front-end-performance-2021-free-pdf-checklist/

Canarytrace isn't a testing framework, but it's a plug-an-play stack for functional testing, web performance testing and availability monitoring web application by user perspective. It’s based on a testing framework WDIO, contain services for live logging, sniffing network of browser, web performance testing and other components for test design, execution, evaluation and investigation of the results.
->>
Canarytrace isn't only a testing framework, but it is a plug-an-play stack for functional testing, web performance testing and monitoring of web application availability from user perspective. It’s based on WDIO testing framework, featuring live logging services, browser network sniffing, web performance testing and other components for designing, execution, evaluation and investigation of the results.

Additional components are responsible for alerting (based on pre-defined thresholds), automatically search for incidents in collected data, reporting, searching and visualisation of data from every run.
->>
Other components are responsible for alerting (based on predefined thresholds), automatically searching for incidents in the data collected, reporting, searching, and visualising data from each run.

Canarytrace collects a lot of non-functional metrics such as FP, FCP, FMP, ResponseTime, TTFB, SpeedScore, WebVitals and more.
->>
Canarytrace collects many non-functional metrics such as FP, FCP, FMP, ResponseTime, TTFB, SpeedScore, WebVitals and many more.

All non-functional metrics are continuously stored to Elasticsearch.
->>
All non-functional metrics data are continuously stored to Elasticsearch.

Standard testing frameworks evaluate functional state of the web application without any information about the non-functional context and behaviour of the web browser. Canarytrace stores a lot of non-functional data from the browser for analyzing the state of the web application. E.g. coverage audit, all requests, all responses, memory usage in the tab with tested web application etc.
->>
Other testing frameworks evaluate functional state of the web application without any information about the non-functional context and behaviour of the web browser. Canarytrace stores many non-functional data metrics from the browser for analysing the state of the web application. E.g. coverage audit, all requests, all responses, memory usage in the tab with tested web application etc.

Easily measure render time of individual parts of your web application with Hero Elements, e.g. when exactly is displayed to suer login button or some banner?
->>
Easily measure render time of individual parts of your web application with Hero Elements, e.g. when exactly is displayed to user login button or some banner?

Elasticsearch stack and Canarytrace Listener take over other activities like alerting by thresholds, reporting to stakeholders and first stage analysis of data from Canarytrace runner.
->>
Elasticsearch stack and Canarytrace Listener take over other activities like alerting based on thresholds, reporting to stakeholders and first stage analysis of data from Canarytrace runner.

Canarytrace Smoke and Canarytrace Smoke Pro is edition without any maintenace. Enter only destination addresses of landing pages, that's all.
->>
Canarytrace Smoke and Canarytrace Smoke Pro are editions without any maintenace. Enter only destination addresses of landing pages, that's all.

All data from Canarytrace runner are continuously stored to Elasticsearch. You can see the results of testing and measurement immediately.
->>
All data from Canarytrace runner are continuously stored into Elasticsearch. You can see the results of testing and measurement immediately.

Kibana is a web application with GUI for viewing data stored in Elasticsearch. Data then can be used in graphs and other visualisations from which you can create custom dashboards for your testers, managers, developers or other team members.
->>
Kibana is a web application with GUI for viewing data stored in Elasticsearch. Data can be used in graphs and other visualisations from which you can create custom dashboards for your testers, managers, developers or other team members.

performance.entries is collection of a part of Web Api Performance and contains events of web page and browser with non-functional metrics and timestamps. Guarantor of this collections is a web browser and whole collection is stored to Elasticsearch.
->>
performance.entries is collection of a part of Web Api Performance and contains events of web page and browser with non-functional metrics and timestamps. Garant of this collections is a web browser and whole collection is stored into Elasticsearch.

Console Intercept is Canarytrace service, which intercept all entries from browser console. Contains message from recommendation level to error level. Again, all data from browser console is stored to elasticsearch.
->>
Console Intercept is Canarytrace service, which intercept all entries from browser console. Contains message from recommendation level to error level. Again, all data from browser console is stored into elasticsearch.

@topribyl documentation was moved to our confluence space:

https://canarytrace.atlassian.net/l/c/pzezHBiG

https://canarytrace.atlassian.net/wiki/spaces/CDD/pages/192348695/Canarytrace+Lifecycle

@topribyl documentation was moved to our confluence space:

https://canarytrace.atlassian.net/l/c/Y1hs0CYG

Reporter: slack

This reporter send mini report to into any slack

• events - save events, alert messaage and additional information to c.listener-events-*
• slack - send events and additional information to slack channel by SLACK_WEBHOOK_URL
• slack-internal - send events and additional information tocanarytrace slack channel listener-events

    rules:
    - type: match
      title: 'Failed check your page!'
      index: c.report
      timeRange: now-1h
      field: passed
      operator: must
      expected: false
      min: 2
      reportLabels:
      - 'fullTitle'
      - 'timestamp'
      reporters:
      - type: slack
        score: 10
        message: An error occurred while checking.
      - type: slack-internal
        score: 10
        message: An error occurred while checking.

Example mini report from internal checks

Example mini report from default rules

Labels seleniumCluster, maxInstances and goog:chromeOptions is variable events and depends on your setup.
->>
Labels seleniumCluster, maxInstances and goog:chromeOptions are variables events and depends on your setup.

Performance entries is intercepted by browser and Canarytrace downloads collection with performance entries when is fired event testStepEnd. All performance entries intercepted between testStepStart and testStepEnd has the same uuidAction such as testStepStart and testStepEnd. This is useful for searching what and how long it took during testStep open home page. E.g. what was downloaded to the browser and how long it took during opening home page tesla.com.
->>
Performance entries are intercepted by browser and Canarytrace downloads collection with performance entries when is fired event testStepEnd. All performance entries intercepted between testStepStart and testStepEnd have the same uuidAction such as testStepStart and testStepEnd. This is useful for searching what and how long it took during testStep open home page. E.g. what was downloaded to the browser and how long it took during opening home page tesla.com.

• https://app.cloudcraft.co/
• https://app.diagrams.net/

https://canarytrace.atlassian.net/wiki/spaces/CDD/pages/193004073/Filebeat+logging

https://canarytrace.atlassian.net/l/c/pG3nXPiT

Email sender / reporter

Email sender is reporter, which send email notification when is threshold is exceeded.

Definition reporter

rules:
  - type: match
    title: 'Functional defect'
    index: c.report
    timeRange: now-1h
    field: passed
    operator: must
    expected: false
    min: 10
    reportLabels:
    - 'fullTitle'
    - 'timestamp'
    reporters:
    - type: email
      score: 50
      message: Web isnt available
      recipients:
      - '[email protected]'
      - '[email protected]'

• type: email is type of reporter
• message send this message in body of email
• recipients is array

Expected form of html email notification

Subject: Canarytrace Notifications

Hello,
your are receiving this email, because your threshold in the rule Functional defect was exceeded.

Functional defect
Web isnt available, total:50

Score: 50, needs improvement.

What now?
Please visit your kibana and check this incident.

Your Canarytrace team

--

• If you wish to stop receiving notifications, please remove your email from rule reporter.
• Please do not reply directly to this email. If you have any questions or comments regarding this email, please contact us at https://canarytrace.com/

Tools

• nodemailer https://nodemailer.com/about/
• email-templates https://www.npmjs.com/package/email-templates
• https://stripo.email/
• https://ethereal.email/

Upgrade scripts and descriptions for:

• Create k8s and deploy Smoke Pro
• Create k8s and deploy Professional

The architecture of Canarytrace is based on dockerized components, which are orchestrated in Kubernetes or OpenShift. Thanks to this approach is easy deploy Canarytrace to in the cloud e.g. AWS, Google Cloud Engine, DigitalOcean, Azure Cloud etc. or in your own datacenter where is possible install of Kubernetes.
->>
The architecture of Canarytrace is based on dockerized components, which are orchestrated in Kubernetes or OpenShift. Thanks to this approach is easy deploy Canarytrace in to cloud e.g. AWS, Google Cloud Engine, DigitalOcean, Azure Cloud etc. or in your own datacenter where is possible install of Kubernetes.

Canarytrace is a stack based on Webdriver.io and uses a hybrid approach to testing of the web application. We use combination of a webdriver and devTools for full control on a web browser during monitoring and measure. Canarytrace isn't a substitute for monitoring tools of lower level e.g. Zabbix or Nagios but complement each other. So, Canarytrace control browser and perform measure and monitoring on top level (E2E) and provides real information about the vitality of the web application and what is about impact on the client.
->>
Canarytrace is a stack based on Webdriver.io and uses a hybrid approach to testing of the web application. We are using combination of a webdriver and devTools for full control on a web browser during monitoring and measure. Canarytrace isn't a substitute for monitoring tools of lower level e.g. Zabbix or Nagios but they complement each other. So, Canarytrace control browser and perform measure and monitoring on top level (E2E) and provides real information about the vitality of the web application and what's the impact on the client.

Canarytrace rotates on very short intervals via Kubernetes and thanks to can run on multiple instances, which is isolated from each other and every instance has fixed resources for reliable results.
->>
Canarytrace rotates on very short intervals via Kubernetes and it runs in multiple instances because of that, which are isolated from each other and every instance has fixed resources for reliable results.

Every run Canarytrace starts of cloning monitor script from git repository directly to Canarytrace container.
->>
Every run of Canarytrace starts with a monitor script cloning from git repository directly to Canarytrace container.

Browser is separated from Canarytrace and has fixed setup of resources. Is very important so that the browser has enough resources. Práce s webovou aplikací nesmí být ovlivněna externími procesy.
->> (tady se ti zatoulala cestina)
Browser is separated from Canarytrace and has fixed setup of resources. Is very important so that the browser has enough resources. Work with web application must not be affected be external processes.

The advantage is that the canarytration has access to all browser activities.
->>
The advantage of this is that the Canarytrace has access to all browser activities.

This glossary contains explain used technologies and terms in the Canarytrace ecosystem.
->> opet copy pasta z glossary?

Some version of elasticsearch return ResponseError: parse_exception and in this case don't use compression.
->>
Some version of elasticsearch return ResponseError: parse_exception. In this case don't use compression.

This print settings of elatsicsearch connection, request and response events and payload to stdout.
->>
This print settings of elasticsearch connection, request and response events and payload to stdout.

You don’t have sufficient resources of your k8s cluster for run Canarytrace instance.
->>
You don’t have sufficient resources on your k8s cluster for run Canarytrace instance.

Describe the bug
ERROR: Version in "./docker-compose.yaml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the services key, or omit the version key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

Deployment / CronJob / CLI command / Docker compose
docker-compose up

version: "3.8"
services:
browser:
image: selenium/standalone-chrome:3.141.59-20200730
ports:
- "4444:4444"
network_mode: "host"
volumes:
- /dev/shm:/dev/shm
canarytrace:
image: quay.io/canarytrace/smoke:3.0.5
depends_on:
- browser
network_mode: "host"
environment:
BASE_URL: 'https://canarytrace.com/;https://www.teststack.cz/'
ELASTIC_CLUSTER: http://localhost:9200

Expected behavior
Run Canarytrace Smoke

Environment information:
Description: Ubuntu 20.04.2 LTS
docker-compose version 1.25.3

Additional context
Add any other context about the problem here.

@topribyl documentation was moved to our confluence space:

https://canarytrace.atlassian.net/l/c/CugxrPQi

This glossary contains explain used technologies and terms in the Canarytrace ecosystem.
->> Co to tady dela, neni to copy paste? Pokud ano melo by se to pravit stejne jako u glossary tzn
This glossary contains an explanation of the technologies and terms used in the Canarytrace ecosystem.

Describe the bug
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating docker_compose_browser_1 ...

ERROR: for docker_compose_browser_1 "host" network_mode is incompatible with port_bindings

ERROR: for browser "host" network_mode is incompatible with port_bindings
Traceback (most recent call last):
File "docker-compose", line 3, in
File "compose/cli/main.py", line 80, in main
File "compose/cli/main.py", line 192, in perform_command
File "compose/metrics/decorator.py", line 18, in wrapper
File "compose/cli/main.py", line 1165, in up
File "compose/cli/main.py", line 1161, in up
File "compose/project.py", line 708, in up
File "compose/parallel.py", line 106, in parallel_execute
File "compose/parallel.py", line 204, in producer
File "compose/project.py", line 694, in do
File "compose/service.py", line 563, in execute_convergence_plan
File "compose/service.py", line 479, in _execute_convergence_create
File "compose/parallel.py", line 106, in parallel_execute
File "compose/parallel.py", line 204, in producer
File "compose/service.py", line 477, in
File "compose/service.py", line 456, in create_and_start
File "compose/service.py", line 333, in create_container
File "compose/service.py", line 937, in _get_container_create_options
File "compose/service.py", line 1069, in _get_container_host_config
File "docker/api/container.py", line 598, in create_host_config
File "docker/types/containers.py", line 339, in init
docker.errors.InvalidArgument: "host" network_mode is incompatible with port_bindings
[4043] Failed to execute script docker-compose

Deployment / CronJob / CLI command / Docker compose

version: "3.8"
services:
  browser:
    image: selenium/standalone-chrome:3.141.59-20200730
    ports:
      - "4444:4444"
    network_mode: "host"
    volumes:
      - /dev/shm:/dev/shm
  canarytrace:
    image: quay.io/canarytrace/smoke:3.0.5
    depends_on:
      - browser
    network_mode: "host"
    environment:
      BASE_URL: 'https://canarytrace.com/;https://www.teststack.cz/'
      ELASTIC_CLUSTER: http://localhost:9200

Expected behavior
Run Canarytrace Smoke

Environment information:
Description: Ubuntu 20.04.2 LTS
docker-compose version 1.28.2, build 67630359
docker-py version: 4.4.1
CPython version: 3.7.9
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019

https://github.com/canarytrace/documentation/blob/master/features.md

Typos

Web applications are tested with functional frameworks ( functional testing ) and by performance tools ( performance testing ). None of these approaches test performance and behavior of browser.
->>
Web applications are tested with functional frameworks ( functional testing ) and by performance tools ( performance testing ). None of these approaches test performance and behaviour of browser.

Browser on desktop or mobile device is a small infrastructure and has different performance and behavior according on resources (eg. CPU, RAM) depending on the device, condition of network and by quality of web application.
->>
A desktop or mobile device browser is a small infrastructure and has different performance and behaviour according on resources (eg. CPU, RAM) depending on the device, condition of network and by quality of web application.

Canarytrace is first stack for in-depth analysis behavior of browser and measure performance web application from end of user perspective.
->>
The Canarytrace is first stack for in-depth analysis behaviour of browser and measure performance web application from end of user perspective.

To get real information about how the web application will work for the user when the infrastructure is under load.
->>
To get real information about how the web application is working for the real user when the infrastructure is under load.

For front-end developers to fine-tune speed metrics and reveal opportunities for improvement performance and quality of web application.
->>
For front-end developers to fine-tune speed metrics and reveal opportunities for improvement performance and web application quality.

Our mission is to build maintenance free stack for analyse web application and web browser in a real-time. Browser and his processes is very complicated for everyone, there are many of them and our mission is provide evidence how the application behaves and how the browser works when simulating a real user.
->>
Our mission is to build maintenance free stack for analyse web application and web browser in a real-time. Browser and its processes are very complicated for everyone, there are many of them and our mission is provide evidence how the application behaves and how the browser works when simulating a real user.

We want to help developers, performance testers and DevOps build and test a new generation of modern applications better with maximum information about quality. We want provide useful information about application availability, about performance from user perspective and about processes inside the browser during a problem with web application.
->>
We want to help developers, performance testers and DevOps build and test a new generation of modern applications better with maximum information about quality. We want to provide useful information about the availability of the application, about performance from user perspective and about processes inside the browser during a problem with web application.

Browser is a small infrastructure, which finally deserves attention, we believe testing needs a lot of and we are here to build a modern stack, a service, and a community that everyone can learn and benefit from.
->>
Browser is a small infrastructure, which finally deserves attention, we believe testing needs a lot of and we are here to build a modern stack. A service. And a community that everyone can learn and benefit from.

We’re solving the hardest pain points of web performance testing and monitoring availability from user perspective.We believe in this mission and hope that you will join us and Canarytrace will become a way for assurance of quality availability your web applications.
->>
We’re solving the hardest pain points of web performance testing and monitoring availability from user perspective. We believe in this mission and hope that you will join us and Canarytrace will become a way for assurance of quality availability your web applications.

Our users are typically developers building web applications using modern JavaScript frameworks, performance testers ensuring quality under load and DevOps ensuring deploy and operation of web application and their high availability.
->>
Our users are typically developers building web applications using modern JavaScript frameworks, performance testers ensuring quality under load and DevOps engineers ensuring deploy and operation of web application and their high availability.

Canarytrace Professional: doing the same thing as Canarytrace Smoke Pro and moreover can monitor and measure performance of user journeys.
->>
Canarytrace Professional: working in simmilar way as Canarytrace Smoke Pro and it can monitor and measure performance of user journeys on top of.

Plug’n'Play stack: Architecture of Canarytrace is designed by Iac ( Infrastructure as code ) principles. Canarytrace and additional components is Dockerized. Very fast onboarding. Very fast deployment of Canarytrace - no lengthy deployment procedures. Deploying to cloud providers such as AWS, DigitalOcean, Azure, or your Kubernetes etc. is matter of the moment.
->>
Plug’n'Play stack: Architecture of Canarytrace is designed by IaC ( Infrastructure as code ) principles. Canarytrace and additional components is Dockerized. Very fast onboarding. Very fast deployment of Canarytrace - no lengthy deployment procedures. Deploying to cloud providers such as AWS, DigitalOcean, Azure, or your Kubernetes etc. within moments.

Testing and monitoring web application during development: Thanks maintenance free you can testing and measure lot of metrics durring development or during switchnig on new version of web application - early testing.
->>
Testing and monitoring web application during development: Due to the fact that Canarytrace is maintenance free you can testing and measure lot of metrics during development or during switching on new version of web application - early testing.

Check landing page every 1min: Canarytrace test and measure your landing page very ofen, e.g. every 1min or every 3min etc. Thanks to this, it catches a lots of errors. Very frequent measurements are useful for immediate and more accurate overview on realy state of your web application.
->>
Check landing page every 1min: Canarytrace test and measure your landing page very often, e.g. every 1min or every 3min etc. Due to this, it catches a lots of errors. Very frequent measurements are useful for immediate and more accurate overview on initial state of your web application.

You will be first to know: You can define lot of thresholds or multi-thresholds from more metrics and event. You will be instantly notified when an issue arises or is anomaly detection was triggered and at the same time you will not be overwhelmed by notifications.
->>
You will be first to know: You can define lot of thresholds or multi-thresholds from more metrics and event. You will be instantly notified when an issue arises or when anomaly is identified and at the same time you will not be overwhelmed by notifications.

Performance audit and Web Performance Testing: Canarytrace use custom implementation of Lighthouse and WebApi for measure web loading speed. We use an isolated web browser for every instace with precissely set resources. Canarytrace measure more than 60 non-functional metrics.
->>
Performance audit and Web Performance Testing: Canarytrace use custom implementation of Lighthouse and WebApi for measure web loading speed. We use an isolated web browser for every instance with precisely set resources. Canarytrace measure more than 60 non-functional metrics.

Run your elasticsearch and kibana and setup them via Canarytrace Installer. Ready to monitoring to 1 minute.
->>
Run your elasticsearch and kibana and setup them via Canarytrace Installer. Ready to monitoring in 1 minute.

And we are at final. Canarytrace continuously send data to elasticsearch and you can show results of all measure in a kibana.
->>
And we are at final. Canarytrace continuously send data to elasticsearch and you can show results of all measurements in a kibana.

@topribyl documentation was moved to our confluence space:

https://canarytrace.atlassian.net/l/c/0v55PcTH

• please check design of homepage documentation https://github.com/canarytrace/documentation
• all links is only placeholders
• check en description
• if you found some bug, please create issue or pull request.

@topribyl documentation was moved to our confluence space:

https://canarytrace.atlassian.net/l/c/n1Z0upvE

Some are required and some are optional.
->>
Some options are mandatory and some are optional.

All options is environment variables and they are used when running in both the docker and kubernetes
->>
All options are environment variables and they are used when running in both the docker and the kubernetes

Use Kubernetes CronJob for better experience, non-stop monitoring and for production use.
->> netusim, co jsi tomto chtel rict???

Some of them are required and some are optional.
->>
Some of them are mandatory and some are optional.

Required
->>
Mandatory parameters/properties

Optional
->>
Optional parameters/properties

Idea

Listener will be automatically search in Elasticsearch indices and look for opportunities for automated alerting by defined rules.

• Canarytrace will supply set of rules by our recommendations #46 /rules/default.yaml
• User can define own rules

Type of rules

• range - search on one index higher or lower values
• match - search on one index exactly expression
• multi-match - can combine range and match type of rules from more indices

Mandatory params

All definition of rules must contains:

• type can be of type range, match, multi-match
• timeRange how old records will be evaluated
• reporter what type of reporter will be triggered

Optional params

Rules match and range can contains:

• min alert is triggered if is found count of matches
• nearby alert is triggered if is count of occurrences found is side by side

params min and nearby isn't applied in multi-match rule

range

• index - in which index it will be searched
• filed - in which field in index will be searched value that is outside the threshold
• operator - can contains gte or lte
• value - expected value

Example:

  - type: range
    description: ''
    index: c.report
    timeRange: now-1h
    field: testStepDuration
    operator: gte
    value: 1000
    min: 10
    nearby: 3
    reporter:
    - type: events
      score: 50
      message: 'description of threshold or recommended'

match

• index - in which index it will be searched
• filed - in which field in index will be searched value that is outside the threshold
• operator - can contains must or must_not
• expected - expected value

Example:

  - type: match
    description: ''
    index: c.report
    timeRange: now-1h
    field: passed
    operator: must
    expected: false
    min: 10
    nearby: 3
    reporter:
    - type: events
      score: 50
      message: 'description of threshold or recommended'

multi-match

• matches - collection of rules type match and / or range. Max. rules 4.

Example:

  - type: multi-match
    description: ''
    timeRange: now-1h
    matches:
    -  type: range
       index: c.performance-entries
       field: responseTime
       operator: lte
       value: 1000
    -  type: match
       index: c.report
       field: sequence
       operator: must
       expected: 9
    reporter:
    - type: events
      score: 50
      message: 'description of threshold or recommended'

Reporters

Reporter is method of alert

    reporter:
    - type: events
      score: 50
      message: 'description of threshold or recommended'

type - can contains events, slack, email, ...
score - weight of the incident

This glossary contains explain used technologies and terms in the Canarytrace ecosystem.
->>
This glossary contains an explanation of the technologies and terms used in the Canarytrace ecosystem.

Web Performance Testing is type of automation test for measure key performance indicators of web application. Metrics such as WebVitals, JS Heap, troughput, count tasks over 100ms, Time to First Byte, Response Time etc. is measured and analysed directly in a browser. In this case is a browser and his APIs and diagnostic tools guarantee for data accuracy.
->>
Web Performance Testing is type of automation test for measuring of key performance indicators of web application. Metrics such as WebVitals, JS Heap, throughput, count tasks over 100ms, Time to First Byte, Response Time etc. it is measured and analysed directly in a browser. In this case is a browser and his APIs and diagnostic tools guarantee for data accuracy.

https://github.com/canarytrace/documentation/blob/master/elk/CanarytraceMonitoringOverview.md

check step by step the documentation