Main

The CNTI Test Catalog is a tool that validates telco application's adherence to cloud native principles and best practices.

This Test Catalog focus area is one part of LF Networking's Cloud Native Telecom Initiative (CNTI) and works closely with the CNTI Best Practices and CNTI Certification focus areas.

To get the CNTI Test Catalog up and running, see the Installation Guide.

Prereqs: kubernetes cluster, wget, curl, helm 3.1.1 or greater on your system already.

1. Install the latest test suite binary: source <(curl -s https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/curl_install.sh)
2. Run setup to prepare the cnf-testsuite: cnf-testsuite setup
3. Pull down an example CNF configuration to try: curl -o cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/example-cnfs/coredns/cnf-testsuite.yml
4. Initialize the test suite for using the CNF: cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
5. Run all of application/workload tests: cnf-testsuite workload

Check out the usage documentation for more info about invoking commands and logging.

The CNTI Test Catalog will inspect CNFs for the following characteristics:

• Configuration - The CNF's configuration should be managed in a declarative manner, using ConfigMaps, Operators, or other declarative interfaces.
• Compatibility, Installability & Upgradability - CNFs should work with any Certified Kubernetes product and any CNI-compatible network that meet their functionality requirements while using standard, in-band deployment tools such as Helm (version 3) charts.
• Microservice - The CNF should be developed and delivered as a microservice.
• State - The CNF's state should be stored in a custom resource definition or a separate database (e.g. etcd) rather than requiring local storage. The CNF should also be resilient to node failure.
• Reliability, Resilience & Availability - CNFs should be reliable, resilient and available to failures inevitable in cloud environments. CNFs should be tested to ensure they are designed to deal with non-carrier-grade shared cloud HW/SW platforms.
• Observability & Diagnostics - CNFs should externalize their internal states in a way that supports metrics, tracing, and logging.
• Security - CNF containers should be isolated from one another and the host. CNFs are to be verified against any common CVE or other vulnerabilities.

See the Complete Test Documentation for a complete overview of the tests.

Welcome! We gladly accept contributions on new tests, example CNFs, updates to documentation, enhancements, bug reports, and more.

• Contributing guide
• Good first issues
• Contributions welcome

• Join the conversation on LFN Tech's Slack channels
  • #cnti-general
  • #cnti-bestpractices
  • #cnti-testcatalog-testsuite
  • #cnti-testsuite-dev
  • #cnti-certification
• Join the weekly CNTI Test Catalog meeting
  • Meetings every Tuesday at 8:00am - 9:00am Pacific Time
  • Meeting minutes are here

CNTI Test Catalog Demo 2021

• Recording
• Slides (PDF)

Crystal in the Cloud: A cloud native journey at Crystal 1.0 Conference 2021

• Recording
• Slides (PDF)

The CNTI Test Catalog leverages upstream tools such as OPA Gatekeeper, Helm linter, and Promtool for testing CNFs. The upstream tool installation, configuration, and versioning has been made repeatable.

The test framework and tests (using the upstream tools) are written in the human-readable, compiled language, Crystal. Common capabilities like dependencies between tests and categories are supported.

Setup of vanilla upstream K8s on Equinix Metal is done with the CNF Testbed platform tool chain, which includes k8s-infra, Kubespray. To add support for other providers, please submit a Pull Request to the CNF Testbed repo.

The CNTI Test Catalog community follows the CNCF Code of Conduct.

The CNTI Test Catalog is available under the Apache 2 license.