install • documentation • examples • we're hiring • chat with us

• Deploy TensorFlow, PyTorch, ONNX, scikit-learn, and other models.
• Define preprocessing and postprocessing steps in Python.
• Configure APIs as realtime or batch.
• Deploy multiple models per API.

• Monitor API performance and track predictions.
• Update APIs with no downtime.
• Stream logs from APIs.
• Perform A/B tests.

• Test locally, scale on your AWS account.
• Autoscale to handle production traffic.
• Reduce cost with spot instances.

Define any real-time or batch inference pipeline as simple Python APIs, regardless of framework.

# predictor.py

from transformers import pipeline

class PythonPredictor:
  def __init__(self, config):
    self.model = pipeline(task="text-generation")

  def predict(self, payload):
    return self.model(payload["text"])[0]

Configure autoscaling, monitoring, compute resources, update strategies, and more.

# cortex.yaml

- name: text-generator
  predictor:
    path: predictor.py
  networking:
    api_gateway: public
  compute:
    gpu: 1
  autoscaling:
    min_replicas: 3

Handle traffic with request-based autoscaling. Minimize spend with spot instances and multi-model APIs.

$ cortex get text-generator

endpoint: https://example.com/text-generator

status   last-update   replicas   requests   latency
live     10h           10         100000     100ms

Integrate Cortex with any data science platform and CI/CD tooling, without changing your workflow.

# predictor.py

import tensorflow
import torch
import transformers
import mlflow

...

Run Cortex on your AWS account (GCP support is coming soon), maintaining control over resource utilization and data access.

# cluster.yaml

region: us-west-2
instance_type: g4dn.xlarge
spot: true
min_instances: 1
max_instances: 5

You don't need to bring your own cluster or containerize your models, Cortex automates your cloud infrastructure.

$ cortex cluster up

confguring networking ...
configuring logging ...
configuring metrics ...
configuring autoscaling ...

cortex is ready!

bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.20/get-cli.sh)"

See our installation guide, then deploy one of our examples or bring your own models to build realtime APIs and batch APIs.