Git Product home page Git Product logo

dagster-cloud-serverless-quickstart's Introduction

Dagster Cloud Serverless Deployment Quickstart

๐Ÿ‘‰ You do not need to use this repo to use Dagster Cloud Serverless. When you create an account on Dagster Cloud, it creates a repo for you with the GitHub actions installed. If you want to deploy another project you can use the "Add code locations" button on the Deployments page. This repo provides an alternative way to create a project linked to Dagster Cloud serverless.

Welcome to your Dagster Cloud sample code repo. Here, you can find the code that's being deployed to your Dagster Cloud instance. For more in-depth information, check out our Serverless docs.

Pushing to production will automatically kick off a workflow which will redeploy your code to your prod deployment.

Creating a pull request will kick off a workflow which will create a new Branch Deployment, an ephemeral deployment where you can test your changes.

Setting up Quickstart Template Manually

If you had Dagster Cloud clone and set up this repo for you, no need to follow these instructions.

Click the Use this Template button and provide details for your new repo.

Screen Shot 2022-07-06 at 7 24 02 AM

Set up secrets

Set up secrets on your newly created repository by navigating to the Settings panel in your repo, clicking Secrets on the sidebar, and selecting Actions. Then, click New repository secret.

Name Description
DAGSTER_CLOUD_API_TOKEN An agent token, for more details see the Dagster Cloud docs.

Update workflows

Replace the ORGANIZATION_NAME in both .github/workflows/deploy.yml and .github/workflows/branch_deployments.yml with your Dagster cloud organization name:

  DAGSTER_CLOUD_URL: "http://ORGANIZATION_NAME.dagster.cloud"

Verify Builds are Successful

At this point, the Workflow should complete successfully. If builds are failing, ensure that your secrets are properly set and that your deployment has finished activating.

Next Steps

Now that your GitHub repository is setup with CI/CD to deploy to Dagster Cloud, you can add your own Dagster code. To run this project locally with dagit first install its local developement dependencies

pip install -e ".[dev]"

Once you've done this, you can run:

dagit

to view this repo in Dagster's UI, Dagit.

You can also copy any existing Dagster examples or quickstart projects into this GitHub repository.

dagster-cloud-serverless-quickstart's People

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

dagster-cloud-serverless-quickstart's Issues

GitHub Action Failing 404

Using this template and replacing the two secrets (DAGSTER_CLOUD_API_TOKEN and ORGANIZATION_ID) result in the following error when running the deploy GitHub workflow:

DagsterCloudHTTPError: 404 Client Error: Not Found for url: 
https://dagster.cloud//prod/graphql: Not Found
Error: No serverless registry information found - your serverless deployment may still be activating

Looks like it might be failing for the same reason here as well. Is this on anyone's radar to get fixed? Or does anyone know of a workaround?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.