Comments (3)
In your dagster.yaml
file, could you try adding an image
key under config
to specify which Docker image to use when launching runs. This image should be the one that contains your Dagster user code.
Example:
run_launcher:
module: dagster_docker
class: DockerRunLauncher
config:
image: "your_image_name:your_tag" # Replace with your actual image name and tag if it haso ne
env_vars:
- DAGSTER_POSTGRES_USER
- DAGSTER_POSTGRES_PASSWORD
- DAGSTER_POSTGRES_DB
network: dagster_network
container_kwargs:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /tmp/io_manager_storage:/tmp/io_manager_storage
from dagster.
Added the image: dagster-user-code
key and value to the dagster.yaml file and the output remains the same.
from dagster.
Wanted to update: I fixed the issue I was having by removing any quotation marks around the "DAGSTER_CURRENT_IMAGE" environment variable in the compose.yaml file. The "dagster_user_code" service now looks like this:
dagster_user_code:
build:
context: .
dockerfile: ./dockerfile_user_code
container_name: dagster_user_code
image: dagster_user_code
environment:
- DAGSTER_POSTGRES_USER=${POSTGRES_USER}
- DAGSTER_POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- DAGSTER_POSTGRES_DB=${POSTGRES_DB}
- DAGSTER_CURRENT_IMAGE=dagster_user_code # THIS WAS THE PROBLEM - ORIGINAL VALUE: '"dagster-user-code"'
ports:
- 4000:4000
networks:
- dagster_network
While this fixed my immediate problem, I believe this issue should remain open. At work, I am using quotations - like this:
dagster_user_code:
build:
context: .
dockerfile: ./Dockerfile_user_code
container_name: docker_user_code
image: docker_user_code_image
ports:
- 4000:4000
networks:
- docker_example_network
environment:
DAGSTER_POSTGRES_USER: "postgres_user"
DAGSTER_POSTGRES_PASSWORD: "postgres_password"
DAGSTER_POSTGRES_DB: "postgres_db"
DAGSTER_CURRENT_IMAGE: "docker_user_code_image" # <<< RIGHT HERE
That code snippet works in my work environment. I'm not sure which is supposed to be correct, but I can't imagine it is working as intended.
from dagster.
Related Issues (20)
- dbt row count breaks on identifier which need quoting
- mlflow integration causes 429 error responses from Databricks when running multiple parallel jobs/ops HOT 2
- Asset daemon breaks when downgrading to 1.7.7 HOT 2
- dbt Freshness Check Fails w/ KeyError Due to "Missing" Metadata HOT 4
- run_status_sensor suddenly starts failing with `Expected non_None value`, possibly caused by ephemeral asset jobs HOT 2
- Monitoring ephemeral runs
- Race condition DuckDB in Dagster Essentials HOT 3
- Support generic type hints
- Asset Versioning with partitioned assets does not detect out-of-sync assets after incrementing code_version HOT 11
- Regression (v1.7.8) - Increased Time Required to start a dbt Project HOT 8
- Pull dagster-databricks Pipes into it's own library to remove dependency on Spark
- [UI] Comma after surname included in catalog greeting
- Add useful context in tooltip for assets outside of the current selection
- [ui][1.7.11] jobs page misbehaving HOT 1
- Overload of warning logs when using dbt-core 1.8.* HOT 9
- Missing imports in `Refactoring assets to use resources` in Dagster Essentials HOT 1
- k8sRunLauncher.runK8sConfig.containerConfig overridden by dagster-user-deployments.deployments[0].resources HOT 4
- Dagster DBT doesn't load list of tags from dbt into dagster
- dbt custom schema not working in dagster HOT 2
- Unexpected behaviour with PermissiveConfig
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dagster.