Comments (2)
Your use case is valid. A single computation in Dagster can produce multiple data assets at once.
However, the fact still stands that each data asset needs to be associated with a unique asset key. This is a framework constraint. Although this was working for you before, this was not valid behavior.
My recommendation for you would be:
- You use a
@multi_asset
to produce your data assets: https://docs.dagster.io/concepts/assets/multi-assets. Your data assets should have unique keys associated with them. - For
dagster-dbt
specific documentation, you can check out https://docs.dagster.io/integrations/dbt/reference#upstream-dependencies for using@multi_asset
with dbt sources.
from dagster.
I used a multi_asset to solve the issue, but what happens now is that dagster runs the dbt CLI command twice instead of once god knows why
from dagster.
Related Issues (20)
- Dagster webserver pages cannot be displayed due to error: "Loading chunk x failed." when using VS Code Remote Explorer HOT 24
- Reconnect Dagster to different slack account for alerts HOT 2
- dagster run in docker container hangs HOT 1
- Possibility to union partitions of multiple PartitionDefinitions
- Inconsistent behaviour for `context.asset_key_for_input` when using `deps` with `key_prefix` HOT 2
- Ability to declaratively set global concurrency key limits
- [Documentation Feedback] Problem on /getting-started/hello-dagster page HOT 1
- [Documentation Feedback] Problem on /getting-started/hello-dagster page HOT 1
- Improve loading speed for Dagit UI filters in each job
- dagster-dbt test failures not respected across asset definitions HOT 1
- Url link in ressource page is incorrect
- Incorrect / persistent new data version flag HOT 2
- Enable batch-writing materialization events during ranged run
- Error on Asset Dependency with Multiple dbt Model Tests after Updating to Dagster 1.6.4 HOT 1
- Unable to add dynamic partition in UI, ok in code
- Auto-layout the asset lineage on the UI
- dbt groups do not update in the UI HOT 1
- gRPC Client does not resolve host name in AWS namespaces HOT 1
- in auto-materialize, handle asset observations with data times but no data versions
- feat: Support multi-line string in run_config HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dagster.