Comments (4)
dberenbaum
commented on June 3, 2024
Could you provide a simplified dvc.yaml to clarify how your pipeline is set up?
from dvc.
EvanKomp
commented on June 3, 2024
stages:
preprocess:
cmd: ./prepare.sh
outs:
- ./data/preprocessing/
predict:
cmd ./predict.sh
params:
- model_type # One of A, B, C
deps:
- ./data/preprocessing/ # THIS ONLY NEEDS TO BE A DEPENDANCY OF `model_type` in [A, B]
outs:
- ./data/predictions/
from dvc.
dberenbaum
commented on June 3, 2024
Unfortunately, I can't think of a good way to do it without creating separate stages/pipelines. If you have some idea of what you would want it to look like, feel free to suggest it here.
from dvc.
EvanKomp
commented on June 3, 2024
Affirmative. Thanks for your work. I think expanding on the yaml like you would with a cache tag would be best. eg.
stages:
preprocess:
cmd: ./prepare.sh
outs:
- ./data/preprocessing/
predict:
cmd ./predict.sh
params:
- model.model_type # One of A, B, C
deps:
- ./data/preprocessing/:
# conditioning syntax
conditions: # these are executable strings with params as local namespace
- 'model.model_type in ["A", "B"]
outs:
- ./data/predictions/
from dvc.
Related Issues (20)
- dvc fetch: hangs forever on "Fetching" HOT 5
- exp run: pygit2 in script does not find active branch and last commit HOT 2
- `dvc repro --pull`: invalidates cache HOT 8
- Dataset delete when using 'dvc add' due to low memory on pc HOT 2
- import: hangs when pulling many files from GCS remote and one fails HOT 2
- bug: running exp with `--queue` always take workspace version for python deps HOT 4
- Feature proposal: `dvc artifacts get --show-url` HOT 1
- dvc.scm.CloneError: SCM error HOT 1
- Config setting studio.url is not respected
- pull: breaks if imported dir ends with "/" HOT 1
- dvc queue status doesn't report active workers
- dvc exp run --run-all: One or two experiments are executed, than it hangs (JSONDecodeError) (similar to #10398)
- dvc metrics diff --all: on same branch is empty HOT 2
- json output is not valid, due to colorized ANSI escape codes HOT 1
- dvc repro: pygit2 update breaking changes HOT 3
- dvc exp apply - fails HOT 3
- plots: Multiple fields result in invalid dvc.yaml HOT 2
- Git credentials does not work on a dvc pull for an imported dvc-git repository HOT 1
- Latest release of DVC breaks object versioned configs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dvc.