Comments (2)
Thanks for the enhancement request @j-tr! One question about your setup: do you wait for these deployment runs to complete at any point in your parent flow?
from prefect.
hello @desertaxle, we are usually waiting for the deployment runs to reach a final state.
i think there are at least two usecases:
- the developer wants to start sub deployments asynchronously and returns from the parent flow before waiting for the subflows to finish. In this case, the current behavior is expected. However, group cancellation of all subflows could still be useful.
- the developer is actually looking for sub flow behavior (flow run as python function, not via run_deployment) but the subflow requires a specific infrastructure that is only available via a different work pool. Therefore, it needs to run as a deployment. In order to get as close as possible to the sub-flow behavior, we use a polling mechanism like in https://github.com/PrefectHQ/prefect/blob/main/src/prefect/deployments/deployments.py#L230-L236 at the end of the parent flow to join all subflows and set the return value of the parent flow according to the states of the sub flows. In this case, it makes no sense to continue the subflows when the parent flow fails or crashes.
See #6689 for a related feature request.
from prefect.
Related Issues (20)
- Running airbyte reset connection job via prefect HOT 1
- Refreshing Work Pools page displays blank page HOT 1
- `test_regression_3803_positive_case_no_relevant_event` is flaky HOT 1
- `ECS Worker` flow run crashes on `RunTask` cause of `Some tags contain invalid characters` HOT 7
- Windows - Prefect GitHub `get_directory` does not clone any files to the `local_path`
- "RuntimeError: is bound to a different event loop" when using prefect client from sync flow in kubernetes HOT 2
- "Inspect" CLI commands should support JSON output HOT 3
- After adding a block as a job variable default to a work pool, its base job template does not load on the Edit page unless I resize to a different breakpoint
- Update all DeploymentSchedule schemas to include new fields
- Add an "Automate" option to the Webhooks three-dot menu in the UI
- Add new fields to deployment schedule forms in the UI
- Update the scheduler to use a deployment schedule's max_scheduled_runs if set
- Make Workers aware of schedule concurrency limits
- Start tracking the schedule that generated a flow run with the `CreatedBy` field
- Add documentation for new deployment schedule fields
- Add support for new fields to `prefect deploy` and prefect.yaml
- Add CLI options to relevant `prefect deployment schedule` commands
- Parameters appear and then disappear when clicking them HOT 2
- anyway to install prefect client side only?
- `client/orchestration.py/update_deployment_schedule` should be update with `active` and `schedule` at once
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prefect.