Comments (3)
PEBBLE:
{{workingDir}}
{{outputDir}}
{{bucketPath}}
ENV:
WORKING_DIR
OUTPUT_DIR
BUCKET_PATH
Rename (gcs/s3/azure)Bucket
to bucket
Move uploadInputFiles to CommandsWrapper (with additionalVars precomputed before scriptRunner.run())
Open questions:
- outputDir -> useless ? Since on cloud providers it is set to the same value as workingDir
- in DOCKER & PROCESS runners everything put in {{outputDir}} are put in the outputFiles. However such operation is costly on cloud providers so we don't want to do this behaviour
To add in doc' -> limitation due to glob patterns hard to handle in remote script runners (we can't list easily files from remote buckets). Currently, remote runners only support hardcoded relative paths to file. We could extend it to prefix but not much further
from kestra.
For cloud runner, outputDir
should be created inside the bucket and all files in this path (via a single prefix query) should be downloaded as outputFiles.
from kestra.
These enhancements has been done, I'll close this one.
from kestra.
Related Issues (20)
- Add ability to restrict HTTP methods for io.kestra.plugin.core.trigger.Webhook HOT 2
- Failed to obtain JDBC Connection HOT 2
- Saved files disappear from the editor when using S3 storage HOT 15
- Setting namespacedFiles.enabled to false in io.kestra.plugin.scripts.python.Script task throws NullPointerException exception HOT 3
- New editor keeps the previous opened namespace file even if I create a new flow
- Deprecated error doesn't explain what was replaced for `LocalFiles` and `docker`
- Issue with Escaping Curly Braces in YAML for Kestra HOT 2
- Fail a flow validation if a Pebble expression contains a `-` in a variable name
- io.kestra.plugin.core.flow.WaitFor: incorrect behaviour when using output of previous task
- Issue with Escaping Curly Braces in YAML for Kestra HOT 3
- Support for secret files HOT 3
- Kestra will go to > 80% CPU and 100% ram if secret in webhook trigger HOT 5
- Image URL with HTTP Request type causes Kestra to crash HOT 3
- Preview PDF files in Outputs Tab HOT 1
- Python dependency caching not working with TTL on S3 storage
- [EE] Refresh the auth token/cookie automatically if the user is still active HOT 3
- Docker Task Runner Icon not displaying HOT 1
- [UI] Allow importing multiple files and folders in the Namespace Files editor HOT 1
- OSS Preview build issue - Failed to load resource: the server responded with a status of 404 ‒ /api/v1/namespaces/company.myteam/files/directory:1 HOT 1
- [UI] Add the Inputs/Outputs Preview functionality for PDF files — for now, kestra displays them as binary objects
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kestra.