Comments (6)
To support native JSON types in Flyte, you propose introducing a new LiteralType.JSON
in FlyteIDL, using binary/bytearray for transport, and allowing custom JSON serializers/deserializers. This approach aims to address issues with current JSON handling via protobuf Struct, particularly with integer representation.
Would you like more detail on how to implement this feature or any potential challenges?
Sources:
This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.
Mention @RunLLM to ask follow-up questions. See runllm.com for more!
from flyte.
Related to #4505
from flyte.
#take
from flyte.
From #4505 (comment), I see the follow Python types that can use Literal.JSON
:
dicts
->JsonPickle
dataclasses
-> UseMashumuro's
encoders/decorders or the object'sto_json
+from_json
(These methods are defined withDataClassJSONMixin
)
@kumare3 Are you considering other Python types that would use Literal.JSON
?
from flyte.
I am saying wholesale migration to json for these unsupported types from struct
from flyte.
from flyte.
Related Issues (20)
- [Housekeeping] Add support for protobuf version 5 in flytekit HOT 1
- [BUG] When triggering a remote LP, Flytekit fails with TypeError: 'NoneType' object is not subscriptable HOT 5
- [BUG] python task retries=n causes "Requests overridden" error log message for interruptible task HOT 1
- [BUG] Handler for .well-known/openid-configuration constructs redirect path incorrectly HOT 2
- [Core feature] LiteralBlob and StructuredDataset metadata HOT 1
- More flexible configuration of SecurityContext for Pods/Containers started by flyte HOT 8
- [Core feature] `@dynamic` should accept all (?) of `@workflow` attributes HOT 2
- [BUG] Tasks from subworkflow calling reference launch plan read cache from different projects HOT 2
- [BUG] New versions of viper breaks config loading HOT 3
- [BUG] nested dynamic won't bind pydantic models or dictionaries as inputs HOT 2
- [Core feature] Build multiple ImageSpec in parallel HOT 2
- [Housekeeping] Distributed Tracing Should Support OTLP Exporters HOT 1
- [Integration] NIM
- [BUG] ArrayNodes downloads all inputs for every subtasks HOT 2
- Flytekit checkpoint improvement- pytorch HOT 1
- [BUG] Union types fail for e.g. two different dataclasses HOT 4
- [BUG] Missing inputs when using datetime.date type hint. HOT 1
- [Core feature] UX improvement: `pyflyte run` includes imported local modules in the target workflow file HOT 2
- [Housekeeping] Remove the need of kwtypes in user code HOT 1
- [Core feature] pyflyte run --remote should support a url HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flyte.