[BUG] `Promise`s are infinite Iterators about flyte HOT 3 CLOSED

Thank you for opening your first issue here! 🛠

from flyte.

The behavior you're encountering with Promise objects in Flyte is due to their design to act as placeholders for values that are not yet computed, rather than iterable objects. The issue you described is similar to a known problem where iterating over a Promise does not behave as expected because Python's __iter__ method is called, which can lead to infinite loops or memory issues if not handled correctly. This is documented in a related Flyte issue where iterating over a list of FlyteFile objects within a dynamic task caused infinite loops and memory growth (Flyte issue #5164).

Promise objects are not designed to be directly iterable. The correct approach is to ensure that the task outputs are fully materialized before attempting operations like extending a list. This can be managed by structuring your workflows and tasks to handle dependencies and outputs appropriately, ensuring that Promise objects are resolved before such operations.

Would you like a more detailed explanation on how to structure your Flyte tasks and workflows to avoid such issues?

Sources:

• Flyte issue #5164

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

from flyte.

Reading through linked issue, I see that this issue was closed by flyteorg/flytekit#2337 already. We just need to update our Flyte version.

from flyte.