[BUG] Tasks from subworkflow calling reference launch plan read cache from different projects about flyte HOT 2 OPEN

The issue you're describing seems to be related to how Flyte handles caching across different projects when using reference launch plans. According to the Flyte documentation on remote caching, cache keys are composed of the Project, Domain, Cache Version, Task Signature, and Task Input Values. This design ensures that task executions are isolated by project and domain to prevent data corruption.

However, your observation indicates that the cache key is being derived from the reference launch plan's identifier, which includes the project, version, domain, and name of the launch plan. This could lead to cache hits across different projects, which is not the intended behavior.

Expected Behavior

Cache keys should be specific to the project and domain of the current task execution, not the reference launch plan. This would ensure that tasks do not inadvertently read cached results from executions in other projects.

Rationale

The current design might be intended to facilitate reusability and efficiency, but it compromises the isolation between projects. Changing the cache key derivation to include the current task's project and domain would align with the documented behavior and ensure data integrity.

Would you like more details on how to potentially address this issue or any specific part of the caching mechanism?

Sources:

• Flyte Remote Caching
• Flyte Issue #3169

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

from flyte.

@ysysys3074 , this is working as intended. This has deep implications about how Flyte entities (e.g. tasks and launch plans) are registered in the backend and how we refer to them. In the specific case of reference entities, they are tied to their original project+domain+name+version.

from flyte.