Comments (7)
@Muhammed-Rashid-07 The @pipeline function executes at composition time, and the execution of the pipeline potentially can happen later. This means that when you run a pipeline, zenml first "runs" your pipeline function code and evaluates which steps are connected to which others. After that zenml does not run that code again, so if you have anything dynamic that needs to go between steps, they should be in the step body, not the pipeline body.
from zenml.
I have solved this issue few months back, the problem was I did list subscription in pipeline. In pipeline it is not possible to do it subscription or any operation. If we have to add anything like that do it in steps and import them into the pipeline. I think that's how it works.
@htahir1 just curious, Why is it not possible to do that in pipeline? Can you explain the reason.
from zenml.
same error
did you find a solution?
from zenml.
I think there needs to be more context on the original error by @Muhammed-Rashid-07 to resolve this.. need the step code
from zenml.
from zenml import step
from tensorflow.keras.datasets import mnist # type: ignore
from tensorflow.keras.utils import to_categorical # type: ignore
from typing_extensions import Tuple, Annotated, Any
import numpy as np
@step
def load_data() -> (
Annotated[Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray], "data"]
):
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape((60000, 28, 28, 1))
x_train = x_train.astype("float32") / 255.0
x_test = x_test.reshape((10000, 28, 28, 1))
x_test = x_test.astype("float32") / 255.0
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)
return x_train, y_train, x_test, y_test
now in my pipleine
from zenml import pipeline
from steps.dataloader import load_data
from steps.modelbuilder import build_model
from steps.trainer import train_model
from steps.eval import evaluate_model
@pipeline
def mnist_pipeline() -> None:
x_train, y_train, x_test, y_test = load_data()
throws the following error
TypeError: cannot unpack non-iterable StepArtifact object
from zenml.
I see. I think you are annotating the step wrongly here:
@step
def load_data() -> (
Annotated[Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray], "data"]
):
Your type annotation is saying "My step has one return type and thats a tuple of 4 things"
But rather it should say
"My step has "four" return types and each have their own annotations:
@step
def load_data() -> (
Tuple[Annotated[np.ndarray, "x_train"], Annotated[np.ndarray, "y_train"], Annotated[np.ndarray, "x_test"], Annotated[np.ndarray, "y_test"]]
):
from zenml.
@Muhammed-Rashid-07 was this enough information for you to solve your problem / move on? If so, I'll close out this issue.
from zenml.
Related Issues (20)
- [BUG]: Race condition Bug HOT 3
- [BUG]: Importing `annotations` from `__future__` breaks pipeline compilation
- [BUG]: Rate limiting Vulnerability HOT 1
- [BUG]: Critical Vulnerability related to file access HOT 1
- [BUG]: Unable to open the quickstart colab notebook HOT 2
- [BUG]: Node Selector doesn"t work on job HOT 1
- Integrate `safetensors` for model serialization HOT 4
- [BUG]: Integration Materializers links not working (Documentation) HOT 5
- [BUG]: THIS RESULTED FROM USING ZENML WITH ML FLOW HOT 1
- [BUG]: File priviledge changing timing, potential TOCTOU HOT 3
- Check if Juypter is installed HOT 2
- [BUG]: {{date}} and {{time}} placeholders not being replaced HOT 2
- [BUG]: Cannot use `zenml init` with templates even after installing `zenml[templates]` HOT 3
- Add Argilla annotator stack component
- [BUG]: Unable to determine source root HOT 4
- [BUG]: GreatExpectations materializer not automatically selected
- [BUG]: Logs are not working HOT 1
- [BUG]: Switch to old version dashboard from 0.57.0 zenml-server HOT 4
- [BUG/Feature Request]: Reusing a step overwrites artifact names HOT 1
- [BUG]: "zenml go" broken for me HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zenml.