Comments (8)
from pipeline.
This would be very useful to have (I was going to suggest this earlier today and so there's a bit more context and background that we might need!):
I'm debugging the build-pipeline example now and I see Tasks, PipelineResources and also the PipelineRun itself. Our pipeline controller performs validation in terms of checking inputs and outputs.
Prior work
Apache Spark has a concept of a directed acylic graph too which defines a bunch of operations that should be run on potentially many worker nodes.
The DAG Execution graph feature, which is available via the Apache Spark web UI, makes it easy to figure out which steps are going to happen after another and makes for an easier understanding of what could be a complex sequence of instructions. Something like this would be perfect
See https://databricks.com/blog/2015/06/22/understanding-your-spark-application-through-visualization.html as well, particularly the Execution DAG image for reference.
A specific problem
I'm seeing:
"message": "PipelineRun default/demo-pipeline-run-1 can't be Run; couldn't resolve all references: unexpected error which should have been caught by Pipeline webhook: pipelineTask tried to use output resource builtImage not present in declared resources",
What I'd want to find out
- So which particular task is it?
- What is looking for that?
- Is the rest of my pipeline run OK?
- Are all references being resolved? Which are and which aren't?
- What depends on what being defined, and in what order?
How could we do it?
-
The command line approach: we'd provide a PipelineRun name, its json will be retrieved and parsed. We'd provide a Docker image or script?
-
Serve a simple webpage (we'd provide yaml that pulls an image, all code under the Knative organisation, perhaps under
ui
: could be a simple Helm chart) and feed in PipelineRuns as json or yaml: I'm assuming a PipelineRun knows all of its own dependencies and the controller responsible for handling it also offers information we could visualize.
We could allow users to pick a PipelineRun (or several) across different namespaces.
We'd then, for each PipelineRun, display all Pipelines and Tasks, all inputs and outputs, and what depends on what.
Some data to play with...
Here's my failed PipelineRun as yaml:
"annotations": {
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"pipeline.knative.dev/v1alpha1\",\"kind\":\"PipelineRun\",\"metadata\":{\"annotations\":{},\"name\":\"demo-pipeline-run-1\",\"namespace\":\"default\"},\"spec\":{\"pipelineRef\":{\"name\":\"demo-pipeline\"},\"resources\":[{\"name\":\"source-repo\",\"resourceRef\":{\"name\":\"skaffold-git\"}},{\"name\":\"web-image\",\"resourceRef\":{\"name\":\"skaffold-image-leeroy-web\"}},{\"name\":\"app-image\",\"resourceRef\":{\"name\":\"skaffold-image-leeroy-app\"}}],\"serviceAccount\":\"default\",\"trigger\":{\"type\":\"manual\"}}}\n"
},
"creationTimestamp": "2019-01-24T14:05:02Z",
"generation": 2,
"name": "demo-pipeline-run-1",
"namespace": "default",
"resourceVersion": "243994",
"selfLink": "/apis/pipeline.knative.dev/v1alpha1/namespaces/default/pipelineruns/demo-pipeline-run-1",
"uid": "0b49222a-1fe1-11e9-9ebd-025000000001"
},
"spec": {
"Status": "",
"generation": 1,
"pipelineRef": {
"name": "demo-pipeline"
},
"resources": [
{
"name": "source-repo",
"resourceRef": {
"name": "skaffold-git"
}
},
{
"name": "web-image",
"resourceRef": {
"name": "skaffold-image-leeroy-web"
}
},
{
"name": "app-image",
"resourceRef": {
"name": "skaffold-image-leeroy-app"
}
}
],
"serviceAccount": "default",
"trigger": {
"type": "manual"
}
},
"status": {
"conditions": [
{
"lastTransitionTime": "2019-01-24T14:05:02Z",
"message": "PipelineRun default/demo-pipeline-run-1 can't be Run; couldn't resolve all references: unexpected error which should have been caught by Pipelinewebhook: pipelineTask tried to use output resource builtImage not present in declared resources",
"reason": "PipelineValidationFailed",
"status": "False",
"type": "Succeeded"
}
]
}
}
Questions
- We'd need to figure out what can be inferred from this (are we providing enough data for such a parser)?
- Do our controller logs tell us more (but I'd rather we not parse those!)?
- Do we need another output vector on our PipelineRun CRD in its own easy to parse format?
from pipeline.
(General CLI Request)
I think it would be great if we could run tasks locally. I think the concept of --inputs-from
from concourse would be interesting to allow the local CLI to grab some values from the remote pipeline to run locally
from pipeline.
@bobcatfish I think we can close this issue and migrate any cli related issues in the new repository yes.
/close
from pipeline.
For an example of what output we'd expect, looking at the example pipelines we would expect that this tool could create an image (e.g. svg) or diagram (e.g. ascii art in the terminal) like the guestbook png.
from pipeline.
/assign
from pipeline.
@vdemeester now that we've got https://github.com/tektoncd/cli I'm thinking we could close this issue, what do you think? :D
from pipeline.
@vdemeester: Closing this issue.
In response to this:
@bobcatfish I think we can close this issue and migrate any cli related issues in the new repository yes.
/close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
from pipeline.
Related Issues (20)
- PipelineRun is canceled prior to its child TaskRuns being canceled. HOT 9
- How to build entry point without SPIRE?
- When performing an upgrade using install.yaml, a pipeline that ran in the past is now running. HOT 1
- The pipeline controller keeps refreshing logs HOT 1
- Error message when PipelineRun is missing parameters required by Pipeline is nonsensical HOT 6
- DryRun Validate should return the mutated Object HOT 2
- Incorrect param type passed to stepaction makes it use default value HOT 2
- Index out of range panic in CheckMissingResultReferences when references are missing
- Controller crashes for certain Pipeline after upgrade HOT 1
- Capture provenance of StepActions in Status HOT 1
- Access `reason` from the execution status of a `taskRun` in `finally`
- Helm chart outdated HOT 3
- Overriding Pipeline timeout does not work HOT 2
- Additional unit tests are needed for Kubernetes native sidecar support HOT 2
- Remote resolvers fail to save ECR cache
- The spec fields of TaskRun and PipelineRun can still be modified during execution
- Allow modify the spec of a PipelineRun when it is in the `PipelineRunPending` state
- A mechanism to remotely obtain runtime parameters of `TaskRun` and `PipelineRun` HOT 4
- Add support for `when` within `matrix.include`. HOT 3
- It is not possible to have the webhook controller _not_ use TLS HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pipeline.