This task is to propose a design and also create a POC tool that demonstrates that des

<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.github

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

For an example of what output we'd expect, looking at <a href="https://github.com/knat

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Design + POC: CLI tool for visualizing pipeline about pipeline HOT 8 CLOSED

tektoncd commented on August 22, 2024

Design + POC: CLI tool for visualizing pipeline

from pipeline.

Comments (8)

bobcatfish commented on August 22, 2024 3

from pipeline.

a-roberts commented on August 22, 2024 1

This would be very useful to have (I was going to suggest this earlier today and so there's a bit more context and background that we might need!):

I'm debugging the build-pipeline example now and I see Tasks, PipelineResources and also the PipelineRun itself. Our pipeline controller performs validation in terms of checking inputs and outputs.

Prior work
Apache Spark has a concept of a directed acylic graph too which defines a bunch of operations that should be run on potentially many worker nodes.

The DAG Execution graph feature, which is available via the Apache Spark web UI, makes it easy to figure out which steps are going to happen after another and makes for an easier understanding of what could be a complex sequence of instructions. Something like this would be perfect

See https://databricks.com/blog/2015/06/22/understanding-your-spark-application-through-visualization.html as well, particularly the Execution DAG image for reference.

A specific problem

I'm seeing:

                "message": "PipelineRun default/demo-pipeline-run-1 can't be Run; couldn't resolve all references: unexpected error which should have been caught by Pipeline webhook: pipelineTask tried to use output resource builtImage not present in declared resources",

What I'd want to find out

So which particular task is it?
What is looking for that?
Is the rest of my pipeline run OK?
Are all references being resolved? Which are and which aren't?
What depends on what being defined, and in what order?

How could we do it?

The command line approach: we'd provide a PipelineRun name, its json will be retrieved and parsed. We'd provide a Docker image or script?
Serve a simple webpage (we'd provide yaml that pulls an image, all code under the Knative organisation, perhaps under ui: could be a simple Helm chart) and feed in PipelineRuns as json or yaml: I'm assuming a PipelineRun knows all of its own dependencies and the controller responsible for handling it also offers information we could visualize.

We could allow users to pick a PipelineRun (or several) across different namespaces.

We'd then, for each PipelineRun, display all Pipelines and Tasks, all inputs and outputs, and what depends on what.

Some data to play with...

Here's my failed PipelineRun as yaml:

                "annotations": {
                    "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"pipeline.knative.dev/v1alpha1\",\"kind\":\"PipelineRun\",\"metadata\":{\"annotations\":{},\"name\":\"demo-pipeline-run-1\",\"namespace\":\"default\"},\"spec\":{\"pipelineRef\":{\"name\":\"demo-pipeline\"},\"resources\":[{\"name\":\"source-repo\",\"resourceRef\":{\"name\":\"skaffold-git\"}},{\"name\":\"web-image\",\"resourceRef\":{\"name\":\"skaffold-image-leeroy-web\"}},{\"name\":\"app-image\",\"resourceRef\":{\"name\":\"skaffold-image-leeroy-app\"}}],\"serviceAccount\":\"default\",\"trigger\":{\"type\":\"manual\"}}}\n"
                },
                "creationTimestamp": "2019-01-24T14:05:02Z",
                "generation": 2,
                "name": "demo-pipeline-run-1",
                "namespace": "default",
                "resourceVersion": "243994",
                "selfLink": "/apis/pipeline.knative.dev/v1alpha1/namespaces/default/pipelineruns/demo-pipeline-run-1",
                "uid": "0b49222a-1fe1-11e9-9ebd-025000000001"
            },
            "spec": {
                "Status": "",
                "generation": 1,
                "pipelineRef": {
                    "name": "demo-pipeline"
                },
                "resources": [
                    {
                        "name": "source-repo",
                        "resourceRef": {
                            "name": "skaffold-git"
                        }
                    },
                    {
                        "name": "web-image",
                        "resourceRef": {
                            "name": "skaffold-image-leeroy-web"
                        }
                    },
                    {
                        "name": "app-image",
                        "resourceRef": {
                            "name": "skaffold-image-leeroy-app"
                        }
                    }
                ],
                "serviceAccount": "default",
                "trigger": {
                    "type": "manual"
                }
            },
            "status": {
                "conditions": [
                    {
                        "lastTransitionTime": "2019-01-24T14:05:02Z",
                        "message": "PipelineRun default/demo-pipeline-run-1 can't be Run; couldn't resolve all references: unexpected error which should have been caught by Pipelinewebhook: pipelineTask tried to use output resource builtImage not present in declared resources",
                        "reason": "PipelineValidationFailed",
                        "status": "False",
                        "type": "Succeeded"
                    }
                ]
            }
        }

Questions

We'd need to figure out what can be inferred from this (are we providing enough data for such a parser)?
Do our controller logs tell us more (but I'd rather we not parse those!)?
Do we need another output vector on our PipelineRun CRD in its own easy to parse format?

from pipeline.

deedubs commented on August 22, 2024 1

(General CLI Request)

I think it would be great if we could run tasks locally. I think the concept of --inputs-from from concourse would be interesting to allow the local CLI to grab some values from the remote pipeline to run locally

from pipeline.

vdemeester commented on August 22, 2024 1

@bobcatfish I think we can close this issue and migrate any cli related issues in the new repository yes.
/close

from pipeline.

bobcatfish commented on August 22, 2024

For an example of what output we'd expect, looking at the example pipelines we would expect that this tool could create an image (e.g. svg) or diagram (e.g. ascii art in the terminal) like the guestbook png.

from pipeline.

vdemeester commented on August 22, 2024

/assign

from pipeline.

bobcatfish commented on August 22, 2024

@vdemeester now that we've got https://github.com/tektoncd/cli I'm thinking we could close this issue, what do you think? :D

from pipeline.

tekton-robot commented on August 22, 2024

@vdemeester: Closing this issue.

In response to this:

@bobcatfish I think we can close this issue and migrate any cli related issues in the new repository yes.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

from pipeline.

Design + POC: CLI tool for visualizing pipeline about pipeline HOT 8 CLOSED

Comments (8)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent