ceph / pulpito-ng Goto Github PK

Think the sentry icon doesn't look like sentry.

Should look more like this:

At the time of this writing, there are currently 6 high severity vulnerabilities when performing npm install on this repository. We need to fix this!

Current progress:

In teuthology-api you can kill a job by inputting the necessary arguments in http://localhost:8082/docs.

Todo:

• 1. pulpito-ng should be able to select and kill individual job or the entire run. Therefore, you should create a UI component that exists next to each job and a kill-run button that kills all the jobs in a run.
• 2. Incorporate some sort of unit-test to this.

Current progress:

In teuthology-api you can authenticate with github (you have to be a member of ceph organization) which gives you permission to use teuthology-suite and teuthology-kill.

Todo

• 1. pulpito-ng should have a login/logout button displaying username (or icon) on top right corner.
• 2. Basically login button calls the login logout endpoint in teuthology-api.
• 3. Incorporate some sort of unit-test to this.

According to the vite documentation, the environment variables should have the prefix VITE_ otherwise it will not be exposed to import.meta.env.

In the original Pulpito, we can click on rows in job tables (on run pages) to expand them, showing the failure reason. We also have a button that expands all of them at once.

MUI's DataGrid keeps that feature locked behind their paid, non-MIT-licensed version. That's not an option for us, so we'd likely need to use a different component to render tables.

There's a pretty complete comparison of the options here: https://www.material-react-table.com/#let's-compare

That project (material-react-table) might do what we want. It looks like they are inching close to a V2 release, with a very large changeset looming. Once that release is out maybe we should try that.

https://pulpito-ng.ceph.com/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7355972/

dead job icon should be red

This an RFE to add a filter for "user", just like you can filter by suite, machine, etc.

Feature to save teuthology-suite configs by a keyname on scheduling page. Later, they can select a config from a list of saved configs in a Select menu or something similar.
Data can be stored in a db served by teuthology-api.

See latest comment before merge for details:

#24

#16 is a good change as it brings our original icon back, but it would be good to have an SVG version so that it could scale to any size and still look nice.

A couple of issues on /runs page:

Issue 1: when using DataGrid filter feature on "user" column, search for "vallari" (for example) and pulpito-ng would send a request to https://paddles.front.sepia.ceph.com/runs/user/vallari/. That request will fail and the page would go blank.
Expected behaviour: filtering happens on client side and no extra requests are made to paddles.

Issue 2: using DataGrid filter feature on any other column except "user" doesn't work. When trying to filter on any other column (example "runtime"), the filter box's "Columns" field changes back to "user" immediately and does not allow to filter by any other field.

We'll need to make sure we have a production-ready container for this.

In local setup, when there's a dot (".") in the url we get 404 not found.

Example run: vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi

To reproduce, start pulpito-ng locally and go to /runs/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi

Add a new button to rerun failed jobs in a test run. The button would appear on run page like "kill run" button does in #51.

We can use existing /suite endpoint of teuthology-api to request for a rerun.
Pulpito-ng can fill the request payload by using the values from the run page data. Like run name (--run-name), rerun-statuses, etc.

Rerun command example:

teuthology-suite --rerun yuriw-2024-06-18_22:03:07-rbd-wip-yuri8-testing-2024-06-18-0723-distro-default-smithi -p 100 --suite-repo https://github.com/vallariag/ceph.git --suite-branch wip-yuri8-testing-2024-06-18-0723 --sha1 9b63bc2c7e1aad253c89228f5672e68aaea112dc -vv

Refer to docs: https://docs.ceph.com/projects/teuthology/en/latest/commands/teuthology-suite.html

To see how to send requests to teuthology-api from pulpito-ng, refer https://github.com/ceph/pulpito-ng/pull/23/files#diff-5f8be92e87d171bfafe635ee330fd0b888ca0871572c4888425bf5476064dc54R28

Reference: #23 (comment)

In teuthology-api project, add a new endpoint which returns default values for teuthology-suite command options. Refer to teuthology-suite docs https://docs.ceph.com/projects/teuthology/en/latest/commands/teuthology-suite.html for possible default values.
(This data would be used in future to populate default values for pulpito-ng's scheduling UI.)

Reference: #23 (comment)

Current progress:

In teuthology-api you can schedule a job by inputting all the variables to teuthology-suite through http://localhost:8082/docs.

Todo:

• 1. pulpito-ng should have a Schedule job section in the left menu tab. The UI should be able to incorporate all arguments, however we should pick suitable component for each arguments, e.g., flags like --dry-run, --verbose, --force-priority should use radio buttons. Some of the arguments should be hidden by default and user can choose to advance filter if need to change those arguments, e.g., --ceph-repo and --suite-repo should by default be https://github.com/ceph/ceph-ci.git.
• 2. --dry-run should pop up a terminal like view and see all the scheduled jobs similar to terminal teuthology.
• 3. Incorporate some sort of unit-test to this.
• 4. Use https://usehooks-ts.com/react-hook/use-local-storage to store user specific config of the suite keys and values

Because of this PR: #49
we might have to wait?

Job definitely didn't start 3 hours ago.

To reproduce this, just schedule a job and look at the job started about section.

The current logic seems to be (correct me if I'm wrong) dark mode by default. To streamline this option, and if it's not too much overhead, it might be good to have the interface remember which setting the user chose last time they logged in.

I seem to remember the concept of user profiles being introduced in the pulpito-ng demo? This could perhaps be a setting associated with a user's profile.

Compared to the old pulpito, pulpito-ng fonts are too small which is not a good user experience.

Think we also need to add something like https://pulpito.ceph.com/nodes/

with Nodes, Node Lock Stats, Node Job Stats.

We should align the date filter box to other widgets

Starting from e.g. /nodes?machine_type=smithi, if you attempt to use the DataGrid filter feature, you're immediately redirected to /nodes, dropping the machine_type query.

On the Run page, it would be good to have the overall Job Stats as shown in this screenshot (the "Job Stats for this page" line):

On the Job page, it would again be good to see an overview of passed, failed, running, queued, etc. jobs as shown by the dark red and green text boxes.

Example link: https://pulpito-ng.ceph.com/runs/yuriw-2023-08-08_14:45:33-rados-wip-yuri6-testing-2023-08-03-0807-pacific-distro-default-smithi

Took about 4 seconds to load. Not crazy bad, but enough to notice.

I visited https://pulpito-ng.ceph.com/ without being on the Ceph Sepia Lab VPN.
The result was a "spinner" icon and there was not obvious error it just sat there making no progress.
I looked at the webdev console and saw it was trying to access, and fail, urls like https://paddles.front.sepia.ceph.com/runs/?page=1

Based on this I figured that this frontend relies on backend services that only are available when logged into the Sepia Lab VPN. A couple thoughts:

1. Don't expose pulpito-ng.ceph.com to the public internet - require the VPN
2. Show an error that says "You need the Sepia Lab VPN" if many api requests fail
3. Enable a mode for it to work fully without the VPN

Thanks!

We shouldn't use the Ceph image for icon in the tab since this will cause confusion with https://ceph.io/en/ tabs

minor annoyance: if you click on a dropdown, say status, and then decide you don't want to filter that way and click outside the dropdown, it chooses a status anyway (queued); I'd expect only a click on the actual status to do that.

Seeing as we receive so much data about jobs in JobList, we can use it to add useful filter options like filter by task name.
Pros:

1. JobList sends 1 request to paddles and gets all of job data. Pagination happens on client side, and so can these filtering on job data.
2. If there is problem with a task, user can filter jobs which include that task so it's easier to inspect jobs that run that task.

POC:
Filtered 18 jobs (out of total 244 jobs) which have "ceph.healthy" task. It was pretty fast too.
The task column shows "number of total tasks" because I'm not sure what's the appropriate data to show there.

I added this in JobList:

  {
    header: "tasks",
    id: "tasks",
    Cell: ({ row }) => {
      return (
        <div>{Object.keys(row.original.tasks || {}).length}</div>
      );
    },
    accessorFn: (row: Job) => (Object.keys(row.tasks || {})),
    size: 150,
    enableColumnFilter: true,
    filterFn: (row, id, filterValue) => {
      return (row.original.tasks?.some(task => filterValue in task)) || false 
    } 
  },

Use-case: searching for jobs setup with cephadm, we can look for "cephadm.install" task.

These buttons don't really work ...