One should be able to delete builds, push a new build request right after the one in progress, ...
without having to stop the bot.

First of all thank you very much for this repo.
I'm trying to build something like IREE's benchmark dashboard but I'm struggling with the demo.
I was able to spot some bugs after lots of debugging (not a node expert):

• sessionSecret should be set to a string in configs/dana.js
• adminUser should be a list in configs/dana.js ? because that's how it's used in src/server.js

• Web socket client only works after uncommenting some code in src/server.js.
• Now I'm at a point where server crashes after I run pullDummyData.js, which creates empty folders and files in configs/db/Test/series.

Can someone from the IREE team share with me a minimal functioning example/demo I would be very grateful.

This will avoid unexpected behavior of bot queue.

Add a page to add/delete/edit annotations on builds that will be show on all annotations.

On the series detail page extend the comment to be able to add an annotation on a build id.

Add on the state view the ability to select multiple rows and do some actions:

• change state for all of them and ask for a comment/build annotation that will be added on all series
• set bug link for all of them

This will allow to invoke a script to process the produced build file.

When series are similar, we should add a way to reduce automatically the number of samples involved in the regression computation. We could adjust automatically the base to keep a maximum of the last 50 or 25 builds in that case?

The regression computation will not be impacted and the computation time of the regression analysis will be better.

Can you tag a release, so that I can make a conda-forge recipe for installing dana with conda? Thanks!

For example one could be interested in looking for all series that matches the regex foo.bar and see how the go on all platforms, with a real time refresh as soon as new data are pushed the the dashboard by the bots.

The page could look a bit like that:

Search: foo.bar

Each newly received build would then be stacked at line 1.

For now a compare is attached to one particular projectId. When a new sample is added in this projectId, the compares are applied. The way it works is that for each serieId that exists in the compare project, the comparison is applied. So we can currently only compare series that have the same serieId.

We have a need to compare series with different serieIds. For example compare "buildTime.linux" with "buildTime.mac". To do this

• we will extend compares to enable a string replacement for a serieId. For example: replace 'linux' with 'mac'.
• we will add a property 'replace in a compare: { searchValue : "linux", newValue:"mac" }, add the logic in analyse.js module to support it and extends the UI on compare management.

The following API, i have used to add value against the single "BuildId" and single "SerieId".

curl -d '{"projectId":"Test","serieId":"serie.dummy.11","sample":{"buildId":1000,"value":748}}'
-H "Content-Type: application/json"
-X POST http://dana.myserver.com/apis/addSample'

• But i am able to add just single sample value against the single "BuildId", when i tries to add another value but it displays an error " serieId already exist, use override flag to override content".

Expected:

• I should be able to add multiple sample value against the single "SerieId".

In statusSeries page, introduce a selector to choose between having a graph showing

• each series starting from its base
• each series according to the last 50 builds (nice to see if some series are missing some builds)

The Dana bot UI allows to see the different steps that were executed for a particular build. For each step we can see:

• if it succeeded or failed
• its options
• the logs (in case it failed)

In addition, it would be nice to also display the duration of each step. It could help catching unexpected slow tasks, get some time stats, ...

When starting the bot with many commits (> 20 for example), the bot queue lists tasks in an unsorted order.

In my case, I see:
2 task1
21 task2
4 task1
5 task2
6 task1
...
10 task1
11 task2
3 task2
...

If a test is failing in triage
-> select confirmed and propose to ad a bug link with a bug report
-> select Fixed and set the analysis base to the last passing

When looking at the bot status page, it would be nice to show active tasks before inactive tasks.

One task should have only one task in the queue, and the base should be updated only when the task is done.
This will avoid to loose some tasks when the bot is shutdown.

Currently the queue takes all builds from one task before beginning with the next.
So if one wants to play many builds on the same repo for multiple tasks, it can take long before results can be compared.

In dana-bot/src/bot.js, the globalBot.limitPatches is the maximum number of CLs to put in the queue when browsing the git log from ToT (HEAD) to the task's base. It is set to 10.

It means that if there are 100 commits between the ToT and the task's base, only the 10 most recent commits will be pushed into the queue (ToT->ToT~9). The 90 other commits (starting from the base) won't be executed.

It would be better to configure this value in dana-bot/configs/bot.js. This way, each bot can set it limit.

Dana only accept single sample value against single "BuildId" . I want to add multiple sample values against a particular "BuildId" , is it possible?
@jillemash @google-admin

The module dana-bot/src/run.js should check that the given working directory (cwd) is valid before executing a command.