ocurrent / current-bench Goto Github PK
View Code? Open in Web Editor NEWExperimental benchmarking infrastructure using OCurrent pipelines
License: Apache License 2.0
Experimental benchmarking infrastructure using OCurrent pipelines
License: Apache License 2.0
Create a POC implementation of a Jupiter notebook fetching the data from our benchmarking backend.
When you click on the benchmark page (from a PR status) you land on a page which shows you outdated results without any clear indication that the benchmark result are still being computed. It's probably better to show either nothing or partial progress (if that's possible), or just the logs currently being produced by the benchmark.
By default benchmark_name
is set to NULL
. We do this even if there are multiple objects produced by make bench
. This most certainly is a mistake and we should require the output to have the benchmark_name
field with multiple benchmark results.
Sometimes changes to benchmarks might be made but since the data is immutable, it shows up as NA. If we load smaller data from the database, this problem of displaying dead benchmarks will go away.
Index benchmarks introduced a change and the PR shows the data for the dead metric as well.
For some of the repos, decompress for example, the benchmarks use a randomly generated file using cat as the input, so current-bench currently fails because the pipeline can handle only json. If we could parse json from the pread log in the pipeline, we could handle extraneous data better.
Multicore OCaml Sandmark environment uses bwrap
and we need to use the --privileged
option with Docker run for the same in pipeline/lib/pipeline.ml
:
- [ "--security-opt"; "seccomp=./aslr_seccomp.json" ]
+ [ "--security-opt"; "seccomp=./aslr_seccomp.json"; "--privileged" ]
Otherwise, we get a Operation not permitted
error.
Is it possible to change the way the raw results are plotted such that they always include 0 on the y-axis (that is [0, y_max+delta]
)? (see screenshot at end)
Right now you have them plotted as [y_min-delta, y_max+delta]
for each run which makes the graphs not easy to interpret across a large number of benchmarks, the reader has to inspect the axis scale because the shape might not be significant. For example the first time_secs plot has a range of [101.9, 102.1]
that is 102 +/- 0.1%
but visually the changes looks proportionally more significant because of the zoomed y-axis. Also when comparing graphs, we could have a graph with range [91, 101]
that could have the same visual shape as [101.9, 102.1]
but the two could have very different implications.
Iโm open to other scaling proposals that also keep things easy to compare: e.g. display as normalised deviation from median/mean with all axes for the plots on a common y-scale. However these other scaling proposals might not be as easy to implement as just making 0
the minimum for all our positive valued metrics.
@shakthimaan requested I file the issue here
Since all the graphs are displayed in the same way right now it could be useful with support for a description of each graph that is displayed below the title in the UI. It could then be possible to include additional information, such as a short explanation of what the graph shows, higher/lower is better etc.
The "ocaml/opam" Docker image name is hard-coded in pipeline/lib/pipeline.ml
. Kindly abstract it out to an environment variable, so that Multicore OCaml can specify ocaml/opam:ubuntu-20.10-ocaml-4.10
.
let dockerfile =
let+ base = Docker.pull ~schedule:weekly "ocaml/opam" in
`Contents (dockerfile ~base)
in
Similar to how it's done in #91
The following request from CommitInfo.res, line 3 is failing:
query ($repoId: String!, $pullNumber: Int, $isMaster: Boolean!) {
lastCommitInfo: benchmarks(
limit: 1
where: {_and: [{pull_number: {_eq: $pullNumber}}, {pull_number: {_is_null: $isMaster}}, {repo_id: {_eq: $repoId}}]}
order_by: [{run_at: desc_nulls_last}]
) {
run_at
pull_number
branch
commit
build_job_id
run_job_id
__typename
}
}
It gives the error:
{
"errors": [
{
"extensions": {
"path": "$.selectionSet.benchmarks.args.where._and[0].pull_number._eq",
"code": "validation-failed"
},
"message": "expected a 32-bit integer for type \"Int\", but found null"
}
]
}
I'm not proficient enough in GraphQL yet to know how to fix this, so for now I'll just monkey patch my data to avoid this case.
Currently, the docker DSL specifies the steps needed to run benchmarks, but for different repositories the packages might differ, someone repositories might need pinning some packages for the benchmarks to work, hence a configurable dockerfile which can be plugged into the pipeline will be useful.
And decide what to do with the repeated entries.
Suggested by @MagnusS
This will allow users to specify benchmarks-only dependencies in opam files:
$ OPAMVAR_bench=1 opam install --deps-only .
depends: [
"logs" {bench}
]
The current alternative approach is adding a separate opam file just for benchmark dependencies.
It would be great to have a label to explain what the Y-axis actually is.
Some benchmarks will require external datasets to work. If the dataset is large it might not be feasible to store it in the git repo of the project. In this case, before running the benchmarks, the data needs to be fetched.
A few notes:
For Multicore OCaml, we provision our hardware with specific configuration, and will re-run the experiments with different settings for compiler analysis. For such cases, we would like to use an ETL tool to just load the benchmark results and use the PostgreSQL and frontend Docker containers only for visualization.
When clicking on a commit id in a graph, this points to a truncated url (which I think works fine because GH support this but would be nice to have the full hash).
But the main issue is that it always point to https://github.com/mirage/index
, e.g. clicking on a commit ID on:
http://autumn.ocamllabs.io/mirage/irmin/pull/1312
leads to:
Every few weeks, autumn runs out of memory and all PRs start failing with Docker exit status 99
. One solution would also be to ensure that the images are on /data and periodically clean them up.
The status 99 is from here:
Fatal error:
opam: "mkdir" failed on /home/opam/.opam/repo/origin: No space left on device
The command '/bin/sh -c opam remote add origin https://opam.ocaml.org' returned a non-zero code: 99
Some benchmarks might run faster than others and it might be useful for the pipeline to report the Quick
benchmarks and report that and run Slow
benchmarks in the background as well.
From #74,
Related to the multi-value datapoints: it would be good to show min/avg/max in the tooltip.
The docker image produced by current-bench to run the benchmarks might not be using an up to date opam repository.
This can be causing errors during package installation and terminating the benchmark execution with "Docker build exited with status 20".
I think the base ocaml-ci images are tracking opam-repository and trigger a rebuild whenever a new version of a dependency is merged in opam-repository. Would be nice if ocaml-bench could do something similar?
See title.
This is a known issue: danvk/dygraphs#506
For new projects that start using current-bench, we might want to have the ability to compute benchmarks for old commits.
Of course, this could result in significant resource usage and should not be done by default.
@tatchi :
When I started the frontend, I was a bit surprised to actually see data from http://autumn.ocamllabs.io. That would be nice to grab the backend URL from env (it's currently hardcoded in the code). Vite allows reading .env files but from what I understand, they should be located at the root of the project (within frontend/ in our case). Maybe we could copy (and rename) the env file e.g development.env within the frontend folder during one of the docker steps? Not sure though if that's a good idea ๐
Another (very simplistic) option is to decide the backend URI based on the the current window location in the app. If it's localhost
, use localhost
, if it's not, use the production URI.
We need to complete the frontend changes for #17.
This is my suggestion on what needs to be done:
When the db is not correctly set up or has an incorrect schema applied, the result insertion will fail with an error similar to the following:
pipeline_1 | Result status PGRES_FATAL_ERROR unexpected (expected status:PGRES_COMMAND_OK); ERROR: relation "benchmarks" does not exist
pipeline_1 | LINE 3: benchmarks(run_at, duration, repo_id, commit, branch, pull...
pipeline_1 | ^
At the same time the job is marked as successful and the PR is marked as "passed".
This is not a benchmark execution problem, but at the very least we should mark the result as undetermined.
When the support for in-progress benchmarks is added, it should be easier to report this kind of errors.
The data from last 50 commits would be a good number, more than that clutters the graph.
This could include different compiler versions and variants:
We're benchmarking Tezos on a Raspberry Pi for a number of reasons. It would be great if there was a notion of "remote runners" - install some software on the Pi that allows it to run the benchmarks, while the rest is running in the cloud somewhere.
Of course, if we go too far in this direction we've recreated other full-fledged CI systems out there (Jenkins, Circle CI, etc).
Thoughts?
Hello,
current-bench fails to start for me.
I'm on Windows 10, running Docker version 20.10.5.
The contents of ./environments/development.env are:
OCAML_BENCH_DOCKER_CPU=1
OCAML_BENCH_DB_PASSWORD=docker
OCAML_BENCH_GRAPHQL_KEY=secret
OCAML_BENCH_TARGET_ARCH=amd64
I run make start-development
and get the following output (errors way down at bottom):
PS C:\Users\d4hin\repos\current-bench> make start-development
docker-compose \
--project-name="current-bench" \
--file=./environments/development.docker-compose.yaml \
--env-file=./development.env \
up \
--remove-orphans \
--build
Building db
failed to get console mode for stdout: The handle is invalid.
[+] Building 11.2s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 151B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/postgres:12 1.7s
=> [internal] load build context 0.0s
=> => transferring context: 87B 0.0s
=> [1/2] FROM docker.io/library/postgres:12@sha256:2f3f78532c9cc5435d1cf 9.2s
=> => resolve docker.io/library/postgres:12@sha256:2f3f78532c9cc5435d1cf 0.0s
=> => sha256:2f3f78532c9cc5435d1cf9c8f5fb1409f9abd43e5b7 1.86kB / 1.86kB 0.0s
=> => sha256:75646c2fb4101d306585c9b106be1dfa7d82720ba 27.14MB / 27.14MB 1.0s
=> => sha256:7e98825f6d67159fc035c0db985619c0635cc7edb89 1.80kB / 1.80kB 0.3s
=> => sha256:6ef141d922bc14123d24ddec34aa11d0f360702a2 10.63kB / 10.63kB 0.0s
=> => sha256:4217f50ff2ca28b7c3dddb25372379c97093df532c7 3.25kB / 3.25kB 0.0s
=> => sha256:2355d0ffeb5531909b8910928bcbaa874e9464657c7 4.18MB / 4.18MB 0.4s
=> => sha256:cfd3ce06be45a467d795e78a52a17330a56a7b85a3b 1.42MB / 1.42MB 0.5s
=> => sha256:c7b7bb83e8f78baa3e4e402b1704b6df53db00bec8b 7.97MB / 7.97MB 0.9s
=> => sha256:c6786930510827ff4067b1103131548886adcd7 391.17kB / 391.17kB 0.6s
=> => sha256:19614baa7ddd150dc7115b541a85cec0e8178fc9c6e62c8 149B / 149B 0.6s
=> => sha256:af508737d8137d59629c343294044cb7f854220431d 3.05kB / 3.05kB 0.7s
=> => sha256:562171e5e21c288ecc3ae718a64e16c2f17902deb 73.34MB / 73.34MB 6.1s
=> => sha256:9a1da7929f851b2f7db64c04a23cf5b2c7377b0db64 9.02kB / 9.02kB 1.0s
=> => extracting sha256:75646c2fb4101d306585c9b106be1dfa7d82720baabe1c75 1.3s
=> => sha256:83302896f1018b739b0cdcdd817cbc82c823cf8651cffd6 200B / 200B 1.2s
=> => sha256:1f2e5b2f6e536af3111e90491c622a3bd93179b70459f09 129B / 129B 1.2s
=> => sha256:8892c210e4afc8a8f8ece7cd0bcccfee3b234dd09b6 4.40kB / 4.40kB 1.4s
=> => sha256:46ddd2f5c65450c2987d3d2526b0a0b7d6f1655d312fb31 120B / 120B 1.4s
=> => extracting sha256:2355d0ffeb5531909b8910928bcbaa874e9464657c71213a 0.2s
=> => extracting sha256:7e98825f6d67159fc035c0db985619c0635cc7edb89aee89 0.0s
=> => extracting sha256:cfd3ce06be45a467d795e78a52a17330a56a7b85a3b3a25b 0.1s
=> => extracting sha256:c7b7bb83e8f78baa3e4e402b1704b6df53db00bec8b57266 0.3s
=> => extracting sha256:c6786930510827ff4067b1103131548886adcd7a25a012ba 0.1s
=> => extracting sha256:19614baa7ddd150dc7115b541a85cec0e8178fc9c6e62c82 0.0s
=> => extracting sha256:af508737d8137d59629c343294044cb7f854220431d23fc4 0.0s
=> => extracting sha256:562171e5e21c288ecc3ae718a64e16c2f17902deb1011cad 2.5s
=> => extracting sha256:9a1da7929f851b2f7db64c04a23cf5b2c7377b0db64b3b67 0.0s
=> => extracting sha256:1f2e5b2f6e536af3111e90491c622a3bd93179b70459f098 0.0s
=> => extracting sha256:83302896f1018b739b0cdcdd817cbc82c823cf8651cffd68 0.0s
=> => extracting sha256:8892c210e4afc8a8f8ece7cd0bcccfee3b234dd09b6b2637 0.0s
=> => extracting sha256:46ddd2f5c65450c2987d3d2526b0a0b7d6f1655d312fb310 0.0s
=> [2/2] COPY ./init.sql /docker-entrypoint-initdb.d/init.sql 0.1s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:65222942325e8b01767e984270bbcbc7150b551d35577 0.0s
=> => naming to docker.io/library/current-bench_db 0.0s
Successfully built 65222942325e8b01767e984270bbcbc7150b551d35577e19aa4f3177c8caa3e6
Pulling graphql-engine (hasura/graphql-engine:v1.3.3)...
v1.3.3: Pulling from hasura/graphql-engine
fcf970b8ac66: Pull complete
Digest: sha256:814e36b355177046195edd795ca561fe762a2ae117129f185a26f9d3f609efb9
Status: Downloaded newer image for hasura/graphql-engine:v1.3.3
Building frontend
failed to get console mode for stdout: The handle is invalid.
[+] Building 42.9s (17/17) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 554B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 52B 0.0s
=> [internal] load metadata for docker.io/library/nginx:stable-alpine 0.7s
=> [internal] load metadata for docker.io/library/node:lts 0.8s
=> [builder 1/7] FROM docker.io/library/node:lts@sha256:f6b9ff4caca9d4f 18.1s
=> => resolve docker.io/library/node:lts@sha256:f6b9ff4caca9d4f0a331a882 0.0s
=> => sha256:abea835c0b3b0ef244d92a1076602ed0891a35b1b77 7.83kB / 7.83kB 0.0s
=> => sha256:da61ad49fa9961d6dfe53dc067fd690133f205498 11.29MB / 11.29MB 0.7s
=> => sha256:af62e04c0f85ed10801df4314d8cb9ba5e6c391f6a3 4.34MB / 4.34MB 0.5s
=> => sha256:f6b9ff4caca9d4f0a331a882e560df242eb332b7bbbed2f 776B / 776B 0.0s
=> => sha256:61e6ae4231ac78eb737a10e70d8925e92e7a35ec0c3 2.21kB / 2.21kB 0.0s
=> => sha256:00168f89dbe8f3c9985e536784c27517f6cc35ea5 45.38MB / 45.38MB 2.9s
=> => sha256:aae835e2c68ef1d48a4ffef72547a79926019aec6 49.79MB / 49.79MB 3.6s
=> => sha256:5254dc317968c943c4321cdbf0364a6a47c8332 214.35MB / 214.35MB 8.3s
=> => sha256:b3f76fdfa8e8784238b141aaa54fa1862aef5348665 4.19kB / 4.19kB 2.9s
=> => extracting sha256:00168f89dbe8f3c9985e536784c27517f6cc35ea56263469 1.9s
=> => sha256:5681b30a17f656444f1bd37c07ee6f594c7e2e26c 34.58MB / 34.58MB 5.5s
=> => sha256:da9a613e2daeaae78b9cd247e52f3f731b701730d7a 2.38MB / 2.38MB 3.9s
=> => sha256:67891041a0a2d985480f4934fc94bc0b180f9f27741bd5c 292B / 292B 4.1s
=> => extracting sha256:da61ad49fa9961d6dfe53dc067fd690133f205498e8045e8 0.4s
=> => extracting sha256:af62e04c0f85ed10801df4314d8cb9ba5e6c391f6a3c1db1 0.2s
=> => extracting sha256:aae835e2c68ef1d48a4ffef72547a79926019aec6a2bbeb5 2.4s
=> => extracting sha256:5254dc317968c943c4321cdbf0364a6a47c833296b3f45e3 7.6s
=> => extracting sha256:b3f76fdfa8e8784238b141aaa54fa1862aef534866543116 0.0s
=> => extracting sha256:5681b30a17f656444f1bd37c07ee6f594c7e2e26c4ec796e 1.6s
=> => extracting sha256:da9a613e2daeaae78b9cd247e52f3f731b701730d7ab8bf6 0.1s
=> => extracting sha256:67891041a0a2d985480f4934fc94bc0b180f9f27741bd5c2 0.0s
=> [stage-1 1/4] FROM docker.io/library/nginx:stable-alpine@sha256:ec3f6 5.7s
=> => resolve docker.io/library/nginx:stable-alpine@sha256:ec3f6e4587a20 0.0s
=> => sha256:ec3f6e4587a2053f406b1b4e027c260e82e4e32c256 1.65kB / 1.65kB 0.0s
=> => sha256:7409b874fed478250a841bc67aa1451df7163543cf6 1.36kB / 1.36kB 0.0s
=> => sha256:d0d03989cd8e044493922e446203a9893eb662a3279 7.98kB / 7.98kB 0.0s
=> => sha256:9b794450f7b6db7c944ba1f4161edb68cb535052fe7 2.82MB / 2.82MB 4.4s
=> => extracting sha256:9b794450f7b6db7c944ba1f4161edb68cb535052fe7db8ac 0.1s
=> => sha256:71c851db865893ed287bbecc2934e756a25e4e78bab 6.67MB / 6.67MB 4.9s
=> => extracting sha256:71c851db865893ed287bbecc2934e756a25e4e78babcd419 0.3s
=> => sha256:fc9aed49b78e299c67c0ee890e26326969df6c564fda2b2 902B / 902B 5.0s
=> => sha256:bb846ca9b45ab911d714856abaca670ed6a798711e73325 665B / 665B 5.1s
=> => sha256:281095db834a17364db464c05304d89414f2ab118283136 600B / 600B 5.0s
=> => extracting sha256:281095db834a17364db464c05304d89414f2ab118283136b 0.0s
=> => extracting sha256:fc9aed49b78e299c67c0ee890e26326969df6c564fda2b28 0.0s
=> => extracting sha256:bb846ca9b45ab911d714856abaca670ed6a798711e73325d 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 332.87kB 0.0s
=> [stage-1 2/4] COPY ./nginx.conf /etc/nginx/nginx.conf 0.2s
=> [stage-1 3/4] RUN rm -rf /usr/share/nginx/html/* 0.3s
=> [builder 2/7] WORKDIR /app 0.4s
=> [builder 3/7] COPY package.json yarn.lock /app/ 0.0s
=> [builder 4/7] RUN yarn install 14.7s
=> [builder 5/7] COPY . /app/ 0.0s
=> [builder 6/7] RUN yarn build 2.1s
=> [builder 7/7] RUN yarn bundle 6.5s
=> [stage-1 4/4] COPY --from=builder /app/dist /usr/share/nginx/html 0.0s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:c536e74e0967b7344ff7988a302a7ae2068e7bdfcd00f 0.0s
=> => naming to docker.io/library/current-bench_frontend 0.0s
Successfully built c536e74e0967b7344ff7988a302a7ae2068e7bdfcd00fa74233b8a508ada4d16
Building pipeline
failed to get console mode for stdout: The handle is invalid.
[+] Building 170.7s (22/22) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.78kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/debian:stable-slim 0.9s
=> [internal] load metadata for docker.io/ocaml/opam:debian-ocaml-4.12 0.9s
=> [internal] load build context 0.0s
=> => transferring context: 54.32kB 0.0s
=> [build 1/8] FROM docker.io/ocaml/opam:debian-ocaml-4.12@sha256:e39cf 49.1s
=> => resolve docker.io/ocaml/opam:debian-ocaml-4.12@sha256:e39cf5ddf42c 0.0s
=> => sha256:004f1eed87df3f75f5e2a1a649fa7edd7f713d130 50.43MB / 50.43MB 2.8s
=> => sha256:b704dfdc404aa80ee420ceb91d1974441bb41b 286.70MB / 286.70MB 20.9s
=> => sha256:dbb52d71bac56d7b6969ac8b0310d33588f700 189.72MB / 189.72MB 14.2s
=> => sha256:e39cf5ddf42ceff246eee0402466a81e47a1c29567a 1.66kB / 1.66kB 0.0s
=> => sha256:7671537e58b4c6a388caee4d445b9a16476fbb3eb6fd1d2 956B / 956B 0.0s
=> => sha256:f7c1669319a4c7bcd2269e33e7bf6950679512069 10.47kB / 10.47kB 0.0s
=> => extracting sha256:004f1eed87df3f75f5e2a1a649fa7edd7f713d1300532fd0 2.2s
=> => extracting sha256:b704dfdc404aa80ee420ceb91d1974441bb41b37fce81fc 20.2s
=> => extracting sha256:dbb52d71bac56d7b6969ac8b0310d33588f7009a3692b933 6.4s
=> [stage-1 1/8] FROM docker.io/library/debian:stable-slim@sha256:f31a1f 6.4s
=> => resolve docker.io/library/debian:stable-slim@sha256:f31a1f9e274ebf 0.0s
=> => sha256:f31a1f9e274ebf9a97b964c9aed3386e1720d56a3d2 1.85kB / 1.85kB 0.0s
=> => sha256:65c240929acadafb6cdf7447d7e3ba1abc214a8b008fe4e 529B / 529B 0.0s
=> => sha256:f052c451335940bea31ab0e58feff9fc657959c4915 1.46kB / 1.46kB 0.0s
=> => sha256:7fa62122c34635d215ecb61e3d515d9eb1676a0f2 27.14MB / 27.14MB 5.0s
=> => extracting sha256:7fa62122c34635d215ecb61e3d515d9eb1676a0f258d352e 1.3s
=> [stage-1 2/8] RUN apt-get update && apt-get install --no-install 26.2s
=> [stage-1 3/8] WORKDIR /app 0.0s
=> [stage-1 4/8] COPY ./aslr_seccomp.json /app/aslr_seccomp.json 0.0s
=> [stage-1 5/8] COPY ./db/migrations /app/db/migrations 0.0s
=> [build 2/8] RUN sudo apt-get update && sudo apt-get install -qq 16.0s
=> [build 3/8] WORKDIR /mnt/project 0.0s
=> [build 4/8] COPY --chown=opam:opam pipeline.opam pipeline.opam 0.0s
=> [build 5/8] RUN opam install -y --deps-only -t . 100.1s
=> [build 6/8] COPY --chown=opam . . 0.0s
=> [build 7/8] RUN sudo chown opam . 0.5s
=> [build 8/8] RUN opam exec -- dune build --profile=release bin/main.ex 2.2s
=> [stage-1 6/8] COPY --from=build /home/opam/.opam/4.12/bin/omigrate /a 0.1s
=> [stage-1 7/8] COPY ./scripts/wait-for /app/bin/wait-for 0.0s
=> [stage-1 8/8] COPY --from=build /mnt/project/_build/default/bin/main. 0.1s
=> exporting to image 1.5s
=> => exporting layers 1.5s
=> => writing image sha256:943cfc7ba022701594600f0d26fd96ada9ff5fbbac566 0.0s
=> => naming to docker.io/library/current-bench_pipeline 0.0s
Successfully built 943cfc7ba022701594600f0d26fd96ada9ff5fbbac566a9c7ad3c80558708e94
Creating current-bench_db_1 ... done
Creating current-bench_pipeline_1 ... done
Creating current-bench_db-migrate_1 ... done
Creating current-bench_graphql-engine_1 ... done
Creating current-bench_frontend_1 ... done
Attaching to current-bench_db_1, current-bench_graphql-engine_1, current-bench_db-migrate_1, current-bench_pipeline_1, current-bench_frontend_1
db-migrate_1 | standard_init_linux.go:219: exec user process caused: no such file or directory
db_1 | The files belonging to this database system will be owned by user "postgres".
db_1 | This user must also own the server process.
db_1 |
db_1 | The database cluster will be initialized with locale "en_US.utf8".
db_1 | The default database encoding has accordingly been set to "UTF8".
db_1 | The default text search configuration will be set to "english".
db_1 |
db_1 | Data page checksums are disabled.
db_1 |
db_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1 | creating subdirectories ... ok
db_1 | selecting dynamic shared memory implementation ... posix
db_1 | selecting default max_connections ... 100
db_1 | selecting default shared_buffers ... 128MB
db_1 | selecting default time zone ... Etc/UTC
db_1 | creating configuration files ... ok
db_1 | running bootstrap script ... ok
db_1 | performing post-bootstrap initialization ... ok
db_1 | syncing data to disk ... ok
db_1 |
db_1 |
db_1 | Success. You can now start the database server using:
db_1 |
db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1 |
graphql-engine_1 | {"type":"startup","timestamp":"2021-04-02T20:12:48.629+0000","level":"info","detail":{"kind":"server_configuration","info":{"live_query_options":{"batch_size":100,"refetch_delay":1},"transaction_isolation":"ISOLATION LEVEL READ COMMITTED","plan_cache_options":{"plan_cache_size":4000},"enabled_log_types":["http-log","websocket-log","startup","webhook-log","query-log"],"server_host":"HostAny","enable_allowlist":false,"log_level":"info","auth_hook_mode":null,"use_prepared_statements":true,"unauth_role":null,"stringify_numeric_types":false,"enabled_apis":["metadata","graphql","config","pgdump"],"enable_telemetry":true,"enable_console":true,"auth_hook":null,"jwt_secret":null,"cors_config":{"allowed_origins":"*","disabled":false,"ws_read_cookie":null},"console_assets_dir":null,"admin_secret_set":true,"port":8080}}}
graphql-engine_1 | {"type":"startup","timestamp":"2021-04-02T20:12:48.629+0000","level":"info","detail":{"kind":"postgres_connection","info":{"retries":1,"database_url":"postgres://docker:...@db:5432/docker"}}}
graphql-engine_1 | {"type":"pg-client","timestamp":"2021-04-02T20:12:48.629+0000","level":"warn","detail":{"message":"postgres
connection failed, retrying(0)."}}
graphql-engine_1 | {"type":"pg-client","timestamp":"2021-04-02T20:12:48.629+0000","level":"warn","detail":{"message":"postgres
connection failed, retrying(1)."}}
graphql-engine_1 | {"type":"startup","timestamp":"2021-04-02T20:12:48.629+0000","level":"error","detail":{"kind":"catalog_migrate","info":{"internal":"could not connect to server: Connection refused\n\tIs the server running on host \"db\" (172.18.0.2) and accepting\n\tTCP/IP connections on port 5432?\n","path":"$","error":"connection error","code":"postgres-error"}}}
graphql-engine_1 | {"internal":"could not connect to server: Connection refused\n\tIs the server running on host \"db\" (172.18.0.2) and accepting\n\tTCP/IP connections on port 5432?\n","path":"$","error":"connection error","code":"postgres-error"}
db_1 | initdb: warning: enabling "trust" authentication for local connections
db_1 | You can change this by editing pg_hba.conf or using the option -A, or
db_1 | --auth-local and --auth-host, the next time you run initdb.
pipeline_1 | application [INFO] Logging ready.
db_1 | waiting for server to start....2021-04-02 20:12:48.618 UTC [47] LOG: starting PostgreSQL 12.6 (Debian 12.6-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
pipeline_1 | application [ERROR] Database connection error:
pipeline_1 | Connection failure: could not connect to server: Connection refused
pipeline_1 | Is the server running on host "db" (172.18.0.2) and accepting
pipeline_1 | TCP/IP connections on port 5432?
pipeline_1 |
db_1 | 2021-04-02 20:12:48.620 UTC [47] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2021-04-02 20:12:48.634 UTC [48] LOG: database system was shut down at 2021-04-02 20:12:48 UTC
pipeline_1 | pipeline: internal error, uncaught exception:
pipeline_1 | Postgresql.Error(_)
pipeline_1 | Raised at Pipeline__Db_util.check_connection in file "lib/db_util.ml", line 10, characters 6-15
pipeline_1 | Called from Pipeline.v in file "lib/pipeline.ml", line 245, characters 2-36
pipeline_1 | Called from Cmdliner_term.app.(fun) in file "cmdliner_term.ml", line 25, characters 19-24
pipeline_1 | Called from Cmdliner.Term.run in file "cmdliner.ml", line 117, characters 32-39
db_1 | 2021-04-02 20:12:48.637 UTC [47] LOG: database system is ready to accept connections
db_1 | done
db_1 | server started
db_1 | CREATE DATABASE
db_1 |
db_1 |
db_1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
db_1 | GRANT
db_1 |
db_1 |
current-bench_db-migrate_1 exited with code 1
db_1 | 2021-04-02 20:12:48.919 UTC [47] LOG: received fast shutdown request
db_1 | waiting for server to shut down....2021-04-02 20:12:48.921 UTC [47] LOG: aborting any active transactions
db_1 | 2021-04-02 20:12:48.922 UTC [47] LOG: background worker "logical replication launcher" (PID 54) exited with exit code 1
db_1 | 2021-04-02 20:12:48.922 UTC [49] LOG: shutting down
db_1 | 2021-04-02 20:12:48.935 UTC [47] LOG: database system is shut down
db_1 | done
db_1 | server stopped
db_1 |
db_1 | PostgreSQL init process complete; ready for start up.
db_1 |
db_1 | 2021-04-02 20:12:49.027 UTC [1] LOG: starting PostgreSQL 12.6 (Debian 12.6-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bitgnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1 | 2021-04-02 20:12:49.028 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2021-04-02 20:12:49.028 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2021-04-02 20:12:49.033 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2021-04-02 20:12:49.043 UTC [84] LOG: database system was shut down at 2021-04-02 20:12:48 UTC
db_1 | 2021-04-02 20:12:49.047 UTC [1] LOG: database system is ready to accept connections
Any help would be greatly appreciated ๐
Hi ๐!
What is the best way to handle the X-Hasura-Admin-Secret
value? It's currently hard-coded into the repo. Can this be used to do anything destructive? I'm not familiar with Hasura but it seems great, and I want to use Hasura Cloud for my own implementation of current-bench. I'll hard code into the client for now, but I don't want some jerk to DOS me if it can be easily avoided ๐
It's difficult to know to what PR a given job belongs to.
Currently, the benchmarks are sorted by the timestamp they were created at which doesn't always result in the correct graph. Sometimes the master & pull request commits are interspersed because the pipeline ran on the master more than once. One way to make it better would be to topologically sort the commits or display master benchmarks first followed by displaying the benchmarks of the commit in the PR.
For Multicore OCaml, we would like to be able to build and run benchmarks on bare metal machines. It will be good to create an OCurrent pipeline just for native builds, while, still using the PostgreSQL database and frontend containers.
There are several hard-coded URI's in the frontend and the pipeline. These should be moved to configuration, but for now they are easy enough to change in my own fork. However, there is another source of hard-coded URI's that I can't locate. My production pipeline (installed as a GH app) is failing. It's giving this warning:
current.github [WARNING] Set d4hines/marigold-tezos
refs/heads/master
902910d2/ocaml-benchmarks to
{"state":"failure","description":"Command \"docker\" \"run\" \"--security-opt\" \"seccomp=./aslr_seccomp.json\" \n\"--tmpfs\" \"/dev/shm:rw,noexec,nosuid,size=4g\" \"--rm\" \"-i\" \"sha256:98a","target_url":"http://autumn.ocamllabs.io/d4hines/marigold-tezos"} failed: 403 Forbidden
{"message":"Resource not accessible by integration","documentation_url":"https://docs.github.com/rest/reference/repos#create-a-commit-status"}
I can't track down where it's getting the URL http://autumn.ocamllabs.io/d4hines/marigold-tezos from. Any ideas?
Hi,
I'm trying to use the production version of the app, but something's not working. My database never gets any inserts. I can't decipher what the problem is, as I don't see any errors in the logs. I tried troubleshooting with print_endline
, but the control flow elludes me - I'm unfamiliar with the let+
and and+
syntax and I think there is magic in there. Any help would be appreciated. Here's the debug logs (hopefully there aren't any secrets in there ๐):
application [INFO] Logging ready.
current.github [DEBUG] API call on https://api.github.com/app/installations
current_web [INFO] Starting web server: (TCP (Port 8081))
current.cache [DEBUG] set: "docker" "pull" "ocaml/opam"
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031300-docker-pull-bcb363.log
current [INFO] Exec: "docker" "pull" "ocaml/opam"
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current.github [DEBUG] Got response:
`List ([`Assoc ([("id", `Int (16069792));
("account",
`Assoc ([("login",
`String ("d4hines"));
("id", `Int (25231383));
("node_id",
`String ("MDQ6VXNlcjI1MjMxMzgz"));
("avatar_url",
`String ("https://avatars.githubusercontent.com/u/25231383?v=4"));
("gravatar_id",
`String (""));
("url",
`String ("https://api.github.com/users/d4hines"));
("html_url",
`String ("https://github.com/d4hines"));
("followers_url",
`String ("https://api.github.com/users/d4hines/followers"));
("following_url",
`String ("https://api.github.com/users/d4hines/following{/other_user}"));
("gists_url",
`String ("https://api.github.com/users/d4hines/gists{/gist_id}"));
("starred_url",
`String ("https://api.github.com/users/d4hines/starred{/owner}{/repo}"));
("subscriptions_url",
`String ("https://api.github.com/users/d4hines/subscriptions"));
("organizations_url",
`String ("https://api.github.com/users/d4hines/orgs"));
("repos_url",
`String ("https://api.github.com/users/d4hines/repos"));
("events_url",
`String ("https://api.github.com/users/d4hines/events{/privacy}"));
("received_events_url",
`String ("https://api.github.com/users/d4hines/received_events"));
("type",
`String ("User"));
("site_admin",
`Bool (false))
]));
("repository_selection",
`String ("selected"));
("access_tokens_url",
`String ("https://api.github.com/app/installations/16069792/access_tokens"));
("repositories_url",
`String ("https://api.github.com/installation/repositories"));
("html_url",
`String ("https://github.com/settings/installations/16069792"));
("app_id", `Int (109289));
("app_slug",
`String ("test-occurent-bench"));
("target_id", `Int (25231383));
("target_type", `String ("User"));
("permissions",
`Assoc ([("contents",
`String ("write"));
("metadata",
`String ("read"));
("statuses",
`String ("write"));
("repository_hooks",
`String ("write"))
]));
("events",
`List ([`String ("create");
`String ("delete");
`String ("fork");
`String ("label");
`String ("push")]));
("created_at",
`String ("2021-04-08T14:39:09.000Z"));
("updated_at",
`String ("2021-04-14T22:52:38.000Z"));
("single_file_name", `Null);
("has_multiple_single_files",
`Bool (false));
("single_file_paths", `List ([]));
("suspended_by", `Null);
("suspended_at", `Null)])
])
current.github [INFO] Found installation 16069792 for "d4hines"
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current.github [INFO] Getting API token
current.github [DEBUG] API call on https://api.github.com/app/installations/16069792/access_tokens
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current.github [DEBUG] Got response:
`Assoc ([("token",
`String ("ghs_qmFkuP4WeBVq3M4ZsRhZbL93hxKec83jmCKQ"));
("expires_at",
`String ("2021-04-15T04:13:02Z"));
("permissions",
`Assoc ([("contents", `String ("write"));
("metadata", `String ("read"));
("repository_hooks",
`String ("write"));
("statuses", `String ("write"))
]));
("repository_selection",
`String ("selected"))
])
current.github [DEBUG] Get repositories for "d4hines" from https://api.github.com/installation/repositories
current [INFO] Exec: "docker" "image" "inspect" "ocaml/opam" "-f"
"{{index .RepoDigests 0}}"
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current.github [DEBUG] Got response:
`Assoc ([("total_count", `Int (1));
("repository_selection",
`String ("selected"));
("repositories",
`List ([`Assoc ([("id", `Int (353847111));
("node_id",
`String ("MDEwOlJlcG9zaXRvcnkzNTM4NDcxMTE="));
("name",
`String ("marigold-tezos"));
("full_name",
`String ("d4hines/marigold-tezos"));
("private",
`Bool (false));
("owner",
`Assoc ([("login",
`String (
"d4hines"));
("id",
`Int (
25231383));
("node_id",
`String (
"MDQ6VXNlcjI1MjMxMzgz"));
("avatar_url",
`String (
"https://avatars.githubusercontent.com/u/25231383?v=4"));
("gravatar_id",
`String (
""));
("url",
`String (
"https://api.github.com/users/d4hines"));
("html_url",
`String (
"https://github.com/d4hines"));
("followers_url",
`String (
"https://api.github.com/users/d4hines/followers"));
("following_url",
`String (
"https://api.github.com/users/d4hines/following{/other_user}"));
("gists_url",
`String (
"https://api.github.com/users/d4hines/gists{/gist_id}"));
("starred_url",
`String (
"https://api.github.com/users/d4hines/starred{/owner}{/repo}"));
("subscriptions_url",
`String (
"https://api.github.com/users/d4hines/subscriptions"));
("organizations_url",
`String (
"https://api.github.com/users/d4hines/orgs"));
("repos_url",
`String (
"https://api.github.com/users/d4hines/repos"));
("events_url",
`String (
"https://api.github.com/users/d4hines/events{/privacy}"));
("received_events_url",
`String (
"https://api.github.com/users/d4hines/received_events"));
("type",
`String (
"User"));
("site_admin",
`Bool (
false))
]));
("html_url",
`String ("https://github.com/d4hines/marigold-tezos"));
("description",
`String ("A mirror of https://gitlab.com/marigold/tezos"));
("fork", `Bool (false));
("url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos"));
("forks_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/forks"));
("keys_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/keys{/key_id}"));
("collaborators_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/collaborators{/collaborator}"));
("teams_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/teams"));
("hooks_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/hooks"));
("issue_events_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/issues/events{/number}"));
("events_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/events"));
("assignees_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/assignees{/user}"));
("branches_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/branches{/branch}"));
("tags_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/tags"));
("blobs_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/git/blobs{/sha}"));
("git_tags_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/git/tags{/sha}"));
("git_refs_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/git/refs{/sha}"));
("trees_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/git/trees{/sha}"));
("statuses_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/statuses/{sha}"));
("languages_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/languages"));
("stargazers_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/stargazers"));
("contributors_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/contributors"));
("subscribers_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/subscribers"));
("subscription_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/subscription"));
("commits_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/commits{/sha}"));
("git_commits_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/git/commits{/sha}"));
("comments_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/comments{/number}"));
("issue_comment_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/issues/comments{/number}"));
("contents_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/contents/{+path}"));
("compare_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/compare/{base}...{head}"));
("merges_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/merges"));
("archive_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/{archive_format}{/ref}"));
("downloads_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/downloads"));
("issues_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/issues{/number}"));
("pulls_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/pulls{/number}"));
("milestones_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/milestones{/number}"));
("notifications_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/notifications{?since,all,participating}"));
("labels_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/labels{/name}"));
("releases_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/releases{/id}"));
("deployments_url",
`String ("https://api.github.com/repos/d4hines/marigold-tezos/deployments"));
("created_at",
`String ("2021-04-01T23:01:10Z"));
("updated_at",
`String ("2021-04-15T03:10:29Z"));
("pushed_at",
`String ("2021-04-15T03:10:27Z"));
("git_url",
`String ("git://github.com/d4hines/marigold-tezos.git"));
("ssh_url",
`String ("[email protected]:d4hines/marigold-tezos.git"));
("clone_url",
`String ("https://github.com/d4hines/marigold-tezos.git"));
("svn_url",
`String ("https://github.com/d4hines/marigold-tezos"));
("homepage",
`String (""));
("size", `Int (101816));
("stargazers_count",
`Int (0));
("watchers_count",
`Int (0));
("language",
`String ("OCaml"));
("has_issues",
`Bool (true));
("has_projects",
`Bool (true));
("has_downloads",
`Bool (true));
("has_wiki",
`Bool (true));
("has_pages",
`Bool (false));
("forks_count",
`Int (0));
("mirror_url", `Null);
("archived",
`Bool (false));
("disabled",
`Bool (false));
("open_issues_count",
`Int (0));
("license",
`Assoc ([("key",
`String (
"other"));
("name",
`String (
"Other"));
("spdx_id",
`String (
"NOASSERTION"));
("url",
`Null);
("node_id",
`String (
"MDc6TGljZW5zZTA="))
]));
("forks", `Int (0));
("open_issues",
`Int (0));
("watchers", `Int (0));
("default_branch",
`String ("master"));
("permissions",
`Assoc ([("admin",
`Bool (
false));
("push",
`Bool (
false));
("pull",
`Bool (
false))
]))
])
]))
])
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current.github [INFO] GraphQL(default_ref): cost:1 remaining:4973 resetAt:2021-04-15T03:15:12Z
current [DEBUG] Evaluating...
current.cache [DEBUG] set: git fetch https://github.com/d4hines/marigold-tezos.git#refs/heads/master (12fa1c421fdef59948cca13109665d11bee31292)
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031301-git-fetch-c2346d.log
current [INFO] Exec: "git" "clone" "--recursive" "-q" "https://github.com/d4hines/marigold-tezos.git"
"/home/d4hines/repos/current-bench/var/git/marigold-tezos.git-ecf80115d780255b0108dda6b07dc18f"
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
12fa1c42/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031301-github-set-status-ac74aa.log
current [INFO] Result: Ok: ()
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
12fa1c42/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current [INFO] Exec: "git" "-C" "/home/d4hines/repos/current-bench/var/git/marigold-tezos.git-ecf80115d780255b0108dda6b07dc18f"
"branch" "-f" "fetch-12fa1c421fdef59948cca13109665d11bee31292"
"12fa1c421fdef59948cca13109665d11bee31292"
current [INFO] Exec: "git" "-C" "/home/d4hines/repos/current-bench/var/git/marigold-tezos.git-ecf80115d780255b0108dda6b07dc18f"
"branch" "-f" "fetch-12fa1c421fdef59948cca13109665d11bee31292"
"12fa1c421fdef59948cca13109665d11bee31292"
current [DEBUG] Evaluating...
current.cache [DEBUG] set: docker build {
"commit":
"12fa1c421fdef59948cca13109665d11bee31292",
"dockerfile": {
"contents":
"0bdebea505c40f2924759d9a5f3888ab"
},
"docker_context": null,
"squash": false,
"build_args": []
}
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031319-docker-build-bfe0e0.log
current [INFO] Exec: "cp" "-a" "--" "/home/d4hines/repos/current-bench/var/git/marigold-tezos.git-ecf80115d780255b0108dda6b07dc18f/.git"
"/tmp/git-checkout10657da"
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
12fa1c42/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
12fa1c42/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [INFO] Exec: "git" "-C" "/tmp/git-checkout10657da" "reset"
"--hard" "12fa1c421fdef59948cca13109665d11bee31292"
current [INFO] Exec: "git" "-C" "/tmp/git-checkout10657da" "submodule"
"update" "--init" "--recursive"
current [INFO] Exec: "docker" "build" "--iidfile" "/tmp/git-checkout10657da/docker-iid"
"--" "/tmp/git-checkout10657da"
current.docker [INFO] Built docker image sha256:a682159e9bb570c8dc189947102d2a332d789c09f9a43ffbbdc7b664ef21038b
current [DEBUG] Switch.turn_off: already off
current [DEBUG] Evaluating...
current.cache [DEBUG] set: "docker" "run" "--security-opt"
"seccomp=./aslr_seccomp.json" "--tmpfs"
"/dev/shm:rw,noexec,nosuid,size=4g" "--rm" "-i"
"sha256:a682159e9bb570c8dc189947102d2a332d789c09f9a43ffbbdc7b664ef21038b"
"/usr/bin/setarch" "x86_64" "--addr-no-randomize"
"make" "bench"
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031322-docker-pread-771b68.log
current [INFO] Exec: "docker" "run" "--security-opt" "seccomp=./aslr_seccomp.json"
"--tmpfs" "/dev/shm:rw,noexec,nosuid,size=4g"
"--rm" "-i" "sha256:a682159e9bb570c8dc189947102d2a332d789c09f9a43ffbbdc7b664ef21038b"
"/usr/bin/setarch" "x86_64" "--addr-no-randomize"
"make" "bench"
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
12fa1c42/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
12fa1c42/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
12fa1c42/ocaml-benchmarks to
{"state":"failure","description":"Command \"docker\" \"run\" \"--security-opt\" \"seccomp=./aslr_seccomp.json\" \n\"--tmpfs\" \"/dev/shm:rw,noexec,nosuid,size=4g\" \"--rm\" \"-i\" \"sha256:a68","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031322-github-set-status-75573b.log
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current_web [INFO] HTTP POST "/webhooks/github"
current.github [INFO] input_webhook: ((headers
((accept */*) (connection close) (content-length 7492)
(content-type application/json) (host ocurrent.marigold.dev:8081)
(user-agent GitHub-Hookshot/64aa546)
(x-github-delivery 8d4ece0a-9d98-11eb-840e-f26b4baf96a4)
(x-github-event push) (x-github-hook-id 290932103)
(x-github-hook-installation-target-id 109289)
(x-github-hook-installation-target-type integration)))
(meth POST) (resource /webhooks/github) (version HTTP_1_1)
(encoding (Fixed 7492)))
current.github [INFO] Got GitHub event "push"
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current.github [INFO] GraphQL(default_ref): cost:1 remaining:4972 resetAt:2021-04-15T03:15:12Z
current [DEBUG] Evaluating...
current.cache [DEBUG] set: git fetch https://github.com/d4hines/marigold-tezos.git#refs/heads/master (e8f2f7784ed31e0343dc26b2d71996a19c5f8ff4)
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031329-git-fetch-affd23.log
current [INFO] Exec: "git" "-C" "/home/d4hines/repos/current-bench/var/git/marigold-tezos.git-ecf80115d780255b0108dda6b07dc18f"
"branch" "-f" "fetch-e8f2f7784ed31e0343dc26b2d71996a19c5f8ff4"
"e8f2f7784ed31e0343dc26b2d71996a19c5f8ff4"
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
e8f2f778/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031329-github-set-status-9f87c3.log
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
e8f2f778/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [INFO] Exec: "git" "-C" "/home/d4hines/repos/current-bench/var/git/marigold-tezos.git-ecf80115d780255b0108dda6b07dc18f"
"fetch" "-f" "https://github.com/d4hines/marigold-tezos.git"
"refs/heads/master"
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current [INFO] Exec: "git" "-C" "/home/d4hines/repos/current-bench/var/git/marigold-tezos.git-ecf80115d780255b0108dda6b07dc18f"
"branch" "-f" "fetch-e8f2f7784ed31e0343dc26b2d71996a19c5f8ff4"
"e8f2f7784ed31e0343dc26b2d71996a19c5f8ff4"
current [DEBUG] Evaluating...
current.cache [DEBUG] set: docker build {
"commit":
"e8f2f7784ed31e0343dc26b2d71996a19c5f8ff4",
"dockerfile": {
"contents":
"0bdebea505c40f2924759d9a5f3888ab"
},
"docker_context": null,
"squash": false,
"build_args": []
}
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031329-docker-build-844220.log
current [INFO] Exec: "cp" "-a" "--" "/home/d4hines/repos/current-bench/var/git/marigold-tezos.git-ecf80115d780255b0108dda6b07dc18f/.git"
"/tmp/git-checkout37965990"
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
e8f2f778/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
e8f2f778/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [INFO] Exec: "git" "-C" "/tmp/git-checkout37965990" "reset"
"--hard" "e8f2f7784ed31e0343dc26b2d71996a19c5f8ff4"
current [INFO] Exec: "git" "-C" "/tmp/git-checkout37965990" "submodule"
"update" "--init" "--recursive"
current [INFO] Exec: "docker" "build" "--iidfile" "/tmp/git-checkout37965990/docker-iid"
"--" "/tmp/git-checkout37965990"
current.docker [INFO] Built docker image sha256:9cf87199b7065eda0b16f547bd861e07736f18a30906d9dd1b564d3692c16a89
current [DEBUG] Switch.turn_off: already off
current [DEBUG] Evaluating...
current.cache [DEBUG] set: "docker" "run" "--security-opt"
"seccomp=./aslr_seccomp.json" "--tmpfs"
"/dev/shm:rw,noexec,nosuid,size=4g" "--rm" "-i"
"sha256:9cf87199b7065eda0b16f547bd861e07736f18a30906d9dd1b564d3692c16a89"
"/usr/bin/setarch" "x86_64" "--addr-no-randomize"
"make" "bench"
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031347-docker-pread-b5b209.log
current [INFO] Exec: "docker" "run" "--security-opt" "seccomp=./aslr_seccomp.json"
"--tmpfs" "/dev/shm:rw,noexec,nosuid,size=4g"
"--rm" "-i" "sha256:9cf87199b7065eda0b16f547bd861e07736f18a30906d9dd1b564d3692c16a89"
"/usr/bin/setarch" "x86_64" "--addr-no-randomize"
"make" "bench"
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
e8f2f778/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
e8f2f778/ocaml-benchmarks to
{"state":"pending","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current.cache [DEBUG] set: Set d4hines/marigold-tezos refs/heads/master
e8f2f778/ocaml-benchmarks to
{"state":"failure","description":"Command \"docker\" \"run\" \"--security-opt\" \"seccomp=./aslr_seccomp.json\" \n\"--tmpfs\" \"/dev/shm:rw,noexec,nosuid,size=4g\" \"--rm\" \"-i\" \"sha256:9cf","target_url":"https://tezos-current-bench.hasura.app/d4hines/marigold-tezos"}
current [INFO] Created new log file at
/home/d4hines/repos/current-bench/var/job/2021-04-15/031347-github-set-status-02564c.log
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event...
current [DEBUG] Evaluating...
current [DEBUG] Waiting for an external event..
Currently we expect the benchmarks to silence all stderr output and only produce valid JSON output on stdout.
This can lead to situations where debugging errors during the benchmark execution is very difficult.
For example, the dune benchmarks are currently failing with:
$ docker run -it --rm current-bench-dune-error-2 make bench
make: *** [Makefile:129: bench] Error 2
But in reality the problem is:
$ dune exec -- ./bench/bench.exe
cloning ocaml-dune/dune-bench
dune (internal) (exit 1)
/home/opam/.opam/4.12/bin/dune build @install --root . &> /dev/null
Fatal error: exception Dune_util.Report_error.Already_reported
Raised at Dune_engine__Scheduler.Run.go in file "src/dune_engine/scheduler.ml", line 790, characters 27-40
Called from Dune__exe__Bench in file "bench/bench.ml", line 131, characters 4-168
We should forward the stderr to the current logs.
The text "Learn" in the nav-bar and the footer is unclear without context and may be confusing to screen readers. You can add a special screen reader text than can help with accessibility.
Relevant code:
Learn
It's useful to see the benchmarks logs when the benchmarks are running or failed (or just if someone wants to reproduce the results and see what have been built). Currently these logs are not accessible.
Currently we use the latest dependencies from opam repository to build, but as the dependency versions may change over time they can affect the benchmark results. It would be useful if it was possible to use the same dependencies for each PR rebuild and/or highlight if there's a dependency change (both over time on branches and on a PR).
Currently the UI shows a +/- signs for the relative comparison values with between master and PRs. It would be nice to be able to know know if higher values are better or worse to add color clues.
One way to achieve this is by adding an extra field to the benchmark output. Another would be to have the ability to set this information in the UI and save it in the db.
From my understanding dygraphs does not have built-in support for vertical labels. We apply a CSS transform to rotate the labels. For some reason the position calculation for labels used in dygraphs results in this strange overlapping.
Another problem I noticed is that the last data point on the graph does not (always?) have a commit label.
There's a Javascript error and the whole page becomes blank when we try to access the benchmark results from a PR that does not exist. This can happen for instance when we manually modify the PR number in the URL.
We should handle that in a better way, like showing a message that tells the user that the PR does not exist (yet).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.