Comments (7)
Hey @jzelinskie, Bencher runs only when there is a code change in .go files or when there is a configuration update. The reason for that is that in a PR, lots of diverse changes can happen but shouldn't be wasting precious machine time and CPU re-benchmarking. We shall document this on https://bencher.orijtech.com/
from spicedb.
We should setup a dedicated worker for benchmarks or explore using a service like bencher to get reliable benchmarks built into our CI workflow.
from spicedb.
Hi, I work on bencher! If you wanted to try it out (it's free!), it should just take a few clicks to install from the GitHub marketplace
We've got some more-detailed installation instructions (with pictures!) as well.
Let me know if I can help with anything :)
from spicedb.
@kirbyquerby I tried enabling it, but we use docker for integration testing in our benchmarks -- this is because we benchmark against various databases.
I see in the bencher configuration you can depend on services based on docker, but can we actually just get access to a docker socket?
from spicedb.
@kirbyquerby @odeke-em Are there any rate limits on bencher? I've been playing with things over at https://github.com/jzelinskie/benchpress and half of my pushes don't run. I was pulling my hair out thinking I had an invalid config, but it's not starting builds even without a config.
I'm experimenting with using docker-in-docker so that our test suite can run unmodified.
from spicedb.
@odeke-em Thanks for the response -- that makes perfect sense.
Do you have any recommendations on how we could get access to the Docker daemon? Our tests use the Docker API to spin up/down images it needs for end-to-end tests. I've been playing with docker-in-docker, but that hasn't seemed to work.
from spicedb.
I'm hesitant to provide direct access to the docker daemon because we run benchmarks on bare metal. This makes it tricky to ensure the state of the machine is reset after a benchmark is run. The current setup provides a intentionally trimmed-down set of docker features to try and minimize the attack surface -- I'm not well-versed enough to understand all the potential pitfalls of allowing arbitrary access to all of docker.
docker-in-docker seems promising, though -- I'll take a look at what it would take to make it work with bencher.
from spicedb.
Related Issues (20)
- Proposal: Health check endpoint for HTTP API HOT 5
- Return Evaluated Caveat Context
- Add ability to toggle certain schema warnings off
- Evaluate replacing ristretto with theine-go
- support normalization of caveat context data
- --datastore-bootstrap-overwrite is additive, not overwriting previous namespaces HOT 5
- v1.34.0 Segfault on ReadRelationships before WriteSchema when using memstore HOT 9
- Unnecessary COALESCE Used
- Maybe a bug or not! Spicedb connect to Pgbouncer report error "prepared statement name is already in use (SQLSTATE 08P01)" HOT 2
- Schema Change Events via Watch API
- wildcard support HOT 4
- Would it be possible to use SpiceDB with sqlite/libsql? HOT 1
- failed to create datastore: failed to load bootstrap files: could not lookup caveat
- SpiceDB version middleware not enabled in `serve-testing` mode
- GOMAXPROCS, GOMEMLIMIT logs emitted on all CLI commands HOT 2
- It would be nice if by specifying some relation we could use it as indirect inverse relation in schema as well HOT 1
- Getting a duplicated transaction exists when trying to do a write request HOT 2
- Add external support for streaming relationships into and out of SpiceDB
- Max connections open flag does not work HOT 2
- Fix up flag descriptions and organize into flagsets HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spicedb.