Comments (5)
I am currently using the following approach to spin up multiple workers per server: thelgevold@52e70e1
from bazel-buildfarm.
The memory reference implementation is designed to be single-host-server, multiple-host-workers. The Workers can be configured to schedule multiple concurrent executions (
) and benefits greatly from a single worker monopolizing a host to the cache coherency of its backing CASFileCache (bazel-buildfarm/examples/worker.config.example
Lines 34 to 38 in d4fefa0
I have run this configuration with the memory server and operationqueue worker across 1000s of cores, and successfully leveraged all of them (https://github.com/werkt/bazel-stress was used at bazelcon with this setup).
from bazel-buildfarm.
@werkt Thanks for clarifying this. I noticed a big performance improvement from switching to this approach.
Thanks again!
from bazel-buildfarm.
I did a quick write-up of the experiment I ran. Sharing in case anyone is interested: http://www.syntaxsuccess.com/viewarticle/bazel-remote-execution
from bazel-buildfarm.
Closing this out, as I believe all of the questions have been answered.
from bazel-buildfarm.
Related Issues (20)
- Buildfarm unit tests are failing in windows at Bazel@HEAD in Downstream CI HOT 1
- Runtime metrics are not reported for match/report result stages
- helm chart: `ingress:` in values.yaml does nothing
- Upgrade jedis to v5, abandon fork
- Redis password is printed to the logs HOT 1
- Blocked thread in RedisShardBackplane HOT 5
- RedisPriorityQueue is malformed
- prometheus metric is malformed HOT 1
- `cas: false` not being respected? HOT 1
- Update to PMD 7.0.0
- How does access control work? HOT 7
- parse CA file with --tls_certificate in bazel does not work HOT 2
- log messages `WARNING: write: uploads/ ... no available workers`. HOT 4
- Work directory should be added automatically as an input directory HOT 3
- Many `WARNING: DEADLINE_EXCEEDED: write` logs from buildfarm-server. HOT 2
- [Bazel CI] gaussian_distribution_test is failing at Bazel@HEAD HOT 1
- No logs HOT 4
- Running a server on Linux and build on MacOS HOT 3
- buildfarm-worker on Windows Server 2022 fails to clean up operation files HOT 2
- if redisUri is empty, java.lang.NullPointerException error occurred
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bazel-buildfarm.