Comments (5)
The worker side of this was implemented in #143, we're waiting on an http implementation I threw together in #144 to use as a grpc endpoint for one or many workers, and #153 will enable this on the server
from bazel-buildfarm.
It is not currently possible to separate this right now, but for only the simple reason that we're using the same endpoint for the operation queue, the CAS, and the Action Cache.
Your use case is definitely sound, but I have a thought about providing the support for the different endpoint types in-worker: How about I just make 3 different configurable endpoint addresses for communication over grpc for each of the services, rather than trying to bake in client support for all manner of CAS/AC communication protocols (http, grpc, hazelcast, redis, memcached, mongo, etc).
I can also quite separately (doesn't even have to be in this repo) provide a grpc hosted server that can speak one of these (separate binaries for separate protos) and facilitate proxy/translation for either activity (CAS or AC).
I think it is an effective strategy to combat complexity to say 'the operation queue workers speak grpc and nothing else.'
from bazel-buildfarm.
Hi @werkt, I'm not sure if that was a comment directed at me.
What I was hoping for was to give a flag to the build farm worker to read/write to some cache backend.
I'm not sure how things work so I don't quite understand the 3 different endpoint configurations or the reason for a proxy.
Thanks!
from bazel-buildfarm.
I believe @werkt is saying that currently there are 3 different kinds of things, all handled on a single port by a single binary. If they were (or could be) on different ports, it would be far easier for others to add support for alternate implementation(s) of the functionality - whether it is a shim to use Bazel Remote Cache in place of the internal CAS, a custom AC backend, or what have you.
from bazel-buildfarm.
We've broken things out at this point, have at it!
from bazel-buildfarm.
Related Issues (20)
- image bazelbuild/buildfarm-worker:v2.7.0 fails to start with "libfuse.so.2: cannot open shared object file: No such file or directory" HOT 8
- First GRPC type storage tries to create Fuse Exec FS
- Buildfarm is failing at Bazel@HEAD
- Add an optional filter to limit artifact sizes by Action HOT 4
- Post Local Clean Java Coverage Builds Against Remote K8s Build Farm Result In Invalid Digest Recieved HOT 6
- Diffrence between execution and CAS shard worker HOT 3
- Why does clang work, but llvm-ar not? HOT 4
- rules_oss_audit fails to install dependencies on mac
- Set up OSSF security scorecards
- poisson_distribution_test is failing with BAZEL@HEAD HOT 2
- External dependency of buildfarm fails with bzlmod
- Redis Hot Shard issue due to DispatchMonitor HashMap
- skipLoad looping can exhaust file path length
- Heuristics for controlling putDirectory (linkedInputDirectories) per action
- "./examples.bf-run start" fails HOT 2
- How to obtain remote system information? HOT 1
- Helm instructions did not work for me HOT 3
- Health check does not seem to work on shard-worker when redis is down HOT 10
- Error occurred during the fetch of repository 'com_grail_bazel_toolchain' HOT 3
- Error occurred during the fetch of repository 'com_grail_bazel_toolchain'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bazel-buildfarm.