Comments (6)
@vianaddsv not at present; we'd be happy to review a PR if this is something you need in the near term for your use-case.
from go-dqlite.
Currently we try to connect to one server at a time until we connect to the leader or until a server tells us who the leader and we try that server next. This strategy has caused some issues in the past in LXD where the whole time budget could be spent attempting to connect to a node that is not responding. I suspect it might also cause issues in microk8s that has stricter timing constraints than LXD.
Instead of that strategy, I would attempt to connect to all servers in parallel in a goroutine, if there is a leader, it should be one of those and we should be able to connect to it. If a server would report that a certain server is the leader and it's not in the list of the servers we are attempting to connect to, I would also attempt to connect to that server.
@stgraber I remember you saying that there are LXD clusters with 80+ nodes, attempting to connect to all 80 at the same time might be a bit too optimistic?
@freeekanayaka Any obvious issues with this approach?
from go-dqlite.
Just a couple of notes:
- Adjusting time budgets could be a good first move: for example the connect (or RPC, once connected) timeout could be made low enough that an unresponsive node won't create significant hiccups
- It seems that we currently attempt all servers found in the store, regardless of their role, probably they should be sorted with voters first, standbys next and spare last, since the leader is highly likely to be found among voters
That might be enough to solve the issues being observed. If that's still not enough, then perhaps you could indeed try adding some degree of parallelism, e.g. attempt to connect to 3 or 5 nodes in parallel at a time (following the role-based servers ordering described above).
from go-dqlite.
Regarding the roles, I believe the app
package does propagate them correctly (see here), but I'm not so sure about LXD, that might need to be fixed.
from go-dqlite.
Yeah, if we do parallel connections, we should do them in batches:
- All voters
- All standby
- Spares in batch of 3
In 99% of cases this would then limit it to just 3 parallel connections and the first batch should either hit the leader or be pointed to it pretty quick.
from go-dqlite.
Is any one working on this?
from go-dqlite.
Related Issues (20)
- Shell: Support for WITH clauses (or more complex queries) HOT 7
- Shell: Provide .help
- Expose WithTracing option in app options HOT 3
- Expose an operation to remove a node from the cluster HOT 2
- RowsAffected not returning correct value for UPDATE HOT 4
- Upgrade from yaml.v2 to yaml.v3
- Issues with sharing a connection between statements (?) HOT 4
- Client Error Handling needs improvement HOT 10
- Jepsen failure: EOF when calling Open HOT 1
- Return error in driver.BeginTx if options aren't used/required or incorrect.
- Role re-shuffling for failure domains
- Shortcomings of sortCandidates
- TestIntegration_LeadershipTransfer CI failures HOT 2
- restore dqlite state from snapshot HOT 1
- Change cluster join addresses at runtime HOT 4
- Understanding problem consistency, scalability and leader HOT 5
- Listener for database updates HOT 2
- go-dqlite doesn't support nested or interleaved Query HOT 4
- Issues with two-node clusters
- Nondeterministic nil Logger dereference in driver
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go-dqlite.