Comments (14)
Hello,
After a short investigation, I found that the error is produced by memberlist package. It may be caused by your operating system configuration.
Check out the code:
https://github.com/hashicorp/go-sockaddr/blob/master/ifaddr.go#L27
https://github.com/hashicorp/memberlist/blob/master/net_transport.go#L148
// GetPrivateIP returns a string with a single IP address that is part of RFC
// 6890 and has a default route. If the system can't determine its IP address
// or find an RFC 6890 IP address, an empty string will be returned instead.
// This function is theeval
equivalent of:
//
//// $ sockaddr eval -r '{{GetPrivateInterfaces | attr "address"}}' ///
It seems that GetPrivateIP returns an empty string as IP for your case. I think this is pretty hard to debug on my side. Very likely, the problem is triggered by your host system configuration.
If you want to find the root cause of the problem, I can help you.
from olric.
@buraksezer Yep, the new consistent hashing algorithms is great, I have already read this paper. Thanks again for everything you've done! 👍
from olric.
@asdf2014 I'm glad to hear that. You feel free to ask any question or give advice about Olric. I have very limited spare time for this project but I'm willing to improve it constantly.
from olric.
Hello, I just checked the tests on master branch and there is no problem on my end. Please give more information about your environment. Output of go env
may be useful.
Olric is developed and tested on Linux and macOS.
from olric.
@buraksezer Thank you for your attention. The following code block is the result of running the go env
command.
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/benedictjin/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/code/gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4z/h5ftkmm90vg5s7j0kv65j0gw0000gp/T/go-build152216936=/tmp/go-build -gno-record-gcc-switches -fno-common"
from olric.
@buraksezer Thanks for the explanation. It finally worked after I removed IfByRFC("6890", privateIfs)
from GetPrivateInterfaces
method.
from olric.
@buraksezer May I ask another question about the architecture of Olric
? How can we ensure that the backup partition dose not appear in the member that owns the primary partition? And why don't we let the consistent
project covers this logic?
from olric.
@buraksezer With the growth of the Olric
cluster, should I increase the value of ReplicationFactor
? And will the increasing operation hurt the Consistency
of cluster?
from olric.
Hi, @buraksezer . When I look at these test cases in dmap_backup_test.go
, I think the two Olric
instance db1
and db2
are bundled into a cluster pair by the peer. Is there any chance to bundle more Olric
clusters together?
from olric.
@buraksezer May I ask another question about the architecture of
Olric
? How can we ensure that the backup partition dose not appear in the member that owns the primary partition? And why don't we let theconsistent
project covers this logic?
FSCK component is responsible maintaining cluster consistency. Unfortunately it has some missing parts. consistent
package should provide a reliable key distribution mechanism between members. Olric calls GetClosestNForPartition
function to find home for backups. That function finds partition's home firstly, then finds closest(suitable for backup) members for the partition.
Here is the code:
https://github.com/buraksezer/consistent/blob/master/consistent.go#L303
As I said before, FSCK is responsible to check inconsistencies and fix problems(including edge-cases) in cluster but It needs more effort to work properly.
from olric.
@buraksezer With the growth of the
Olric
cluster, should I increase the value ofReplicationFactor
? And will the increasing operation hurt theConsistency
of cluster?
ReplicationFactor
is a configuration parameter of consistent
package. It's not about backup count on Olric cluster. It's about key distribution and average key count on a member. According to consistent
documentation: Members are replicated on consistent hash ring. This number means that a member how many times replicated on the ring.
Please note that consistent
package doesn't aware about keys. Olric distributes partitions among members and keys are distributed among partitions by using modulo operator.
hkey = hash(key)
partID = hkey % config.partitionCount
Cluster configuration cannot be changed after initialization. So you should design your cluster properly. If your requirements change along the way, you need to reset your cluster with the new configuration.
I need to test Olric with some fake data to know the exact effect of ReplicationFactor
on operation. Now, the whole project is just experimental.
from olric.
Hi, @buraksezer . When I look at these test cases in
dmap_backup_test.go
, I think the twoOlric
instancedb1
anddb2
are bundled into a cluster pair by the peer. Is there any chance to bundle moreOlric
clusters together?
An Olric node needs to be seeded at initialization phase. Even if you have a cluster with 100 nodes, you just need to give address one of the nodes. It should discover the other nodes automatically. memberlist package provides this functionality.
Olric only supports static configuration at startup. In future, it may support more way to discover node.
https://github.com/buraksezer/olric/blob/master/config.go#L106
By the way, I designed Olric by reading Hazelcast's code and documentation. Their operations and deployment guide is a great way to learn about running distributed systems on production.
https://hazelcast.com/resources/hazelcast-deployment-operations-guide/
from olric.
@buraksezer Thanks a lot. One more thing is that the ReplicationFactor
parameters of all of members are same for now. Is there a chance assign a different ReplicationFactor
to each members, and then we can fix the heterogeneity problem of machine.
from olric.
Normally consistent hashing algorithms doesn't care about heterogeneity. In Olric, I use a different algorithm to overcome that problem. It's defined in a paper by Googlers.
https://arxiv.org/abs/1608.01350
https://ai.googleblog.com/2017/04/consistent-hashing-with-bounded-loads.html
In order to manage heterogeneity problem, you can use different combinations of ReplicationFactor
and Load
parameters in consistent
package. The following sample code from consistent
package can be very useful for experimenting:
https://github.com/buraksezer/consistent/blob/master/_examples/load_distribution.go
from olric.
Related Issues (20)
- panic: runtime error: integer divide by zero in setLRUEvictionStats HOT 1
- panic on args HOT 1
- Panic on start when BindAddr cannot be resolved
- Data race when getting stats in embedded mode
- remove dtopic listener didn't work well in cluster mode
- DNS based service discovery HOT 8
- Fast restart will cause data lost HOT 1
- Flaky integration test
- Using olric from PHP programs HOT 3
- storage overhead per object HOT 3
- Allow developer to pass his/her logging framework of choice HOT 9
- Use of DM Lock with Embedded Member HOT 2
- Failed to fetch the latest version of the routing table
- Can't reliably get PubSub - "no available client found"
- Data race in cluster setup using Olric v0.5.5 HOT 1
- Performance issues for production usage HOT 3
- Entire cluster fail to serve requests when one pod fails to restart with DMap creation operation timeout HOT 4
- Potential lock contention: Coordinator is not able to publish routing table to new node which causes it fails to be bootstrapped HOT 1
- Olric Clustered over different Kubernetes Clusters HOT 1
- panic: DMapPipeline.execOnPartition
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from olric.