styxlab / kubernetes-mongodb-shard Goto Github PK

View Code? Open in Web Editor NEW

37.0 5.0 15.0 28 KB

Deploy a mongodb sharded cluster on kubernetes.

License: MIT License

Makefile 5.27% JavaScript 2.51% Shell 92.22%

kubernetes-cluster shard mongodb-shard replicas

kubernetes-mongodb-shard's People

Contributors

Stargazers

Watchers

Forkers

froy001 samgarr jljlpch denistsoi pepuscz jkulak richmondwang bitlogic killheaven dudaka peterwillcn yeeezsh cugbtang minhhungit nguyenduong21

kubernetes-mongodb-shard's Issues

Question

How to add new mongo shard to this setup? Is it supported if the current nodes are out of space? Thanks.

automate.sh fails

hello , i get this error can you help ? i am running this on CentOS atomic hosts

src/automate.sh
01: Initialize replication set rs01 on node 01
Execute command on pod mongodb-shard-node01-1268161559-xdv4z and container rsp01-node01
Unable to use a TTY - input is not a terminal or the right kind of file
error: Internal error occurred: error executing command in container: container not found ("rsp01-node01")
make: *** [run] Error 1
[root@atomic01 kubernetes-mongodb-shard]#

Having this error

The authenticity of host 'k8s-agent-303974a5-0 (xx.xxx.0.4)' can't be established.
ECDSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxM4Qk1UmqpaiSpVk89b+4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'k8s-agent-303974a5-0,xx.xxx.0.4' (ECDSA) to the list of known hosts.
Permission denied (publickey).

replSetReconfig should only be run on PRIMARY, but my state is REMOVED ...

There are a few errors that seem prevent this from working successfully.

$ make run
src/create.sh
Create Deployment on machine 1**.6**.2**.2** (node01)...
service "mongodb-node01" created
deployment.extensions "mongodb-shard-node01" created
Create Deployment on machine 1**.6**.2**.1** (node02)...
service "mongodb-node02" created
deployment.extensions "mongodb-shard-node02" created
Create Deployment on machine 1**.9**.1**.1** (node03)...
service "mongodb-node03" created
deployment.extensions "mongodb-shard-node03" created
Create Deployment on machine 2**.1**.2**.1** (node04)...
service "mongodb-node04" created
deployment.extensions "mongodb-shard-node04" created
Create Deployment on machine 2**.1**.2**.5** (node05)...
service "mongodb-node05" created
deployment.extensions "mongodb-shard-node05" created
sleep 120
src/automate.sh
01: Initialize replication set rs01 on node 01
Execute command on pod mongodb-shard-node01-6db647f94b-6cz52 and container rsp01-node01
Unable to use a TTY - input is not a terminal or the right kind of file
MongoDB shell version: 3.2.20
connecting to: 127.0.0.1:27020/test
{
	"info2" : "no configuration specified. Using a default configuration for the set",
	"me" : "mongodb-shard-node01-6db647f94b-6cz52:27020",
	"info" : "try querying local.system.replset to see current configuration",
	"ok" : 0,
	"errmsg" : "already initialized",
	"code" : 23
}
{
	"_id" : "rs01",
	"version" : 1,
	"protocolVersion" : NumberLong(1),
	"members" : [
		{
			"_id" : 0,
			"host" : "mongodb-shard-node01-6db647f94b-v9s5n:27020",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

			},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		}
	],
	"settings" : {
		"chainingAllowed" : true,
		"heartbeatIntervalMillis" : 2000,
		"heartbeatTimeoutSecs" : 10,
		"electionTimeoutMillis" : 10000,
		"getLastErrorModes" : {

		},
		"getLastErrorDefaults" : {
			"w" : 1,
			"wtimeout" : 0
		},
		"replicaSetId" : ObjectId("5b524669dc9071c34d154519")
	}
}
mongodb-node01.default.svc.cluster.local:27020
5
{
	"ok" : 0,
	"errmsg" : "No host described in new configuration 24891 for replica set rs01 maps to this node",
	"code" : 103
}
{
	"ok" : 0,
	"errmsg" : "replSetReconfig should only be run on PRIMARY, but my state is REMOVED; use the \"force\" argument to override",
	"code" : 10107
}
{
	"ok" : 0,
	"errmsg" : "replSetReconfig should only be run on PRIMARY, but my state is REMOVED; use the \"force\" argument to override",
	"code" : 10107
}
bye
Initialize Config Server Replication Set
Unable to use a TTY - input is not a terminal or the right kind of file
MongoDB shell version: 3.2.20
connecting to: 127.0.0.1:27018/test
2018-07-20T20:37:41.140+0000 E QUERY    [thread1] SyntaxError: missing ] after element list @(shell):5:2

bye
command terminated with exit code 1
make: *** [run] Error 1

./build/node01-deployment.yaml": services "mongodb-node01" already exists

hello

through trial and errors i have managed to delete the deployment because i want to start over with this. please see the logs below and tell me what can i do to keep this going.

kubectl delete service mongodb-node01
service "mongodb-node01" deleted
[root@atomic01 kubernetes-mongodb-shard]# make run
src/create.sh
Create Deployment on machine atomic02 (node01)...
service "mongodb-node01" created
Error from server (AlreadyExists): error when creating "./build/node01-deployment.yaml": deployments.extensions "mongodb-shard-node01" already exists
make: *** [run] Error 1
[root@atomic01 kubernetes-mongodb-shard]#

Kubernetes 1.10: connection refused when executing 'mongo' command

Cluster: 5 nodes on DigitalOcean
Kubernetes version: 1.10

Deployment appears to be successful:

$ kubectl get pods
NAME                                    READY     STATUS    RESTARTS   AGE
mongodb-shard-node01-6db647f94b-dmd9m   5/5       Running   0          32m
mongodb-shard-node02-65d9c95848-ts4wq   5/5       Running   0          32m
mongodb-shard-node03-7c6fc4c959-6jstm   5/5       Running   0          32m
mongodb-shard-node04-5c6fb98596-kjmnp   5/5       Running   0          32m
mongodb-shard-node05-7684db7866-59j2r   5/5       Running   0          32m

Trying execute mongo on the first node simply hangs. Nodes 2 through 5 report connection refused when debug mode is enabled:

$ DEBUG=1 kubectl exec -ti mongodb-shard-node03-7c6fc4c959-6jstm -c mgs01-node03 mongo
I0719 14:54:54.754234   92967 logs.go:49] (0xc4206b8210) (0xc4205df9a0) Create stream
I0719 14:54:54.754473   92967 logs.go:49] (0xc4206b8210) (0xc4205df9a0) Stream added, broadcasting: 1
I0719 14:54:54.831555   92967 logs.go:49] (0xc4206b8210) Reply frame received for 1
I0719 14:54:54.831607   92967 logs.go:49] (0xc4206b8210) (0xc420526000) Create stream
I0719 14:54:54.831621   92967 logs.go:49] (0xc4206b8210) (0xc420526000) Stream added, broadcasting: 3
I0719 14:54:54.901498   92967 logs.go:49] (0xc4206b8210) Reply frame received for 3
I0719 14:54:54.901558   92967 logs.go:49] (0xc4206b8210) (0xc420692000) Create stream
I0719 14:54:54.901573   92967 logs.go:49] (0xc4206b8210) (0xc420692000) Stream added, broadcasting: 5
I0719 14:54:54.977888   92967 logs.go:49] (0xc4206b8210) Reply frame received for 5
I0719 14:54:54.977940   92967 logs.go:49] (0xc4206b8210) (0xc4205260a0) Create stream
I0719 14:54:54.977958   92967 logs.go:49] (0xc4206b8210) (0xc4205260a0) Stream added, broadcasting: 7
I0719 14:54:55.048296   92967 logs.go:49] (0xc4206b8210) Reply frame received for 7
I0719 14:54:55.048505   92967 logs.go:49] (0xc4205260a0) (7) Writing data frame
I0719 14:54:55.056748   92967 logs.go:49] (0xc4206b8210) Data frame received for 5
I0719 14:54:55.056775   92967 logs.go:49] (0xc420692000) (5) Data frame handling
I0719 14:54:55.355209   92967 logs.go:49] (0xc4206b8210) Data frame received for 5
I0719 14:54:55.355234   92967 logs.go:49] (0xc420692000) (5) Data frame handling
I0719 14:54:55.355247   92967 logs.go:49] (0xc420692000) (5) Data frame sent
MongoDB shell version: 3.2.20I0719 14:54:55.364324   92967 logs.go:49] (0xc4206b8210) Data frame received for 5
I0719 14:54:55.364354   92967 logs.go:49] (0xc420692000) (5) Data frame handling
I0719 14:54:55.364378   92967 logs.go:49] (0xc420692000) (5) Data frame sent
I0719 14:54:55.597671   92967 logs.go:49] (0xc4206b8210) Data frame received for 5
I0719 14:54:55.597698   92967 logs.go:49] (0xc420692000) (5) Data frame handling
I0719 14:54:55.597714   92967 logs.go:49] (0xc420692000) (5) Data frame sent
connecting to: test
I0719 14:54:55.599020   92967 logs.go:49] (0xc4206b8210) Data frame received for 5
I0719 14:54:55.599042   92967 logs.go:49] (0xc420692000) (5) Data frame handling
I0719 14:54:55.599054   92967 logs.go:49] (0xc420692000) (5) Data frame sent
2018-07-19T19:54:54.964+0000 W NETWORK  [thread1] Failed to connect to 127.0.0.1:27017, in(checking socket for error after poll), reason: errno:111 Connection refused
I0719 14:54:55.615411   92967 logs.go:49] (0xc4206b8210) Data frame received for 5
I0719 14:54:55.615446   92967 logs.go:49] (0xc420692000) (5) Data frame handling
I0719 14:54:55.615467   92967 logs.go:49] (0xc420692000) (5) Data frame sent
2018-07-19T19:54:54.973+0000 E QUERY    [thread1] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed :
connect@src/mongo/shell/mongo.js:229:14
@(connect):1:6
I0719 14:54:55.615818   92967 logs.go:49] (0xc4206b8210) Data frame received for 5
I0719 14:54:55.615841   92967 logs.go:49] (0xc420692000) (5) Data frame handling
I0719 14:54:55.615858   92967 logs.go:49] (0xc420692000) (5) Data frame sent
exception: connect failed
I0719 14:54:55.666426   92967 logs.go:49] (0xc4206b8210) Data frame received for 1
I0719 14:54:55.666468   92967 logs.go:49] (0xc4206b8210) (0xc420526000) Stream removed, broadcasting: 3
I0719 14:54:55.666501   92967 logs.go:49] (0xc4205df9a0) (1) Data frame handling
I0719 14:54:55.666514   92967 logs.go:49] (0xc4205df9a0) (1) Data frame sent
I0719 14:54:55.666561   92967 logs.go:49] (0xc4206b8210) (0xc420526000) Stream removed, broadcasting: 3
I0719 14:54:55.666604   92967 logs.go:49] (0xc4206b8210) (0xc420692000) Stream removed, broadcasting: 5
I0719 14:54:55.666648   92967 logs.go:49] (0xc4206b8210) (0xc4205260a0) Stream removed, broadcasting: 7
I0719 14:54:55.666680   92967 logs.go:49] (0xc4206b8210) (0xc4205df9a0) Stream removed, broadcasting: 1
I0719 14:54:55.666708   92967 logs.go:49] (0xc4206b8210) Go away received
I0719 14:54:55.666938   92967 logs.go:49] (0xc4206b8210) (0xc4205df9a0) Stream removed, broadcasting: 1
I0719 14:54:55.666961   92967 logs.go:49] (0xc4206b8210) (0xc420526000) Stream removed, broadcasting: 3
I0719 14:54:55.666979   92967 logs.go:49] (0xc4206b8210) (0xc420692000) Stream removed, broadcasting: 5
I0719 14:54:55.666997   92967 logs.go:49] (0xc4206b8210) (0xc4205260a0) Stream removed, broadcasting: 7
command terminated with exit code 1

$ kubectl describe pod mongodb-shard-node05-7684db7866-59j2r
Name:           mongodb-shard-node05-7684db7866-59j2r
Namespace:      default
Node:           2**.1**.2**.5**/1**.1**.2**.1**
Start Time:     Thu, 19 Jul 2018 14:21:58 -0500
Labels:         app=mongodb-shard-node05
                pod-template-hash=3240863422
                role=mongoshard
                tier=backend
Annotations:    <none>
Status:         Running
IP:             1**.2**.1**.2**
Controlled By:  ReplicaSet/mongodb-shard-node05-7684db7866
Containers:
  arb04-node05:
    Container ID:  docker://0a4f0607e3845b58b0e3b969340d596c8131852e31c5f56ed3cd97196d7048e7
    Image:         mongo:3.2
    Image ID:      docker-pullable://mongo@sha256:2ea3783f7ee258361a4264ea1d5b69b79e66abef97a26d2e92f493331368c9c9
    Port:          27019/TCP
    Host Port:     0/TCP
    Args:
      --storageEngine
      wiredTiger
      --replSet
      rs04
      --port
      27019
      --noprealloc
      --smallfiles
    State:          Running
      Started:      Thu, 19 Jul 2018 14:22:07 -0500
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /data/db from db-rs04 (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-qcxbq (ro)
  rss03-node05:
    Container ID:  docker://4dc9a2ef25fbc9714bec1724ec7002b1cc7635653d94a9a47e55726de5d7e743
    Image:         mongo:3.2
    Image ID:      docker-pullable://mongo@sha256:2ea3783f7ee258361a4264ea1d5b69b79e66abef97a26d2e92f493331368c9c9
    Port:          27021/TCP
    Host Port:     0/TCP
    Args:
      --storageEngine
      wiredTiger
      --replSet
      rs03
      --port
      27021
      --noprealloc
      --smallfiles
    State:          Running
      Started:      Thu, 19 Jul 2018 14:22:07 -0500
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /data/db from db-rs03 (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-qcxbq (ro)
  rsp02-node05:
    Container ID:  docker://0e976ecdf8abb138f6b3c806184a0aadce2a2c229efaf8aa612a2dc290358194
    Image:         mongo:3.2
    Image ID:      docker-pullable://mongo@sha256:2ea3783f7ee258361a4264ea1d5b69b79e66abef97a26d2e92f493331368c9c9
    Port:          27020/TCP
    Host Port:     0/TCP
    Args:
      --storageEngine
      wiredTiger
      --replSet
      rs02
      --port
      27020
      --noprealloc
      --smallfiles
    State:          Running
      Started:      Thu, 19 Jul 2018 14:22:07 -0500
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /data/db from db-rs02 (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-qcxbq (ro)
  cfg01-node05:
    Container ID:  docker://108f9231bc43bcb4ed2e56a40b1cbdf0dd382f1fc6bb7327d45b93645940a483
    Image:         mongo:3.2
    Image ID:      docker-pullable://mongo@sha256:2ea3783f7ee258361a4264ea1d5b69b79e66abef97a26d2e92f493331368c9c9
    Port:          27018/TCP
    Host Port:     0/TCP
    Args:
      --storageEngine
      wiredTiger
      --configsvr
      --replSet
      configReplSet01
      --port
      27018
      --noprealloc
      --smallfiles
    State:          Running
      Started:      Thu, 19 Jul 2018 14:22:08 -0500
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /data/db from db-cfg (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-qcxbq (ro)
  mgs01-node05:
    Container ID:  docker://02cdbf90cdfe50a22e570ed0e7b1f86612ef4e3f29bc03986aaf76b9178a7fc5
    Image:         mongo:3.2
    Image ID:      docker-pullable://mongo@sha256:2ea3783f7ee258361a4264ea1d5b69b79e66abef97a26d2e92f493331368c9c9
    Port:          27017/TCP
    Host Port:     0/TCP
    Command:
      mongos
    Args:
      --configdb
      configReplSet01/mongodb-node01.default.svc.cluster.local:27018,mongodb-node02.default.svc.cluster.local:27018,mongodb-node03.default.svc.cluster.local:27018,mongodb-node04.default.svc.cluster.local:27018,mongodb-node05.default.svc.cluster.local:27018
      --port
      27017
    State:          Running
      Started:      Thu, 19 Jul 2018 14:22:08 -0500
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-qcxbq (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          True
  PodScheduled   True
Volumes:
  db-cfg:
    Type:          HostPath (bare host directory volume)
    Path:          /data/mongodb/db-cfg
    HostPathType:
  db-rs02:
    Type:          HostPath (bare host directory volume)
    Path:          /data/mongodb/db-rs02
    HostPathType:
  db-rs03:
    Type:          HostPath (bare host directory volume)
    Path:          /data/mongodb/db-rs03
    HostPathType:
  db-rs04:
    Type:          HostPath (bare host directory volume)
    Path:          /data/mongodb/db-rs04
    HostPathType:
  default-token-qcxbq:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-qcxbq
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  kubernetes.io/hostname=2**.1**.2**.5**
Tolerations:     <none>
Events:
  Type    Reason                 Age   From                     Message
  ----    ------                 ----  ----                     -------
  Normal  Scheduled              41m   default-scheduler        Successfully assigned mongodb-shard-node05-7684db7866-59j2r to 2**.1**.2**.5**
  Normal  SuccessfulMountVolume  41m   kubelet, 2**.1**.2**.5**  MountVolume.SetUp succeeded for volume "db-rs04"
  Normal  SuccessfulMountVolume  41m   kubelet, 2**.1**.2**.5**  MountVolume.SetUp succeeded for volume "db-rs03"
  Normal  SuccessfulMountVolume  41m   kubelet, 2**.1**.2**.5**  MountVolume.SetUp succeeded for volume "db-cfg"
  Normal  SuccessfulMountVolume  41m   kubelet, 2**.1**.2**.5**  MountVolume.SetUp succeeded for volume "db-rs02"
  Normal  SuccessfulMountVolume  41m   kubelet, 2**.1**.2**.5**  MountVolume.SetUp succeeded for volume "default-token-qcxbq"
  Normal  Pulling                41m   kubelet, 2**.1**.2**.5**  pulling image "mongo:3.2"
  Normal  Pulled                 41m   kubelet, 2**.1**.2**.5**  Container image "mongo:3.2" already present on machine
  Normal  Created                41m   kubelet, 2**.1**.2**.5**  Created container
  Normal  Started                41m   kubelet, 2**.1**.2**.5**  Started container
  Normal  Pulled                 41m   kubelet, 2**.1**.2**.5**  Container image "mongo:3.2" already present on machine
  Normal  Pulled                 41m   kubelet, 2**.1**.2**.5**  Successfully pulled image "mongo:3.2"
  Normal  Started                41m   kubelet, 2**.1**.2**.5**  Started container
  Normal  Pulled                 41m   kubelet, 2**.1**.2**.5**  Container image "mongo:3.2" already present on machine
  Normal  Created                41m   kubelet, 2**.1**.2**.5**  Created container
  Normal  Started                41m   kubelet, 2**.1**.2**.5**  Started container
  Normal  Created                41m   kubelet, 2**.1**.2**.5**  Created container
  Normal  Created                41m   kubelet, 2**.1**.2**.5**  Created container
  Normal  Started                41m   kubelet, 2**.1**.2**.5**  Started container
  Normal  Pulled                 41m   kubelet, 2**.1**.2**.5**  Container image "mongo:3.2" already present on machine
  Normal  Created                41m   kubelet, 2**.1**.2**.5**  Created container
  Normal  Started                41m   kubelet, 2**.1**.2**.5**  Started container

I was able to connect and execute mongo on my regular MongoDB replica cluster which is on the same Kubernetes cluster.

Any thoughts would be more than appreciated! :)

generate.sh creates invalid JSON

Running into this error:

Initialize Config Server Replication Set
Unable to use a TTY - input is not a terminal or the right kind of file
MongoDB shell version: 3.2.20
connecting to: 127.0.0.1:27018/test
2018-07-20T20:37:41.140+0000 E QUERY    [thread1] SyntaxError: missing ] after element list @(shell):5:2

bye

I noticed that cfg01-init.js looks like:

rs.initiate( {
   _id: "configReplSet01",
   configsvr: true,
   members: [
tt{ _id: 0, host: "mongodb-node01.default.svc.cluster.local:27018" },ntt{ _id: 1, host: "mongodb-node02.default.svc.cluster.local:27018" },ntt{ _id: 2, host: "mongodb-node03.default.svc.cluster.local:27018" },ntt{ _id: 3, host: "mongodb-node04.default.svc.cluster.local:27018" },ntt{ _id: 4, host: "mongodb-node05.default.svc.cluster.local:27018" }
   ]
} )

The newline (\n) and tab (\t) are being converted to n and t (respectively) instead of being maintained. This appears to be with the current usage of sed.

		CONFIG_SERVERS_SERVICES_JS="${CONFIG_SERVERS_SERVICES_JS},\n\t\t{ _id: ${ID}, host: \"mongodb-node${NODENUM}.default.svc.cluster.local:${PORT}\" }"

Can easily be changed to:

		CONFIG_SERVERS_SERVICES_JS="${CONFIG_SERVERS_SERVICES_JS},{ _id: ${ID}, host: \"mongodb-node${NODENUM}.default.svc.cluster.local:${PORT}\" }"

On a side note, nobody really cares if there are tabs or newlines in there since JSON doesn't rely on them. Sure, it makes things look pretty but maybe utilize a JSON parser or validator instead of trying to manually accomplish this kind of pretty-printing.

How to expose mongos externally?

I followed along and have everything running. Under the consume section of the readme it shows the lines for the exposed services but the only address it shows is the cluster-ip. The external-ip shows none. Is there something in the configuration that maps the mongos server to an external IP? I ask because when I try to connect to the listed servers from robomongo for example, I cannot connect using the cluster ip address or the address of the node.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.