rubenafo / docker-spark-cluster Goto Github PK

I followed the steps laid out to get the cluster running. However the last step I keep getting network issues. What am I missing, where should the network be set up?

Must be a rookie issue :)

Below output on bash when running ./cluster.sh deploy

d0a34aebc25e
b275e68a1eb1
d396b6f66f84
5620e240a15c
Error: No such network: sparknet
Error response from daemon: could not find plugin bridge in v1 plugin registry: plugin not found

Starting nodes master and worker nodes ...
b3845db3e039c4b19f0fcf8c392d332e7ea14e5b5a4eb6163b9a22bf8ec5bfe4
docker: Error response from daemon: network sparknet not found.
3df880ee2c12782f63e25720b0766871cbcad450fde70aa7545162cd5ca39454
docker: Error response from daemon: network sparknet not found.
b209f7b1cbdfe5e58d995c7b8a5bb8bb7a8ade217609d5b9a9ff7ccc50794e46
docker: Error response from daemon: network sparknet not found.
e34b422b13a75a1693ef359fd451b42b09cc6db342182c8337446fdaa7f019b6
docker: Error response from daemon: network sparknet not found.
Formatting hdfs ...
Error response from daemon: Container b3845db3e039c4b19f0fcf8c392d332e7ea14e5b5a4eb6163b9a22bf8ec5bfe4 is not running
Error response from daemon: network sparknet not found
Error response from daemon: network sparknet not found
Error response from daemon: network sparknet not found
Error response from daemon: network sparknet not found
Error: failed to start containers: nodemaster, node2, node3, node4
Starting hdfs ...
Error response from daemon: Container b3845db3e039c4b19f0fcf8c392d332e7ea14e5b5a4eb6163b9a22bf8ec5bfe4 is not running
Starting yarn ...
Error response from daemon: Container b3845db3e039c4b19f0fcf8c392d332e7ea14e5b5a4eb6163b9a22bf8ec5bfe4 is not running
Starting Spark ...
Error response from daemon: Container b3845db3e039c4b19f0fcf8c392d332e7ea14e5b5a4eb6163b9a22bf8ec5bfe4 is not running
Error response from daemon: Container 3df880ee2c12782f63e25720b0766871cbcad450fde70aa7545162cd5ca39454 is not running
Error response from daemon: Container b209f7b1cbdfe5e58d995c7b8a5bb8bb7a8ade217609d5b9a9ff7ccc50794e46 is not running
Error response from daemon: Container e34b422b13a75a1693ef359fd451b42b09cc6db342182c8337446fdaa7f019b6 is not running
Hadoop info @ nodemaster: http://:8088/cluster
Spark info @ nodemater : http://:8080/
DFS Health @ nodemaster : http://:9870/dfshealth.html

Greetings ! Can you help with error on UNPROTECTED PRIVATE KEY FILE please??

attaching error messages...

Thanks!
Regards
Raj

When I run the command sudo docker attach node4, nothing happens except waiting.
And after I run the command ctrl + c, this container is disappeared.

The tutorial as far as i see allow creating multinodes but all of them on single computer.
Did you try to build one using network of physical computers?

the docker rm at deployment ie removing existing working docker container which should not happen

  docker rm -f `docker ps -aq` # delete old containers
  docker network rm sparknet
  docker network create --driver bridge sparknet # create custom network```