communication between containers across clusters about worker HOT 5 CLOSED

There is no built-in service discovery in Nebula (yet), if you have 2 micro-services named app-a & app-b that has to talk with each other you have 3 options I can think of right now:

1. Have them all run in the same pod by running them on the same worker docker run -e APP_NAME="app-a,app-b" ... <your_worker_manager_container>, this will ensure they are both running on the same server so you only have to bind the containers to the host and you can access it through the host's ports\ip, I think adding a user networking option and\or a default Nebula user network will make that option even simpler as it will allow container name DNS resolution inside each pod\instance automatically
2. Have each microservice treated as it's own service, setting up a cluster of load balanced instances for each microservice and have them communicate with each other through said LB's.
3. set up your own service discovery with Consul\ETCD\etc...

Nebula really have 2 different use cases I can think of, the first being IoT\distrubuted deployments which will lend itself better to the first option (no need having your smart fridge talking to your smart lightbalbs, but you might want to have the smart fridge what-to-order onboard DB talk with the screen component), the second is large scale high traffic apps that are distributed to fewer regions which will lend it self better to the second option.

I would be interested to hear about others thoughts\suggestions on the subject as there are a billion ways to network things together and while I would prefer giving all the options and have everybody choose what they use it might be more realistic choosing a default method and focusing on that with other methods being a more secondary method.

from worker.

I think something like calico plugin will be one of the approaches for enabling this app-based overlay networking. https://docs.projectcalico.org/v2.4/introduction/

With this, we may be able to get a user networking working for apps that need to communicate with each other.

from worker.

I can see Calico & other overlay networks being used as an optional add-on but I don't think it should be made a baseline feature as there are a lot of use cases which don't require any host-spanning network communications & for those use cases they will just add unneeded complicity.

If we implement the ability to have Docker user networks we could also leverage docker network plug-ins to have the overlay option configured by the user to use his preferred tool should he feel the need (the user will also have to install said plugin on the workers on it's own & any plugin prerequisites such as ETCD).

The thing that worries me with Calico (& other overlay networks for that matter) is that I don't really know of any successful usage of them outside of VPC\private environments & Nebula is designed to manage workers across the internet, that being said as a purely optional plug-in giving users the choice to make their own trade-off decisions isn't a bad idea, just as long as it's tested to work at very large scales distributed across the globe first and properly documented.

from worker.

Think we have a plan here, feel free to reopen if any other ideas come up that aren't covered in the network\plugin option tickets created as a result of this discussion.

Personally the plugin support is something I prioritize rather low down the totem pole but if someone feels like handling the implementation it will of course go faster.

from worker.

Partial solution implemented in 1.1.0, Nebula now uses Docker networks so you can have networks manually created with whatever plugin you wish then start the containers and config the nebula-app to use that network.

from worker.