Comments about Multi-GPU and Torch about deepframeworks HOT 1 OPEN

Thanks @sirotenko

1. Agreed that distributed training of CNNs is harder. However, keep in mind that: (a) although there aren't many empirical evidences, 1-bit (or 2/4-bit) quantization of the gradient is a generic and cool technique for distributed training overall; (b) We don't have good benchmarks even single-node training, let alone distributed training; hence I don't provide ratings in the multi-GPU performance area (although I personally believe that CNTK would easily win).
2. Model deployment. It's not about whether something can be done but whether it can be done easily and fits well with the rest of the production pipeline. As an evidence, my friend had trouble deploying a trained Torch model on Android.
3. It's true that using Torch, you may need to use many modules. I personally like it because it keeps the architecture modular and compact. For example, most NN toolkits would embed SGD, etc. but in Torch, SGD is contained in the optim package and the expected scope of optim isn't limited to just NN or ML.

from deepframeworks.