Distributed Word2Vec about deeplearning4j HOT 6 CLOSED

Deep Learning can sometimes take a bit of time.

That's why we have a parallelized Hadoop-yarn version

https://github.com/jpatanooga/Metronome

which I'm working on moving over into:

https://github.com/agibsonccc/java-deeplearning/tree/master/hadoop-yarn

On Thu, Aug 7, 2014 at 1:59 AM, Samiur rahman [email protected]
wrote:

Training a Word2Vec model on a large corpus (approximately 3 billion
english words) takes a significant amount of time. On a Core i7- 8 core
machine, it took approximately 24 hours to train. Distributing this will
significantly reduce training time.

—
Reply to this email directly or view it on GitHub
#44.

from deeplearning4j.

I will also be creating a quick and dirty akka version of this. It will be pretty straight forward. I have another engineer looking at GPUs this week. Getting that incorporated should help a little bit with benchmarks and all that. As of right now unfortunately, we're still in the stabilization phase, once we have a stable API and something easy to build on top of, finetuning (even with JNI if need be) will be pretty straightforward.

from deeplearning4j.

I surfaced one bottleneck that in retrospect was obvious. Check out

https://github.com/agibsonccc/java-deeplearning/tree/api-overhaul

when you get the chance. It's not distributed (still have other things to work on) but should be a performance improvement.

from deeplearning4j.

It looks like Apache Spark is adopting this in MLLib for Spark 1.1. It might be worth making the generated models compatible

from deeplearning4j.

Update on this: I've implemented a general purpose distributed system that will make neural nets a plugin. The word2vec plugin is almost done. I've already added the baseline feedforward nets and simpler things like word count/tfidf. Please check the 0.0.3.3 branch.

from deeplearning4j.

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

from deeplearning4j.