Computation on CondMat_network example takes too much time about largevis HOT 5 CLOSED

Are you sure you can run 4 threads on your machine? However, the memory consumption shouldn't be that high. I will do some tests.

If you are in linux, run lscpu. Then to obtain the total number of threads consider the nex info:
Threads(s) per core: 2
Core(s) per socket: 4
Socket(s): 1

In this example, you can run max. 2x4x1 = 8 threads

from largevis.

Are you using the python wrapper for your test?

from largevis.

In my machine, 8 x Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60 GHz CondMat_network takes (Note: I used the C++ implementation directly)

2 threads: ~16m 27s
4 threads: ~8m 12s
8 threads: ~5m 33s

All of them using 391mb RAM.
The algorithm is CPU intensive, so it is OK a 90-100% CPU usage.

from largevis.

Thanks, the numbers that you gave in the last comment are comparable to mine, when taking into account number of cores/processor_speed/ number_of_threads.

For the other questions: I used the c++ implementation directly (no python wrapper). The first comment was not clear from beginning to end, but in principle you can run a large number of threads on any machine (unless limited by application). However, after a certain number of threads the performances will degrade due to ''context switch'' time. In my case with 2 threads the CPU was under-utilized so I tried 4 and it was fully utilized (and the program ran faster).

from largevis.

I will close this for now, as it is not really an issue. It is more important that it is efficient on large datasets (compared to bh-tSNE).

from largevis.