georgedimitriadis / spikesorting_tsne Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
I'm attempting to run TSNE on a machine in the cloud, but I'm having trouble. This machine has successfully run tensorflow operations before, so I know the Nvidia drivers are set up properly. I'm running Nvidia Driver Version: 390.30, numba version 0.39.0, and pyculib 1.0.2, all on a Tesla K80. Do you have any insight into why this error would be happening? Thanks in advance!
LOADING UP THE GPU
Loading matrix time: 0.010 s
ITERATION NUMBER: 1
Making the dot products Time: 0.008 s
Summing the dot products on the GPU Time: 0.178 s
cuBLAS gemm Time: 0.174 s
Number of sorting segments = 6
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/spikesorting_tsne/gpu.py", line 189, in _segment_sort_transposed_distances_get_knns
sorting.segmented_sort(keys=keys, vals=values, segments=segments)
File "/opt/conda/lib/python3.6/site-packages/pyculib/sorting/segsort.py", line 115, in segmented_sort
_segmentedsort(d_keys, d_vals, d_segments, stream)
File "/opt/conda/lib/python3.6/contextlib.py", line 88, in __exit__
next(self.gen)
File "/opt/conda/lib/python3.6/site-packages/pyculib/sorting/segsort.py", line 84, in _autodevice
dptr.copy_to_host(ary, stream=stream)
File "/opt/conda/lib/python3.6/site-packages/numba/cuda/cudadrv/devices.py", line 212, in _require_cuda_context
return fn(*args, **kws)
File "/opt/conda/lib/python3.6/site-packages/numba/cuda/cudadrv/devicearray.py", line 241, in copy_to_host
_driver.device_to_host(hostary, self, self.alloc_size, stream=stream)
File "/opt/conda/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 1606, in device_to_host
fn(host_pointer(dst), device_pointer(src), size, *varargs)
File "/opt/conda/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 288, in safe_cuda_api_call
self._check_error(fname, retcode)
File "/opt/conda/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 323, in _check_error
raise CudaAPIError(retcode, msg)
numba.cuda.cudadrv.driver.CudaAPIError: [700] Call to cuMemcpyDtoH results in UNKNOWN_CUDA_ERROR
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/spikesorting_tsne/gpu.py", line 294, in calculate_knn_distances
number_of_sorts=number_of_sorts, verbose=verbose)
File "/opt/conda/lib/python3.6/site-packages/spikesorting_tsne/gpu.py", line 198, in _segment_sort_transposed_distances_get_knns
print(' Sorted {} of {} segments of this iteration'.format(i, p.shape[0] - 1))
File "/opt/conda/lib/python3.6/site-packages/spikesorting_tsne/stopwatch.py", line 82, in __exit__
raise type
TypeError: __init__() missing 2 required positional arguments: 'code' and 'msg'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/spikesorting_tsne/gpu.py", line 302, in calculate_knn_distances
' ITERATIONS')
File "/opt/conda/lib/python3.6/site-packages/spikesorting_tsne/stopwatch.py", line 82, in __exit__
raise type
TypeError
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.