juliagpu / cudanativelib.jl Goto Github PK

View Code? Open in Web Editor NEW

12.0 12.0 6.0 42 KB

License: Other

Julia 100.00%

cuda ffi julia

cudanativelib.jl's People

Contributors

Stargazers

Watchers

Forkers

waltsims tkelman stjordanis ranocha

cudanativelib.jl's Issues

Support for device level CUBLAS

Early support was added with 500d471, but this currently only includes the helper function an non of the actual kernel functions.

This relies on stack allocation within C++ and then returning the struct to Julia (I have to check that this generates correct code)

Ideally the code would look more like this:

cublasCreate(handle) = icxx"cublasCreate($handle);"
@target ptx function f()
   r_handle = Ref{cublasHandle_t}()
   cublasCreate(r_handle)
   handle = r_handle[]
 end

But this won't work till we have stack allocated references... curand.jl works around this by doing heap allocation.

@MichaelOhlrogge this is still a WIP and I have to figure out the proper way for a few thins, but let me know what you think.

TODO:

Verify that the current approach with stack allocation (in C++) and returning values works. (It probably doesn't) Ideally we have stack allocation support in Julia
Link against cublas_static.a
Implement Level 1
Implement Level 2
Implement Level 3
Expose higlevel api similar to CUBLAS.jl

@kshyatt If Cxx.jl wouldn't be such a heavy dependency we could use exactly the same code for CUBLAS.jl proper.

Precompilation

Precompilation is currently disabled, because Cxx.jl currently doesn't support it. This makes loading quite slow

time CUDA_HOME="/opt/cuda-7.0" ./julia -e "import CUDAnativelib"

real	0m16.622s
user	0m16.547s
sys	0m0.840s

x-ref: JuliaInterop/Cxx.jl#181

LLVM 4.0 requirement for CUDA 8.0

I have been testing out the update branch of CUDAnativelib.jl

despite having LLVM 4.0 installed on my system, Pkg.build("CUDAnativelib") builds with LLVM3.9 (also installed on my system)

here are the steps I have taken in this matter:

when running this sample code:

#!/usr/local/bin/julia

#requirementsTest.jl

using CUDAdrv
using CUDAnative
using CUDAnativelib
using .CURANDkernel
using Base.Test

dev=CuDevice(0)
ctx=CuContext(dev)

destroy(ctx)

wsimson@m03map28:~/WalkingOnSpheres.jl$ ./requirementTest 
ERROR: LoadError: LoadError: LoadError: For CUDA 8.0+ we require at least LLVM 4.0
Stacktrace:
 [1] error(::String) at ./error.jl:21
 [2] include_from_node1(::String) at ./loading.jl:539
 [3] include(::String) at ./sysimg.jl:14
 [4] include_from_node1(::String) at ./loading.jl:539
while loading /home/wsimson/.julia/v0.6/CUDAnativelib/src/codegen.jl, in expression starting on line 11
while loading /home/wsimson/.julia/v0.6/CUDAnativelib/src/CUDAnativelib.jl, in expression starting on line 38
while loading /home/wsimson/WalkingOnSpheres.jl/requirementTest, in expression starting on line 5

This is how the issue came to my attention. I then took the following steps:

julia> ENV["USE_SYSTEM_LLVM"] = "1"
"1"

Julia> ENV["TRACE"] = "1"
"1"


julia> Pkg.build("CUDAnativelib")
INFO: Building LLVM
DEBUG: Looking for libLLVM in /home/wsimson/julia/usr/bin/../lib/julia
DEBUG: Looking for llvm-config in /home/wsimson/julia/usr/bin
DEBUG: Looking for llvm-config in /home/wsimson/julia/usr/bin/../tools
DEBUG: - 3.9.1 at /home/wsimson/julia/usr/bin/../tools/llvm-config
DEBUG: Looking for libLLVM in /home/wsimson/julia/usr/lib
DEBUG: - v3.9.1 at /home/wsimson/julia/usr/lib/libLLVM-3.9.1.so
DEBUG: - v3.9.1 at /home/wsimson/julia/usr/lib/libLLVM-3.9.so
DEBUG: Looking for llvm-config in /usr/local/cuda-8.0/bin
DEBUG: Looking for llvm-config in /usr/local/sbin
DEBUG: Looking for llvm-config in /usr/local/bin
DEBUG: Looking for llvm-config in /usr/sbin
DEBUG: Looking for llvm-config in /usr/bin
DEBUG: - 3.8.0 at /usr/bin/llvm-config-3.8
DEBUG: - 4.0.0 at /usr/bin/llvm-config-4.0
DEBUG: Looking for libLLVM in /usr/lib/llvm-3.8/lib
DEBUG: - v3.8.0 at /usr/lib/llvm-3.8/lib/libLLVM-3.8.0.so
DEBUG: - v3.8.0 at /usr/lib/llvm-3.8/lib/libLLVM-3.8.so
DEBUG: Looking for libLLVM in /usr/lib/llvm-4.0/lib
DEBUG: - v4.0.0 at /usr/lib/llvm-4.0/lib/libLLVM-4.0.0.so
DEBUG: - v4.0.0 at /usr/lib/llvm-4.0/lib/libLLVM-4.0.so
DEBUG: Looking for llvm-config in /sbin
DEBUG: Looking for llvm-config in /bin
DEBUG: Looking for llvm-config in /usr/games
DEBUG: Looking for llvm-config in /usr/local/games
INFO: Found 3 unique LLVM installations
INFO: Performing source build of LLVM extras library
DEBUG: Building for LLVM v3.9.1 at /home/wsimson/julia/usr/lib/libLLVM-3.9.1.so using /home/wsimson/julia/usr/bin/../tools/llvm-config
`make clean`
rm -f libLLVM_extra.so ./IR/Metadata.o ./IR/Core.o ./IR/Pass.o ./Target.o ./Transforms/IPO.o ./Target/NVPTX.o
`make -j9`
g++ -I/home/wsimson/julia/usr/include -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/wsimson/julia/usr/lib\" -I/home/wsimson/julia/usr/include/julia -fPIC -I/home/wsimson/julia/usr/include   -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_NVPTX -DTARGET_X86 -D_GLIBCXX_USE_CXX11_ABI=1  -c -o IR/Metadata.o IR/Metadata.cpp
g++ -I/home/wsimson/julia/usr/include -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/wsimson/julia/usr/lib\" -I/home/wsimson/julia/usr/include/julia -fPIC -I/home/wsimson/julia/usr/include   -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_NVPTX -DTARGET_X86 -D_GLIBCXX_USE_CXX11_ABI=1  -c -o IR/Core.o IR/Core.cpp
g++ -I/home/wsimson/julia/usr/include -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/wsimson/julia/usr/lib\" -I/home/wsimson/julia/usr/include/julia -fPIC -I/home/wsimson/julia/usr/include   -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_NVPTX -DTARGET_X86 -D_GLIBCXX_USE_CXX11_ABI=1  -c -o IR/Pass.o IR/Pass.cpp
g++ -I/home/wsimson/julia/usr/include -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/wsimson/julia/usr/lib\" -I/home/wsimson/julia/usr/include/julia -fPIC -I/home/wsimson/julia/usr/include   -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_NVPTX -DTARGET_X86 -D_GLIBCXX_USE_CXX11_ABI=1  -c -o Target.o Target.cpp
g++ -I/home/wsimson/julia/usr/include -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/wsimson/julia/usr/lib\" -I/home/wsimson/julia/usr/include/julia -fPIC -I/home/wsimson/julia/usr/include   -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_NVPTX -DTARGET_X86 -D_GLIBCXX_USE_CXX11_ABI=1  -c -o Transforms/IPO.o Transforms/IPO.cpp
g++ -I/home/wsimson/julia/usr/include -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/wsimson/julia/usr/lib\" -I/home/wsimson/julia/usr/include/julia -fPIC -I/home/wsimson/julia/usr/include   -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_NVPTX -DTARGET_X86 -D_GLIBCXX_USE_CXX11_ABI=1  -c -o Target/NVPTX.o Target/NVPTX.cpp
g++ -I/home/wsimson/julia/usr/include -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/wsimson/julia/usr/lib\" -I/home/wsimson/julia/usr/include/julia -fPIC -I/home/wsimson/julia/usr/include   -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_NVPTX -DTARGET_X86 -D_GLIBCXX_USE_CXX11_ABI=1 -L/home/wsimson/julia/usr/lib  "-Wl,-rpath,/home/wsimson/julia/usr/lib/" -L/home/wsimson/julia/usr/lib  -shared IR/Metadata.o IR/Core.o IR/Pass.o Target.o Transforms/IPO.o Target/NVPTX.o -L/home/wsimson/julia/usr/lib  "-Wl,-rpath,/home/wsimson/julia/usr/lib/" -L/home/wsimson/julia/usr/lib -lrt -ldl -lpthread -lm -l:libLLVM-3.9.1.so -o libLLVM_extra.so
=================================================================================[ ERROR: LLVM ]==================================================================================

LoadError: AssertionError: exclusive access mode does not match requested type of LLVM library (run with TRACE=1 and file an issue)
while loading /home/wsimson/.julia/v0.6/LLVM/deps/build.jl, in expression starting on line 223

==================================================================================================================================================================================
INFO: Building CUDAnative
INFO: Building Cxx
writing path.jl file
Tuning for julia installation at /home/wsimson/julia/usr/bin with sources possibly at /home/wsimson/julia/usr/bin/../..
INFO: Building julia source build
Not building debug library because corresponding julia DEBUG library does not exist.
To build, simply run the build again once the library at
/libjulia-debug.so
has been built.

=================================================================================[ BUILD ERRORS ]=================================================================================

WARNING: LLVM had build errors.

 - packages with build errors remain installed in /home/wsimson/.julia/v0.6
 - build the package(s) and all dependencies with `Pkg.build("LLVM")`
 - build a single package by running its `deps/build.jl` script

==================================================================================================================================================================================

have I made a mistake along the way?

Interfacing with device libraries that use reference passing

In many cases the NVIDIA apis work by passing references.
We currently don't support that since we can't have stack-allocated Refs. For curand I worked around this by using the CUDA malloc and free primitives (see

CUDAnativelib.jl/src/curand.jl

Lines 19 to 21 in 500d471

 export new, delete 

 new(::Type{curandState_t}) = icxx"return (curandState_t*) malloc(sizeof(curandState_t));" 

 delete(x) = icxx"free($x);"

),
but that is not a very satisfying solution.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

	export new, delete
	new(::Type{curandState_t}) = icxx"return (curandState_t*) malloc(sizeof(curandState_t));"
	delete(x) = icxx"free($x);"