Git Product home page Git Product logo

cbindinggen.jl's Introduction

CBinding.jl

Features of this package are now available as https://github.com/analytech-solutions/CBinding.jl

CBindingGen.jl

Build Status

Automatically generate Julia bindings to C API's! We are developing CBindingGen.jl and CBinding.jl to support the use of arbitrary local C libraries, such as those provided by your Linux distribution, from Julia.

Usage

CBindingGen.jl seeks to be a comprehensive and automatic C bindings generation framework. The bindings utilize the CBinding.jl capabilities to precisely interface Julia with C.

This package should only be used at package build time when you wish to generate bindings to a particular C library on the system or one built and installed at build time. The generated bindings file can then be included from your package code. Bindings files created by this package should not be committed with your package and they are not meant for editing by lowly humans once generated. Se let's get started with an example!

Generating

To start, you must add CBinding = "^0.9" as a dependency to your package, and CBindingGen = "^0.3,^0.4" as a build dependency. CBindingGen.jl relies on the artifacts distributed with LLVM_jll for providing a libclang.so library and header files for your system, so we will use those to demonstrate. The following code shows what is necessary to generate bindings to libclang.so, and something like it would normally be placed in your package's deps/build.jl file. (another example found in PLCTag.jl)

using CBindingGen
import LLVM_jll

incdir = joinpath(dirname(dirname(LLVM_jll.libclang_path)), "include")
hdrs = map(hdr -> joinpath("clang-c", hdr), readdir(joinpath(incdir, "clang-c")))

cvts = convert_headers(hdrs, args = ["-I", incdir]) do cursor
	header = CodeLocation(cursor).file
	name   = string(cursor)
	
	# only wrap the libclang headers
	startswith(header, "$(incdir)/") || return false
	
	# ignore function that uses time_t since we don't know what time_t is yet
	name == "clang_getFileTime" && return false
	
	return true
end

open("bindings.jl", "w+") do io
	generate(io, LLVM_jll.libclang_path => cvts)
end

The convert_headers function takes an array of header files and the command line arguments, args. Any include directories, compiler options, or preprocessor definitions would be provided in args in the same way they would be used in your clang command line. An important detail of convert_headers is the filter function provided, here provided with the do syntax. The filter function allows you fine-grained control over what is converted to Julia as the C AST is traversed. In our example, we filter out any C constructs not defined within the header files we are interested in.

We use CodeLocation(cursor) to get the file, line, and col for the start of the C expression, while CodeRange(cursor) can be used to get the start and stop locations of the expression. Additionally, string(cursor) will get the "spelling" of the expression if you wish to filter particular C symbols.

The result of convert_headers is an array of Converted objects. Converted objects contain the Julia expression strings, as expr, and comments for storing exportable symbols an their comments ported from C.

Finally, the generate function is used to write the converted expressions for one or more libraries into a bindings file.

Loading

In order to load the bindings file within your package, it is best to define a baremodule within your package module to encapsulate the bindings. The namespace within the baremodule will have only a very few symbols that could conflict with those from C. CBinding provides ๐ฃ๐ฅ (\bfj<tab>\bfl<tab>) to provide access to the CBinding types and macros without increasing the chance of naming conflicts. (another example found in PLCTag.jl)

module MyModule
	baremodule LibClang
		using CBinding: ๐ฃ๐ฅ
		
		const size_t = ๐ฃ๐ฅ.Csize_t
		๐ฃ๐ฅ.Base.include((๐ฃ๐ฅ.@__MODULE__), "pre-bindings.jl"))  # <-handwritten
		๐ฃ๐ฅ.Base.include((๐ฃ๐ฅ.@__MODULE__), ๐ฃ๐ฅ.joinpath(๐ฃ๐ฅ.dirname(๐ฃ๐ฅ.@__DIR__), "deps", "libclang.jl"))  # <-generated
		๐ฃ๐ฅ.Base.include((๐ฃ๐ฅ.@__MODULE__), "post-bindings.jl"))  # <-handwritten
	end
	
	# other module code, such as high-level Julian code wrapping the bindings...
end

Next is a section defining dependencies of the bindings and should be composed of hand-written code or imported packages that export the required symbols. Finishing the bindings module is the inclusion of the bindings file generated at build-time.

Help/Comments

CBindingGen.jl automatically imports comments from the C header files into Julia's @doc string syntax. If you find that C header comments are not imported, you should try adding -fparse-all-comments to the list of args in your call to convert_headers.

julia> import MyModule

help?> MyModule.LibClang.clang_getClangVersion
  ๐ฃ๐ฅ.@cextern clang_getClangVersion()::CXString

  Return a version string, suitable for showing to a user, but not intended to be parsed (the format is not guaranteed to be stable).

  Reference
  ===========

  Index.h:5482 (./include/clang-c/Index.h:5482:25)

Using

Use the generated bindings as you would any hand-written or generated ccall and cglobal wrappers. Remember, this is very low-level interfacing, and segmentation faults can result from misuse.

julia> cxstr = MyModule.LibClang.clang_getClangVersion();

julia> unsafe_string(MyModule.LibClang.clang_getCString(cxstr))
"clang version 8.0.1 "

julia> MyModule.LibClang.clang_disposeString(cxstr)

cbindinggen.jl's People

Contributors

juliatagbot avatar krrutkow avatar sloede avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

krrutkow sloede

cbindinggen.jl's Issues

Confusion between macro and function

I created some code to generate bindings for GNU Gettext library and it generated incorrect bindings. Any C macro that expands to a function name is translated into a Julia macro that expands to another macro. For instance:

	๐ฃ๐ฅ.@doc """
	## Reference
	
	[libintl.h:358](./..\\..\\..\\..\\.julia\\artifacts\\c38e0f0854753b42d208bf7f62aafa3100ba3880\\include\\libintl.h:358:10)
	""" :(@libintl_printf)
	export @libintl_printf
	macro libintl_printf() return quote (@__printf__) end end

        # [...]

	๐ฃ๐ฅ.@doc """
	```
	๐ฃ๐ฅ.@cextern __printf__(var\"?1?\"::๐ฃ๐ฅ.Ptr{๐ฃ๐ฅ.Cconst(๐ฃ๐ฅ.Cchar)}, var\"?vararg?\"...)::๐ฃ๐ฅ.Cint
	```
	
	## Reference
	
	[libintl.h:361](./..\\..\\..\\..\\.julia\\artifacts\\c38e0f0854753b42d208bf7f62aafa3100ba3880\\include\\libintl.h:361:12)
	""" __printf__

	๐ฃ๐ฅ.@cextern __printf__(var"?1?"::๐ฃ๐ฅ.Ptr{๐ฃ๐ฅ.Cconst(๐ฃ๐ฅ.Cchar)}, var"?vararg?"...)::๐ฃ๐ฅ.Cint

But the macro @__printf__ doesn't exist:

julia> LibGettext.@libintl_printf
ERROR: UndefVarError: @__printf__ not defined

The same problem occurs with gettext, ngettext, etc.

Avoid `UndefVarError: FILE not defined` errors

I followed the example

startswith(header, "$(incdir)/") || return false

to only generate wrappers for library headers, not system headers. However, is there a simple way to get external dependencies like the Ptr{FILE} to work, which is the reason (I assume) for the error I get during pre-compilation:

[ Info: Precompiling P4est [7d669430-f675-4ae7-b43e-fab78ec5a902]
ERROR: LoadError: LoadError: LoadError: UndefVarError: FILE not defined
Stacktrace:
 [1] top-level scope at /home/mschlott/.julia/packages/CBinding/Vcw7p/src/cbindings.jl:92
 [2] eval(::Module, ::Any) at ./boot.jl:331
 [3] top-level scope at /home/mschlott/.julia/packages/CBinding/Vcw7p/src/cbindings.jl:801
 [4] include(::Function, ::Module, ::String) at ./Base.jl:380
 [5] include(::Module, ::String) at ./Base.jl:368
 [6] top-level scope at /home/mschlott/gdrive/work/code/p4est/P4est.jll/src/libp4est.jl:5
 [7] include(::Function, ::Module, ::String) at ./Base.jl:380
 [8] include at ./Base.jl:368 [inlined]
 [9] include(::String) at /home/mschlott/gdrive/work/code/p4est/P4est.jll/src/P4est.jl:1
 [10] top-level scope at /home/mschlott/gdrive/work/code/p4est/P4est.jll/src/P4est.jl:3
 [11] include(::Function, ::Module, ::String) at ./Base.jl:380
 [12] include(::Module, ::String) at ./Base.jl:368
 [13] top-level scope at none:2
 [14] eval at ./boot.jl:331 [inlined]
 [15] eval(::Expr) at ./client.jl:467
 [16] top-level scope at ./none:3
in expression starting at /home/mschlott/gdrive/work/code/p4est/P4est.jll/deps/libp4est.jl:4
in expression starting at /home/mschlott/gdrive/work/code/p4est/P4est.jll/src/libp4est.jl:5
in expression starting at /home/mschlott/gdrive/work/code/p4est/P4est.jll/src/P4est.jl:3
ERROR: Failed to precompile P4est [7d669430-f675-4ae7-b43e-fab78ec5a902] to /home/mschlott/.julia/compiled/v1.5/P4est/OxTLc_z5FAb.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1290
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Change propagation of type names

The workaround for the lack of forward declarations in Julia propagates the type name at the type definition up through typedefs and such. Within a package this works, but when another package needs to use the higher-level typedef type names, they don't exist.

How to avoid "Method definition overwritten" warnings for forward declarations

First of all, great work @krrutkow, thanks for putting in all the effort, and also for providing CBinding.jl as well!

I'm experiencing some issues while using CBindingGen.jl and I am not sure whether it is a user question or a bug, thus I don't know if I should be asking here or on Discourse. Please let me know if I you feel like Discourse would be more appropriate!

I am trying to auto-generate bindings for the p4est library. One of the issues I am experiencing there is that they forward declare (for some simple macro shenanigans) a function sc_extern_c_hack_3 (and _4) multiple times, never providing a definition for it (it is a hack after all):

https://github.com/cburstedde/libsc/blob/7089d44934f34b60d6f63dded2cf774eebbfe1c9/src/sc.h#L110-L118

However, CBindingGen.jl), generates a C function binding each time, like so

  ๐ฃ๐ฅ.@doc """                                                                                           

๐ฃ๐ฅ.@cextern sc_extern_c_hack_3()::๐ฃ๐ฅ.Cvoid

                                                                                                      
## Reference                                                                                          
                                                                                                  
[sc_containers.h:50](~/.julia/artifacts/9347f3407298e3ccce2b12eb83235b4d80ef8e54/include/sc_containers.h:50:1)
""" sc_extern_c_hack_3                                                                            
export sc_extern_c_hack_3                                                                         
๐ฃ๐ฅ.@cextern sc_extern_c_hack_3()::๐ฃ๐ฅ.Cvoid

Thus my question is: Is there a way to avoid generating a binding multiple times (i.e., like some sort of unique pass before the generation process)? And if not, what would be the best way to get rid of it - maybe filtering for it like here:

name == "clang_getFileTime" && return false

?

How to debug an error with "Errors encountered parsing headers, unable to proceed with conversion"

While trying to generate bindings for a C library in deps/build.jl, I cannot get past the call to convert_headers, since it always throws an error as

โ”‚ ERROR: LoadError: Errors encountered parsing headers, unable to proceed with conversion
โ”‚ Stacktrace:
โ”‚  [1] error(::String) at ./error.jl:33
โ”‚  [2] (::CBindingGen.var"#18#21"{var"#1#2",Array{String,1}})(::String) at /home/mschlott/.julia/packages/CBindingGen/BiQcs/src/CBindingGen.jl:193
โ”‚  [3] mktempdir(::CBindingGen.var"#18#21"{var"#1#2",Array{String,1}}, ::String; prefix::String) at ./file.jl:709
โ”‚  [4] mktempdir(::Function, ::String) at ./file.jl:707 (repeats 2 times)
โ”‚  [5] convert_headers(::Function, ::Array{String,1}; args::Array{String,1}) at /home/mschlott/.julia/packages/CBindingGen/BiQcs/src/CBindingGen.jl:169
โ”‚  [6] top-level scope at /mnt/ssd/home/mschlott/code/SEAL.jl/deps/build.jl:56
โ”‚  [7] include(::String) at ./client.jl:457
โ”‚  [8] top-level scope at none:5
โ”‚ in expression starting at /mnt/ssd/home/mschlott/code/SEAL.jl/deps/build.jl:56

I have tried looking at CBindingGen.jl:193, but I cannot figure out how to make LibClang tell me what is actually the problem (e.g., is it a bad include directory, or is it something in the headers themselves etc.). Can you give me advice on how to proceed here, i.e., how to debug the error that caused this message?

Test failing for `F3`

Conversion of the struct F3ret *p redeclares the F3ret struct.

typedef struct F3ret { int i; float f; } (*F3[3])(union { int i; float f; } x[1], struct F3ret *p);

Support conversion of preprocessor directives (macros)

#define CONST (42)
#define FUNC(x,y) ((x)*(y))

should become something like

const CONST = 42
const FUNC = (x,y) -> x*y

A tricky situation occurs when this is done:

enum E {
   E_1 = 0
   #define E_1 0
};

But assuming the symbol already exists, we can also assume they are equivalent values, perhaps displaying a warning.

Error with imported docs that contain `$`

One of the C header files for which I used CBindingGen.jl contained $ signs, which caused Julia to produce numerous errors like this

syntax: invalid interpolation syntax: "$ "

Is there a way to disable comment generation and/or to automatically convert $ in docstrings to something innocuous(, e.g., `)?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.