Unable to find installation candidates for libclang (16.0.0) on ubuntu:22.04 amd64.
Probably missing the Source Distribution

As libclang-15.0.6.1 works as expected

I try to parse code that include comment with Japanese and Chinese.

when I want to get all tokens spelling in a method by iterate tokens in cursor.get_tokens(), it will raise error when it try to decode some string into UTF-8 in default.

what if I got a token and want to decode it into other format like "shiftJIS"?
can add some args like "encoding" when return token.spelling ?

Failed installing in docker container: registry.baidubce.com/paddlepaddle/paddle:2.2.2-gpu-cuda11.2-cudnn8
dockerfile is here: https://registry.hub.docker.com/layers/paddlepaddle/paddle/2.2.2-gpu-cuda11.2-cudnn8/images/sha256-09603d35054766f0d328ab4368a5eb8dbf92d0802041308992df1db41601e817?context=explore

Hey @sighingnow, I noticed the libclang package on PyPI includes copies of the libclang shared library, though only for a handful of the platform tags supported on PyPI. Recently the clang-format wheel was switched over to build using a combination of cibuildwheel (easy builds for wheels across a wide variety of platforms) and scikit-build (easy cross-platform building, single CMake "script" works for all platforms).

Some of the advantages that came from this approach:

• cibuildwheel builds wheels on GitHub Actions (other CI platforms also supported) both natively and using QEMU; the resulting wheels include: Linux x86_64, i686, s390x, and aarch64 architectures for both manylinux (glibc) and musllinux platforms, macOS universal binaries (x86_64 and arm64), and Windows 64-bit/32-bit
• scikit-build works well for building things from the LLVM project (single CMakeLists.txt using ExternalProject_add works on all platforms)
• sdist is available; piwheels will try to build Raspberry PI compatible wheels, and it could be a route to pip installing on BSD systems (with some wait time for compiling)
• GitHub actions can automate building + uploading wheels to PyPI when a release is tagged

If there's interest (or you need help with maintaining the libclang package), I'm willing start working on this for getting libclang 13.0.0 wheels published -- once the changes are made it should be easy to update for future LLVM releases.

Hi @sighingnow, thanks for creating such a helpful package. I try to parse a simple c++ code and get its AST using libclang. But the problem is that it runs well on MacOs but fails on Windows. The parsing code and c++ code are exactly the same on both platforms. Here is my case.

c++ code code.cpp:

#include <iostream>
int main() {
  int n;
  n = 0;
  return n;
}

python parsing code:

import clang.cindex as cc
cc.Config.set_library_path('D:\\Python3.7.3\\Lib\\site-packages\\clang\\native')

def walk(node):
    print(node.kind, node.spelling)
    for child in node.get_children():
        walk(child)

index = cc.Index.create()
tu = index.parse('./code.cpp')
root = tu.cursor
walk(root)

When I run this on MacOS, it returns a concise result (library_path is set correctly on both platforms):

CursorKind.TRANSLATION_UNIT code.cpp
CursorKind.FUNCTION_DECL main
CursorKind.COMPOUND_STMT 
CursorKind.DECL_STMT 
CursorKind.VAR_DECL n
CursorKind.BINARY_OPERATOR 
CursorKind.DECL_REF_EXPR n
CursorKind.INTEGER_LITERAL 
CursorKind.RETURN_STMT 
CursorKind.UNEXPOSED_EXPR n
CursorKind.DECL_REF_EXPR n

Process finished with exit code 0

But on Windows, it fails with ValueError: Unknown template argument kind 280 and weird AST output:

D:\Python3.7.3\python.exe "F:/PyCharm Project/codesim/delete.py"
CursorKind.TRANSLATION_UNIT ./test/code4.cpp
CursorKind.UNEXPOSED_DECL 
CursorKind.UNEXPOSED_DECL 
CursorKind.UNEXPOSED_DECL 
CursorKind.TYPEDEF_DECL uintptr_t
CursorKind.TYPEDEF_DECL va_list
CursorKind.FUNCTION_DECL __va_start
CursorKind.PARM_DECL 
CursorKind.TYPE_REF va_list
CursorKind.UNEXPOSED_DECL 
CursorKind.CLASS_TEMPLATE __vcrt_va_list_is_reference
CursorKind.TEMPLATE_TYPE_PARAMETER _Ty
CursorKind.ENUM_DECL 
CursorKind.ENUM_CONSTANT_DECL __the_value
CursorKind.CXX_BOOL_LITERAL_EXPR 
CursorKind.CLASS_TEMPLATE_PARTIAL_SPECIALIZATION __vcrt_va_list_is_reference
CursorKind.TEMPLATE_TYPE_PARAMETER _Ty
CursorKind.TYPE_REF _Ty
CursorKind.ENUM_DECL 
CursorKind.ENUM_CONSTANT_DECL __the_value
CursorKind.CXX_BOOL_LITERAL_EXPR 
...... (too much lines)
Traceback (most recent call last):
  File "F:/PyCharm Project/codesim/delete.py", line 16, in <module>
    walk(root)
  File "F:/PyCharm Project/codesim/delete.py", line 9, in walk
    walk(child)
  File "F:/PyCharm Project/codesim/delete.py", line 9, in walk
    walk(child)
  File "F:/PyCharm Project/codesim/delete.py", line 9, in walk
    walk(child)
  [Previous line repeated 2 more times]
  File "F:/PyCharm Project/codesim/delete.py", line 7, in walk
    print(node.kind, node.spelling)
  File "D:\Python3.7.3\lib\site-packages\clang\cindex.py", line 1541, in kind
    return CursorKind.from_id(self._kind_id)
  File "D:\Python3.7.3\lib\site-packages\clang\cindex.py", line 650, in from_id
    raise ValueError('Unknown template argument kind %d' % id)
ValueError: Unknown template argument kind 280

My libclang version is 12.0.0
clang version is 11.0
python version is 3.7
How can I solve this problem on Windows so that I can get the same output as what on MacOS? Thanks!

Hi,
I've noticed that the actions configured only compiles and uploads artifacts; have you considered using the official PyPI publisher action pypa/gh-action-pypi-publish to automate uploading to PyPI? You can test the workflow against TestPyPI first, and it does require an API key secret to be configured first.

After updating libclang from version 12.0.0 to 13.0.0, this error appeared

Traceback (most recent call last):
  File "cxx_parser.py", line 150, in generate_file
    translation_unit = index.parse(input_filename, args=['-std=c++14'])
  File "C:\Python\lib\site-packages\clang\cindex.py", line 2721, in parse
    return TranslationUnit.from_source(path, args, unsaved_files, options,
  File "C:\Python\lib\site-packages\clang\cindex.py", line 2836, in from_source
    raise TranslationUnitLoadError("Error parsing translation unit.")
clang.cindex.TranslationUnitLoadError: Error parsing translation unit.

First of all many thanks for this work, it is an extremely helpful resource.

Core error is using ValueError: Unknown template argument kind 280

At first sight this looks like a mismatch between the cursor kind enumerations here

which are in turn different to those defined here

and those specified in the Python interface here

but, oddly, this error is only triggered if <bit> is included.

I'm stuck with clang 10 on my Ubuntu machine so cannot test there (no member named 'bit_cast' in namespace 'std')

How best to proceed?

Many thanks

Jerry

// minimal test case https://en.cppreference.com/w/cpp/numeric/bit_cast
#include <cstdint>
// this works until <bit> is included
// uncomment to see libclang error
// #include <bit>
 
constexpr double f64v = 19880124.0; 
constexpr auto u64v = std::bit_cast<std::uint64_t>(f64v);
static_assert( std::bit_cast<double>(u64v) == f64v ); // round-trip
 
constexpr std::uint64_t u64v2 = 0x3fe9000000000000ull;
constexpr auto f64v2 = std::bit_cast<double>(u64v2);
static_assert( std::bit_cast<std::uint64_t>(f64v2) == u64v2 ); // round-trip
 
int main()
{
	return 0;
}

Stack traceback with #include <bit>

python cindex-dump.py test.cpp -std=c++20
('diags', [])
Traceback (most recent call last):
  File "cindex-dump.py", line 86, in <module>
    main()
  File "cindex-dump.py", line 83, in main
    pprint(('nodes', get_info(tu.cursor)))
  File "cindex-dump.py", line 43, in get_info
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in <listcomp>
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in get_info
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in <listcomp>
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in get_info
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in <listcomp>
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in get_info
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in <listcomp>
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in get_info
    for c in node.get_children()]
  File "cindex-dump.py", line 43, in <listcomp>
    for c in node.get_children()]
  File "cindex-dump.py", line 45, in get_info
    'kind' : node.kind,
  File "R:\apps\python3\lib\site-packages\clang\cindex.py", line 1541, in kind
    return CursorKind.from_id(self._kind_id)
  File "R:\apps\python3\lib\site-packages\clang\cindex.py", line 650, in from_id
    raise ValueError('Unknown template argument kind %d' % id)
ValueError: Unknown template argument kind 280

System details (Windows 10 FWIW):

clang -v
clang version 13.0.1
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: r:\apps\llvm\bin

pip show libclang
Name: libclang
Version: 13.0.0
Summary: Clang Python Bindings, mirrored from the official LLVM repo: https://github.com/llvm/llvm-project/tree/main/clang/bindings/python, to make the installation process easier.
Home-page: https://github.com/sighingnow/libclang
Author: Tao He
Author-email: [email protected]
License: Apache License 2.0
Location: r:\apps\python3\lib\site-packages

Tensorflow started to depend on libclang which broke builds on AArch64 architecture.

Please consider building wheel for AArch64 (manylinux2014 container is present for it).

I am trying to build tensorflow for arm32. Could you please provide instructions for existing platforms, so that interested parties can try to build for unsupported platforms like arm32. I am not very familiar with llvm build infrastructure and how these bindings are generated. I tried to look at scripts under the .github folder for aarch64. It seems to expect some code to be present which needs to compiled. It would be of great help, if you can add some documentation on which repo needs to be checked out and what does work folder represent and all. Thanks for providing these bindings.

In 14.0.6, this platform was macosx_11_0_arm64. Not seeing one 15.0.6.1. Are there plans to add one?

Hello, I've encountered an issue while parsing the AST of header files using the libclang library. It seems that all class and namespace CursorKind are recognized as VAR_DECL. Is there any solution to this problem?The correct solution is to use CursorKind.CLASS_DECL for classes and CursorKind.NAMESPACE_DECL for namespaces.Looking forward to your reply！

Your package installs a top level module called “tests” in addition to the clang module.

The problem is that tests contains an __init__.py and is therefore identified as package by this line:

Please take steps to exclude it, thanks.

https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#using-find-or-find-packages

Say you have a function like this:

__attribute__((always_inline)) void test(int a) {

}

If you use libclang to iterate over the children it becomes an UNEXPOSED_ATTRIBUTE.
However, I don't know how to inspect it further because its tokens are empty and spelling is the empty string.

For the case of libc's malloc function I get the tokens:

['extern', 'void', '*', 'malloc', '(', 'size_t', '__size', ')', '__THROW', '__attribute_malloc__']

and the parameter has the tokens:

['size_t', '__size']

However when I look at the type of the parameter with param.type I get TypeKind.INT instead of something resembling size_t.

This seems to be missing from the API, my workaround is looking at the tokens but it's not very elegant.

I'm experiencing some strange behavior while parsing the following Objective-C code:

code = """@class NSObject, SomeProtocol;

@interface OtherClass : NSObject {
    id* testVar;
    id*<SomeProtocol> objWithSomeMethod;
    NSObject*<SomeProtocol> objWithSomeMethod2;
    NSObject<SomeProtocol> objWithSomeMethod3;
}
@end"""

translation_unit = i.parse("test.h", unsaved_files=(("test.h", code),), args=("-x", "objective-c"))

# Access instance variable declarations
instance_cursors = list(list(translation_unit.cursor.get_children())[3].get_children())

testvar_cursor, objwithsomemethod_cursor, objwithsomemethod2_cursor, objwithsomemethod3_cursor = instance_cursors

print(f"testVar: {testvar_cursor.kind=} {testvar_cursor.objc_type_encoding=} {testvar_cursor.type.kind=}  {testvar_cursor.displayname=}")

print(f"objWithSomeMethod: {objwithsomemethod_cursor.kind=} {objwithsomemethod_cursor.objc_type_encoding=} {objwithsomemethod_cursor.type.kind=}  {objwithsomemethod_cursor.displayname=}")

print(f"objWithSomeMethod2: {objwithsomemethod2_cursor.kind=} {objwithsomemethod2_cursor.objc_type_encoding=} {objwithsomemethod2_cursor.type.kind=}  {objwithsomemethod2_cursor.displayname=}")

print(f"objWithSomeMethod3: {objwithsomemethod3_cursor.kind=} {objwithsomemethod3_cursor.objc_type_encoding=} {objwithsomemethod3_cursor.type.kind=}  {objwithsomemethod3_cursor.displayname=}")

Result:

testVar: testvar_cursor.kind=CursorKind.OBJC_IVAR_DECL testvar_cursor.objc_type_encoding='^@' testvar_cursor.type.kind=TypeKind.POINTER  testvar_cursor.displayname='test'
objWithSomeMethod: objwithsomemethod_cursor.kind=CursorKind.OBJC_IVAR_DECL objwithsomemethod_cursor.objc_type_encoding='^@' objwithsomemethod_cursor.type.kind=TypeKind.POINTER  objwithsomemethod_cursor.displayname=''
objWithSomeMethod2: objwithsomemethod2_cursor.kind=CursorKind.OBJC_IVAR_DECL objwithsomemethod2_cursor.objc_type_encoding='@' objwithsomemethod2_cursor.type.kind=TypeKind.OBJCOBJECTPOINTER  objwithsomemethod2_cursor.displayname=''
objWithSomeMethod3: objwithsomemethod3_cursor.kind=CursorKind.OBJC_IVAR_DECL objwithsomemethod3_cursor.objc_type_encoding='{NSObject=}' objwithsomemethod3_cursor.type.kind=TypeKind.OBJCINTERFACE  objwithsomemethod3_cursor.displayname='objWithSomeMethod3'

It would appear that the cursor displayname (and spelling) properties are empty when the cursor type is POINTER or OBJCOBJECTPOINTER and the type contains Objective-C protocol information. The regular protocol-less pointer testVar seems to work fine.

Additionally, I see no way to access the protocol modifier on the type declaration of the variables (the angle brackets <SomeProtocol>). The variable types are reported as simple pointers without any protocol information that I can identify.

I've also included objWithSomeMethod3 which is not a pointer (OBJCINTERFACE) but has a protocol defined. The displayname works fine however protocol info is likewise unavailable.

Is this behavior intended or am I missing something?

Thanks.

I just tried installing libclang on two different machines, and got two different results.

Linux happily accepted pip3 install libclang, but on MacOS, I get:

Collecting libclang
ERROR: Could not find a version that satisfies the requirement libclang (from versions: none)
ERROR: No matching distribution found for libclang

However, both OSes will accept pip3 install clang. The problem then comes when trying to use libclang, which results in the error:

Linux:

libclang-11.so: cannot open shared object file: No such file or directory. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

MacOS:

clang.cindex.LibclangError: dlopen(libclang.dylib, 6): image not found. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

I have been able to repeat this on three different Mac machines, and two different Linux machines.

Hi.
In your project you have a badge of supported python versions (probably generated from the description in pypy). My project using your library for a long time and the CI (with a lot of test coverage) on python 3.9 and everything works - you can pick up information about supported python with a clear conscience.
I also used python 3.10 on windows in this context, but due to the fact that I do not run automatic tests for this version myself, I do not give any guarantees, but I am telling you.

Just wanted to say thanks for this - very convenient and useful - thanks for putting in the work. Also the build scripts are very useful for figuring out how to cross-compile (I paraphrased them here).

This package cannot be installed with poetry since a few hours. I think this might me related to the update made for #18 (comment) without version bump.

$ poetry init
$ poetry add libclang

  • Installing libclang (12.0.0): Failed

  RuntimeError

  Retrieved digest for link libclang-12.0.0-2-py2.py3-none-win_amd64.whl(sha256:46414009fcee8375ba64ea6c2c43c5b80a63e3a8b679f4293e00aa605b7265aa) not in poetry.lock metadata ['sha256:3b0585dbdc3f3b372340f1efe7c28bf4a5b658d2fb8dc0544a7ef0d2bef40618', 'sha256:6df2f8a2cb75181e3c1e99e5dfca2fd44bb7f84ed12d5112541e03c10384f306', 'sha256:b828cb52cf3f02fb0e0a8fddb9dece7e2ed006f8b4d54ee811cef0471d414367', 'sha256:275126823c60ab5c9fae6a433cbb6a47e4d1b5f668a985fbd6065553dbc7efcc', 'sha256:fadad3bf5fbab50c996eb151adc58c3a7cbee45a9135060c416be7b640372112']

Installing an older version still works

$ poetry add "libclang=<12.0.0"

Hello @sighingnow

This one is a bit of a mystery. Running my libclang based project from the command line (i.e. $python cppuml.py) works exactly as expected. If I create an executable file with `cx-freeze' it then fails with the following message:

clang.cindex.LibclangError: libclang-14.so: cannot open shared object file: No such file or directory. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

Any thoughts on this? Please ask if you require more information. Additionally just say so f this is an upstream problem!

The same build process works perfectly on Windows (providing a path to the clang binaries is set).

MTIA.

$ pip show libclang
Name: libclang
Version: 13.0.0
Summary: Clang Python Bindings, mirrored from the official LLVM repo: https://github.com/llvm/llvm-project/tree/main/clang/bindings/python, to make the installation process easier.
Home-page: https://github.com/sighingnow/libclang
Author: Tao He
Author-email: [email protected]
License: Apache License 2.0
Location: /home/moi/.local/lib/python3.8/site-packages

$uname -a
Linux ub-20 5.13.0-39-generic #44~20.04.1-Ubuntu SMP Thu Mar 24 16:43:35 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
$ which clang
/usr/bin/clang
$ clang -v
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;@m64
Selected multilib: .;@m64

At the outset, I would like to point out that this is not a bug, but rather an improvement in the quality of life category. You may not want to do this, but then I need information to work around it for myself.
The whole thing needs a bit of explanation.

Problem:
No clang tool can parse all glibc out-of-box. From what I've noticed the main problem is stddef.h. Glibc does not include this file as a physical dependency, it is an internal to the compiler. However, when trying to parse a library, clang encounters this include in many cases. If we can't solve it, we have a problem. Perhaps some other file/files have a similar surprise, but I don't know about it.

How others deal with it
Most users, apart from tools, install the whole clang and either use its standard library for parsing or just add the problematic file to the include path.
An interesting case uses Qt Creator. It has the whole std clang in its sources and adds a problematic file from its sources to the paths (clang model). You can see it as we click to view the include sources. Working under gcc and clicking on include vector, string etc. we will be transferred to the preview of our compiler's std sources, but if we click on stddef.h the IDE will show us its internal clang std file it uses :)

What do I propose
I came across this problem when I wanted to add full support for "external modules" together with the std library in my project https://github.com/JhnW/devana. I can include std clang in the project for my own needs.
However, it seems to me that it is better that clang stds (or at least stddef.h) come to us together with your package. Since the purpose of this package is to facilitate the installation of native dependencies, the std needed to parse the full gcc code is in this definition.
Additionally, it's a version management issue. It is best to use the version of clang std lib that is related to the current clang version from which clanglib comes out. So it's nice to have version matching managed by libclang.

If you want to add this dependency, a simple function returning the current dependency path would be appreciated. If not, just let me know, I will maintain such a dependency as part of my project (although as I wrote, from the version maintenance point of view, this is not the best).

Given Foo.cpp

export module Foo;
export void foo(int)
{
};

I get

Kind UnexposedDecl Spelling
Kind UnexposedDecl Spelling

when visitChildren is used on the TU.

Given Bar.cpp

export module Bar;
export import :PartA;
export import :PartB;

I get

Kind UnexposedDecl Spelling
Kind ModuleImport Spelling Bar:PartA
Kind ModuleImport Spelling Bar:PartB

when visitChildren is used on the TU i.e. there is some C++20 module support.

Your package claims that

To make the libclang available without install the LLVM toolkits, this package provides bundled static-linked libclang shared library for different platforms, which, should work well on OSX, Windows, as well as usual Linux distributions.

However, downloading it with pip install clang and trying it out immediately yields the error I put in the title of this issue. This is all I am running:

docker run --rm debian:buster bash -c 'apt-get update && apt-get install --no-install-recommends -y python3 python3-pip && pip3 install clang && python3 -c "import clang.cindex; index = clang.cindex.Index.create()"'
# clang.cindex.LibclangError: libclang-11.so: cannot open shared object file: No such file or directory. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

I did some digging to find out why I was getting this error, and it seems like the PyPI package doesn't contain or install the libclang shared object at all?

The macOS arm64 wheel artifact for 16.0.6 (libclang-16.0.6-py2.py3-none-macosx_11_0_arm64.whl) seems to contain a RECORD hash mismatch. The WHEEL file has a different hash than reported in the RECORD file:

...
libclang-16.0.6.dist-info/WHEEL,sha256=tLrGdGXpMbYUGbL5K7EBSnH6WAXho1SlyqpGJluoubM,138
...

tLrGdGXpMbYUGbL5K7EBSnH6WAXho1SlyqpGJluoubM is not the correct hash for the WHEEL file in the archive.

I am trying to install a TF Nightly wheel that I built from source, and it seems that TensorFlow requires libclang>=9.0.1. However, there is not an aarch64 or universal2 macOS wheel available for download on pypi, so my pip install is failing.

I see in #11 that there is now a Linux wheel, and it would be great to have one for macOS as well. I'd be glad to help out, but I'm not sure what generating the new wheel would require.

Steps taken:

❯ python3.9 -m venv ~/tensorflow_blank_39
❯ source ~/tensorflow_blank_39/bin/activate
❯ pip3 install --upgrade pip numpy wheel
Requirement already satisfied: pip in /Users/tylerdavis/tensorflow_blank_39/lib/python3.9/site-packages (21.2.3)
Collecting pip
  Using cached pip-21.3.1-py3-none-any.whl (1.7 MB)
Collecting numpy
  Using cached numpy-1.21.3-cp39-cp39-macosx_11_0_arm64.whl (12.4 MB)
Collecting wheel
  Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, pip, numpy
  Attempting uninstall: pip
    Found existing installation: pip 21.2.3
    Uninstalling pip-21.2.3:
      Successfully uninstalled pip-21.2.3
Successfully installed numpy-1.21.3 pip-21.3.1 wheel-0.37.0
❯ python3 install keras-preprocessing --no-deps
/Users/tylerdavis/tensorflow_blank_39/bin/python3: can't open file '/private/tmp/tensorflow_pkg/install': [Errno 2] No such file or directory
❯ python3 install keras-preprocessing --no-deps
❯ ls
tf_nightly-2.8.0-cp39-cp39-macosx_12_0_arm64.whl
❯ pip3 install keras-preprocessing --no-deps
Collecting keras-preprocessing
  Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Installing collected packages: keras-preprocessing
Successfully installed keras-preprocessing-1.1.2
❯ pip3 install ./tf_nightly-2.8.0-cp39-cp39-macosx_12_0_arm64.whl
Processing ./tf_nightly-2.8.0-cp39-cp39-macosx_12_0_arm64.whl
Collecting six>=1.12.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt>=1.11.0
  Using cached wrapt-1.13.3.tar.gz (48 kB)
  Preparing metadata (setup.py) ... done
Collecting astunparse>=1.6.0
  Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
ERROR: Could not find a version that satisfies the requirement libclang>=9.0.1 (from tf-nightly) (from versions: none)
ERROR: No matching distribution found for libclang>=9.0.1

Hi, this seems to be a really useful package, however, it seems I can't get it to work. When I run the script below, I get the following error. PS: I have libclang.so in /usr/lib/libclang.so.

#!/usr/bin/env python

import os
from clang.cindex import Config

Config.library_path = os.path.join(
    os.path.dirname(os.path.realpath(__file__)), "native"
)

print("Hello world")

Traceback (most recent call last):
  File "/home/yiannis/git/esbmc-ai/./testclang.py", line 4, in <module>
    from clang.cindex import Config
ModuleNotFoundError: No module named 'clang.cindex'

For context, I installed the libclang Python package using pipenv. pipenv install libclang.

Description

I was testing out this library with some dummy code you can find here, and I came across a segmentation fault when calling clang.cindex.Type.get_size() on a cursor node of kind CursorKind.DECL_REF_EXPR. It doesn't really make sense to get the size, but I think it should just return 0 or None.

Clang Version

Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: arm64-apple-darwin21.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

To Reproduce

git clone https://github.com/plmwd/libclang-test.git
cd libclang-test
git checkout 65d236ee200eff80c3094e76e93b0358a1490636

python3.10 -m venv .venv
pip install -r requirements.txt
python libclang-test

Hi All! This is a great piece of software 🆒 but missing citation instructions 🤔 .
Let me propose to add a CITATION.cff file as recommended to all valuable projects.

https://citation-file-format.github.io/

Hi.
I have problem in Windows Server 2019 environment. I'm using it by GitHub Actions to setup windows tests (unfortunately, it is not possible to run Windows 10).
I have following log:

Run python -m pip install --upgrade pip
  python -m pip install --upgrade pip
  python -m pip install -r requirements.txt
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    pythonLocation: C:\hostedtoolcache\windows\Python\3.9.6\x64
Requirement already satisfied: pip in c:\hostedtoolcache\windows\python\3.9.6\x64\lib\site-packages (21.1.3)
ERROR: Could not find a version that satisfies the requirement libclang==12.0.0 (from versions: 9.0.1, 10.0.1.0, 11.0.0, 11.0.1, 11.1.0)
ERROR: No matching distribution found for libclang==12.0.0
Error: Process completed with exit code 1.

Do you want provide this version for Windows platform?

The documentation states that get_num_template_arguments() is a member function of Cursor.
However, it is currently a member function of Cursor.type.
This should be mentioned in the documentation, as currently it seems Cursor.get_num_template_arguments() doesn't work as it (always?) returns -1.

See this commit for example:
llvm/llvm-project@bb83f8e

The cursorkinds doesn't match between the binary libclang.dll and the bindings enum in cindex.py.

When the cursor is pointing to any Binary Operator for example like ( + or = ).
The spelling property of the Cursor class is not working.
It is not returning the Operator. As you can see in the following screenshots that i have tried to printout the cursor.spelling but for that it returns nothing and the numbers are the location line number.
I have provided the code and the output of the problem.

I am fairly new to using this package and followed the install installation instructions (pip install and changing the call variable in cindex.py file). After running the below code

import clang.cindex
index = clang.cindex.Index.create()
index.parse("test.cpp")

I get the TranslationUnitLoadError. Any guidance would be greatly appreciated.

I am using an M1 Mac, btw

The latest manylinux2014_aarch64.whl for libclang 16.0.6 is broken, as it needs a minimum version of GLIBC_2.29. Essentially, this is actually a manylinux_2_29 wheel.

You can quickly search for this issue by running strings on the libclang.so shared library file:

# download link taken from https://pypi.org/project/libclang/#files
wget https://files.pythonhosted.org/packages/d0/66/b4e67a05a00b54dab6af06e04b3e5dd8231c2b7a363df2f86f1a47808321/libclang-16.0.6-py2.py3-none-manylinux2014_aarch64.whl
unzip libclang-16.0.6-py2.py3-none-manylinux2014_aarch64.whl
# search for GLIBC_2.29 string
strings libclang-16.0.6.data/platlib/clang/native/libclang.so | grep 'GLIBC_2.29'

It looks like this repo is building the aarch64 wheels using Ubuntu 20.04, see

Since Ubuntu 20.04 uses v2.31 of glibc, they should instead be manylinux_2_31 wheels.

How to fix

Easy way (remove manylinx2014 support)

Change the manylinux_2014 flags to manylinux_2_31. This fill fix future package releases. However, anybody on an old operating system will still download the old broken manylinux_2014 wheels.

Hard fix (compile on manylinux2014)

Use something like pypa/cibuildwheel to build the wheels for you on the official pypa/manylinux2014 docker image. This will let you build all of the Linux wheels in a single GitHub Action, but the pypa/manylinux2014 image is based on CentOS, not Ubuntu, so you'd have to replace all of the apt instructions with yum/dnf instructions.

give an example

bool file_exists(const std::string &file) {
    std::ifstream infile(file);
    if (!infile.is_open()) {
        return false;
    }
    infile.close();
    return true;
}

bool read_file_to_string(const std::string file_path, std::string &str) {
    if (!file_exists(file_path)) {
        return false;
    }
    std::ifstream file{file_path};
    str = std::string{std::istreambuf_iterator<char>(file), std::istreambuf_iterator<char>()};
    file.close();
    return true;
}

config

Config.set_library_file('/usr/local/Cellar/llvm@14/14.0.6/lib/libclang.dylib')

use code

print(f"{'  ' * level} {str(node.kind)}: {node.spelling}")
for child in node.get_children():
    self.print_ast(child, code_source, file_path, level + 1)

Unable to output read_ file_ to_ String called file_ exists

method parse should be annoted to return TranslationUnit

property cursor should be annotated to return Cursor class

I just gave 2 examples but there are many more instances where return type annotations would be helpful (especially while prototyping the code because without those PyCharm autocomplete is not working properly)

Are there plans to continue manylinux support for x86_64? musllinux doesn't appear to be picked up as an installation candidate in my environment and the sdist is also missing as a backup

Hi, I reopen this issue.

I am encountering the same problem. For some files/headers, the libclang misinterpret size_t type.

I am in an Ubuntu 20.04 (Dockerized) with libclang==14.0.6 (installed from pip).

$ uname -a
Linux d3718923266e 5.15.0-73-generic #80~20.04.1-Ubuntu SMP Wed May 17 14:58:14 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

I made a minimal example here.
You should just unzip and run ./sizet_error.py

To note that clang-12 seems to get the correct AST.

$ clang-12 --version
Ubuntu clang version 12.0.0-3ubuntu1~20.04.5
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ clang-12 -I. -Xclang -ast-dump -fsyntax-only min.cc | grep aom_uleb_encode_fixed_size
| | | `-FunctionDecl 0x20ae310 <line:60:1, line:62:50> line:60:5 aom_uleb_encode_fixed_size 'int (uint64_t, size_

Hi 👋

Thanks for this nice package!

Version 11.0.0 fails with:

  File "/usr/local/lib/python3.7/site-packages/clang/cindex.py", line 2699, in create
    return Index(conf.lib.clang_createIndex(excludeDecls, 0))
  File "/usr/local/lib/python3.7/site-packages/clang/cindex.py", line 212, in __get__
    value = self.wrapped(instance)
  File "/usr/local/lib/python3.7/site-packages/clang/cindex.py", line 4147, in lib
    lib = self.get_cindex_library()
  File "/usr/local/lib/python3.7/site-packages/clang/cindex.py", line 4178, in get_cindex_library
    raise LibclangError(msg)
clang.cindex.LibclangError: dlopen(/usr/local/lib/python3.7/site-packages/clang/native/libclang.dylib, 6): no suitable image found.  Did find:
	/usr/local/lib/python3.7/site-packages/clang/native/libclang.dylib: unknown file type, first eight bytes: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
	/usr/local/lib/python3.7/site-packages/clang/native/libclang.dylib: unknown file type, first eight bytes: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

also running file on it gives "data"

However with libclang==10.0.1.0 it works find and also file identifies the file as: Mach-O 64-bit dynamically linked shared library x86_64

🙏

Hi

I have installed a python package for clang and libclang version 14.0.6 and tried using it for one of the project i am working on

but i am getting following error
clang.cindex.LibclangError: libclang-14.so: cannot open shared object file: No such file or directory. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

I am not getting this error when using LLVM library and setting LIBCLANG_PATH variable pointing to linclang.so.14 file

any other setup i have to do if i want to use python libclang package instead of llvm library??

Hello, I am having some difficulties with a project on an M1 that depends on tensorflow-macos which uses libclang. I have solved this for now by pinning version 13. I can provide a more detailed example if it would be helpful, but the easiest way to see the issue is to compare the files available in version 13.0.0 with version 14.0.1.

For all those arch/os/libc combinations where you do not provide wheel files source tarball should be present on Pypi.

This allows anyone to build libclang with simple "pip install libclang" (assuming that all toolchains and libraries are present).

Hi, folks!

As the title suggests, I'm getting an incorrect spelling for CALL_EXPR arguments, but only when the call site was created by expanding a macro.

Here's a Python test that shows what I'm talking about:

import clang.cindex
import tempfile
import unittest

class TestLibclang(unittest.TestCase):
    def test_repro_libclang_quirk(self):
        """Document the libclang quirk related to macro expansion. Issue
        lowRISC/opentitan#19438. When we use libclang to find a call site that
        was created by a macro expansion, the call site's arguments' tokens are
        incorrect. They seem to point into the non-preprocessed translation
        unit.
        """

        C_SRC_WITH_MACRO_EXPANSION = b"""
#define CONST_FOO 0
#define CALL_MAGIC(name) magic(CONST_##name)
void magic(int register_offset);
void entry_point(void) {
  CALL_MAGIC(FOO);
}
"""
        INCORRECT_ARG_TOKENS = [
            '0', '#', 'define', 'CALL_MAGIC', '(', 'name', ')', 'magic', '(',
            'CONST_', '##', 'name', ')', 'void', 'magic', '(', 'int',
            'register_offset', ')', ';', 'void', 'entry_point', '(', 'void',
            ')', '{', 'CALL_MAGIC', '(', 'FOO', ')'
        ]

        index = clang.cindex.Index.create()

        with tempfile.NamedTemporaryFile(suffix=".c") as tf:
            tf.write(C_SRC_WITH_MACRO_EXPANSION)
            tf.flush()

            translation_unit = index.parse(tf.name, args=[])

            [cursor] = [
                c for c in translation_unit.cursor.walk_preorder()
                if c.kind == clang.cindex.CursorKind.CALL_EXPR and
                c.displayname == 'magic'
            ]

            [arg] = list(cursor.get_arguments())

            tokens = [t.spelling for t in arg.get_tokens()]

            # These assertions may feel a little backwards because the purpose
            # of this test is to document the unwanted behavior.
            self.assertEqual(tokens, INCORRECT_ARG_TOKENS)
            self.assertNotEqual(tokens, ['CONST_FOO'])


if __name__ == "__main__":
    unittest.main()

Hello,

I was wondering how hard it would be to add support for Pyodide.

https://pyodide.org/en/stable/development/new-packages.html

I've also enquired about this on the pyodide repo.

Thanks

When the cursor traverses to _flashlightable, the macro UFlashlightableComponent macro expansion error occurs during get_tokens()

Hi.
The problem I am about to describe was first encountered by my contributor, then by me. Everything is fine on CI and other environments.
I'm using version 16.0.6, Ubuntu 22, but I know the problem occurred with the contributor on Windows as well. I also checked version 15 and 14, same thing. But to the point.
Until now, if I want to parse a file containing a header or another header from the standard library, libclang looks for e.g. gcc in the system library. It has always worked like this, for two years, on CI, during my development, etc. Nothing unusual.

Suddenly lbclang stopped detecting standard headers automatically.

Interestingly, I saw it happen with my own eyes. I was working on correcting one problem, I ran tests, everything was fine. I reverted a few files in the repository, I ran tests again - and suddenly it didn't work. I did not make any changes to the configuration, I did not update the OS.
I also checked on a separate venv, it's happening again.
The only good news is that once the error "latches" it persists continuously in a given environment, it is not random until it first occurs.

Hello,

could you upload a wheel for this arch please?
If something is blocking for build, I can help with it.

Thanks.
Pierrick

Hi.
Great package, speed up configuration python libclang only for paring C++.
Unfortunately in my case, Ubuntu 20, it is not complete out of box. It need install libncurses5 from standard distribution repository.
I know that add it to package will be bad idea, but i think section know issues with OS version and/or "sudo apt install libncurses5" command will improve the quality of package.