libgit2 / pygit2 Goto Github PK

View Code? Open in Web Editor NEW

1.6K 70.0 386.0 8.2 MB

Python bindings for libgit2

Home Page: https://www.pygit2.org/

License: Other

Shell 0.74% Python 50.37% C 48.88% Makefile 0.01%

python git

pygit2's Introduction

pygit2 - libgit2 bindings in Python

Bindings to the libgit2 shared library, implements Git plumbing. Supports Python 3.9 to 3.12 and PyPy3 7.3+

License: GPLv2 with linking exception

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License, version 2, as published by the Free Software Foundation.

In addition to the permissions in the GNU General Public License, the authors give you unlimited permission to link the compiled version of this file into combinations with other programs, and to distribute those combinations without any restriction coming from the use of this file. (The General Public License restrictions do apply in other respects; for example, they cover modification of the file, and distribution when not linked into a combined executable.)

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; see the file COPYING. If not, write to the Free Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

pygit2's People

Contributors

Stargazers

Watchers

Forkers

sanga szktty saschpe dunhamsteve versmisse jszakmeister t3rm1n4l zoranzaric enzbang deegrayve ambakshi carlosmn mike-perdide schu fuyasing tail kevingessner encukou erikvanzijst sigmaris chengxiangfei sophiayue liucuicui lygstate gafton tsuraan bos ravenac95 timtadh petrhosek pistacchio alex179ohm cboos dnet miri64 vaab bendavis78 pkroma ferengee edavis wking delanne davesque richo dmvrtx meteoritt jun66j5 smaftoul zluis masdude sixpi davidfischer gnauhznauy pavlenko-volodymyr xtao drodriguez heynemann agmarianna delphinus gitsan kharazi theojulienne metatoaster edgarchen douban gmaccarone emeusel eminence letolabs ymzy salehqt jplana oscarsj victorgp felipellrocha aspiers shinningstar ahrensburger folpindo java66liu beeble yamateh guoqiangyang takluyver goodwillcoding ircubic ipmcc rek001 flowroute mduggan arnovanlumig opbeat mrasskazov kyriakosoikonomakos feisuzhu sourcelair soasme lyoniionly drwrong point-line-surface-body

pygit2's Issues

pygit build failure with libgit2 master branch

Trying to compile the pygit2 master branch (using the libgit2 master branch), I get errors like:

pygit2.c:593:5: error: too many arguments to function ‘git_signature_new’

It works with libgit2 development branch. It is expected ?

Segfaults during tests on OS X, Python 2.6.6

Running 10.6.8, libraries installed via MacPorts. Python 2.6.6, also installed via MacPorts. Using commit 7b8ae0e (HEAD as of time of writing).

libgit2 installed per instructions, at commit b2e60e4eb5efc2d8ab78371ecd85e8a1f482134d.

python setup.py install succeeded.

python setup.py test yielded:

test_new_repo (test.test_repository.NewRepositoryTest) ... Segmentation fault

I tried using libgit2 at v0.14.0 instead. That yielded:

test_new_commit (test.test_commit.CommitTest) ... Segmentation fault

I tried using pygit2 at v0.14.0 instead. That yielded:

test_write (test.test_repository.RepositoryTest) ... Segmentation fault

The build tool is not make but waf

Your doc suggests using make and make install to build libgit2, but the build tool used is actually waf
./waf configure --prefix=$PWD/release ./waf ./waf install

Compilation fails with python 2.5

PyVarObject_HEAD_INIT isn't defined in python 2.5, so compilation fails

Maybe this will help: biopython/biopython@d4eaf57

FSF address incorrect in COPYING

In the COPYING file the address of the FSF is no long correct.

The new address is:
Free Software Foundation
51 Franklin Street, Fifth Floor
Boston, MA 02110-1301
USA
Source: http://www.fsf.org/about/contact/

New GPLv2 file: http://www.gnu.org/licenses/gpl-2.0.txt

Note that version 3 of the GPL does not contain this information any-more, I guess to prevent this situation in the future.
See: http://www.gnu.org/licenses/gpl-3.0.txt

GPL License Question

Hi,

As the answer to this question might be interesting for others as well, I make it public using the issue system.

Pygit2 uses the same license as libgit2, the GPL with linkage exception. The problem I see is that this exception doesn't apply to the usage of pygit2 within other python programs and libraries which are not necessarily licensed under GPL.

For instance, I would be interested in using pygit2 in one of my projects, gitdb, which in turn can be used by git-python. Both are licensed under NewBSD, so I am worried that using pygit2 would compromise this license and force it into the GPL.

Could you please clarify the situation ?
Thank you,
Sebastian

Use default user if author/committer is None

To make a commit one as to precise an author and a committer, these argument cannot be null (None), but if they are (None) could git used the information set at the system level in ~/.gitconfig?

It would be nice to see libgit2/pygit2 support this.

building on OSX 10.6 fails

Here is the process:

☺ git clone git://github.com/libgit2/pygit2.git

☺ python setup.py install
running install
running build
running build_py
copying pygit2/init.py -> build/lib.macosx-10.5-intel-2.7/pygit2
copying pygit2/utils.py -> build/lib.macosx-10.5-intel-2.7/pygit2
running build_ext
building '_pygit2' extension
/usr/bin/gcc-4.2 -fno-strict-aliasing -arch i386 -arch x86_64 -O3 -w -pipe -march=core2 -msse4.1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/include -Iinclude -I/usr/local/Cellar/python/2.7.2/include/python2.7 -c src/pygit2.c -o build/temp.macosx-10.5-intel-2.7/src/pygit2.o
In file included from src/pygit2.c:32:
include/pygit2/types.h:60: error: expected specifier-qualifier-list before ‘git_diff_list’
src/pygit2.c: In function ‘moduleinit’:
src/pygit2.c:246: error: ‘GIT_DIFF_NORMAL’ undeclared (first use in this function)
src/pygit2.c:246: error: (Each undeclared identifier is reported only once
src/pygit2.c:246: error: for each function it appears in.)
src/pygit2.c:247: error: ‘GIT_DIFF_REVERSE’ undeclared (first use in this function)
src/pygit2.c:248: error: ‘GIT_DIFF_FORCE_TEXT’ undeclared (first use in this function)
src/pygit2.c:250: error: ‘GIT_DIFF_IGNORE_WHITESPACE’ undeclared (first use in this function)
src/pygit2.c:252: error: ‘GIT_DIFF_IGNORE_WHITESPACE_CHANGE’ undeclared (first use in this function)
src/pygit2.c:254: error: ‘GIT_DIFF_IGNORE_WHITESPACE_EOL’ undeclared (first use in this function)
src/pygit2.c:256: error: ‘GIT_DIFF_IGNORE_SUBMODULES’ undeclared (first use in this function)
src/pygit2.c:257: error: ‘GIT_DIFF_PATIENCE’ undeclared (first use in this function)
src/pygit2.c:259: error: ‘GIT_DIFF_INCLUDE_IGNORED’ undeclared (first use in this function)
src/pygit2.c:261: error: ‘GIT_DIFF_INCLUDE_UNTRACKED’ undeclared (first use in this function)
src/pygit2.c:263: error: ‘GIT_DIFF_INCLUDE_UNMODIFIED’ undeclared (first use in this function)
src/pygit2.c:265: error: ‘GIT_DIFF_RECURSE_UNTRACKED_DIRS’ undeclared (first use in this function)
src/pygit2.c:269: error: ‘GIT_DIFF_FILE_VALID_OID’ undeclared (first use in this function)
src/pygit2.c:271: error: ‘GIT_DIFF_FILE_FREE_PATH’ undeclared (first use in this function)
src/pygit2.c:272: error: ‘GIT_DIFF_FILE_BINARY’ undeclared (first use in this function)
src/pygit2.c:274: error: ‘GIT_DIFF_FILE_NOT_BINARY’ undeclared (first use in this function)
src/pygit2.c:276: error: ‘GIT_DIFF_FILE_FREE_DATA’ undeclared (first use in this function)
src/pygit2.c:278: error: ‘GIT_DIFF_FILE_UNMAP_DATA’ undeclared (first use in this function)
src/pygit2.c:281: error: ‘GIT_DELTA_UNMODIFIED’ undeclared (first use in this function)
src/pygit2.c:282: error: ‘GIT_DELTA_ADDED’ undeclared (first use in this function)
src/pygit2.c:283: error: ‘GIT_DELTA_DELETED’ undeclared (first use in this function)
src/pygit2.c:284: error: ‘GIT_DELTA_MODIFIED’ undeclared (first use in this function)
src/pygit2.c:285: error: ‘GIT_DELTA_RENAMED’ undeclared (first use in this function)
src/pygit2.c:286: error: ‘GIT_DELTA_COPIED’ undeclared (first use in this function)
src/pygit2.c:287: error: ‘GIT_DELTA_IGNORED’ undeclared (first use in this function)
src/pygit2.c:288: error: ‘GIT_DELTA_UNTRACKED’ undeclared (first use in this function)
src/pygit2.c:291: error: ‘GIT_DIFF_LINE_CONTEXT’ undeclared (first use in this function)
src/pygit2.c:293: error: ‘GIT_DIFF_LINE_ADDITION’ undeclared (first use in this function)
src/pygit2.c:295: error: ‘GIT_DIFF_LINE_DELETION’ undeclared (first use in this function)
src/pygit2.c:297: error: ‘GIT_DIFF_LINE_ADD_EOFNL’ undeclared (first use in this function)
src/pygit2.c:299: error: ‘GIT_DIFF_LINE_DEL_EOFNL’ undeclared (first use in this function)
src/pygit2.c:301: error: ‘GIT_DIFF_LINE_FILE_HDR’ undeclared (first use in this function)
src/pygit2.c:303: error: ‘GIT_DIFF_LINE_HUNK_HDR’ undeclared (first use in this function)
src/pygit2.c:304: error: ‘GIT_DIFF_LINE_BINARY’ undeclared (first use in this function)
In file included from src/pygit2.c:32:
include/pygit2/types.h:60: error: expected specifier-qualifier-list before ‘git_diff_list’
src/pygit2.c: In function ‘moduleinit’:
src/pygit2.c:246: error: ‘GIT_DIFF_NORMAL’ undeclared (first use in this function)
src/pygit2.c:246: error: (Each undeclared identifier is reported only once
src/pygit2.c:246: error: for each function it appears in.)
src/pygit2.c:247: error: ‘GIT_DIFF_REVERSE’ undeclared (first use in this function)
src/pygit2.c:248: error: ‘GIT_DIFF_FORCE_TEXT’ undeclared (first use in this function)
src/pygit2.c:250: error: ‘GIT_DIFF_IGNORE_WHITESPACE’ undeclared (first use in this function)
src/pygit2.c:252: error: ‘GIT_DIFF_IGNORE_WHITESPACE_CHANGE’ undeclared (first use in this function)
src/pygit2.c:254: error: ‘GIT_DIFF_IGNORE_WHITESPACE_EOL’ undeclared (first use in this function)
src/pygit2.c:256: error: ‘GIT_DIFF_IGNORE_SUBMODULES’ undeclared (first use in this function)
src/pygit2.c:257: error: ‘GIT_DIFF_PATIENCE’ undeclared (first use in this function)
src/pygit2.c:259: error: ‘GIT_DIFF_INCLUDE_IGNORED’ undeclared (first use in this function)
src/pygit2.c:261: error: ‘GIT_DIFF_INCLUDE_UNTRACKED’ undeclared (first use in this function)
src/pygit2.c:263: error: ‘GIT_DIFF_INCLUDE_UNMODIFIED’ undeclared (first use in this function)
src/pygit2.c:265: error: ‘GIT_DIFF_RECURSE_UNTRACKED_DIRS’ undeclared (first use in this function)
src/pygit2.c:269: error: ‘GIT_DIFF_FILE_VALID_OID’ undeclared (first use in this function)
src/pygit2.c:271: error: ‘GIT_DIFF_FILE_FREE_PATH’ undeclared (first use in this function)
src/pygit2.c:272: error: ‘GIT_DIFF_FILE_BINARY’ undeclared (first use in this function)
src/pygit2.c:274: error: ‘GIT_DIFF_FILE_NOT_BINARY’ undeclared (first use in this function)
src/pygit2.c:276: error: ‘GIT_DIFF_FILE_FREE_DATA’ undeclared (first use in this function)
src/pygit2.c:278: error: ‘GIT_DIFF_FILE_UNMAP_DATA’ undeclared (first use in this function)
src/pygit2.c:281: error: ‘GIT_DELTA_UNMODIFIED’ undeclared (first use in this function)
src/pygit2.c:282: error: ‘GIT_DELTA_ADDED’ undeclared (first use in this function)
src/pygit2.c:283: error: ‘GIT_DELTA_DELETED’ undeclared (first use in this function)
src/pygit2.c:284: error: ‘GIT_DELTA_MODIFIED’ undeclared (first use in this function)
src/pygit2.c:285: error: ‘GIT_DELTA_RENAMED’ undeclared (first use in this function)
src/pygit2.c:286: error: ‘GIT_DELTA_COPIED’ undeclared (first use in this function)
src/pygit2.c:287: error: ‘GIT_DELTA_IGNORED’ undeclared (first use in this function)
src/pygit2.c:288: error: ‘GIT_DELTA_UNTRACKED’ undeclared (first use in this function)
src/pygit2.c:291: error: ‘GIT_DIFF_LINE_CONTEXT’ undeclared (first use in this function)
src/pygit2.c:293: error: ‘GIT_DIFF_LINE_ADDITION’ undeclared (first use in this function)
src/pygit2.c:295: error: ‘GIT_DIFF_LINE_DELETION’ undeclared (first use in this function)
src/pygit2.c:297: error: ‘GIT_DIFF_LINE_ADD_EOFNL’ undeclared (first use in this function)
src/pygit2.c:299: error: ‘GIT_DIFF_LINE_DEL_EOFNL’ undeclared (first use in this function)
src/pygit2.c:301: error: ‘GIT_DIFF_LINE_FILE_HDR’ undeclared (first use in this function)
src/pygit2.c:303: error: ‘GIT_DIFF_LINE_HUNK_HDR’ undeclared (first use in this function)
src/pygit2.c:304: error: ‘GIT_DIFF_LINE_BINARY’ undeclared (first use in this function)
lipo: can't open input file: /var/folders/it/itEAjVAME2GWcW-Q6QxMBU+++TI/-Tmp-//ccu38n1P.out (No such file or directory)
error: command '/usr/bin/gcc-4.2' failed with exit status 1

state of pygit2

With git_revparse_sigle I think we could improve object lookups a little bit. For example repo['HEAD'] or repo.lookup_object('HEAD') would be much more convenient to use instead of repo.revparse_single('HEAD'). At the moment the pygit2 api looks a little bit confusing because we just write bindings for c functions. I would like to have a clean well-structured pythonic Repository class instead of just a blind collection of git_* functions.

Something like the following (of course written in c...):

class Repository:
    def __init__():
        self.head = None
        self.path = ''
        self.workdir = ''

        self.is_detached = False
        self.is_orphan = False
        self.is_empty = False
        self.is_bare = False

        self.submodules = []
        self.references = []
        self.branches = []
        self.tags = []
        self.remotes = []
        self.history = []                    #rev walker

        self.config = Config()
        self.index = Index()
        self.status = Status()
        self.reflog = RefLog()

    def create_blob_from_buffer():
        pass

    def create_blob_from_file():
        pass

    def create_tree_from_dir():
        pass

    def create_tree_from_builder()
        pass

    def create_commit():
        pass

    def create_reference():
        pass

    def create_tag():
        pass

    def create_branch():
        pass

    def lookup_object():
        pass

    def lookup_reference():
        pass

    def pack_references():
        pass

    def checkout():
        pass

    def reset():
        pass

    def read_raw():
        pass

    def write_raw():
        pass

I don't know if it's really necessary to create all objects through Repository... As well I don't like the naming differences for object creations: repo.create_X or repo.TreeBuilder().write().

What do you think? Some suggestions?

tests failing and import failing

setup.py install succeeds

setup.py test gives output:-

running test
running egg_info
writing pygit2.egg-info/PKG-INFO
writing top-level names to pygit2.egg-info/top_level.txt
writing dependency_links to pygit2.egg-info/dependency_links.txt
reading manifest file 'pygit2.egg-info/SOURCES.txt'
writing manifest file 'pygit2.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.5-i386-2.7/pygit2.so ->
Traceback (most recent call last):
File "setup.py", line 88, in
**kwargs
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site-packages/setuptools/command/test.py", line 137, in run
self.with_project_on_sys_path(self.run_tests)
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site-packages/setuptools/command/test.py", line 117, in with_project_on_sys_path
func()
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site-packages/setuptools/command/test.py", line 146, in run_tests
testLoader = loader_class()
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/unittest/main.py", line 94, in init
self.parseArgs(argv)
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/unittest/main.py", line 149, in parseArgs
self.createTests()
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/unittest/main.py", line 158, in createTests
self.module)
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/unittest/loader.py", line 113, in loadTestsFromName
test = obj()
File "/my/code/git/pygit2/test/init.py", line 42, in test_suite
return unittest.defaultTestLoader.loadTestsFromNames(modules)
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/unittest/loader.py", line 100, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_blob'

import gives:-
ImportError Traceback (most recent call last)
/my/code/git/pygit2/ in ()
----> 1 import pygit2

ImportError: dlopen(./pygit2.so, 2): Symbol not found: _git_commit_author
Referenced from: /my/code/git/pygit2/pygit2.so
Expected in: dynamic lookup

HELP!

Making things reentrant

Hi there,

I'm using pygit over NFS and the network latency that builds up during long DAG or Tree traversals, reading and following objects on disk is becoming a real problem.

Since almost all time is spent waiting on IO, I'd like to see pygit release the GIL whenever calling a libgit2 function that hits the disk. That way I could parallellize these IO intensive operations.

I've experimented a bit adding Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS around git_object_lookup calls and while I can see multiple CPU cores spinning up concurrently, I'm consistently crashing the interpreter with segfaults.

Is there any global state I should be aware of?

Cheers,
Erik

include directories incorrect in pygit2.c

When trying to install I found that header files being looked for were in /usr/include/git2, but the file pygit2.c is attempting to include files in /usr/include/git. For example "#include <git/commit.h>" on line 29. Only after changing "git" to "git2" was I able to install pygit2.

File README.rst not included in sdist distribution

The file README.rst (referenced in setup.py) is not included when the sdist distribution is built. The following patch will add a MANIFEST.in file so README.rst will be included:

diff --git a/MANIFEST.in b/MANIFEST.in
new file mode 100644
index 0000000..bb37a27
--- /dev/null
+++ b/MANIFEST.in
@@ -0,0 +1 @@
+include *.rst

error C2275: 'PyObject' : illegal use of this type as an expression

Installing in Windows crashes in setup.py

python setup.py install
at Microsoft Visual Studio 9.0\VC\BIN\cl.exe ...
pygit2.c(496) : error C2275: 'PyObject' : illegal use of this type as an expression

Error is caused by variable declaration in the middle of the method in Repository_read;

Fix is to move the declaration to the beginning of the method

static PyObject *
Repository_read(Repository *self, PyObject *py_hex)
{
    git_oid oid;
    int err;
    git_odb_object *obj;
    size_t len;
    PyObject* tuple; // add declaration in line 487

...

    tuple = Py_BuildValue( // Remove tuple declaration in line 497

pygit2 is only compatible with libgit2 version 0.10

Hi,

I noticed that the bindings only work with version 0.10 of libgit2 currently. This is okay, but I think it should be stated in the Readme file to be less confusing for early adopters.

Thank you,
Sebastian

pygit2 cant build in git1.8.0

Hi,
In git 1.8.0, pygit2 cant build. Error info:

src/pygit2/config.c: In function ‘Config_foreach’:
src/pygit2/config.c:261:13: warning: passing argument 2 of ‘git_config_foreach’ from incompatible pointer type [enabled by default]
In file included from /usr/include/git2.h:38:0,
from include/pygit2/error.h:33,
from src/pygit2/config.c:30:
/usr/include/git2/config.h:420:17: note: expected ‘int ()(const struct git_config_entry *, void *)’ but argument is of type ‘int ()(const char , const char *, void *)’
src/pygit2/config.c: In function ‘Config_add_file’:
src/pygit2/config.c:276:5: error: too few arguments to function ‘git_config_add_file_ondisk’
In file included from /usr/include/git2.h:38:0,
from include/pygit2/error.h:33,
from src/pygit2/config.c:30:
/usr/include/git2/config.h:206:17: note: declared here
src/pygit2/config.c: In function ‘Config_get_multivar’:
src/pygit2/config.c:311:13: warning: passing argument 4 of ‘git_config_get_multivar’ from incompatible pointer type [enabled by default]
In file included from /usr/include/git2.h:38:0,
from include/pygit2/error.h:33,
from src/pygit2/config.c:30:
/usr/include/git2/config.h:339:17: note: expected ‘int ()(const struct git_config_entry , void *)’ but argument is of type ‘int ()(const char *, void *)’
error: command 'gcc' failed with exit status 1

Trivial test failures due to OS X's tmp path handling

OS X sometimes puts tmp paths in /private, with symlinks. As a result, with current master, three of the tests fail on OS X:

FAIL: test_get_path (test.test_repository.RepositoryTest)

Traceback (most recent call last):
File "/Volumes/Data/src/varia/pygit2/test/test_repository.py", line 96, in test_get_path
self.assertEqual(directory, expected)
AssertionError: u'/private/tmp/tmpFjAax_/testrepo.git' != u'/tmp/tmpFjAax_/testrepo.git'

FAIL: test_get_path (test.test_repository.RepositoryTest_II)

Traceback (most recent call last):
File "/Volumes/Data/src/varia/pygit2/test/test_repository.py", line 108, in test_get_path
self.assertEqual(directory, expected)
AssertionError: u'/private/tmp/tmpazfPEk/testrepo/.git' != u'/tmp/tmpazfPEk/testrepo/.git'

FAIL: test_get_workdir (test.test_repository.RepositoryTest_II)

Traceback (most recent call last):
File "/Volumes/Data/src/varia/pygit2/test/test_repository.py", line 113, in test_get_workdir
self.assertEqual(directory, expected)
AssertionError: u'/private/tmp/tmpkKyjZq/testrepo' != u'/tmp/tmpkKyjZq/testrepo'

Ran 51 tests in 1.838s

FAILED (failures=3)

The fix is simple: Use realpath instead of abspath in test_repository.py, which will resolve the symlinks. Pull request coming soon.

Document adding files to a repository

After a long time trying to figure out how to populate a newly created repository, it looks like I got something that works.

https://gist.github.com/3750225

But, TBH it's a mix up of copy/paste and guessing (libgit2/pygit2 is not super explicit), so I'd be glad if someone could comment on it.

Thanks in advance

It's seems that circular reference happened between Index and Repository.

I think Index need weak reference to Repository.
Or just don't do that.

D:\CI\bld\vcs\pygit2>python setup.py test --args="test.test_index.IndexTest"
running test
running build
running build_ext
open file c:/users/dreamkxd/appdata/local/temp/tmp05dgg4/testrepo/.git/config with fd 3
close fd:3, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmp05dgg4/testrepo/.git/index with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitattributes with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmp05dgg4/testrepo/bye.txt with fd 3
close fd:3, result:0
create c:/users/dreamkxd/appdata/local/temp/tmp05dgg4/testrepo/.git/objects/tmp_object_git2_a47136 with fd:3
git_futils_mktmp c:/users/dreamkxd/appdata/local/temp/tmp05dgg4/testrepo/.git/objects/tmp_object_git2_a47136 c:/users/dreamkxd/appdata/local/temp/tmp0
5dgg4/testrepo/.git/objects/tmp_object fd:3
close fd:3, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x0295B6E8>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x0295B710>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x0295B6E8>)
.open file c:/users/dreamkxd/appdata/local/temp/tmpne9z4t/testrepo/.git/config with fd 3
close fd:3, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpne9z4t/testrepo/.git/index with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpne9z4t/testrepo/.git/index with fd 3
close fd:3, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99508>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99530>)
.open file c:/users/dreamkxd/appdata/local/temp/tmp85t50n/testrepo/.git/config with fd 3
close fd:3, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmp85t50n/testrepo/.git/index with fd 3
close fd:3, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A995F8>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99670>)
.open file c:/users/dreamkxd/appdata/local/temp/tmpwfbfbk/testrepo/.git/config with fd 3
close fd:3, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpwfbfbk/testrepo/.git/index with fd 3
close fd:3, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99508>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A995A8>)
.open file c:/users/dreamkxd/appdata/local/temp/tmpwt93xs/testrepo/.git/config with fd 3
close fd:3, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpwt93xs/testrepo/.git/index with fd 3
close fd:3, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A995F8>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99760>)
.open file c:/users/dreamkxd/appdata/local/temp/tmpcsggi1/testrepo/.git/config with fd 3
close fd:3, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpcsggi1/testrepo/.git/index with fd 3
close fd:3, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99508>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99648>)
.open file c:/users/dreamkxd/appdata/local/temp/tmpdu2d9r/testrepo/.git/config with fd 3
close fd:3, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpdu2d9r/testrepo/.git/index with fd 3
close fd:3, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A995F8>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99800>)
.open file c:/users/dreamkxd/appdata/local/temp/tmp5v9zd7/testrepo/.git/config with fd 3
close fd:3, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 3
close fd:3, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmp5v9zd7/testrepo/.git/index with fd 3
close fd:3, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmp5v9zd7/testrepo/.git/objects/pack/pack-e6fbc15b315a0eab6005c44e5b7054b1f0043f39.idx with fd 3
p_mmap 0x304eb08
close fd:3, result:-1
open file c:/users/dreamkxd/appdata/local/temp/tmp5v9zd7/testrepo/.git/objects/pack/pack-e6fbc15b315a0eab6005c44e5b7054b1f0043f39.pack with fd 3
p_mmap 0x304da9c
create c:/users/dreamkxd/appdata/local/temp/tmp5v9zd7/testrepo/.git/objects/tmp_object_git2_a47136 with fd:4
git_futils_mktmp c:/users/dreamkxd/appdata/local/temp/tmp5v9zd7/testrepo/.git/objects/tmp_object_git2_a47136 c:/users/dreamkxd/appdata/local/temp/tmp5
v9zd7/testrepo/.git/objects/tmp_object fd:4
close fd:4, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmp5v9zd7/testrepo/.git/index with fd 4
close fd:4, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99508>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02A99530>)
Eopen file c:/users/dreamkxd/appdata/local/temp/tmpfo2eyt/testrepo/.git/config with fd 4
close fd:4, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 4
close fd:4, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 4
close fd:4, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpfo2eyt/testrepo/.git/index with fd 4
close fd:4, result:0
open file C:/Program Files (x86)/Git/etc/gitattributes with fd 4
close fd:4, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpfo2eyt/testrepo/bye.txt with fd 4
close fd:4, result:0
create c:/users/dreamkxd/appdata/local/temp/tmpfo2eyt/testrepo/.git/objects/tmp_object_git2_a47136 with fd:4
git_futils_mktmp c:/users/dreamkxd/appdata/local/temp/tmpfo2eyt/testrepo/.git/objects/tmp_object_git2_a47136 c:/users/dreamkxd/appdata/local/temp/tmpf
o2eyt/testrepo/.git/objects/tmp_object fd:4
close fd:4, result:0
Open c:/users/dreamkxd/appdata/local/temp/tmpfo2eyt/testrepo/.git/index.lock with 4
close fd:4, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpfo2eyt/testrepo/.git/index with fd 4
close fd:4, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02ADC1C0>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02ADC1E8>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02ADC1C0>)
.open file c:/users/dreamkxd/appdata/local/temp/tmpen3wj1/testrepo/.git/config with fd 4
close fd:4, result:0
open file D:\CI\docs\home\lyg/.gitconfig with fd 4
close fd:4, result:0
open file C:/Program Files (x86)/Git/etc/gitconfig with fd 4
close fd:4, result:0
open file c:/users/dreamkxd/appdata/local/temp/tmpen3wj1/testrepo/.git/index with fd 4
close fd:4, result:0
Tear down
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02ADC0F8>)
(<class 'WindowsError'>, WindowsError(5, 'Access is denied'), <traceback object at 0x02ADC0D0>)
.
======================================================================
ERROR: test_read_tree (test.test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python32\lib\shutil.py", line 281, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: 'c:\\users\\dreamkxd\\appdata\\local\\temp\\tmp5v9zd7\\testrepo\\.git\\objects\\pack\\pack-e6fbc15b315a0eab6
005c44e5b7054b1f0043f39.idx'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 73, in tearDown
    rmtree(self._temp_dir)
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 60, in rmtree
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "C:\Python32\lib\shutil.py", line 278, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python32\lib\shutil.py", line 278, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python32\lib\shutil.py", line 278, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python32\lib\shutil.py", line 278, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python32\lib\shutil.py", line 283, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 60, in <lambda>
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 49, in force_rm_handle
    remove_path(path)
WindowsError: [Error 5] Access is denied: 'c:\\users\\dreamkxd\\appdata\\local\\temp\\tmp5v9zd7\\testrepo\\.git\\objects\\pack\\pack-e6fbc15b315a0eab6
005c44e5b7054b1f0043f39.idx'

----------------------------------------------------------------------
Ran 10 tests in 2.280s

FAILED (errors=1)
Free repository 0x3031888
Free repository 0x2fbe6c0
Free repository 0x2fbfef8
Free repository 0x3039510
Free repository 0x303d168
Free repository 0x303d9f0
Free repository 0x30413f8
Free repository 0x3048ab0
p_munmap 0x304da9c
close fd:3, result:-1
p_munmap 0x304eb08
Free map 0x304eb08
Free repository 0x304c580
Free repository 0x304ea10

D:\CI\bld\vcs\pygit2>

post-install test fails

I just installed libgit2 and pygit2 on Ubuntu 12.04. The installation seems to proceed fine, but the test is failing with the following trace:

running test
running build
running build_ext
Traceback (most recent call last):
  File "setup.py", line 147, in <module>
    cmdclass=cmdclass)
  File "/usr/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "setup.py", line 80, in run
    unittest.main(module=None, defaultTest='test.test_suite', argv=test_argv)
  File "/usr/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/usr/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
  File "/usr/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
  File "/usr/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.7/unittest/loader.py", line 113, in loadTestsFromName
    test = obj()
  File "/home/basus/Public/pygit2/test/__init__.py", line 42, in test_suite
    return unittest.defaultTestLoader.loadTestsFromNames(modules)
  File "/usr/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.7/unittest/loader.py", line 100, in loadTestsFromName
    parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_blob'

Write a benchmark

The purpose is to measure whether a change improves the performance, degrades it, or changes nothing. So to help making decisions.

Installation error under MacOS Lion

Installing the latest version under Lion (having both the 86x and 64bit version of libgit2 installed) results in several errors that make the setup fail:

ld: duplicate symbol _BlobType in build/temp.macosx-10.7-intel-2.7/src/pygit2/blob.o and build/temp.macosx-10.7-intel-2.7/src/pygit2.o for architecture i386

To solve the problem I had to declare extern all the PyTypeObject defined in pygit2.c:

extern PyTypeObject RepositoryType;
extern PyTypeObject ObjectType;
extern PyTypeObject CommitType;
extern PyTypeObject DiffType;
extern PyTypeObject HunkType;
extern PyTypeObject TreeType;
extern PyTypeObject TreeBuilderType;
extern PyTypeObject TreeEntryType;
extern PyTypeObject TreeIterType;
extern PyTypeObject BlobType;
extern PyTypeObject TagType;
extern PyTypeObject IndexType;
extern PyTypeObject IndexEntryType;
extern PyTypeObject IndexIterType;
extern PyTypeObject WalkerType;
extern PyTypeObject ReferenceType;
extern PyTypeObject RefLogIterType;
extern PyTypeObject RefLogEntryType;
extern PyTypeObject SignatureType;

Custom Backends?

Is there any plan to add custom backends into pygit2? What Rugged has (Rugged::Backend) looks really nice; it would be great to be able to do that in pygit2, but the entire pygit2 Repository API seems to depend on everything working on a local filesystem.

Unittest didn't passed under Windows.

Updated test result:


D:\CI\bld\vcs\pygit2>python setup.py test
running test
running build
running build_py
running build_ext
.............E....................................E...........
======================================================================
ERROR: test_read_tree (test.test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python32\lib\shutil.py", line 281, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: 'c:\\users\\dreamkxd\\appdata\\local\\temp\\tmpjzh5wp\\testrepo\\.git\\objects\\pack\\pack-e6fbc15b315a0eab6005c44e5b7054b1f0043f39.idx'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 65, in tearDown
    rmtree(self._temp_dir)
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 54, in rmtree
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "C:\Python32\lib\shutil.py", line 278, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python32\lib\shutil.py", line 278, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python32\lib\shutil.py", line 278, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python32\lib\shutil.py", line 278, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python32\lib\shutil.py", line 283, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 54, in <lambda>
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 47, in force_rm_handle
    remove_path(path)
WindowsError: [Error 5] Access is denied: 'c:\\users\\dreamkxd\\appdata\\local\\temp\\tmpjzh5wp\\testrepo\\.git\\objects\\pack\\pack-e6fbc15b315a0eab6005c44e5b7054b1f0043f39.idx'

======================================================================
ERROR: test_new_tag (test.test_tag.TagTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_tag.py", line 70, in test_new_tag
    tagger, message)
pygit2.GitError: af431: Failed to create tag
        - Failed to retrieve tag reference
        - Failed to lookup reference
        - Failed to lookup reference
        - Failed to lookup reference from packfile
        - Failed to load packed references
        - Failed to parse OID of packed reference
        - Failed to lookup loose reference

----------------------------------------------------------------------
Ran 62 tests in 5.182s

FAILED (errors=2)

D:\CI\bld\vcs\pygit2>

Inconsistent results when iterating and indexing Tree

When iterating over the entries in a Tree, it seems like some entries appear in the iteration, but can't be retrieved by using the entry name as an index to the Tree object. I noticed this when trying to read the Git repository for the Linux kernel. There are some tree entries in 'include/linux' which show the problem. The tree entries / directories named 'can', 'hdlc', 'i2c', 'input', 'netfilter_bridge', 'rtc' and 'wimax' all show up in the list of entries when iterating over the tree for the directory 'include/linux', but can't be retrieved by, e.g., tree['can'] - this throws a KeyError.

I uploaded a small gist which shows up the problem: https://gist.github.com/3bcc088234d2de488258

One common feature to all of these is that there are also files which have the directory name as a prefix in the tree, e.g. 'can.h', 'hdlc.h'. Other entries, including directories like 'dvb', can be retrieved from the Tree object by name.

As far as I can see, Tree objects are supposed to be index-able by the names of entries in the Tree, as well as by a numeric index. Assuming this is correct, it looks like a bug in either pygit2 or libgit2.

I'm using the latest revisions of pygit2 (28b1deb) and libgit2 (77bb37eb47647180c0ed30517f6c6e4c219d131a). The operating system is Ubuntu 10.04.3, and the Python / GCC versions are:

$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) 
[GCC 4.4.3] on linux2

$ gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.4.3-4ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --program-suffix=-4.4 --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-plugin --enable-objc-gc --enable-targets=all --disable-werror --with-arch-32=i486 --with-tune=generic --enable-checking=release --build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)

commit metadata is incorrectly assumed to be UTF-8

git has not always supported encodings, and so there are commits "in the wild" that contain 8-bit non-UTF-8 messages.

Here's an example from the git tree itself that uses ISO-8859-1.

Unfortunately, pygit2 assumes and enforces UTF-8 encodings, in C, at read time. This means that commit metadata that is not UTF-8 causes a UnicodeDecodeError to be thrown, in such a way that there is no possible workaround in Python.

As far as I can tell, there are just a few possible sensible approaches:

Add a message_raw field to Commit and Tag objects that supplies the raw bytes and allows an app to decide how to decode them. This would give a decent fallback. Do the same for author and committer Signature objects.
Perform decoding in lenient or replace mode. I don't like this because it can lead to silent data loss, which is slightly worse than the current "unrecoverable exception" approach :-)

Builtin doc strings do not have argument information.

The doc strings for the API do not seem to have the argument information. eg.

|  status(...)
|      Reads the status of the repository and returns a dictionary with file paths as keys and status flags as values.
|      See pygit2.GIT_STATUS_*.
|
|  status_file(...)
|      Returns the status of the given file path.
|
|  walk(...)
|      Generator that traverses the history starting from the given commit.
|
|  write(...)
|      Write raw object data into the repository. First arg is the object
|      type, the second one a buffer with data. Return the object id (sha)
|      of the created object.
|

It would be helpful if such information was provided.

Commit access to tree and parents oids

We need something like Commit.tree_oid, that we can use instead of Commit.tree.oid, which would fail if the tree object is missing in the database. The same about Commit.parents.

Support for Notes

Since 0.17 libgit2 has a notes API https://github.com/libgit2/libgit2/blob/master/include/git2/notes.h it would be great if pygit could implement this functionality!

Linker fails when building on Windows

Hey, I have been trying all day to get pygit2 running on my machine. I successfully built libgit2 (a few times by now, with different options) but I simply can’t get pygit2 working. I keep getting unresolved symbols when linking, and I can’t really figure out why.

Below you can see the link command being invoked and some of the error messages (I truncated them, as it was basically repeating the same over and over again just with a different symbol/object name):

c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\amd64\link.exe /DLL /nologo /INCREMENTAL:NO "/LIBPATH:C:\Program Files\libgit2\lib" "/LIBPATH:C:\Program Files\Python32\libs" "/LIBPATH:C:\Program Files\Python32\PCbuild\amd64" git2.lib /EXPORT:PyInit__pygit2 build\temp.win-amd64-3.2\Release\src\pygit2.obj build\temp.win-amd64-3.2\Release\src\pygit2\blob.obj build\temp.win-amd64-3.2\Release\src\pygit2\commit.obj build\temp.win-amd64-3.2\Release\src\pygit2\config.obj build\temp.win-amd64-3.2\Release\src\pygit2\diff.obj build\temp.win-amd64-3.2\Release\src\pygit2\error.obj build\temp.win-amd64-3.2\Release\src\pygit2\index.obj build\temp.win-amd64-3.2\Release\src\pygit2\object.obj build\temp.win-amd64-3.2\Release\src\pygit2\oid.obj build\temp.win-amd64-3.2\Release\src\pygit2\reference.obj build\temp.win-amd64-3.2\Release\src\pygit2\repository.obj build\temp.win-amd64-3.2\Release\src\pygit2\signature.obj build\temp.win-amd64-3.2\Release\src\pygit2\tag.obj build\temp.win-amd64-3.2\Release\src\pygit2\tree.obj build\temp.win-amd64-3.2\Release\src\pygit2\utils.obj build\temp.win-amd64-3.2\Release\src\pygit2\walker.obj /OUT:build\lib.win-amd64-3.2\_pygit2.pyd /IMPLIB:build\temp.win-amd64-3.2\Release\src\_pygit2.lib /MANIFESTFILE:build\temp.win-amd64-3.2\Release\src\_pygit2.pyd.manifest
pygit2.obj : warning LNK4197: export 'PyInit__pygit2' specified multiple times; using first specification
   Creating library build\temp.win-amd64-3.2\Release\src\_pygit2.lib and object build\temp.win-amd64-3.2\Release\src\_pygit2.exp
pygit2.obj : error LNK2019: unresolved external symbol git_repository_free referenced in function init_repository
repository.obj : error LNK2001: unresolved external symbol git_repository_free
pygit2.obj : error LNK2019: unresolved external symbol git_repository_init referenced in function init_repository
pygit2.obj : error LNK2019: unresolved external symbol git_repository_discover referenced in function discover_repository
commit.obj : error LNK2019: unresolved external symbol git_commit_message_encoding referenced in function Commit_get_message_encoding
commit.obj : error LNK2019: unresolved external symbol git_commit_message referenced in function Commit_get_message
commit.obj : error LNK2019: unresolved external symbol git_commit_time referenced in function Commit_get_commit_time

[…] (this goes on for another 150 lines)

build\lib.win-amd64-3.2\_pygit2.pyd : fatal error LNK1120: 131 unresolved externals
error: command '"c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\amd64\link.exe"' failed with exit status 1120

I’m using Visual Studio 10 (2010) to build on Windows 7 x64 with Python 3.2.2 (x64); libgit2 was built with the same configurations and works fine.

Do you have any idea why this happens and what I can do to complete the build?

implement git_tree_entry_bypath

It would be handy to be able to do this:

import pygit2
repo = pygit2.Repository('/tmp/test_repo/.git')

# make a file
blob = repo.create_blob('Test Content')

# put the file in a dir
tb = repo.TreeBuilder()
tb.insert('test.txt', blob, 100644)
dir_id = tb.write()

# put the dir in a "root" tree
tb_root = repo.TreeBuilder()
tb_root.insert('test_dir', dir_id, 100755)
root_id = tb_root.write()

root_tree = repo[root_id]

print root_tree['test_dir/test.txt']

Traceback:

Traceback (most recent call last):
  File "test.py", line 21, in <module>
    print root_tree['test_dir/test.txt']
KeyError: 'test_dir/test.txt'

Error_set_str crashes if Repository.status_file is called on a directory

If I try to call Repository.status_file on a directory, giterr_last() returns NULL, which gets assigned to error in src/pygit2/error.c line 104. In line 105, a member of the git_error struct is dereferenced, but since error is a null pointer, the library crashes with a segmentation fault. See the example below, using ee28132 and libgit2/libgit2@81eecc3.

/tmp ➤ git init test
Initialized empty Git repository in /tmp/test/.git/
/tmp ➤ cd test
test ➤ mkdir foo
test ➤ touch foo/bar
test ➤ git add .
test ➤ git commit -m "foobar"
[master (root-commit) 104e1fa] foobar
 0 files changed
 create mode 100644 foo/bar
test ➤ gdb python
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/python...(no debugging symbols found)...done.
(gdb) run -c "from pygit2 import Repository; Repository('.git').status_file('foo')"
Starting program: /usr/bin/python -c "from pygit2 import Repository; Repository('.git').status_file('foo')"
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
Error_set_str (err=-5, str=0x9973e0 "foo") at src/pygit2/error.c:105
105         return PyErr_Format(Error_type(err), "%s: %s", str, error->message);
(gdb) print error
$1 = (const git_error *) 0x0

Suggestion: Use Cython

Cython is a Python-like language that gets compiled to C extensions for Python. It's mainly used for speeding up Python code and wrapping C libraries.

For pygit2, Cython would have these advantages:

python 3 compatibility
smaller, cleaner code (much less boilerplate; easier to write long docstrings)
easier maintenance (much DRYer than Python C-API; automatic refcounting)

And some disadvantages:

it's a complete rewrite
extra build-time dependency (source releases could come with the compiled .c file, though)
the language might not be as familiar as C (but it's not that hard to learn)

Do you think it's worth it?

There's a cython branch in my fork; I got the test suite to pass in Pythons 2.5–3.2.

pygit2 won't compile on MacOS Lion

Hi, I followed the installation tutorial but can't get past the setup.py install step. I'm using Mac OS X 10.7.2.

Here's the last error message I get:

lipo: can't figure out the architecture type of: /var/folders/sy/r8t7px154vs3rb9dfvt3hw1m0000gn/T//cc2CqvAs.out
error: command 'gcc-4.2' failed with exit status 1

Implement some stuff in Python rather than C

As raised in issue #57, it would be nice to implement some stuff in Python rather than C, specifically extending the API of basic types (like commits).

The example from issue #57 is to add the Commit.get_subject method.

several small c files instead of one big 3727 line monster

Is there any good reason to have one big c-extension file with 3727 lines instead of several small ones?

Not compatible with pip/virtualenv

When installing via pip within a virtualenv, I get this error:

$ pip install pygit2
Downloading/unpacking pygit2
  Downloading pygit2-0.17.1.tar.gz (40Kb): 40Kb downloaded
  Running setup.py egg_info for package pygit2

Installing collected packages: pygit2
  Running setup.py install for pygit2
    building '_pygit2' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/pygit2.c -o build/temp.linux-x86_64-2.7/src/pygit2.o
    src/pygit2.c:31:26: fatal error: pygit2/error.h: No such file or directory
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools;__file__='/home/ben/Projects/Sandbox/gittest/src/build/pygit2/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-DwY3wr-record/install-record.txt:
    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/pygit2

copying pygit2/utils.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/__init__.py -> build/lib.linux-x86_64-2.7/pygit2

running build_ext

building '_pygit2' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/src

creating build/temp.linux-x86_64-2.7/src/pygit2

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/pygit2.c -o build/temp.linux-x86_64-2.7/src/pygit2.o

src/pygit2.c:31:26: fatal error: pygit2/error.h: No such file or directory

compilation terminated.

error: command 'gcc' failed with exit status 1

----------------------------------------
Command /usr/bin/python -c "import setuptools;__file__='/home/ben/Projects/Sandbox/gittest/src/build/pygit2/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-DwY3wr-record/install-record.txt failed with error code 1 in /home/ben/Projects/Sandbox/gittest/src/build/pygit2
Storing complete log in /home/ben/.pip/pip.log

pygit tests don't work

sjuul@vaio:~/projects/pygit2$ sudo python setup.py test
running test
running build
running build_py
running build_ext
Traceback (most recent call last):
File "setup.py", line 185, in
cmdclass=cmdclass)
File "/usr/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 93, in run
unittest.main(module=None, defaultTest='test.test_suite', argv=test_argv)
File "/usr/lib/python2.7/unittest/main.py", line 94, in init
self.parseArgs(argv)
File "/usr/lib/python2.7/unittest/main.py", line 149, in parseArgs
self.createTests()
File "/usr/lib/python2.7/unittest/main.py", line 158, in createTests
self.module)
File "/usr/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib/python2.7/unittest/loader.py", line 113, in loadTestsFromName
test = obj()
File "/home/sjuul/projects/pygit2/test/init.py", line 42, in test_suite
return unittest.defaultTestLoader.loadTestsFromNames(modules)
File "/usr/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib/python2.7/unittest/loader.py", line 100, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_blob'

Test Failure (AttributeError) with Python 2.7

When I run make test, it dies with AttributeError: 'module' object has no attribute 'test_blob'.
This is on OS X 10.6.6, with Python 2.7.1 (Python.org binary), and building against libgit2/libgit2@88eb05e.

Here's the full traceback:
(lgh)erik@Avatar ~/Sandbox/lgh/pygit2 $ python setup.py test
running test
running egg_info
writing pygit2.egg-info/PKG-INFO
writing top-level names to pygit2.egg-info/top_level.txt
writing dependency_links to pygit2.egg-info/dependency_links.txt
reading manifest file 'pygit2.egg-info/SOURCES.txt'
writing manifest file 'pygit2.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.6-intel-2.7/pygit2.so ->
Traceback (most recent call last):
File "setup.py", line 87, in
**kwargs
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/Users/erik/Sandbox/lgh/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/test.py", line 137, in run
self.with_project_on_sys_path(self.run_tests)
File "/Users/erik/Sandbox/lgh/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/test.py", line 117, in with_project_on_sys_path
func()
File "/Users/erik/Sandbox/lgh/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/test.py", line 146, in run_tests
testLoader = loader_class()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 94, in init
self.parseArgs(argv)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 149, in parseArgs
self.createTests()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 158, in createTests
self.module)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 113, in loadTestsFromName
test = obj()
File "/Users/erik/Sandbox/lgh/pygit2/test/init.py", line 41, in test_suite
return unittest.defaultTestLoader.loadTestsFromNames(modules)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 100, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_blob'

Support Tree Diffs

I haven't written a pure C extension for Python before otherwise I would try my hand at this right now. libgit2 supports diffing between trees. "We" should add this to pygit2. Here is the function libgit2 provides:

git_tree_diff(git_tree *old, git_tree *newer, git_tree_diff_cb cb, void *data )

    Diff two trees

    Compare two trees. For each difference in the trees, the callback
    will be called with a git_tree_diff_data filled with the relevant
    information.

    @param old the "old" tree
    @param newer the "newer" tree
    @param cb callback
    @param data data to give to the callback
    @return GIT_SUCCESS or an error code

It generates a diff between the two given trees. I propose the following interface in pygit2:

class Tree(object):

    def diff(self, other):
        return pygit2.tree_diff(self, other)

with tree_diff being a thin wrapper around git_tree_diff. There may be a better API for this but this seems to be a natural way one could do it. If I have time over the next week I may try my hand at it but I don't promise good code.

Clone command in README

The command to clone pygit2 probably should look like this:
git clone git://github.com/libgit2/pygit2.git

python3 build fails using 2to3

On the version tagged 0.17.0, I copide the files to a dedicated py3 folder, I then ran
2to3 --write --nobackups py3

I then get:
$ /usr/bin/python3 setup.py build
Traceback (most recent call last):
File "setup.py", line 129, in
long_description = readme.read()
File "/usr/lib64/python3.2/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 6643: ordinal not in range(128)
error: Bad exit status from /var/tmp/rpm-tmp.FsMGsW (%build)

Apparently something in the README.rst is not appreciated.

unittest failed.

D:\CI\bld\vcs\pygit2>cd ..

D:\CI\bld\vcs>build-pygit2.bat

D:\CI\bld\vcs\pygit2>call python setup.py test  & cd /d D:\CI\bld\vcs
running test
running build
running build_ext
building 'pygit2' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -ID:\CI\Tools\vcs\libgit2\include -IC:\Python27\
include -IC:\Python27\PC /Tcpygit2.c /Fobuild\temp.win32-2.7\Release\pygit2.obj
pygit2.c
pygit2.c(1277) : warning C4244: 'function' : conversion from 'git_time_t' to 'long', possible loss of data
pygit2.c(3014) : warning C4244: 'function' : conversion from 'const git_time_t' to 'long', possible loss of data
creating build\lib.win32-2.7
c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:D:\CI\Tools\vcs\libgit2\lib /LIBPATH:C:\Pytho
n27\libs /LIBPATH:C:\Python27\PCbuild git2.lib /EXPORT:initpygit2 build\temp.win32-2.7\Release\pygit2.obj /OUT:build\lib.win32-2.7\pygit2.pyd /IMPLIB:
build\temp.win32-2.7\Release\pygit2.lib /MANIFESTFILE:build\temp.win32-2.7\Release\pygit2.pyd.manifest
   Creating library build\temp.win32-2.7\Release\pygit2.lib and object build\temp.win32-2.7\Release\pygit2.exp
C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\mt.exe -nologo -manifest build\temp.win32-2.7\Release\pygit2.pyd.manifest -outputresource:build\lib.
win32-2.7\pygit2.pyd;2
..EEEEE........E..............E.....E...........EE......E...E....EEEE
======================================================================
ERROR: test_new_commit (test.test_commit.CommitTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_commit.py", line 96, in test_new_commit
    self.assertEqual(COMMIT_SHA, commit.parents[0].hex)
AttributeError: 'str' object has no attribute 'hex'

======================================================================
ERROR: test_new_commit (test.test_commit.CommitTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 70, in tearDown
    rmtree(self._temp_dir)
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 59, in rmtree
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 249, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 59, in <lambda>
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 48, in force_rm_handle
    remove_path(path)
WindowsError: [Error 5] Access is denied: 'c:\\users\\dreamkxd\\appdata\\local\\temp\\tmpwhd2mw\\testrepo.git\\objects\\pack\\pack-822653eb59791a6df71
4f8aa5fbf9f1c1951478e.idx'

======================================================================
ERROR: test_new_commit_encoding (test.test_commit.CommitTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_commit.py", line 122, in test_new_commit_encoding
    self.assertEqual(COMMIT_SHA, commit.parents[0].hex)
AttributeError: 'str' object has no attribute 'hex'

======================================================================
ERROR: test_new_commit_encoding (test.test_commit.CommitTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 70, in tearDown
    rmtree(self._temp_dir)
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 59, in rmtree
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 249, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 59, in <lambda>
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 48, in force_rm_handle
    remove_path(path)
WindowsError: [Error 5] Access is denied: 'c:\\users\\dreamkxd\\appdata\\local\\temp\\tmpvweb_e\\testrepo.git\\objects\\pack\\pack-822653eb59791a6df71
4f8aa5fbf9f1c1951478e.idx'

======================================================================
ERROR: test_read_commit (test.test_commit.CommitTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_commit.py", line 51, in test_read_commit
    parents[0].hex)
AttributeError: 'str' object has no attribute 'hex'

======================================================================
ERROR: test_read_tree (test.test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 70, in tearDown
    rmtree(self._temp_dir)
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 59, in rmtree
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 249, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 59, in <lambda>
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 48, in force_rm_handle
    remove_path(path)
WindowsError: [Error 5] Access is denied: 'c:\\users\\dreamkxd\\appdata\\local\\temp\\tmpsnhj2z\\testrepo\\.git\\objects\\pack\\pack-e6fbc15b315a0eab6
005c44e5b7054b1f0043f39.idx'

======================================================================
ERROR: test_reference_set_sha_prefix (test.test_refs.ReferencesTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_refs.py", line 90, in test_reference_set_sha_prefix
    reference.oid = NEW_COMMIT[0:6]
NotImplementedError: short strings not yet supported

======================================================================
ERROR: test_contains (test.test_repository.RepositoryTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_repository.py", line 79, in test_contains
    self.assertTrue(A_HEX_SHA[0:10] in self.repo)
NotImplementedError: short strings not yet supported

======================================================================
ERROR: test_hide_prefix (test.test_revwalk.WalkerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_revwalk.py", line 69, in test_hide_prefix
    walker.hide('4ec4389a')
NotImplementedError: short strings not yet supported

======================================================================
ERROR: test_hide_prefix (test.test_revwalk.WalkerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 70, in tearDown
    rmtree(self._temp_dir)
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 59, in rmtree
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 244, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\lib\shutil.py", line 249, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 59, in <lambda>
    shutil.rmtree(path, onerror=lambda func, path, e: force_rm_handle(func, path, e))
  File "D:\CI\bld\vcs\pygit2\test\utils.py", line 48, in force_rm_handle
    remove_path(path)
WindowsError: [Error 5] Access is denied: 'c:\\users\\dreamkxd\\appdata\\local\\temp\\tmpmzrcrd\\testrepo\\.git\\objects\\pack\\pack-e6fbc15b315a0eab6
005c44e5b7054b1f0043f39.idx'

======================================================================
ERROR: test_new_tag (test.test_tag.TagTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_tag.py", line 72, in test_new_tag
    tagger, message)
GitError: af431: Failed to create tag
        - Failed to retrieve tag reference
        - Failed to lookup reference
        - Failed to lookup reference
        - Failed to lookup reference from packfile
        - Failed to load packed references
        - Failed to parse OID of packed reference
        - Failed to lookup loose reference

======================================================================
ERROR: test_read_subtree (test.test_tree.TreeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_tree.py", line 78, in test_read_subtree
    subtree = subtree_entry.to_object()
AttributeError: 'pygit2.TreeEntry' object has no attribute 'to_object'

======================================================================
ERROR: test_new_empty_treebuilder (test.test_treebuilder.TreeBuilderTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_treebuilder.py", line 45, in test_new_empty_treebuilder
    bld = self.repo.TreeBuilder()
AttributeError: 'pygit2.Repository' object has no attribute 'TreeBuilder'

======================================================================
ERROR: test_noop_treebuilder (test.test_treebuilder.TreeBuilderTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_treebuilder.py", line 49, in test_noop_treebuilder
    bld = self.repo.TreeBuilder(TREE_SHA)
AttributeError: 'pygit2.Repository' object has no attribute 'TreeBuilder'

======================================================================
ERROR: test_noop_treebuilder_from_tree (test.test_treebuilder.TreeBuilderTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_treebuilder.py", line 55, in test_noop_treebuilder_from_tree
    bld = self.repo.TreeBuilder(tree)
AttributeError: 'pygit2.Repository' object has no attribute 'TreeBuilder'

======================================================================
ERROR: test_rebuild_treebuilder (test.test_treebuilder.TreeBuilderTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\CI\bld\vcs\pygit2\test\test_treebuilder.py", line 61, in test_rebuild_treebuilder
    bld = self.repo.TreeBuilder()
AttributeError: 'pygit2.Repository' object has no attribute 'TreeBuilder'

----------------------------------------------------------------------
Ran 66 tests in 13.749s

FAILED (errors=16)

D:\CI\bld\vcs>

error building

i get this error when i tried to build pygit2 on freebsd box:

    [khine@awakwe:~/sandboxes/pygit2] ~/.virtualenvs/itools/bin/python setup.py install
    /usr/local/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'package'
      warnings.warn(msg)
    running install
    running build
    running build_ext
    building '_pygit2' extension
    cc -fno-strict-aliasing -O2 -pipe -fno-strict-aliasing -DNDEBUG -O2 -pipe -fno-strict-aliasing -fPIC -I/usr/local/include -Iinclude -I/usr/local/include/python2.7 -c src/pygit2.c -o build/temp.freebsd-9.0-RELEASE-amd64-2.7/src/pygit2.o
    cc -fno-strict-aliasing -O2 -pipe -fno-strict-aliasing -DNDEBUG -O2 -pipe -fno-strict-aliasing -fPIC -I/usr/local/include -Iinclude -I/usr/local/include/python2.7 -c src/pygit2/error.c -o build/temp.freebsd-9.0-RELEASE-amd64-2.7/src/pygit2/error.o
    src/pygit2/error.c: In function 'Error_type':
    src/pygit2/error.c:18: error: 'GIT_EAMBIGUOUS' undeclared (first use in this function)
    src/pygit2/error.c:18: error: (Each undeclared identifier is reported only once
    src/pygit2/error.c:18: error: for each function it appears in.)
    src/pygit2/error.c:22: error: 'GIT_EBUFS' undeclared (first use in this function)
    src/pygit2/error.c:26: error: 'GIT_PASSTHROUGH' undeclared (first use in this function)
    src/pygit2/error.c:30: error: 'GIT_REVWALKOVER' undeclared (first use in this function)
    src/pygit2/error.c:35: warning: initialization makes pointer from integer without a cast
    src/pygit2/error.c:36: error: request for member 'klass' in something not a structure or union
    src/pygit2/error.c:37: error: 'GITERR_NOMEMORY' undeclared (first use in this function)
    src/pygit2/error.c:39: error: 'GITERR_OS' undeclared (first use in this function)
    src/pygit2/error.c:41: error: 'GITERR_INVALID' undeclared (first use in this function)
    src/pygit2/error.c: In function 'Error_set':
    src/pygit2/error.c:56: warning: initialization makes pointer from integer without a cast
    src/pygit2/error.c:57: error: request for member 'message' in something not a structure or union
    src/pygit2/error.c: In function 'Error_set_str':
    src/pygit2/error.c:71: warning: initialization makes pointer from integer without a cast
    src/pygit2/error.c:72: error: request for member 'message' in something not a structure or union
    error: command 'cc' failed with exit status 1

and the same error on MacOSX

    (itools)☺  python setup.py install                                                                                                                         * master 872f4dbitools"
    running install
    running build
    running build_py
    creating build/lib.macosx-10.5-intel-2.7/pygit2
    copying pygit2/__init__.py -> build/lib.macosx-10.5-intel-2.7/pygit2
    copying pygit2/utils.py -> build/lib.macosx-10.5-intel-2.7/pygit2
    running build_ext
    building '_pygit2' extension
    creating build/temp.macosx-10.5-intel-2.7/src
    creating build/temp.macosx-10.5-intel-2.7/src/pygit2
    /usr/bin/gcc-4.2 -fno-strict-aliasing -arch i386 -arch x86_64 -O3 -w -pipe -march=core2 -msse4.1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/include -Iinclude -I/usr/local/Cellar/python/2.7.2/include/python2.7 -c src/pygit2.c -o build/temp.macosx-10.5-intel-2.7/src/pygit2.o
    In file included from src/pygit2.c:33:
    include/pygit2/types.h:32: error: expected specifier-qualifier-list before ‘git_diff_list’
    src/pygit2.c: In function ‘moduleinit’:
    src/pygit2.c:240: error: ‘GIT_DIFF_NORMAL’ undeclared (first use in this function)
    src/pygit2.c:240: error: (Each undeclared identifier is reported only once
    src/pygit2.c:240: error: for each function it appears in.)
    src/pygit2.c:241: error: ‘GIT_DIFF_REVERSE’ undeclared (first use in this function)
    src/pygit2.c:242: error: ‘GIT_DIFF_FORCE_TEXT’ undeclared (first use in this function)
    src/pygit2.c:244: error: ‘GIT_DIFF_IGNORE_WHITESPACE’ undeclared (first use in this function)
    src/pygit2.c:246: error: ‘GIT_DIFF_IGNORE_WHITESPACE_CHANGE’ undeclared (first use in this function)
    src/pygit2.c:248: error: ‘GIT_DIFF_IGNORE_WHITESPACE_EOL’ undeclared (first use in this function)
    src/pygit2.c:250: error: ‘GIT_DIFF_IGNORE_SUBMODULES’ undeclared (first use in this function)
    src/pygit2.c:251: error: ‘GIT_DIFF_PATIENCE’ undeclared (first use in this function)
    src/pygit2.c:253: error: ‘GIT_DIFF_INCLUDE_IGNORED’ undeclared (first use in this function)
    src/pygit2.c:255: error: ‘GIT_DIFF_INCLUDE_UNTRACKED’ undeclared (first use in this function)
    src/pygit2.c:257: error: ‘GIT_DIFF_INCLUDE_UNMODIFIED’ undeclared (first use in this function)
    src/pygit2.c:259: error: ‘GIT_DIFF_RECURSE_UNTRACKED_DIRS’ undeclared (first use in this function)
    src/pygit2.c:263: error: ‘GIT_DIFF_FILE_VALID_OID’ undeclared (first use in this function)
    src/pygit2.c:265: error: ‘GIT_DIFF_FILE_FREE_PATH’ undeclared (first use in this function)
    src/pygit2.c:266: error: ‘GIT_DIFF_FILE_BINARY’ undeclared (first use in this function)
    src/pygit2.c:268: error: ‘GIT_DIFF_FILE_NOT_BINARY’ undeclared (first use in this function)
    src/pygit2.c:270: error: ‘GIT_DIFF_FILE_FREE_DATA’ undeclared (first use in this function)
    src/pygit2.c:272: error: ‘GIT_DIFF_FILE_UNMAP_DATA’ undeclared (first use in this function)
    src/pygit2.c:275: error: ‘GIT_DELTA_UNMODIFIED’ undeclared (first use in this function)
    src/pygit2.c:276: error: ‘GIT_DELTA_ADDED’ undeclared (first use in this function)
    src/pygit2.c:277: error: ‘GIT_DELTA_DELETED’ undeclared (first use in this function)
    src/pygit2.c:278: error: ‘GIT_DELTA_MODIFIED’ undeclared (first use in this function)
    src/pygit2.c:279: error: ‘GIT_DELTA_RENAMED’ undeclared (first use in this function)
    src/pygit2.c:280: error: ‘GIT_DELTA_COPIED’ undeclared (first use in this function)
    src/pygit2.c:281: error: ‘GIT_DELTA_IGNORED’ undeclared (first use in this function)
    src/pygit2.c:282: error: ‘GIT_DELTA_UNTRACKED’ undeclared (first use in this function)
    src/pygit2.c:285: error: ‘GIT_DIFF_LINE_CONTEXT’ undeclared (first use in this function)
    src/pygit2.c:287: error: ‘GIT_DIFF_LINE_ADDITION’ undeclared (first use in this function)
    src/pygit2.c:289: error: ‘GIT_DIFF_LINE_DELETION’ undeclared (first use in this function)
    src/pygit2.c:291: error: ‘GIT_DIFF_LINE_ADD_EOFNL’ undeclared (first use in this function)
    src/pygit2.c:293: error: ‘GIT_DIFF_LINE_DEL_EOFNL’ undeclared (first use in this function)
    src/pygit2.c:295: error: ‘GIT_DIFF_LINE_FILE_HDR’ undeclared (first use in this function)
    src/pygit2.c:297: error: ‘GIT_DIFF_LINE_HUNK_HDR’ undeclared (first use in this function)
    src/pygit2.c:298: error: ‘GIT_DIFF_LINE_BINARY’ undeclared (first use in this function)
    In file included from src/pygit2.c:33:
    include/pygit2/types.h:32: error: expected specifier-qualifier-list before ‘git_diff_list’
    src/pygit2.c: In function ‘moduleinit’:
    src/pygit2.c:240: error: ‘GIT_DIFF_NORMAL’ undeclared (first use in this function)
    src/pygit2.c:240: error: (Each undeclared identifier is reported only once
    src/pygit2.c:240: error: for each function it appears in.)
    src/pygit2.c:241: error: ‘GIT_DIFF_REVERSE’ undeclared (first use in this function)
    src/pygit2.c:242: error: ‘GIT_DIFF_FORCE_TEXT’ undeclared (first use in this function)
    src/pygit2.c:244: error: ‘GIT_DIFF_IGNORE_WHITESPACE’ undeclared (first use in this function)
    src/pygit2.c:246: error: ‘GIT_DIFF_IGNORE_WHITESPACE_CHANGE’ undeclared (first use in this function)
    src/pygit2.c:248: error: ‘GIT_DIFF_IGNORE_WHITESPACE_EOL’ undeclared (first use in this function)
    src/pygit2.c:250: error: ‘GIT_DIFF_IGNORE_SUBMODULES’ undeclared (first use in this function)
    src/pygit2.c:251: error: ‘GIT_DIFF_PATIENCE’ undeclared (first use in this function)
    src/pygit2.c:253: error: ‘GIT_DIFF_INCLUDE_IGNORED’ undeclared (first use in this function)
    src/pygit2.c:255: error: ‘GIT_DIFF_INCLUDE_UNTRACKED’ undeclared (first use in this function)
    src/pygit2.c:257: error: ‘GIT_DIFF_INCLUDE_UNMODIFIED’ undeclared (first use in this function)
    src/pygit2.c:259: error: ‘GIT_DIFF_RECURSE_UNTRACKED_DIRS’ undeclared (first use in this function)
    src/pygit2.c:263: error: ‘GIT_DIFF_FILE_VALID_OID’ undeclared (first use in this function)
    src/pygit2.c:265: error: ‘GIT_DIFF_FILE_FREE_PATH’ undeclared (first use in this function)
    src/pygit2.c:266: error: ‘GIT_DIFF_FILE_BINARY’ undeclared (first use in this function)
    src/pygit2.c:268: error: ‘GIT_DIFF_FILE_NOT_BINARY’ undeclared (first use in this function)
    src/pygit2.c:270: error: ‘GIT_DIFF_FILE_FREE_DATA’ undeclared (first use in this function)
    src/pygit2.c:272: error: ‘GIT_DIFF_FILE_UNMAP_DATA’ undeclared (first use in this function)
    src/pygit2.c:275: error: ‘GIT_DELTA_UNMODIFIED’ undeclared (first use in this function)
    src/pygit2.c:276: error: ‘GIT_DELTA_ADDED’ undeclared (first use in this function)
    src/pygit2.c:277: error: ‘GIT_DELTA_DELETED’ undeclared (first use in this function)
    src/pygit2.c:278: error: ‘GIT_DELTA_MODIFIED’ undeclared (first use in this function)
    src/pygit2.c:279: error: ‘GIT_DELTA_RENAMED’ undeclared (first use in this function)
    src/pygit2.c:280: error: ‘GIT_DELTA_COPIED’ undeclared (first use in this function)
    src/pygit2.c:281: error: ‘GIT_DELTA_IGNORED’ undeclared (first use in this function)
    src/pygit2.c:282: error: ‘GIT_DELTA_UNTRACKED’ undeclared (first use in this function)
    src/pygit2.c:285: error: ‘GIT_DIFF_LINE_CONTEXT’ undeclared (first use in this function)
    src/pygit2.c:287: error: ‘GIT_DIFF_LINE_ADDITION’ undeclared (first use in this function)
    src/pygit2.c:289: error: ‘GIT_DIFF_LINE_DELETION’ undeclared (first use in this function)
    src/pygit2.c:291: error: ‘GIT_DIFF_LINE_ADD_EOFNL’ undeclared (first use in this function)
    src/pygit2.c:293: error: ‘GIT_DIFF_LINE_DEL_EOFNL’ undeclared (first use in this function)
    src/pygit2.c:295: error: ‘GIT_DIFF_LINE_FILE_HDR’ undeclared (first use in this function)
    src/pygit2.c:297: error: ‘GIT_DIFF_LINE_HUNK_HDR’ undeclared (first use in this function)
    src/pygit2.c:298: error: ‘GIT_DIFF_LINE_BINARY’ undeclared (first use in this function)
    lipo: can't open input file: /var/folders/it/itEAjVAME2GWcW-Q6QxMBU+++TI/-Tmp-//ccc1l2qB.out (No such file or directory)
    error: command '/usr/bin/gcc-4.2' failed with exit status 1

on both machines release 0.16.2 works

commit 2bbf62f

but after this it does not work

I have libgit2-0.16.0_1 installed

Testsuite failure on windows

The testsuite does not work on windows.

pygit2 fails if the repository path contains \ one easy workaround in the testsuite is to replace them by / but I think this should be supported by pygit2/libgit2

On my machine, shutil.rmtree fails with 'Access denied' for half of the tests. If I force ignore_errors to True, all tests are OK.

See the workaround in f477901

install fails on Mac OSX

Installed latest libgit2, zlib and openssl. Using this pygit2 revision, I can't get it to install:

bash-3.2$ sudo python setup.py install
running install
running bdist_egg
running egg_info
writing pygit2.egg-info/PKG-INFO
writing top-level names to pygit2.egg-info/top_level.txt
writing dependency_links to pygit2.egg-info/dependency_links.txt
reading manifest file 'pygit2.egg-info/SOURCES.txt'
writing manifest file 'pygit2.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.6-universal/egg
running install_lib
running build_ext
building 'pygit2' extension
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I/usr/local/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c pygit2.c -o build/temp.macosx-10.6-universal-2.6/pygit2.o
pygit2.c: In function ‘Repository_walk’:
pygit2.c:356: error: void value not ignored as it ought to be
pygit2.c: In function ‘Walker_sort’:
pygit2.c:1838: error: void value not ignored as it ought to be
pygit2.c: In function ‘Repository_walk’:
pygit2.c:356: error: void value not ignored as it ought to be
pygit2.c: In function ‘Walker_sort’:
pygit2.c:1838: error: void value not ignored as it ought to be
pygit2.c: In function ‘Repository_walk’:
pygit2.c:356: error: void value not ignored as it ought to be
pygit2.c: In function ‘Walker_sort’:
pygit2.c:1838: error: void value not ignored as it ought to be
lipo: can't open input file: /var/tmp//ccSxjred.out (No such file or directory)
error: command 'gcc-4.2' failed with exit status 1

Segmentation fault caused by Index destructor

This is a bit of an odd case, but here's a script that I've been using to reproduce the issue: https://gist.github.com/1124259. This segfaults when it attempts to access the tree of the second Repository instance that gets created, and it crashes right after do_foo() when running Python with debugging. I've been able to reproduce the issue in numerous environments (Ubuntu 10.04 64-bit with packaged Python 2.6.5 with and without debugging, OSX 10.6 with Python 2.6.7 installed with Macports, and OSX 10.7 with system Python) using HEAD of master for libgit2 and pygit2 as of this writing (commits 03d88ed and 28b1deb respectively).

From what I can tell, Index_dealloc is trying to decrement the reference count for its pointer to Repository, however this wasn't a new object that was created (it was set to the Repository itself on the Index created in Repository_get_index). I removed the line from Index_dealloc, and I stopped seeing the issues. I'll create a pull request with the patch.

Proper support for text/byte strings

Properly accept and return values as text strings, or byte strings, or both. Whatever is appropriate in each case.

Pygit cant find libgit

When I import pygit2 I get the following error:

sjuul@vaio:~/projects/pygit2$ python
Python 2.7.3 (default, Aug 1 2012, 05:16:07)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.

from pygit2 import *
Traceback (most recent call last):
File "", line 1, in
File "pygit2/init.py", line 29, in
from _pygit2 import *
ImportError: libgit2.so.0: cannot open shared object file: No such file or directory

But when I search my folders it's right there where it's supposed to be

sjuul@vaio:~/projects/pygit2$ sudo find / -name "libgit2.so.0"
/usr/local/lib/libgit2.so.0

Doesn't work with latest development branch of libgit2

On libgit2/libgit2@0838ccc :

~/git/pygit2 master )$ python setup.py build
running build
running build_ext
building 'pygit2' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -I/usr/include/python2.6 -c pygit2.c -o build/temp.linux-x86_64-2.6/pygit2.o
pygit2.c: In function ‘Repository_create_commit’:
pygit2.c:501: warning: passing argument 7 of ‘git_commit_create’ from incompatible pointer type
/usr/local/include/git2/commit.h:233: note: expected ‘const struct git_tree *’ but argument is of type ‘struct git_oid *’
pygit2.c:501: warning: passing argument 9 of ‘git_commit_create’ from incompatible pointer type
/usr/local/include/git2/commit.h:233: note: expected ‘const struct git_commit **’ but argument is of type ‘const struct git_oid **’
pygit2.c: In function ‘Repository_create_tag’:
pygit2.c:527: warning: passing argument 4 of ‘git_tag_create’ from incompatible pointer type
/usr/local/include/git2/tag.h:183: note: expected ‘const struct git_object *’ but argument is of type ‘struct git_oid *’
pygit2.c:527: warning: passing argument 5 of ‘git_tag_create’ makes pointer from integer without a cast
/usr/local/include/git2/tag.h:183: note: expected ‘const struct git_signature *’ but argument is of type ‘int’
pygit2.c:527: warning: passing argument 6 of ‘git_tag_create’ from incompatible pointer type
/usr/local/include/git2/tag.h:183: note: expected ‘const char *’ but argument is of type ‘struct git_signature *’
pygit2.c:527: warning: passing argument 7 of ‘git_tag_create’ makes integer from pointer without a cast
/usr/local/include/git2/tag.h:183: note: expected ‘int’ but argument is of type ‘char *’
pygit2.c: In function ‘Repository_create_reference’:
pygit2.c:609: error: too few arguments to function ‘git_reference_create_oid’
pygit2.c: In function ‘Repository_create_symbolic_reference’:
pygit2.c:629: error: too few arguments to function ‘git_reference_create_symbolic’
pygit2.c: In function ‘Reference_rename’:
pygit2.c:1981: error: too few arguments to function ‘git_reference_rename’
error: command 'gcc' failed with exit status 1