zenhack / go.notmuch Goto Github PK

View Code? Open in Web Editor NEW

20.0 10.0 9.0 117 KB

Go language bindings for notmuch mail

License: GNU General Public License v3.0

Go 99.25% Makefile 0.75%

go.notmuch's People

Stargazers

Watchers

Forkers

kalbasit plumpmath msp301 brunnre8 yzzyx stbenjam 00mjk nvedant

go.notmuch's Issues

Rewrite notmuchmail in Go

Today I'm wondering if it would be a much better investment to write the core of notmuch (indexing and searching) using bleve or ElasticSearch (Best would be a swappable backend). Thoughts?

Deprecated notmuch database open function

go.notmuch throw this warning during the compilation. Maybe we should update the to the new binding?

# github.com/zenhack/go.notmuch
cgo-gcc-prolog: In function ‘_cgo_c02956f23830_Cfunc_notmuch_database_open’:
cgo-gcc-prolog:347:2: warning: ‘notmuch_database_open’ is deprecated: function deprecated as of libnotmuch 5.4 [-Wdeprecated-declarations]
In file included from /build/go/pkg/mod/github.com/brunnre8/[email protected]/db.go:9:
/usr/include/notmuch.h:332:1: note: declared here
  332 | notmuch_database_open (const char *path,
      | ^~~~~~~~~~~~~~~~~~~~~

core on .Replies

This is the stack trace:

goroutine 1 [syscall]:
runtime.cgocall(0x5254d0, 0xc00010f5b8)
	/usr/lib/go-1.17/src/runtime/cgocall.go:156 +0x5c fp=0xc00010f590 sp=0xc00010f558 pc=0x40e33c
github.com/zenhack/go%2enotmuch._Cfunc_notmuch_message_get_replies(0x7fd25403ee10)
	_cgo_gotypes.go:698 +0x4d fp=0xc00010f5b8 sp=0xc00010f590 pc=0x4ae2cd
github.com/zenhack/go%2enotmuch.(*Message).Replies.func1(0x2)
	/home/mdt/go/pkg/mod/github.com/zenhack/[email protected]/message.go:42 +0x46 fp=0xc00010f5f0 sp=0xc00010f5b8 pc=0x4afe46
github.com/zenhack/go%2enotmuch.(*Message).Replies(0xc0000a1560)
	/home/mdt/go/pkg/mod/github.com/zenhack/[email protected]/message.go:42 +0x25 fp=0xc00010f620 sp=0xc00010f5f0 pc=0x4afd45
main.r(0xc0000a0e70)
	/home/mdt/Source/emdete/gomail/core.go:14 +0x105 fp=0xc00010f688 sp=0xc00010f620 pc=0x520945
main.x(0xc, {0xc0000fc360, 0x10})
	/home/mdt/Source/emdete/gomail/core.go:34 +0x16a fp=0xc00010f758 sp=0xc00010f688 pc=0x520c2a
main.(*Threads).notifyThreadsThread(0x6fff20, {0x58fd70, 0xc000176240})
	/home/mdt/Source/emdete/gomail/threads.go:263 +0x108 fp=0xc00010f788 sp=0xc00010f758 pc=0x524888
main.(*Threads).EventHandler(0x6fff20, {0x58fd70, 0xc000176240}, {0x58b800, 0xc000394860})
	/home/mdt/Source/emdete/gomail/threads.go:147 +0x378 fp=0xc00010f7d8 sp=0xc00010f788 pc=0x523eb8
main.main()
	/home/mdt/Source/emdete/gomail/main.go:101 +0x6cc fp=0xc00010ff80 sp=0xc00010f7d8 pc=0x5227ec
runtime.main()
	/usr/lib/go-1.17/src/runtime/proc.go:255 +0x227 fp=0xc00010ffe0 sp=0xc00010ff80 pc=0x43f7e7
runtime.goexit()
	/usr/lib/go-1.17/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc00010ffe8 sp=0xc00010ffe0 pc=0x46c241

this is the code used:

package main

import (
	"log"
	"github.com/zenhack/go.notmuch"
)

func r(messages *notmuch.Messages) {
	log.Printf("-- SAMPLE enter r")
	message := &notmuch.Message{}
	for messages.Next(&message) {
		defer message.Close()
		log.Printf("%v", message)
		if replies, err := message.Replies(); err == nil {
			defer replies.Close()
			r(replies)
		}
	}
}

func x(db *notmuch.DB, id string) {
	log.Printf("-- SAMPLE BEGIN %s", id)
	nmq := db.NewQuery(id)
	defer nmq.Close()
	if 1 != nmq.CountThreads() {
		panic("thread not found")
	}
	if threads, err := nmq.Threads(); err != nil {
		panic(err)
	} else {
		var thread *notmuch.Thread
		for threads.Next(&thread) {
			defer thread.Close()
			r(thread.Messages())
		}
	}
	log.Printf("-- SAMPLE END")
}

it seems to try to retrieve more mails than available. the thread as 10 mails and it cores after that.

forgot to mention: astroid opens that same thread just fine so notmuch has no problems with that.

MoveToNext return bool

Hey Ian,

So I was looking at func (t *Threads) MoveToNext() and func (t *Threads) Get() and I feel that MoveToNext() should return a bool (or better return an error) instead of having to call func (t *Threads) Valid().

I have a use case for using notmuch and I feel that there go bindings are quite C-like and not idiomatic. Thank you for your time on this.

Cross Compile does not work

I developed a small email program based on your awesome library 💯. Thanks alot for your work! It was such a hassle dealing with IMAP and then Maildir until I found notmuch.

Now I want to deploy to a small server, so I have to cross compile for linux on a mac.

go build works.
GOOS=linux GOARCH=amd64 go build does not work (undefined: notmuch.DB).
commenting out the notmuch parts and running GOOS=linux GOARCH=amd64 go build again works.
CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build gives me a bunch of errors about the sqlite3-bindings.
using xgo xgo --targets=linux/amd64 . fails with the following message:

...

# github.com/zenhack/go.notmuch
/go/pkg/mod/github.com/zenhack/[email protected]/configList.go:9:22: fatal error: notmuch.h: No such file or directory
 // #include <notmuch.h>
                      ^
compilation terminated.
2020/05/28 13:54:27 Failed to cross compile package: exit status 2.

xgo --deps=https://notmuchmail.org/releases/LATEST-notmuch-0.29.3.tar.xz --targets=linux/amd64 . still fails with the above message.

To be honest, I don't know what to try now 🤷‍♂️. The last option is pulling the source code onto a Linux machine and then compiling there. But this is also annoying.

@zenhack Do you have any ideas?

.Slice() methods on iterator types.

...Which returns a slice of the sequence.

Particularly with tags we're re-implementing this repeatedly in the test-suite.

Random panics whenever the GC is invoked

While working on my project (which is BTW now accessible at Gmuch) I discovered that go.notmuch is panicking whenever the GC is invoked (either by runtime.GC() or by Go itself). This has happened when I asked Gmuch to load the latest 5000 threads:

kalbasit@cratos ~/code/src/github.com/gmuch/gmuch [master *] ± % time http POST localhost:7080/query q=tag:inbox limit:=5000

http: error: ConnectionError: ('Connection aborted.', BadStatusLine("''",))
http --print=HhBb POST localhost:7080/query q=tag:inbox limit:=5000  0.13s user 0.06s system 3% cpu 5.556 total

and the panic:

kalbasit@cratos ~/code/src/github.com/gmuch/gmuch [master *] ± % go run cmd/gmuch/main.go --db-path=/Users/kalbasit/.mail
listen=:7080 caller=main.go:72 msg=HTTP addr=:7080
listen=:7080 caller=log.go:43 method=thread id=0000000000020ab3 took=30.917227ms
listen=:7080 caller=log.go:26 method=query query=tag:inbox offset=0 limit=100 took=483.272983ms
listen=:7080 caller=log.go:43 method=thread id=0000000000020c5c took=22.619513ms
SIGABRT: abort
PC=0x7fff9589a0ae m=7
signal arrived during cgo execution

goroutine 19 [syscall, locked to thread]:
runtime.cgocall(0x4002fc0, 0xc82002af18, 0x0)
        /usr/local/Cellar/go/1.5.1/libexec/src/runtime/cgocall.go:120 +0x11b fp=0xc82002aee8 sp=0xc82002aeb8
github.com/gmuch/gmuch/vendor/github.com/zenhack/go%2enotmuch._Cfunc_notmuch_tags_destroy(0x64062d0)
        github.com/gmuch/gmuch/vendor/github.com/zenhack/go.notmuch/_obj/_cgo_gotypes.go:689 +0x31 fp=0xc82002af18 sp=0xc82002aee8
github.com/gmuch/gmuch/vendor/github.com/zenhack/go%2enotmuch.(*Message).Tags.func1(0xc82012bc20)
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/vendor/github.com/zenhack/go.notmuch/message.go:88 +0x24 fp=0xc82002af28 sp=0xc82002af18
runtime.call32(0x0, 0x44fe278, 0xc820104030, 0x1000000010)
        /usr/local/Cellar/go/1.5.1/libexec/src/runtime/asm_amd64.s:437 +0x3e fp=0xc82002af50 sp=0xc82002af28
runtime.runfinq()
        /usr/local/Cellar/go/1.5.1/libexec/src/runtime/mfinal.go:196 +0x292 fp=0xc82002afc0 sp=0xc82002af50
runtime.goexit()
        /usr/local/Cellar/go/1.5.1/libexec/src/runtime/asm_amd64.s:1696 +0x1 fp=0xc82002afc8 sp=0xc82002afc0
created by runtime.createfing
        /usr/local/Cellar/go/1.5.1/libexec/src/runtime/mfinal.go:135 +0x60

goroutine 1 [IO wait]:
net.runtime_pollWait(0x4f45e30, 0x72, 0xc8200780a0)
        /usr/local/Cellar/go/1.5.1/libexec/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8200e6920, 0x72, 0x0, 0x0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8200e6920, 0x0, 0x0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).accept(0xc8200e68c0, 0x0, 0x4f45f28, 0xc8203d4700)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/fd_unix.go:408 +0x27c
net.(*TCPListener).AcceptTCP(0xc8200900b0, 0x4054720, 0x0, 0x0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/tcpsock_posix.go:254 +0x4d
net/http.tcpKeepAliveListener.Accept(0xc8200900b0, 0x0, 0x0, 0x0, 0x0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/http/server.go:2135 +0x41
net/http.(*Server).Serve(0xc820076900, 0x4f45ef0, 0xc8200900b0, 0x0, 0x0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/http/server.go:1887 +0xb3
net/http.(*Server).ListenAndServe(0xc820076900, 0x0, 0x0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/http/server.go:1877 +0x136
net/http.ListenAndServe(0x4449470, 0x5, 0x4f44e58, 0xc82007fda0, 0x0, 0x0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/http/server.go:1967 +0x8f
main.main()
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/cmd/gmuch/main.go:73 +0xed7

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /usr/local/Cellar/go/1.5.1/libexec/src/runtime/asm_amd64.s:1696 +0x1

goroutine 32 [runnable, locked to thread]:
github.com/gmuch/gmuch/vendor/github.com/zenhack/go%2enotmuch._Cfunc_notmuch_threads_get(0x4d589f0, 0x6430990)
        github.com/gmuch/gmuch/vendor/github.com/zenhack/go.notmuch/_obj/_cgo_gotypes.go:881 +0x36
github.com/gmuch/gmuch/vendor/github.com/zenhack/go%2enotmuch.(*Threads).get(0xc8203d10e0, 0xc800000001)
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/vendor/github.com/zenhack/go.notmuch/threads.go:35 +0x28
github.com/gmuch/gmuch/vendor/github.com/zenhack/go%2enotmuch.(*Threads).Next(0xc8203d10e0, 0xc82004f648, 0x3)
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/vendor/github.com/zenhack/go.notmuch/threads.go:29 +0x47
github.com/gmuch/gmuch.(*Gmuch).Query(0xc820093ae0, 0xc8203d10c0, 0x9, 0x0, 0x1388, 0x0, 0x0, 0x0)
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/query.go:31 +0x24e
github.com/gmuch/gmuch/server.logMiddleware.Query(0x4f449c0, 0xc82007faa0, 0x4f44b00, 0xc820093ae0, 0xc8203d10c0, 0x9, 0x0, 0x1388, 0x0, 0x0, ...)
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/server/log.go:37 +0x155
github.com/gmuch/gmuch/server.(*logMiddleware).Query(0xc820093b20, 0xc8203d10c0, 0x9, 0x0, 0x1388, 0x4f44ad0, 0x0, 0x0)
        <autogenerated>:5 +0xd3
github.com/gmuch/gmuch/server.instrumentingMiddleware.Query(0x4f44a40, 0xc820079e30, 0x4f44ad0, 0xc820093ac0, 0x4f44b30, 0xc820093b20, 0xc8203d10c0, 0x9, 0x0, 0x1388, ...)
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/server/instrumenting.go:35 +0x151
github.com/gmuch/gmuch/server.(*instrumentingMiddleware).Query(0xc82007fd40, 0xc8203d10c0, 0x9, 0x0, 0x1388, 0xc8203d4780, 0x0, 0x0)
        <autogenerated>:3 +0xd9
github.com/gmuch/gmuch.EndpointenizeQuery.func1(0x4f46160, 0xc8203913c0, 0x43df420, 0xc8203d47a0, 0x0, 0x0, 0x0, 0x0)
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/endpoint.go:13 +0x116
github.com/gmuch/gmuch/vendor/github.com/go-kit/kit/transport/http.Server.ServeHTTP(0x4f44b90, 0xc820078bd0, 0xc820093b40, 0x44fe210, 0x44fe220, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /Users/kalbasit/code/src/github.com/gmuch/gmuch/vendor/github.com/go-kit/kit/transport/http/server.go:93 +0x418
github.com/gmuch/gmuch/vendor/github.com/go-kit/kit/transport/http.(*Server).ServeHTTP(0xc8200e6380, 0x4f460f8, 0xc8200c7550, 0xc8203d62a0)
        <autogenerated>:2 +0xd1
net/http.(*ServeMux).ServeHTTP(0xc82007fda0, 0x4f460f8, 0xc8200c7550, 0xc8203d62a0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/http/server.go:1699 +0x17d
net/http.serverHandler.ServeHTTP(0xc820076900, 0x4f460f8, 0xc8200c7550, 0xc8203d62a0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/http/server.go:1862 +0x19e
net/http.(*conn).serve(0xc8200c74a0)
        /usr/local/Cellar/go/1.5.1/libexec/src/net/http/server.go:1361 +0xbee
created by net/http.(*Server).Serve
        /usr/local/Cellar/go/1.5.1/libexec/src/net/http/server.go:1910 +0x3f6

rax    0x0
rbx    0x6
rcx    0x70000030fd58
rdx    0x0
rdi    0x1e03
rsi    0x6
rbp    0x70000030fd80
rsp    0x70000030fd58
r8     0xc820068480
r9     0xc82002aeb0
r10    0x8000000
r11    0x206
r12    0x64062d0
r13    0x61660089df10
r14    0x700000310000
r15    0x0
rip    0x7fff9589a0ae
rflags 0x206
cs     0x7
fs     0x0
gs     0x0
exit status 2

I made a reproducible test by randomly invoking the GC in the tests. See kalbasit/go.notmuch@491e3a3

Probably unmaintained

The maintainer of this repo sadly passed away, chances are it won't be maintained in the future.

sources, with no guarantees obviously:

There's a deadlock somewhere.

https://travis-ci.org/zenhack/go.notmuch/builds/85976525

The tests passed fine on my desktop, and the branch I pushed to before merging to master. I can't repeat the failure on my desktop, but I seem to be able to do so on one of my other machines. I've yet to dig into this.

Adding duplicate messages in AddMessage() is not handled properly

In db.go, method AddMessage, the check for errors does not handle duplicate messages the same way that the notmuch-library does.

A quick look in lib/add-message.cc in the notmuch source-code
https://git.notmuchmail.org/git?p=notmuch;a=blob;f=lib/add-message.cc;h=485debad225239211e9a6f165f24be519080a0ed;hb=HEAD#l574 shows that the message information is set on success or if a duplicate message has been found.

The relevant section being:

    if (message) {
	if ((ret == NOTMUCH_STATUS_SUCCESS ||
	     ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
	    *message_ret = message;
	else
	    notmuch_message_destroy (message);
    }

We therefore have to update the AddMessage-function to both make sure that the gc cleans our message up, and also make it available to the caller.

Document how to run the tests

The testsuit as is fails if one clones the repo as is.
It can't even open up the database at first: Error opening database at /home/reto/sourcecode/go.notmuch/fixtures/database-v1/.notmuch: No such file or directory

Now, even if one does create that database, it still fails as all the emails it expects are not present, yielding errors like

--- FAIL: TestFindMessage (0.00s)
    db_test.go:158: db.FindMessage("[email protected]"): unexpected error: not found

Could you please add the corresponding messages you expect in the test to the repo?

zenhack / go.notmuch Goto Github PK

go.notmuch's People

Stargazers

Watchers

Forkers

go.notmuch's Issues

Rewrite notmuchmail in Go

Deprecated notmuch database open function

core on .Replies

MoveToNext return bool

Cross Compile does not work

.Slice() methods on iterator types.

Random panics whenever the GC is invoked

Probably unmaintained

There's a deadlock somewhere.

Adding duplicate messages in AddMessage() is not handled properly

Document how to run the tests

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent