__in filter combined with punctuation drops valid results about xapian-haystack HOT 3 CLOSED

My suspicion here is it's the combination of xapian.Query.OP_PHRASE in the SearchQuery._phrase_query method and a term with punctuation as there is already tests demonstrating the use of __in with phrases (http://github.com/notanumber/xapian-haystack/blob/master/tests/xapian_tests/tests/xapian_query.py#L126) and punctation in terms (http://github.com/notanumber/xapian-haystack/blob/master/tests/xapian_tests/tests/xapian_backend.py#L212).

Will need to setup a new test-case that combines both and go from there.

from xapian-haystack.

As a temporary fix, for anyone that encounters this issue, please replace punctuation in the term list passed to filter(foo__in).

from xapian-haystack.

IMO this occurs because we are using Xapian.text_generator to build the index, and it "eats" some punctuation to improve searching. This means that any query of the form __exact (which __in uses) can return incorrect results.

One way I'm seeing to fix this is to also store the exact terms (with position information).

from xapian-haystack.