Using Tongrams about tongrams HOT 6 CLOSED

It is not difficult, but currently the library does not support it yet.
If you don't care about the efficiency of the operation, one option is to proceed by scan: retrieve all children of a given prefix and just select the top-k.
For most queries, it will probably be efficiency too because of the few children to be examined.
A more complicated solution could use additional RMQ data structures.

from tongrams.

Hi.
The data structures storing the weights as counts support the lookup operation that, given a ngram, returns its associated weight.
The data structures storing the weights as prob/backoff support, instead, the score operation.
See details in the README (Benchmark section) and in the relevant paper.
Currently, top-k queries are not supported and there is no python binding.

from tongrams.

Thanks @jermp, how difficult would it be to modify things to support some top-k?
The k doesn't have to be unlimited. For example if I have an ngram-prefix, finding the best 5 or 10 words right after the prefix would be highly useful.

from tongrams.

Thanks @jermp, I see the lookup() operation in trie_count_lm.h, but what's the right way to scan all children of a given prefix?

from tongrams.

Currently, there is no method that allows you to do this but I can implement that.

from tongrams.

That would be fantastic.

from tongrams.