jjfiv / irene Goto Github PK

So that we can have a python/whatever API

Terrier is another powerful open-source search engine.
Once the multi-module version of this repo lands (#8) we could start on an irene-terrier backend. I think this should be fairly straightforward, given we can probably implement Match operators and a Models for our black-box scoring trees.

https://github.com/terrier-org/terrier-core/tree/5.x/modules/core/src/main/java/org/terrier/matching/matchops

Creating an issue so I can group together commits that work on this task.

Creating an issue so I can group together commits that work on this task.

We can't support Galago's hacky #rm traversal (#4), because I want the QExpr language to actually be independently useful from any retrieval backend implementation.

Want people like @daltonj who have existing libraries to generate string queries to be able to ingest them.

Conflicts a little with #3, but ultimately this belongs in the irene-galago subproject.

Causing work to not build in #4.

With #4 we ought to be able to support a front-end like that of Galago, which will render detailed queries and present results, snippets, etc.

• results
• Jetty versions with Galago
• parsing of Galago queries #4
• How to render Lucene's more free-form documents?
• snippets

Frequency of import/typealias in src/main.

  18 import org.lemurproject.galago.utility.Parameters
   4 import org.lemurproject.galago.utility.StreamCreator
   4 import org.lemurproject.galago.core.eval.QueryResults
   3 import org.lemurproject.galago.core.eval.SimpleEvalDoc
   2 import org.lemurproject.galago.utility.MathUtils
   2 import org.lemurproject.galago.core.retrieval.LocalRetrieval
   2 import org.lemurproject.galago.core.index.stats.FieldStatistics
   2 import org.lemurproject.galago.core.eval.metric.QueryEvaluatorFactory
   2 import org.lemurproject.galago.core.eval.QueryJudgments
   1 typealias GResults = org.lemurproject.galago.core.retrieval.Results
   1 typealias GExpr = org.lemurproject.galago.core.retrieval.query.Node
   1 typealias GDocArgs = org.lemurproject.galago.core.parse.Document.DocumentComponents
   1 typealias GDoc = org.lemurproject.galago.core.parse.Document
   1 import org.lemurproject.galago.utility.lists.Ranked
   1 import org.lemurproject.galago.utility.StringPooler
   1 import org.lemurproject.galago.core.util.WordLists
   1 import org.lemurproject.galago.core.retrieval.query.NodeParameters
   1 import org.lemurproject.galago.core.retrieval.query.AnnotatedNode
   1 import org.lemurproject.galago.core.retrieval.processing.ScoringContext
   1 import org.lemurproject.galago.core.retrieval.iterator.TransformIterator
   1 import org.lemurproject.galago.core.retrieval.iterator.ScoreIterator
   1 import org.lemurproject.galago.core.retrieval.iterator.CountIterator
   1 import org.lemurproject.galago.core.retrieval.Retrieval
   1 import org.lemurproject.galago.core.parse.TagTokenizer
   1 import org.lemurproject.galago.core.index.stats.NodeStatistics
   1 import org.lemurproject.galago.core.index.corpus.CorpusReader
   1 import org.lemurproject.galago.core.eval.metric.QueryEvaluator
   1 import org.lemurproject.galago.core.eval.QuerySetJudgments

• Crashes when a segment does not have any results -- this returns a null Scorer, I think.

This blocks everything when it is done.

I'm envisioning the following sub-projects.

• irene-lang No dependencies, just our CountStats and QExpr classes and our query compilation passes.
• irene-ltr depends irene-lang, our LTRDoc and LTRDocField and re-rank context scoring. Regressions against Galago can be done in test scope.
• irene-galago depends on both galago-core and irene-lang. Galago's Nodes <-> our QExpr will be supported here.
• irene-lucene depends on both irene-lang and lucene.
• irene-full depends on all projects, has graphical front-end, etc.

Once we split out the language to different models (#8), the language will be pure Kotlin and compilable with both Kotlin-js and Kotlin-native.

This means (potentially) we can run re-ranking queries in the browser for the user and pursue an efficient Indri (et al?) backend that shares code without paying the JNI price to use indri-java.