Pyserini package/module structure: unnecessary nesting? about pyserini HOT 6 CLOSED

I think the submodule was originally there to mirror the java structure:

• pyserini.search corresponds to java/io/anserini/search and would contain every module that we want to bridge from there.
• pysearch.py was intended to mirror SimpleSearcher.java, which is the only module we have bridged to pyserini for now

If we don’t plan to extend anything else over from java/io/anserini/search in the near future, we can probably get rid of the submodule? Otherwise I'd second @zeynepakkalyoncu’s suggestion to perhaps rename pysearch.py to something like simple_searcher.py.

from pyserini.

I actually think the current setup is fine. As long as we want to keep a main "source" folder (pyserini) and individual modules (collection, index, etc.), which we should, the "extra" nested layer is necessary in Python. Would it help to rename pysearch.py to directly reflect the class it contains (simple_searcher.py instead of pysearch.py?

from pyserini.

I don't know Python very well, so please correct me - but here's my understanding: we have pyserini/search/pysearch.py:

• pyserini is the package
• pyserini.search being the sub-package.
• pysearch is the name of the module
• SimpleSearcher is the name of a class in the module.

I guess what I'm suggesting is that we don't need the sub-package? So

• pyserini is the package
• search is the name of the module
• SimpleSearcher is the name of a class in the module.

I think this means renaming pyserini/search/pysearch.py to pyserini/search.py and that's it?

from pyserini.

@emmileaf

If we don’t plan to extend anything else over from java/io/anserini/search in the near future, we can probably get rid of the submodule?

Do you actually mean sub-package instead of sub-module? Or do I have the Python terminology wrong?

I think the main difference here is that in Java, every class needs to be in it's own file, whereas in Python, a file can have multiple classes... so I think this means Python can have one level shallower nesting? We can have pyserini/search.py and for other classes in Anserini we want to bridge, just throw in search.py also?

rename pysearch.py to something like simple_searcher.py

I don't think this would work, because pyserini/collection/pycollection.py has multiple classes in it. So under my proposal, pyserini/collection/pycollection.py would be renamed pyserini/collection.py (with all the same classes inside).

from pyserini.

Do you actually mean sub-package instead of sub-module? Or do I have the Python terminology wrong?

My bad, you are correct! I meant to say the sub-package 🤦‍♀

I think the main difference here is that in Java, every class needs to be in it's own file, whereas in Python, a file can have multiple classes... so I think this means Python can have one level shallower nesting? We can have pyserini/search.py and for other classes in Anserini we want to bridge, just throw in search.py also?

Sure, that works also! When I first set this up I was unsure of whether we'd want to bridge, say, the search/topicreader classes into pyserini/search/topicreader.py one day and keep it under the search sub-package, but with the few classes pyserini has for now it might look a bit excessive.

However, if we collapse pyserini.search into search.py, does that mean also collapsing other sub-packages as well, likepyserini.collection into collection.py and so on? Perhaps the sub-packages can just help the repo look cleaner, in case the modules eventually grow into longer files?

from pyserini.

@emmileaf re: future proofing when modules grow too big. Agreed.
You've convinced me - let's keep the sub-packages. You were right all along! :)

Closing issue.

from pyserini.