jjfiv / fastrank Goto Github PK

Write a wrapper for the library with argparse and friends that matches Ranklib's command-line behavior for these two models (i.e., -ranker 4 is CA, -ranker 8 is RF, all others error.)

Hi everyone,

I've a question regarding running Coordinate Ascent. I noticed from the colab file https://colab.research.google.com/drive/1IjF7yTin1XaNO_6mBNxAoQYTmF0nckk1 that the model is trained using NDCG. Is it possible to train the model using a user-defined metric? Or how can we make the model optimize for different metrics, e.g. MAP or ERR?

Thanks!

Replace evaluate_mean with evaluate_bootstrap (to-write) in all places it is computationally feasible.

John, I'm not clear on how to make a dataset. E.g. I have a dataframe with 3 columns: "qid", "features", "label" with types string, np.array and int, how do I make it into a dataset?

The token username password is not working for me right now :(

#3 will help.

#1

Requires: #2 #3
Enables: #1

We were using cffi in order to build fewer wheels, but with the new CI setup introduced in #54 being relatively successful, we can try out PyO3 again; as the need to rebuild for every python version exists in the build matrix already.

This should be doable as a patch release.

I think there's a segfault happening in github-actions for windows, but I don't have a windows machine right now.

So that we have some sense it hasn't crashed without the gigantic table output...

Which will allow #2 and eventually #1.

Raised in #46.

Depends on #3.

All randomness I'm aware of derives from "seed" but for some reason the same seed trains differently. Needs more investigation.

See macOS-latest run for #31

/usr/local/lib/python3.7/dist-packages/fastrank/fastrank/init.py in ()
4 from .ffi import ffi
5
----> 6 lib = ffi.dlopen(os.path.join(os.path.dirname(file), 'native.so'), 4098)
7 del os

OSError: cannot load library '/usr/local/lib/python3.7/dist-packages/fastrank/fastrank/native.so': /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /usr/local/lib/python3.7/dist-packages/fastrank/fastrank/native.so)

Requires: #2 #3 #1

So we don't need to expose mutable datasets to Python, although I suppose it could return a "new" dataset... either way needs thinking.

Depends on the ability to add random features, maybe: #2

Related to #1

• Support CModel
• Support loading of qrels in python
• Support qrels->json, json->qrels
• Support Evaluate(cdataset, cmodel, evaluator_name: str) -> Dict[qid, score]
• Support feature removal/sampling for dataset

Want to take arbitrary y_pred along with qids, y_true and an optional qrel to have reliable (pessimistic) mAP, ndcg, and mrr.

DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.

https://dl.acm.org/citation.cfm?id=3130346

pip install fastrank

Fails on M1 macs. I have a built wheel that you could upload to pypi, if you like.

Maybe connected to #2. Definitely requires #3.

Be smarter instead of burning a column.

Related to #1 deferred for #15