uniir's Introduction

UniIR

This repo is under construction. Please stay tuned.

🌐 Homepage | 🤗 Dataset | 📖 arXiv | GitHub

This repo contains the codebase for the paper "UniIR: Training and Benchmarking Universal Multimodal Information Retrievers"

UniIR Models

Universal multimodal Information Retrieval (UniIR)

Training

Environment Setup

Scripts

Evaluation

Environment Setup

Scripts

Model Zoo

M-BEIR

Multimodal BEnchmark for Instructed Retrieval (M-BEIR)

M-BEIR Dataset Structure

mbeir_data
├── mbeir_train.jsonl
├── mbeir_val.jsonl
├── mbeir_test.jsonl
├── mbeir_instructions.tsv
├── mbeir_images
│   ├── oven_images
│   ├── infoseek_images
│   ├── edis_images
│   ├── webqa_images
│   ├── mscoco_images
...

UniIR Instructions

M-BEIR format

M-BEIR Statistics

M-BEIR Downloading

Contact

Cong Wei: [email protected]
Yang Chen: [email protected]
Alan Ritter: [email protected]
Wenhu Chen: [email protected]

Citation

BibTeX:

@misc{wei2023uniir,
      title={UniIR: Training and Benchmarking Universal Multimodal Information Retrievers},
      author={Cong Wei and Yang Chen and Haonan Chen and Hexiang Hu and Ge Zhang and Jie Fu and Alan Ritter and Wenhu Chen},
      year={2023},
      eprint={2311.17136},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Recommend Projects