Category Builder

This repository contains data and code for the Category Builder system.

Category Builder can do set expansion while dealing robustly with polysemy. See category_builder_paper.pdf in this directory.

Installation

Download code and data using git. You will need to have installed git lfs.

git clone https://github.com/google/categorybuilder
cd categorybuilder
git lfs pull

Initialization

Note: This will take a few minutes to initalize. An sqlite database of about 5.7 GB is produced.

Note: The initialization uses alive_progress to show progress. You can get it by 'pip3 install alive_progress'.

python initialize.py

How to use Category Builder

python category_builder.py ford nixon
python category_builder.py --rho=2 --n=20 ford chevy

The seeds to expand are provided on the command line as positional arguments and should be lowercase. Compound names (e.g., "New York") should be quoted.

python category_builder.py chicago "new york"

Example Output

How to do analogies

The same system can solve analogies such as "What is the mount everest of africa?"

python analogy.py "mount everest" africa

Note that these are harder than "proportional" analogies such as "hand:glove::foot:?". People don't need to be provided the first term ("hand") and can answer "What is the glove for a foot?"

Example Output

The items are labeled B and C because analogies are often shown as A:B::C:D.

How to run the evaluation suite

python3 eval_set_expansion.py eval_data/cat_eval_data/nfl-teams

python3 eval_analogy.py eval_data/analogy_eval_data/questions-words.txt