Collection of various names datasets.

There are currently 15 sources which produces:

There is no guarantee of the accuracy of the data, ie the associated gender & origin, as it will vary (significantly) source to source.

To use the data you can read it straight from github into pandas:

import pandas as pd

names_df = pd.read_csv("https://raw.githubusercontent.com/diabolical-ninja/AllTheNames/main/names.csv", sep = "|")

Found in names.csv

This project uses poetry for dependency management. This assumes python 3.9+. It may well work for lower versions but has not been tested.

First setup the environment by running:

poetry install

You can then generate a dataset of interest from the command line via:

poetry run python src/data_collection/<generation_script_run>.py

To build all sources, run:

poetry run python all_the_names.py

You should see a file names.csv appear in the root directory.

This process will depend on your network speed but on my 2021 MBP & 50Mbps internet connection it takes roughly 2min & 10sec

Contributions are absolutely welcomed and encouraged. To do so, raise a PR with the new datasource and its integration into the names dataset and schema.

Ideas:

• Some datasets have a "frequency" measure of sorts. It would be cool to include that & have a mechanism to standardise/normalise the value across all datasets
• Standardise origin so that difference sources better match. Eg USA, United States of America, US of A, etc are all consistently presented as the same location

If you encounter any problems, please file an issue along with a detailed description.