Task description

This task involves reducing number of data points to plot on Message Embedding visualizer. As our current dataset is supposed to have more than 1 M data points which possibly can't fit in the given space.

We can use different features given in the dataset to come up with strategy to create filters to reduce the datapoints.
For example, you could filter models like GPT3.5, Vicuna, etc. or filter by language like English.

Current datasets to explore

https://huggingface.co/datasets/lmsys/lmsys-chat-1m

TO DO

• Filtering and saving relevant data files.
• Cleaning downloaded CSVs, by converting conversation rows to message rows
• Create embeddings and store in CSV for each message
• Save 3D/2D representation of embedding vector in the CSV for later use.