lintool / cs-big-cows Goto Github PK

It would be nice to add mappings to:

• Google scholar profiles
• Semantic scholar profiles
• DBLP profiles
• etc.

For Google Scholar profiles, here's a good start: https://lintool.github.io/bigcows/index-stratosphere.html

Note that this only needs to be done once... I bet we might even be able to coax ChatGPT into automating it...

https://github.com/lintool/art-science-empirical-cs-2023f/blob/main/project-vis.md
Visualizing ACM fellows using different embeddings + context for exploratory data analysis

First step to build a RAG system from ACM fellows and turings data for Question and Answering using LLM model.
We want a backend system for applying embedding and do vector search (for information retrieval) across these publications data and try to analyze the retrieval score.

Basically part 2 and 3 of this schema

The latest scraper PR has implementation to gather publications data for certain author in ACM Fellows and Turings
#14
references_search under paper_search.py has basic implementation for getting references/citations using sequential batch requests. So, the task is to improve and test this functionality to be more accurate, fast and consistent + testing with some outputs.

Current script dblp_scraper.py only support for one author input per execution. We want to refactor this so it can automate the scraper for entire authors in acm_fellows.csv and acm_turings.csv.

Things need to be considered:

• Fault tolerance considering we called the API thousand times (~5 api calls/author), need checkpoint implementation in case we want to run the script several times instead in 1 go
• Rate Limiting on SemanticScholar, current rate limit is https://github.com/allenai/s2-folks/blob/main/API_RELEASE_NOTES.md