romanmaksy / etterna-ratings-distribution Goto Github PK

user profile pic url is available from the existing leaderboards/leaderboards request, so to make this happen just need to extract that in similar way we extract username/flag data in python script to add to csv, and then render this on frontend when a user is highlighted.

Currently GitHub actions runs python script to update CSV and commits the changes - this bloats the repo size and pollutes commit history so would be much better to have the script push the files to something like an s3 bucket and then have the frontend fetch the latest file from that.

The following run failed, see here for logs
https://github.com/romanmaksy/etterna-ratings-distribution/runs/3911539825?check_suite_focus=true

Suspect that this is due to status code check being AFTER trying to load json from res.text, hopefully easy fix. May want to add more robust error handling here in general, since currently the code is extremely optimistic.

extend csv generation script, ensuring the script completes fast enough to not time-out github actions, and rate limited enough to be nice to etterna servers. Will involve following steps:

1. set up rate limiting framework
2. fetch userid for each user in existing csv we don't have an id for
3. for each userid, check the last updated date we have on file, if it's within some threshold, fetch new last score submission date from score/userScores, and store this and fetch date in csv

note that for the above, will probably want to do one big scrape overnight or something to get inital data, and then be smart about not fetching stuff we already have when running on github actions.

once we have this data in csv, add date range UI to allow user to filter by how recently the user has submitted a score.

site dies when min higher than max - should improve handling for when we filter to zero results, and also subscribe to min/max change events separately to clamp the opposing control in range

we now have country data in the generated csv, need to add a user control + filter for this on frontend. might be cool to show flag icon next to user name when highlighting too if possible.