sfirke / annarborvotes Goto Github PK

After scraping about 3,000-5,000 name combinations, RStudio has about 3.5GB of memory tied up, despite the reported usage being only 100MB (see http://stackoverflow.com/q/31385964/4470365). I thought this was due to growing a data.frame row-by-row but even when pre-allocating the size, the problem persists.

It would be much better if users could scrape say 15,000 names at a time rather than doing blocks of 3k and restarting RStudio after each one. But I don't know what is causing this.

The most recent timestamp of a tweet can be a retweet, just saw someone with last tweet = 2006 because in October 2015 they retweeted that 2006 tweet. One quick fix would be to grab the max of say the last 5 tweet timestamps.

This should be its own function - right now there's a skeleton for it but it's just empty space

Would be a fun and informative statistic to gather.

both have a blank value for "time_last_tweet" returned by the scraping function though some have thousands of tweets. Prioritize these restricted users as if their last tweet was 12 months ago, not as if they have never tweeted.

If on August 3rd 2015 a user's last tweet was August 21 2014, Twitter will display that as "August 21" not August 21 2014 the way it would for July 21 2014. Running this function in on August 3rd 2015 would put this August 21 tweet at the top of the months list, before July, when it should go last.

Low priority.