archive all comments about ltc HOT 6 CLOSED

So I'm working on this but the query I'm using to find missing comments is slow as HELL (think a hour at most slow)

https://github.com/boehs/ltc/blob/master/analysis/missingcomments.pgsql

Need to find something better, I assume the time researching a faster query will actually be shorter than resuming

from ltc.

streaming makes this like a tiny issue

from ltc.

• theroretical number of comments: 319458
• actual comments: 319672
• missing comments: -214
• negative comments (might reduce sum) -420
• which means 206 are unaccounted for

from ltc.

forgot disqus

from ltc.

Disqus has like an interesting API

forums/listPosts

Possible fields of interest

• message: html message
• likes/dislikes
  • points (likes-dislikes?)
• author
  • name
• thread
  • link
• createdAt
• thread (another request?)

And less interesting fields

• author
  • url
  • location
  • username

Notes

• Not sure the best way to identify where the comment was made, I think I need to make another request to get the thread which I really don't want to do but
  • actually you can use related: thread which is awesome
• limit=100 can save 5x requests (default is 20)

from ltc.

we've archived 98055 disqus comments. Probably ~3% was missed due to extensive filtering

from ltc.