Congressional Record Data about congress HOT 6 CLOSED

Hi. My (GovTrack) CR scraper hasn't worked properly in a long time.

@konklone can probably point you to Sunlight's Capitol Words parser.

from congress.

Last I knew, @drinks was working on detethering the Capitol Words parser from Solr and making it work as a standalone thing so we could offer bulk data, but I'm not sure how far he's gotten. You might try opening a ticket there, or perhaps my mentioning his name will summon him like a ghost.

from congress.

But of course. That work is long finished, at least to the degree that I'd intended for Capitol Words' needs, though the data hasn't been re-parsed and uploaded yet. The endgame artifacts will be raw text, some flavor of XML (lacking bioguide resolution), and finally json that's ready to be ingested by elasticsearch. It may make sense, though, to provide a more space-thrifty version of the final output documents in json or yaml; ours will have a bunch of nlp cruft. What data points would you be interested in?

from congress.

Thank you for your responses.
What I like about GovTrack data is that each speaking section has the 'speaker' attribute (e.g., here), and there is a separate people.xml file with a lot of attributes, such as party, role, state, district, etc., which help uniquely identify legislators. That is, one can construct a mapping between speakers and their speeches. In this respect GovTrack is great. However, not all documents from the THOMAS system have been parsed.
Does this kind of parsing require a lot of manual coding?

from congress.

GovTrack's people.xml has been transformed and expanded into this project:
https://github.com/unitedstates/congress-legislators

Also, take a look at Capitol Words' API and see if it can't do what you need. It provides speaker IDs (bioguide IDs) which you can link to the congress-legislators project I linked to above.

from congress.

OK, thanks for the discussion.

from congress.