heylobby_me's Introduction

HeyLobby.ME

The Maine Commission on Gov. Ethics and Election Practices has done an abysmal job putting lobbyist information on the internet. Let's change that, eh?

Next steps

Scrape the current form for legislative documents in the various legislative sessions and get them into our own database
Scrape lobbyist and client data from the same form
Associate the lobbyist with the legislation

heylobby_me's People

Contributors

Watchers

heylobby_me's Issues

Create a database schema for lobbyist data

This is harder than it might seem, as you first have to figure out what folks are going to be looking for. Namely you have:

Lobbyist organizations
Individual lobbyists
Legislation lobbied for
Lobbyist reports
Lobbyist declarations of no lobbying (registered lobbyist must attest to not having lobbied)

Setting this all up reasonably and modeling it will be tricky.

Find a way to scrape lobbyist, client and legislation data into the database

The URL to post against is here:

https://secure.mainecampaignfinance.com/PublicSite/SearchPages/LobbyingInfoSearch.aspx

Effectively what one needs to do is populate a list of legislation from the tag in the page and then just repeatedly hit the form grabbing the POST response and loading it in our own database.

Recommend Projects