Git Product home page Git Product logo

usaspending-api's Introduction

USAspending API

Build Status Test Coverage Code Climate

This API is utilized by USAspending.gov to obtain all federal spending data which is open source and provided to the public as part of the DATA Act.

USAspending Landing Page

Creating a Development Environment

Ensure the following dependencies are installed and working prior to continuing:

Requirements

  • PostgreSQL 10.x (with a dedicated data_store_api database)
  • direnv
    • For Mac OSX, be sure to put the hook in your ~/.bash_profile, not ~/.bashrc
  • Bash or another Unix Shell equivalent
  • Command line package manager
    • Windows' WSL bash uses apt-get
    • OSX users will use Homebrew

Cloning the Repository

Now, navigate to the base file directory where you will store the USAspending repositories

$ mkdir -p usaspending && cd usaspending
$ git clone https://github.com/fedspendingtransparency/usaspending-api.git
$ cd usaspending-api

Database Setup

There are two options for how you want to setup your database in order to run the API. You can:
1. Use your own local postgres database for the API to use.
2. Create an empty directory on your localhost where all the database files will persist and use the docker-compose file to bring up a containerized postgres database

Option 1: Using a Locally Hosted Postgres Database

Create a Local postgres database called 'data_store_api' and either create a new username and password for the database or use all the defaults. For help, consult:

Make sure to grant whatever user you created for the data_store api database superuser permissions or some scripts will not work:

postgres=# ALTER ROLE <<role/user you created>> WITH SUPERUSER;

Option 2: Using the Docker Compose Postgres Database

See below for basic setup instructions. For help with Docker Compose:

Database Setup and Initialization w/Docker Compose

  • None of these commands will rebuild a Docker image! Use --build if you make changes to the code or want to rebuild the image before running the up steps.

  • If you run a local database, set POSTGRES_HOST in .env to host.docker.internal. POSTGRES_PORT should be changed if it isn't 5432.

    • docker-compose up usaspending-db will create and run a Postgres database in the POSTGRES_CLUSTER_DIR specified in the .env configuration file. We recommend using a folder outside of the usaspending-api project directory so it does not get copied to other containers in subsequent steps.

    • docker-compose up usaspending-db-migrate will run Django migrations: https://docs.djangoproject.com/en/2.2/topics/migrations/.

    • docker-compose up usaspending-ref-data will load essential reference data (agencies, program activity codes, CFDA program data, country codes, and others).

    • docker-compose up usaspending-db-sql, then docker-compose up usaspending-db-init will provision the custom materialized views which are required by certain API endpoints.

Manual Database Setup

  • docker-compose.yaml contains the shell commands necessary to set up the database manually, if you prefer to have a more custom environment.

Running the API

docker-compose up usaspending-api

  • You can update environment variables in settings.py (buckets, elasticsearch, local paths) and they will be mounted and used when you run this.

The application will now be available at http://localhost:8000.

Using the API

In your local development environment, available API endpoints may be found at http://localhost:8000/docs/endpoints

Deployed production API endpoints and docs are found by following links here: https://api.usaspending.gov

Loading Data

Note: it is possible to run ad-hoc commands out of a Docker container once you get the hang of it, see the comments in the Dockerfile.

For details on loading reference data, DATA Act Broker submissions, and current USAspending data into the API, see loading_data.md.

For details on how our data loaders modify incoming data, see data_changes.md.

Public Domain License

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest. got

usaspending-api's People

Contributors

afrasier avatar alburde1 avatar alexjajabah avatar bray-michael-bah avatar brianzito avatar bsweger avatar catherinedevlin avatar dpb-bah avatar dylanbah avatar edshields avatar emily-brents avatar isaacray avatar jdodd2 avatar justinle avatar kaitlin avatar kbard avatar kco-kjackson avatar klrbah avatar kwhickey avatar maxwellkendall avatar melvindunn avatar miahunsicker avatar michaeldhess avatar nmonga91 avatar phroiland avatar playmakervi avatar russellbodine avatar shinson avatar tony-sappe avatar willkjackson avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.