Git Product home page Git Product logo

cat-bus-api's Introduction

Cat Bus API ๐ŸšŒ

Cat Bus

Simple API wrapper for Austin's Capmetro GTFS data.

Stack

  1. Python 3
  2. Docker handles standing up the MySQL database and Flask app
  3. Flask server handles API requests
  4. Swagger defines API and provides a UI for exploring the API
  5. PeeWee is a lightweight Python ORM I used to make it easier to model and query data
  6. Pandas is a package for Python I like using for manipulating CSV data

Set up

This API requires Docker to run locally.

  1. docker-compose build && docker-compose up to build and bring up the project.
  2. From the root directory, pip install -r requirements.txt. This pretty much only installs Pandas and PeeWee, which is needed to backfill data in the MySQL database.
  3. Run the backfill: python backfill.py. Note: Backfilling stops, routes, and trips is pretty fast, but stop times will take a while. Drink some tea. ๐Ÿต

Run the server

The API server is a Flask app that uses the Swagger spec. Using docker-compose up should bring up both the database and the Flask app.

  1. The Flask app and API should be running on localhost:5000
  2. You should be able to curl the API on http://localhost:5000/api or use an app like POSTMAN, or the Swagger UI link below.

Swagger

Swagger UI

Go to http://localhost:5000/api/ui/ on your browser to check out the spec and make test calls!

Challenges / To Do's

I was using MySQL version 8, but I had some compatibility issues being able to connect Flask to the MySQL 8 version of the DB due to how MySQL hashes passwords now. Articles I found online suggested downgrading to MySQL 5.

To Do's:

  • Find workaround for MySQL 8 pw bug
  • Set env var for db pw on container

Database design and queries

Design

I tried to mimick the GTFS txt files as much as possible in my database, and I made some decisions I probably wouldn't have made with what I know now, but I've gone too far :)

"gone too far gif"

For example, when I wrote my SQL schema, I decided to name route IDs as route_id everywhere, even in the routes table. This means that now I have routes.route_id instead of routes.id. I made that decision to remain consistent with the nomenclature of the txt files, but in retrospect, I would change that in the future to the more standard SQL practice of table.id.

Queries

I thought it would be good to document the ORM + SQL queries for comparison purposes. Doing the join was much easier to do in MySQL than in PeeWee, so I hope this helps anyone looking for examples.

List routes

Routes.select().limit(LIMIT).offset(OFFSET)

MYSQL:

select * from routes limit X OFFSET X

Get a route

Routes.select().where(Routes.route_id == ID)

MYSQL:

select * from routes where route_id = ID

List all trips for a route

Trips.select().where(Trips.route_id == ID)

MYSQL:

select * from trips where route_id = ID

List stops by trip ID

This was tricker to do in the ORM than in pure SQL!

(Stop_Times
        .select(Stop_Times, Stops, Trips)
        .join(Stops, on=(Stop_Times.stop_id == Stops.stop_id).alias('stop'))
        .switch(Stop_Times)
        .join(Trips, on=(Stop_Times.trip_id == Trips.trip_id).alias('trip'))
        .where(Stop_Times.trip_id == trip_id, Trips.route_id == route_id).limit(limit).offset(offset))

MYSQL:

select arrival_time, departure_time, stops.stop_id, stops.stop_name, stop_times.trip_id, trips.trip_headsign 
from stop_times 
inner join stops on stop_times.stop_id = stops.stop_id 
inner join trips on stop_times.trip_id = trips.trip_id  
where stop_times.trip_id = ID;

cat-bus-api's People

Watchers

 avatar

Forkers

r-woosley

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.