The bestnest-api from codeforamerica

Action Analytics

for any behavior we want users to do, eg leave a review, visit detail pages, etc, we should log these actions.

use google analytics
also track view/routing changes as navigation

ensure indexes for landlord view

instrument response times

maybe add new relic?

look at slow db queries report

generic survey storage mechanism

be able to capture structured user data about a home
design read/write API
persistence mechanism
add as internal service
expose POST http endpoint
store all user responses
last write wins resolution

Ensure that CORS is enabled on every request

We're going to use GH pages in the beginning to host our front end client so CORS needs to be enabled on all requests

Load code violation data with the correct dates

Right now the data we recieved from IT doesn't have correct dates. Therefore, all the violations show up with incorrect dates inside of the application.

Let's get the correct data and load it in

ensure cors preflight is enabled on all routes

seeing occasional cors errors from bestnest-app (most recently on violations route)

Ensure that code violation data has valid dates

Right now, no dates are being returned by the API for code violation data. For the Summit demo, we absolutely need to have dates in there to show when the violations happened.

Include utility cost data from HES API in a /utilities endpoint

Implement Engine Light

Get Erica's Engine Light implemented in the API

Add schools data service to API

API needs to return state and city of the homes

Right now, there is an underlying assumption that all homes we're talking about are in Chatt, TN when that might not be the case. Let's make sure to return state and city within the API response so that the client can be more generic and we can ensure that street view images are more accurate

set up prod environment

prod heroku
prod db
db mirror script prod -> dev

Add API route for getting all properties from a landlord

So we want to have a view that shows all properties from a given landlord, so let's first have an endpoint that given a landlord ID of some kind returns all the homes owned by that landlord

Part of this Trello card: https://trello.com/c/55N6xg5w/331-other-homes-from-same-landlord

Dependency injection

So that @jden doesn't introduce more race conditions, let's use some DI around our database promises

rename /buildings* endpoints to /homes*

to be consistent with the terms we're using in https://github.com/codeforamerica/bestnest-app/ front-end

Add parcel owner data

set up dev db

use mongohq initially for fast

sessions and users

users get a session upon first coming to site
users can authenticate with a name and an email address
userId stored with all user-submitted content (comments/reviews, structured content, etc)

Upgrade mongo to larger production instance

Fix capitalization in data

from codeforamerica/bestnest-app#55

Addresses are appearing in all caps. Let's process the data so that we capitalize only the first letter of streets, towns, etc. both for the main address and for the owner information.

It's also kind of involved since the data is coming in from the source in all caps.

We might be able to get most of the way there by simply Title Casing all of the things.

todo:

document which datasets are affected
per dataset, try to find a good, scriptable solution.

spec search functionality

GET /search?q=:searchString

searchString should be an address, but it's gonna be user-entered, so who knows what it might be :)

response: a JSON array of the top n matches (n = 5? what would be useful? I'm thinking a smaller number because an address is a pretty specific search term)

{
  results: [
    {
      id: '924',
      address: '123 Mullholland Dr'.
      href: 'http://api.com/homes/5'
    }
  ]
}

questions

is there any additional information we need to return in a search result?

Landlord properties returned aren't in Chattanooga

Take a gander at http://dev.api.bestnestapp.com/landlords/PATTEN%20INVESTORS%20LLC. The address returned is for Sherman Oaks, California. Clearly we are using the certified mailing addresses of the landlord instead of properties they own in Chattanooga

set up prod and staging deploy environments

set up on heroku

use app names:

cfa-rentfacts-api-prod
cfa-rentfacts-api-staging

Add building square footage to database

lots of duplicate records for some landlords

see http://dev.bestnestapp.com/#/landlords/HATHCOCK%20RUTH%20H%20%26%20TIMOTHY%20S

Prefetch HES API data on summary endpoint

Because the HES API can take so long, we need to prefetch the energy cost estimate from that API when our app makes the summary request. The results should be cached so that subsequent requests for that property are fast

Foreign key/unique ID for land owners

Right now the API is using the string name of the landlord to determine what other properties are owned by that entity.

This feels wrong, what should we use as our key? Should we generate one on the data import?

return state and city along with addresses

this will require #14 as the current parcel base layer does not have city information (we can pretty much assume tennessee for all current data)

return UGC structured data in summary endpoint

Using a last write wins resolution policy, have the UGC content return in the summary endpoint as a set of key/value pairs as documented in codeforamerica/bestnest-app#82

Add project monitor per Erica's request

We'll need to add this to our travis.yml

notifications:
http://project-monitor.codeforamerica.org/projects/19c25e73-95dc-451d-a221-12a0a9008d9f/status

Just storing this here so I don't forget, will try and do shortly

API response for summary view

Proposal for the API response and the data that we'll need as far as our current design goes:

{
  'address': '2964 E 3rd Street',
  'parcel_id': 'TF1234567890',
  'owner_information': {
    'name': 'Gina Sakich',
    'street_address': '5559 Little Debbie Parkway',
    'city': 'Chattanooga',
    'state': 'TN',
    'phone': '4443453434'
  }
  'utility_estimates': {
    'electricity': {
      'average_monthly_cost': 90
    },
    'water': {
      'average_monthly_cost': 35
    }
  },
  'code_violations': [
    {
      'description': 'Abandoned/inoperable vehicle',
      'date_issued': '09-12-2014'
      'current_status': 'outstanding'
    },
    {
      'description': 'Overgrowth on an unoccupied lot',
      'date_issued': '01-23-2010',
      'current_status': 'resolved'
    }
  ],
  'schools': {
    'elementary': 'Allen Elementary',
    'middle': 'East Ridge Middle',
    'high': 'East Ridge High'
  },
  'public_transit': ,
  'crime': ,
}

@jden & @GiselleS - I'm not sure we yet know exactly what data should come along with public_transit and crime but we can update that as those views are fleshed out.

Thoughts on if we should include more/different information?

no user when saving comments

2014-08-26T17:27:10.220566+00:00 heroku[router]: at=info method=POST path="/comments" host=dev.api.bestnestapp.com request_id=9b68a530-a132-4c13-a155-64803f10617c fwd=
"54.215.190.2" dyno=web.2 connect=1ms service=4ms status=400 bytes=603                                                                                                 
2014-08-26T17:27:10.220709+00:00 app[web.2]: 10.6.103.177 - - [Tue, 26 Aug 2014 17:27:10 GMT] "POST /comments HTTP/1.1" 400 5 "-" "Rested/2009 CFNetwork/673.3 Darwin/1
3.3.0 (x86_64) (MacBookAir6%2C2)"                                                                                                                                      
2014-08-26T17:27:10.222973+00:00 app[web.2]: error: Error: No user                                                                                                     
2014-08-26T17:27:10.222977+00:00 app[web.2]:     at ensure (/app/comments.js:34:13)                                                                                    
2014-08-26T17:27:10.222979+00:00 app[web.2]:     at Object.post (/app/comments.js:8:16)                                                                                
2014-08-26T17:27:10.222980+00:00 app[web.2]:     at /app/http.js:83:21                                                                                                 
2014-08-26T17:27:10.222982+00:00 app[web.2]:     at /app/respondWith.js:9:16                                                                                           
2014-08-26T17:27:10.222983+00:00 app[web.2]:     at tryCatch1 (/app/node_modules/polyfill-promise/node_modules/bluebird/js/main/util.js:63:19)                         
2014-08-26T17:27:10.222985+00:00 app[web.2]:     at Promise$_callHandler [as _callHandler] (/app/node_modules/polyfill-promise/node_modules/bluebird/js/main/promise.js
:695:13)                                                                                                                                                               
2014-08-26T17:27:10.222987+00:00 app[web.2]:     at Promise$_settlePromiseFromHandler [as _settlePromiseFromHandler] (/app/node_modules/polyfill-promise/node_modules/b
luebird/js/main/promise.js:711:18)                                                                                                                                     
2014-08-26T17:27:10.222989+00:00 app[web.2]:     at Promise$_settlePromiseAt [as _settlePromiseAt] (/app/node_modules/polyfill-promise/node_modules/bluebird/js/main/pr
omise.js:868:14)                                                                                                                                                       
2014-08-26T17:27:10.222990+00:00 app[web.2]:     at Async$_consumeFunctionBuffer [as _consumeFunctionBuffer] (/app/node_modules/polyfill-promise/node_modules/bluebird/
js/main/async.js:74:12)                                                                                                                                                
2014-08-26T17:27:10.222993+00:00 app[web.2]:     at Async$consumeFunctionBuffer (/app/node_modules/polyfill-promise/node_modules/bluebird/js/main/async.js:37:14)      
2014-08-26T17:27:10.222995+00:00 app[web.2]:     at /app/node_modules/newrelic/node_modules/continuation-local-storage/node_modules/async-listener/glue.js:188:31

implement /search endpoint

depends on #6

create /search endpoint
create search functionality
- figure out address matching / suggestion

Code violations URL

homes/:id/violations should return a list of all the code violations that have occured at that property sorted descending by date

finish setting up travis ci -> chat integration

implement summary endpoint

render response as specced in #2
http endpoint responds at /homes/:homeId
test renderer

security audit

sanitize all user submitted content
usw.

rename all the things

we've got a lot of stuff to update to bestnest. i'm gonna let it simmer a few days before going through everything

make HES wrapper endpoint

a read-only endpoint the app can use to access HES functionality

unclear if it will be part of the summary response or a separate endpoint - that should be clarified in #56

object format for this response will be depend on #56

endpoint:
/home/:homeId/energy

format:

{
  "annualCost": 1000,
  "annualkWh": 3200
}

note that after testing this api is extremely slow due to the 3rd party api response times

2014-08-11T21:01:30.246751+00:00 app[web.2]: 10.238.162.205 - - [Mon, 11 Aug 2014 21:01:30 GMT] "GET /landlords/SMITH%20TYLER HTTP/1.1" 404 5 "http://dev.bestnestapp.com/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36"
2014-08-11T21:01:30.247659+00:00 app[web.2]: error: NotFoundError: Query returned no results and no default value specified: homes {"properties.parcel":"136A C 031"}
2014-08-11T21:01:30.247662+00:00 app[web.2]:     at tryCatch1 (/app/node_modules/minq/node_modules/bluebird/js/main/util.js:85:19)
2014-08-11T21:01:30.247665+00:00 app[web.2]:     at Promise$_settlePromiseFromHandler [as _settlePromiseFromHandler] (/app/node_modules/minq/node_modules/bluebird/js/main/promise.js:693:18)
2014-08-11T21:01:30.247669+00:00 app[web.2]:     at Promise$_fulfillPromises [as _fulfillPromises] (/app/node_modules/minq/node_modules/bluebird/js/main/promise.js:976:14)
2014-08-11T21:01:30.247664+00:00 app[web.2]:     at Promise$_callHandler [as _callHandler] (/app/node_modules/minq/node_modules/bluebird/js/main/promise.js:677:13)
2014-08-11T21:01:30.247671+00:00 app[web.2]:     at Async$_consumeFunctionBuffer [as _consumeFunctionBuffer] (/app/node_modules/minq/node_modules/bluebird/js/main/async.js:63:12)
2014-08-11T21:01:30.247667+00:00 app[web.2]:     at Promise$_settlePromiseAt [as _settlePromiseAt] (/app/node_modules/minq/node_modules/bluebird/js/main/promise.js:875:14)
2014-08-11T21:01:30.247672+00:00 app[web.2]:     at Async$consumeFunctionBuffer (/app/node_modules/minq/node_modules/bluebird/js/main/async.js:36:14)
2014-08-11T21:01:30.247674+00:00 app[web.2]:     at /app/node_modules/newrelic/node_modules/continuation-local-storage/node_modules/async-listener/glue.js:188:31

codeforamerica / bestnest-api Goto Github PK

bestnest-api's Introduction

bestnest-api

installation

running

environment variables

database

framework

contributing

license

bestnest-api's People

Contributors

Stargazers

Watchers

Forkers

bestnest-api's Issues

questions

Recommend Projects

Recommend Topics

Recommend Org