hackforla / 311-data Goto Github PK

Overview

Create a draft of a pie chart for the 311 data

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Enable docker file building for ease of use for opposite team

Action Items

1. Create basic docker file for frontend application
2. Allow environment variable injection
3. Include in CI workflows

Resources/Instructions

Docker file for react apps

Overview

We have an existing self-sustained javascript prototype that interacts with socrata data. We need to migrate the socrata interaction to the backend and utilize the new react front end for the display layer

Action Items

Migrate display logic (Showing yearly breakdown of # of calls by request type) into react

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Create a draft of a bar graph for the 311 data

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

The control state will be the driving state for the individual components. This includes all values represented in the header. On any control change, all child components will update accordingly.

Action Items

1. Centralize state across application with Redux
2. Plan all/starter control values [startYear, endYear, NC, etc...]
3. Create a pattern for integrating redux across the componentry and application (i.e duck, rails, etc.)
4. MVP+ Implement Redux libraries to reduce boilerplate reducer/action code

Verification Items

• Utilize react hooks useState() where applicable
• Ensure functional components used as much as possible
• All controls should be connected/accessible from their corresponding child components
  • This could be actually functional or at least console.log() the new values to prove functionality

Resources/Instructions

React hooks with controlled input

Overview

Create TreeMap

Action Items

• Vol of Reqs Overall
• Vol of Reqs Per NC (color code by top reqs)
• Start and End time (date)

Overview

REPLACE THIS TEXT -Text here that clearly states the purpose of this issue in 2 sentences or less.

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Existing README has a lot of unnecessary information, it should be cleaned up to more accurately describe the 311-data project

Overview

Organize Project Board

Action Items

Define Milestones
Follow up on Assigned Issues

Overview

Modify Map component to adhere to a standard across components and connect props to header controls

Action Items

1. Migrate heavy logic into helper functions that can be utilized in other components
2. Add listener for zoom level
   - In support of plotting little dots/parcels based on zoom
3. Convert to functional component if possible?

Resources/Instructions

Overview

To remain lean and clean, we should remove the dependency on facebook's create-react-app and instead utilize vanilla react. App will remain the same, backing infrastructure is the only component being swapped out

Action Items

1. Remove Create react app existence
2. Replace with wepback, babel

Verification Items

• Application starts with npm start
• Application dev runs with npm run dev
• Application tests run with npm test
• Application builds with npm build
  • Coverage report is output
  • Can run through interactive testing session when run from terminal
  • Can be run from CI with export CI=true && npm test
• CI/CD works

Resources/Instructions

Create react app

Overview

Json payload that defines the average time to service request closure as defined by the control-filters

Action Items

• Talk with @gennaer or @ryanmswan to get the pandas/sql function used to generate the time to close report that was demoed during the...demo
• Update the time to close module with the necessary components to pull the data from the postgres instance
• Pass required parameters from the get/post body on the api side to the time to close module
• Return either the dataframe/json/dict to the api side and determine if further conversion is necessary
• Result should represent a mapping of

{ 
  "RequestType": "AverageTimeToClose"
} 

to start

• Build out possibility of including more information such as

{
  "RequestType" : { 
     "averageTimeToClose": "48H", 
     "Max": "78H", 
     "Min": "1H" 
    }
} 

or something similar

Resources/Instructions

Reach out to @gennaer or @ryanmswan for the proper calculation/logic behind time to close

Overview

Finish uploading 311 data into Jupyter Notebook. Analyze data by adding needed variables into dataframes and grouping features by relevance. Make any figures that may be of potential interest to the neighborhood councils.

Action Items

-Add additional variables to dataframes

• Create visuals(heat maps, bar graphs, line graphs, etc) that have any potential for interest

Resources/Instructions

Use Jupyter Notebook as a development environment. Will most likely be using the Pandas and Numpy python library for data analysis. Matplotlib and Bokeh python library will be used for data visualization.

Overview

Create a Heatmap

Action Items

• Total Reqs
• By Population
• By Type of Req
• Start and End time (date)

Overview

Write dummy tests for the backend to begin the CICD validation

Overview

Create the MVP for the Request Frequncy visualization

Action Items

1. Fill in 311-data/src/components/Frequency/Frequency.jsx with a overlapping line graph
2. Utilize a dummy json array for data, ideally pull example dummy data from data team
3. React to changes from parent state in header controls. see #76
   - This can simply be console.log() for now

Verification Items

• RequestFrequency graph fills in defined space
• Chart is responsive with browser resize
• Chart data closely resembles expected format
• Component is utilizing props from header controls in #76

Resources/Instructions

Reach out to data team for expected json payload

Overview

The columns in the 311 dataset need a model class to map to. This will utilize the ORM features of flask-sqlalchemy

Action Items

Create db related folder in backend
Create create models.py
Create object class to map each column from dataset

Resources/Instructions

https://stackoverflow.com/a/9695045
https://data.lacity.org/A-Well-Run-City/MyLA311-Service-Request-Data-2018/h65r-yf5i

Overview

Create a draft of a line graph for the 311 data

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Create Map with Pins

Action Items

Map with Pins (Dev - Josh)

• Filter by Reqs
• By NC
• By Census Track
• Start and End time (date

Resources/Instructions

https://plot.ly/

Overview

REPLACE THIS TEXT -Text here that clearly states the purpose of this issue in 2 sentences or less.

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Frontend related github actions are running a lot of unnecessary tasks over and over, this will reduce the redundancy and speed up builds.

Action Items

1. Only build once, remove all extra build steps

Overview

Draft out a heat map for the 311 data

Overview

REPLACE THIS TEXT -Text here that clearly states the purpose of this issue in 2 sentences or less.

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

A list somewhere that will look like trackable goals aka initiatives, that when clicked will adjust the request type to demostrate functionality

Action Items

Create List of fancy material cards
Populate with dummy profile and verbage
Connect cards with different target request types
When clicked, adjusts chart data to a different filter

Resources/Instructions

Overview

Create Candles (Dev - Angie)

Action Items

• Avg Time to Close Req per NC
• Start and End time (date)

Overview

REPLACE THIS TEXT -Text here that clearly states the purpose of this issue in 2 sentences or less.

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Create the MVP for the Time To Close visualization

Action Items

1. Fill in 311-data/src/components/TimeToClose/TimeToClose.jsx with a horizontal bar chart
2. Utilize a dummy json array for data, ideally pull example dummy data from data team
3. React to changes from parent state in header controls. see #76
   - This can simply be console.log() for now

Verification Items

• Horizontal bar chart fills in defined space
• Chart is responsive with browser resize
• Chart data closely resembles expected format
• Component is utilizing props from header controls in #76

Resources/Instructions

Reach out to data team for expected json payload

Overview

Pull data from socrata sources(Most likely from pre-downloaded csv's) into a docker postgres database
This will be a one-off script and configurable to pull all data, only pull year X or pull year x,y,z
Due to the way socrata is setup....this will be based on the data source-id
Should be extensible to be converted into a cron-job sort of feature

Action Items

1. If you dont have a local version of settings.cfg create that

• cp repo-root/server/src/settings.example.cfg repo-root/server/src/settings.cfg

2. Put in a new config option into the settings.cfg for the db connection string

• DB_CONNECTION_STRING = "postgresql://REDACTED:REDACTED@localhost:5432/311-user"
• The settings.example.cfg will have the redacted version since that will be checked in....the settings.cfg will have the actual auth values
• info on postgres

3. Create the /import_data endpoint in Flask/Sanic

• see example

4. Create /services/db_service.py
5. Mimic the previous seed function with the real cleanup script

Resources/Instructions

Previous barebones seeder function
Previous Seed endpoint

Overview

Create interface definition for inbound and outbound requests to database

Action Items

Create interface
Define contracts

Resources/Instructions

Overview

Enable docker file building for ease of use for opposite team

Action Items

1. Create basic docker file for backend application
2. Allow environment variable injection
3. Include in CI workflows

Resources/Instructions

Docker file for flask apps

Overview

Take a look through the dataAnalysis branch and remove all non-essentials
Also consolidate existing functions/scripts into one unified class

This will support bringing the cleaning script into the backend server

Please rewrite these:

How they currently tracking initiatives that exists, Do you have a list of past or present initiatives?

Whats wrong with it and what’s good with it?

Who creates the initiatives?

Exciting set of success measures (reduced calls)?

Are there other resources besides the 311 that you use to drive initiatives?

Is there any chance that initiatives might overlap?

Other dashboard views?

Do neightboood councils do cross jurisdiction diff (delta)?

Rates of change

Overview

All visualizations need to have an export feature

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Create backend github actions to ensure smooth CI/CD

Action Items

1. WF will include Build and Test
2. Deploy is TBD until team resolves deployment strategy....current option is heroku

Resources/Instructions

Overview

Have a docker compose or a convience script that will ensure a full environment to experiment with or develop against

Action Items

Services will include
[ ] Frontend (React)

• Requires a local version of .env, taken from example.env
  [ ] Backend (Flask/Sanic)
• Requires a local version of settings.cfg, taken from settings.example.cfg
  [ ] Persistence (Postgres)
• Required configuration is TBD

Upon executing docker-compose up (Or executing a script) all above services will start
Required configuration will also be checked before starting up

Overview

Prototype & Usability Testing

Action Items

tbd

Overview

Begin the barebones flask server for frontend/backend communication

Action Items

1. Create a hello world flask server
2. Create a /sample-data endpoint that returns a simple json array of data
3. Create a settings.default.cfg
4. Add entry into ignore file for settings.cfg

Resources/Instructions

Creating settings file for flask app

Overview

Gather distinct request types from 311 dataset and show them on the prototype

Action Items

1. Get distinct request types
2. Create dropdown selector
3. Update chart data on select

Resources/Instructions

311 dataset

Overview

We need to have a working READme file to easily on-board new team members.

Follow example from Example repo

Action Items

• Add the following information as a comment to this issue:
  • Identify what information needs to be collected.
  • Identify who holds each piece of information.
  • Collect information.

Resources/Instructions

PUT LINK TO README.md FILE HERE

Overview

REPLACE THIS TEXT -Text here that clearly states the purpose of this issue in 2 sentences or less.

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

Overview

Github can post activity to the slack channel (or even a separate but related slack channel). There are several ways to implement this, discuss pros and cons of each and decide what our setup will be.

Action Items

Review Resources

Resources/Instructions

Example of general project channel where github posts directly: Heart's slack channel, scroll back till you see a post or go directly to a post

Shared housing has two slack channels:
#shared-housing-lafh
#shared-housing-dev
They have decided to segregate the dev traffic from all other slack traffic on their project. The github messages are only setup to go to the dev channel.

This is to send the email with the notes from our Dev Jam to the potential attendee list.

Overview

Put the existing sanic application behind a gunicorn wrapper to enable multi-worker deployment
OR
Utilize the workers argument to app.run to allow asynchronicity with Sanic

Overview

Utilize the count() soda function to aggregate on count of rows that fulfill the query

Action Items

Remove the bare request to get the 311 dataset
Replace request with a filtered count request to populate the chart data

Resources/Instructions

https://dev.socrata.com/docs/functions/count.html

Overview

Rolling Roster of Team Participants.

Action Items

Add or update your personal info to the team roster.

Resources/Instructions

Roster

Overview

When importing subsets of a yearly 311 dataset(~500MB) pandas didnt like large datasets without datatype associations. The import/seed functionality needs to support larger sets.

Action Items

Hook up the models.py and flask-sqlalchemy from #28
Download a full year dataset from https://data.lacity.org/A-Well-Run-City/MyLA311-Service-Request-Data-2018/h65r-yf5i
Fix/adjust /seed_db endpoint and respective functions to support the full import into postresql

Resources/Instructions

Sample dataset https://data.lacity.org/A-Well-Run-City/MyLA311-Service-Request-Data-2018/h65r-yf5i
Will most likely look similar to this

Overview

X axis represents the time frames selected, for now just 2016 and 2017

Action Items

Change the X axis to represent the time frames selected, for now just 2016 and 2017

Resources/Instructions

N/A

Overview

REPLACE THIS TEXT -Text here that clearly states the purpose of this issue in 2 sentences or less.

Action Items

REPLACE THIS TEXT -If this is the beginning of the task this is most likely something to be researched and documented.

REPLACE THIS TEXT -If the issue has already been researched, and the course of action is clear, this will describe the steps. However, if the steps can be divided into tasks for more than one person, we recommend dividing it up into separate issues, or assigning it as a pair programming task.

Resources/Instructions

REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.