projectsidewalk / sidewalkwebpage Goto Github PK

Welcome to Project Sidewalk! Project Sidewalk is an open source project aimed at mapping and assessing every sidewalk in the world using remote crowdsourcing, artificial intelligence, and online satellite & streetscape imagery.

If you use or reference Project Sidewalk in your research, please cite:

Manaswi Saha, Michael Saugstad, Hanuma Teja Maddali, Aileen Zeng, Ryan Holland, Steven Bower, Aditya Dash, Sage Chen, Anthony Li, Kotaro Hara, and Jon Froehlich. 2019. Project Sidewalk: A Web-based Crowdsourcing Tool for Collecting Sidewalk Accessibility Data At Scale. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, Paper 62, 1–14. https://doi.org/10.1145/3290605.3300292

Want a Project Sidewalk server set up for your city/municipality? You can read about things we consider when choosing new deployment cities on our Wiki including geographic diversity, presence of local advocates, funding, etc. You can also read some past discussions here, here, and here.

If you would like to suggest that we deploy in your city/municipality, please email us at [email protected]!

If you run into any problems during setup, check the Docker troubleshooting wiki page and the Github issues tagged as "Dev Environment". If you don't find any answers there, then post in the "core" or "intern" channels on Slack! We prefer posting to channels vs. DMs to Mikey to enable all of us to help each other.

Here are the instructions to run Project Sidewalk locally for the first time. If you've already run through this list and gotten Project Sidewalk to run locally on your machine, but you just want to run it again (e.g., after a machine restart), then type make dev in the root SidewalkWebpage directory.

Make sure Docker is running on your machine. You should see a Docker whale in your Mac or Windows tray. You can also configure Docker to run at startup (see Preferences).

On Windows, we recommend Windows Powershell (built in to Win10). On Mac, use the basic terminal or, even better, iTerm2. On Linux (or if you're using WSL2 on Windows), the default Linux Shell (such as Bash) is a great choice.

1. Email Mikey ([email protected]) and ask for a database dump, a Mapbox API key, and a Google Maps API key & secret (if you are not part of our team, you'll have to create a Google Maps API key yourself).

2. If your computer has an Apple Silicon (M1 or M2) chip, then you should modify the platform line in the docker-compose.yml, changing it to linux/arm64.

3. Modify the MAPBOX_API_KEY, GOOGLE_MAPS_API_KEY, and GOOGLE_MAPS_SECRET lines in the docker-compose.yml using the keys and secret you've acquired.

4. Modify the SIDEWALK_CITY_ID line in the docker-compose.yml to use the ID of the appropriate city, listed here (it's the city that matches your database dump, so check the name of the db dump file).

5. Modify the DATABASE_USER line in the docker-compose.yml, replacing "sidewalk" with the username from the table linked above.

6. Rename the database dump file that you got from Mikey to "<database_user>-dump" (using the name from the prev step) and put it in the db/ directory (other files in this dir include init.sh and import-dump.sh).

7. From the root SidewalkWebpage dir, run make dev. This will take time (20-30 mins or more depending on your Internet connection) as the command downloads the docker images, spins up the containers, and opens a Docker shell into the webpage container in that same terminal. The containers (running Ubuntu Stretch) will have all the necessary packages and tools so no installation is necessary. This command also initializes the database, though we still need to import the data. Successful output of this command will look like:

   Successfully built [container-id]
   Successfully tagged projectsidewalk/web:latest
   WARNING: Image for service web was built because it did not already exist. 
   To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
   root@[container-id]:/home#
   

8. Run make import-dump db=<database_user> (needs to be the same thing you set for DATABASE_USER) from the root project directory outside the Docker shell (from a new Ubuntu terminal). This may take a while depending on the size of the dump. Don't panic if this step fails :) and consult the Docker Troubleshooting wiki. Check the output carefully. If it looks like there are errors, do not skip to the next step, check the wiki and ask Mikey if you don't find solutions in there.

9. Run npm start from inside the Docker shell (the terminal where you ran make dev). If this is your first time running the command, everything will need to be compiled. So, it may take 5+ minutes initially, but will be orders of magnitude faster in the future (~10 secs).

   The behavior of npm start is dictated by what start is supposed to do as defined in package.json file. As per the current code, running this command will run grunt watch & sbt compile "~ run" (the ~ here is triggered execution that allows for the server to run in watch mode). This should start the web server. Successful output of this command will look like:

   > grunt watch & sbt clean "~ run"
   
   Running "watch" task
   Waiting...
   [info] Loading project definition from /home/project
   [info] Set current project to sidewalk-webpage (in build file:/home/)
   [success] Total time: 78 s, completed Dec 20, 2018 8:06:19 AM
   [info] Updating {file:/home/}root...
   [info] Resolving it.geosolutions.jaiext.errordiffusion#jt-errordiffusion;1.0.8 .[info] Resolving org.fusesource.jansi#jansi;1.4 ...
   [info] Done updating.
   
   --- (Running the application, auto-reloading is enabled) ---
   
   [info] play - Listening for HTTP on /0.0.0.0:9000
   
   (Server started, use Ctrl+D to stop and go back to the console...)
   

10. Head on over to your browser and navigate to localhost:9000 (or try 127.0.0.1:9000). This should display the Project Sidewalk webpage. It might take time to load initially.

1. Acquire another database dump, put it in the db/ directory, and rename it to "<database_user>-dump", using the appropriate database user from this table.
2. Run make import-dump db=<db_user> (using the name from the prev step) from the root project directory outside the Docker shell.
3. Update the DATABASE_USER variable in the docker-compose.yml to the same name.
4. Modify the SIDEWALK_CITY_ID line in docker-compose.yml to use the appropriate ID from this table.
5. Rerun make dev.
6. To switch back and forth between databases going forward, you will need to close the Docker shell (if you ran make dev, that just means running exit in that terminal), update the DATABASE_USER and SIDEWALK_CITY_ID, and rerun make dev.

1. SSH into containers: To ssh into the containers, run make ssh target=[web|db]. Note that [web|db] is not a literal syntax, it specifies which container you would want to ssh into. For example, you can do make ssh target=web.

We recommend the IntelliJ IDEA IDE for development. You should be able to get a student license to get the "ultimate" edition of IntelliJ IDEA.

1. Download and install from here, and make sure to get a student license, which you should use during the install proces!
2. On the first run of IntelliJ IDEA, you'll want to choose the 'Open' option (not the 'New Project') option, and choose the folder that holds the Project Sidewalk code (it is probably called SidewalkWebpage, it should be one level above the app/ directory).
3. Now let's hide some build files so that searching through your code is easier/faster. Go to File -> Project Structure and click on Modules in the left sidebar. You should have a module there with the Project Sidewalk code; if you don't, try to make one and let me know how it goes!
   1. For each of the directories below, right click on it and click 'Sources' to mark it as source code:
      • app
      • conf
      • db
      • project
      • public
   2. Then for each of these directories, right click on it and click 'Excluded' so that it doesn't show up in search, so that you hopefully don't accidentally edit these files!
      • Any other top-level directory that you didn't mark as 'Sources'
      • project/project
      • project/target
      • public/fonts
      • public/javascripts/Admin/build
      • public/javascripts/Gallery/build
      • public/javascripts/Help/build
      • public/javascripts/Progress/build
      • public/javascripts/PSMap/build
      • public/javascripts/SVLabel/build
      • public/javascripts/SVValidate/build
4. We then recommend installing a few plugins. To do so, go to File -> Settings. Select the Plugins option on the left sidebar and then Marketplace (on top menubar). For each of the following plugins, enter their name in the "search area" (textfield next to magnifying glass), find the plugin, and click Install: Play 2 Routes, i18n support, HOCON, and Scala (if you haven't already). You will then need to restart IntelliJ to install the plugins.
5. Finally, let's install a Java SDK to help with syntax highlighting in Scala/Java. Open File -> Project Structure and click on Project in the left sidebar. Under SDK, choose Add SDK -> Download JDK. Choosing version 1.8 is necessary. The default "Amazon Corretto" version works, but presumably others should as well. Click Download. Then set the Language Level to 8. It may take a bit of time to install Java.

To look at and run queries on your database, you will want to install a database client. Valentina Studio is a good cross-platform database client. People also like using Postico for Mac or PGAdmin on Windows/Mac.

You'll connect to the database using the following credentials:

Host: localhost:5432
User: postgres
Password: sidewalk
Database: sidewalk

1. Before making changes, check out our style guide and process for contributing new code wiki pages.

2. If you make any changes to the build.sbt or the configs, you'd need to press Ctrl+D and then sbt clean and then npm start from inside the Docker shell.

3. If you make any changes to the views or other scala files, these changes will be automatically picked up by sbt. You'd need to reload the browser once the compilation finishes. For example, a change to index.scala.html file results in:

   [info] Compiling 1 Scala source to /home/target/scala-2.10/classes...
   [success] Compiled in 260s
   
   --- (RELOAD) ---
   
   [info] play - Shutdown application default Akka system.
   [info] play - database [default] connected at jdbc:postgresql://db:5432/sidewalk
   [info] play - Starting application default Akka system.
   [info] play - Application started (Dev)
   [success] Compiled in 124s
   

4. If you make any changes to the assets (look in Gruntfile.js under watch block), these changes will be picked up by grunt. You'd need to reload the browser once the compilation finishes. For example, a change to public/javascripts/HELP/src/tableOfContents.js file results in (output has been trimmed):

   >> File "public/javascripts/Help/src/tableOfContents.js" changed.
   Running "concat:dist_audit" (concat) task
   Running "concat:dist_progress" (concat) task
   Running "concat:dist_admin" (concat) task
   Running "concat:dist_help" (concat) task
   Running "concat:dist_validate" (concat) task
   Running "concat_css:dist_audit" (concat_css) task
   File "public/javascripts/SVLabel/build/SVLabel.css" created.
   Running "concat_css:dist_validate" (concat_css) task
   File "public/javascripts/SVValidate/build/SVValidate.css" created.
   
   Done.
   Completed in 23.905s at Thu Dec 20 2018 09:31:45 GMT+0000 (Coordinated Universal Time) - Waiting...
   
   [success] Compiled in 5s