This repo hosts code for the Data Commons website.
Data Commons is an open knowledge graph that provides a unified view across multiple public data sets and statistics. We've bootstrapped the graph with lots of data from US Census, CDC, NOAA, etc., and through collaborations with the New York Botanical Garden, Opportunity Insights, and more. However, Data Commons is meant to be for community, by the community. We are excited to work with you to make public data accessible to everyone.
To see the extent of data we have today, browse the graph using our browser.
In https://github.com/datacommonsorg/website, click on "Fork" button to fork the repo.
Clone your forked repo to your desktop.
Add datacommonsorg/website repo as a remote:
git remote add dc https://github.com/datacommonsorg/website.git
Every time when you want to send a Pull Request, do the following steps:
git checkout master
git pull dc master
git checkout -b new_branch_name
# Make some code change
git add .
# Run tests
./run_test.sh -a
git commit -m "commit message"
# If tests pass
git push -u origin new_branch_name
Then in your forked repo, you can send a Pull Request. If this is your first time contributing to a Google Open Source project, you may need to follow the steps in CONTRIBUTING.md.
Wait for approval of the Pull Request and merge the change.
This repo contains DataCommons mixer repo as a submodule.
When mixer upstream is updated, and you want to track the change:
git submodule foreach git pull origin master
Contact Data Commons team to get permission for access of secret manager.
Contact Data Commons team to get permission for GCP application credential. Then run the following command once
gcloud config set project datcom-browser-staging && \
gcloud auth application-default login
This will generate a json file in your local machine, record the path and set the environment variable in your ~/.bash_profile or ~/.bashrc file
export GOOGLE_APPLICATION_CREDENTIALS=<JSON_CREDENTIAL_PATH>
Before running client-side tests and packages, install nodejs
Before running the tests, install the browser and webdriver. Here we recommend you use Google Chrome browser and ChromeDriver.
-
Chrome browser can be downloaded here.
-
ChromeDriver can be downloaded here, or you can download it using package manager directly:
npm install chromedriver
If using Linux system, you can run the following commands to download Chrome browser and ChromeDriver, this will also include the path setup:
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo dpkg -i google-chrome-stable_current_amd64.deb; sudo apt-get -fy install
CHROMEDRIVERV=$(curl https://chromedriver.storage.googleapis.com/LATEST_RELEASE)
wget https://chromedriver.storage.googleapis.com/${CHROMEDRIVERV}/chromedriver_linux64.zip
unset CHROMEDRIVERV
unzip chromedriver_linux64.zip
sudo mv chromedriver /usr/bin/chromedriver
sudo chown root:root /usr/bin/chromedriver
sudo chmod +x /usr/bin/chromedriver
Note: Make sure that your ChromeDriver version is compatible with your local Google Chrome version. You can change view the lastet ChromeDriver version here.
./run_test.sh -a
cd static
npm test testfilename -- -u
./run_npm.sh
This will watch static files change and re-build on code edit.
Start the flask webserver locally at localhost:8080
./run_flask.sh
Install "Air" for Go server auto reload
go get -u github.com/cosmtrek/air
Start the Go webserver locally at localhost:7070
air
Another way to start the server locally is to use Docker.
Make sure you have installed Docker Desktop.
Build Docker image
docker build -t datacommonsorg-website .
Run it locally
docker run \
-p 8080:8080 \
-e GOOGLE_APPLICATION_CREDENTIALS=<JSON_CREDENTIAL_PATH> \
-v $GOOGLE_APPLICATION_CREDENTIALS:<JSON_CREDENTIAL_PATH>:ro \
datacommonsorg-website
This file is stored in GCS bucket: datcom-browser-prod.appspot.com (for prod) and datcom-browser-staging.appspot.com (for local and staging). To update this file, please contact Data Commons team.
Check Redis instance information.
gcloud config set core/project datcom-browser-staging(datcom-browser-prod)
gcloud redis instances describe redis-instance --region=us-west2
Apache 2.0
For general questions or issues, please open an issue on our
issues page. For all other
questions, please send an email to [email protected]
.
Note - This is not an officially supported Google product.