climate-data-aggregator's Introduction
We're working on the rhok.org project for the worldbank Problem definition: http://www.rhok.org/problems/web-based-interface-tabulating-daily-global-climate-dataset-0 Project manager: Hrishi Patel Github handle: hppatel hrishi (dot) p (dot) patel (at) gmail (dot) com Status as of 2:00 AM 12/5/2011 Some useful knowledge we've gained: - Can import data into sqlite db, but only one file at a time and it needs to be unzipped first - SQLite DB grows at a rate of approx. 10 MB per day of data imported without indices - Have prototype web interface, but it's extremely basic and will probably not work in practice (can't use it to get lat-long easily for instance) - Tried using PostGreSQL instead of sqlite, works but appears to be much slower, at least on Amir's machine with default settings Next steps: - Get web version up and running. For now, all we need is a google map we can click on that will pass lat-long back to the server - Get a REST endpoint up and running for the server that accepts above lat-long requests and returns all the data available for that point - Import 6 years of data into the DB Data: - Unfortunately, the data is 13 GB of zipped ASCII files, so it's not easy to distribute. Several of the committers for this project have copies though. If you would like to get a copy, please contact one of us. Samples of the unzipped data files can be found in /data/test/ .
climate-data-aggregator's People
climate-data-aggregator's Issues
server.py uses copyrighted code
We should refactor server.py ASAP to not use copyrighted code (either write our own or find an open source version).
Document visualization code
at min throw out the usage options from the CLI
Make spaces vs. tabs consistent in python source files
Right now, we have a mixture of .py files using either spaces or tabs depending on who originally created them. While python doesn't mind this, so long as you're consistent within the same file, it would be nice to decide on a standard so we don't have to keep switching editors back and forth.
My vote is for spaces. I find them easier to work with, and according to the python style guide http://www.python.org/dev/peps/pep-0008/ they are much preferred over tabs. I would, however, like to get other people's opinions on this, particularly skykooler's as he's the other main python dev, before I go around changing everything.
Let me know what you guys think in the comments.
Handle null data ("-999.000") when mapping
Null data should be differentiated from "low" values. See #FLAG in perl code.
Consider transparency.
Perl image rendering bounds are swapped
The image the perl .png renderer produces is tall and thin instead of short and wide
Points on the web interface show up in the middle of the ocean
When you click on the OL map, it will find points in the middle of oceans but not on land masses. However, the visualization Amir did shows the points seem to correspond directly to land masses.
Port shell scripts to python
It would be nice if we could import the various shell scripts we have to python. One of the eventual goals of this project is porting it to a Windows/ArcGIS environment, which means we'll probably have to do this eventually anyway. In the meantime, it will be much easier to use if everything is in the same language.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.