pygeo / pycmbs Goto Github PK

At the moment, a lot of print statements are written to the console. Would be good to develop a proper error/warning handling, including different verbose levels.

Use already existing python modules for this.

check again installation via pip; seems not working yet properly!

It seems that the bounding box cutting does not work properly, as the cell_area is not usable afterwards.

Here is a code that can be used to reproduce the problem:

"""
reproducing a bug when masking areas
"""


from pycmbs.examples import download
from pycmbs.region import Region
from pycmbs.mapping import map_plot
import matplotlib.pyplot as plt
from pycmbs.plots import LinePlot

plt.close('all')

air = download.get_sample_file(name='air')
europe = Region(-10., 40., 25., 75., 'europe', type='latlon')

air.get_aoi_lat_lon(europe)
map_plot(air)

air.cut_bounding_box()
map_plot(air, title='cutted')

# this now raises an error
L=LinePlot()
L.plot(air)

plt.show()

Replace parser for config file by a parser similar to the one used for the INI files

jsbach_T63_GR15_4tiles_1992.nc is missing, no masking possible. Can one distribute it along with the package?
I would add a possibility to provide land sea mask path in the configuration file

In principle the benchmarking part of pycmbs is an entirely separate code. It should be therefore split from the core part of the code into an own project

It would be nice to have a routine like map_movie, which is basically similar to the map_plot routine, but which would create a movie automatically

Fails on mpi-m servers. Only one year specified in configuration file in this case.

ensmean_cfc
    Doing Hovmoeller plot ...
Traceback (most recent call last):
  File "./pycmbs-benchmarking.py", line 663, in <module>
    main()
  File "./pycmbs-benchmarking.py", line 574, in main
    eval(cmd)
  File "<string>", line 1, in <module>
  File "/data/share/mpiles/TRS/m300036/pycmbs/pycmbs/benchmarking/analysis.py", line 269, in cfc_analysis
    plot_options=plot_options, actvar='cfc', regions=regions)
  File "/data/share/mpiles/TRS/m300036/pycmbs/pycmbs/benchmarking/analysis.py", line 1331, in main_analysis
    use_basemap=use_basemap, shift_lon=shift_lon, interval=interval, regions=regions, GM_HT_clim=GM_HT_clim)
  File "/data/share/mpiles/TRS/m300036/pycmbs/pycmbs/benchmarking/analysis.py", line 797, in generic_analysis
    tmp = tmp.interp_time(pl.date2num(tref))
  File "/data/share/mpiles/TRS/m300036/pycmbs/pycmbs/data.py", line 2116, in interp_time
    d = np.asarray([datetime.datetime(x.year, x.month, x.day, x.hour, x.minute, x.second, 0, pytz.UTC) for x in d])
AttributeError: 'numpy.float64' object has no attribute 'year'

generate a git-hook for automatic compilation of polygon_utils on pull

http://www.manpagez.com/man/5/githooks/
http://stackoverflow.com/questions/5623208/how-to-execute-a-command-right-after-a-fetch-or-pull-command-in-git

make Report class more flexible. It would be good if the report class would support more output formats. One way to implement this is by writing the reports in markdown and then use tools like pandoc to convert on the fly to different formats.

ensure better test coverage for model different components

implement again the contours option for the map_plot wrapper

Implement a script that allows automatic installation with cartopy support using virtual machines.

There is no module index html page when building docs

When plotting projected data, it can happen that an empty "stripe" is visible around the 0degree longitude. This is typically due to a missing value at the longitude of 0deg.

In Basemap, this is captured by using add_cyclic. It would be necessary to include something similar also in the plotting backend to get always proper plots.

It seems that the get_zonal_mean() routine does not perform proper area weighting.

When using e.g. with CERES radiation data the result is the same as a non-weighted dataset.
I could not figure out quickly where the problem is actualyl coming from. Needs more thorough analysis and unit test implementation

Since release of version 1.1.2 of netCDF4, the pip based installation does not work.
At the moment v1.1.1 is therefore still used in travis.yml for automatic testing.

Once proper installation is possible, then travis.yml needs to be updated.

See also issue raised in netCDF4 repository: Unidata/netcdf4-python#319

when aggregating raw data to monthly means and the input data has much gaps in time, it can happen that monthly mean values are based only on a few (even single) data point in time.

To avoid this, on should be able to specify in the configuration a threshold which needs to be exceeded before monthly composite is generated.

When writing a report to a specific directory, then all files and figures are automatlically stored as fig0000001 ... fig00000N.png

If we have now multiple reports ending up in the same directory, then the figures will be overwritten! As this is a major problem, one should be able to set a uniwue prefix to the figure filename, to allow to identify to which report/tex file it belongs to.

Check if pycmbs-benchmarking.py works in the isolated environment ( with visible site-packages ).

make sure that sphinx generates and includes graphs using sample data

Build documentation after successful testing by travis-ci and publish it automatically to github-pages.

Show_timeseries and show_histogram in map_plot function; are they working?

with the following message:

(ipythondev)m300028@alexnotebook:pycmbs$ make build_docs
python setup.py build_sphinx
running build_sphinx
Running Sphinx v1.2b3
/home/m300028/virtualenvs/ipythondev/local/lib/python2.7/site-packages/matplotlib-1.4.x-py2.7-linux-x86_64.egg/matplotlib/cbook.py:126: MatplotlibDeprecationWarning:
The Sphinx extension ipython_console_highlighting has moved from
matplotlib to IPython, and its use in matplotlib is deprecated.
Change your import from 'matplotlib.sphinxext.ipython_directive' to
'IPython.sphinxext.ipython_directive.
warnings.warn(message, mplDeprecation, stacklevel=1)
Traceback (most recent call last):
File "setup.py", line 37, in
license="MIT license")
File "/usr/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/sphinx/setup_command.py", line 143, in run
freshenv=self.fresh_env)
File "/usr/local/lib/python2.7/dist-packages/sphinx/application.py", line 119, in init
self.setup_extension(extension)
File "/usr/local/lib/python2.7/dist-packages/sphinx/application.py", line 283, in setup_extension
mod = import(extension, None, None, ['setup'])
File "/home/m300028/virtualenvs/ipythondev/local/lib/python2.7/site-packages/matplotlib-1.4.x-py2.7-linux-x86_64.egg/matplotlib/sphinxext/ipython_directive.py", line 534, in
class IpythonDirective(Directive):
File "/home/m300028/virtualenvs/ipythondev/local/lib/python2.7/site-packages/matplotlib-1.4.x-py2.7-linux-x86_64.egg/matplotlib/sphinxext/ipython_directive.py", line 546, in IpythonDirective
shell = EmbeddedSphinxShell()
File "/home/m300028/virtualenvs/ipythondev/local/lib/python2.7/site-packages/matplotlib-1.4.x-py2.7-linux-x86_64.egg/matplotlib/sphinxext/ipython_directive.py", line 208, in init
self.cout = io.StringIO()
AttributeError: 'module' object has no attribute 'StringIO'
make: *** [build_docs] Error 1

When applying cut_bounding_box() it seems that the bbox is only cutted in the y direction but not in the x-direction. Would need to be checked.

Problem occurs in demo.ipynb

Generate a tagged version and upload to PyPi

Improve the Gallery and examples by putting them into ipynb

Use
a) examples in example directory
b) the following examples

Basic data analysis

Temporal mean

Calculating the temporal mean field of a variable is as simple as::

    air.timmean()

Spatial mean

The (area weighted) spatial mean is obtained as::

air.fldmean()

Masking an area

You probably want to work only on particular regions. The following script shows you how to easily to this.

.. plot:: ../../pycmbs/examples/05_mask_a_region.py
  :include-source:

Temporal slicing

If you want to perform a temporal subsetting of the data, this can be done as follows::

# temporal subsetting using existing start/stop dates
import datetime.datetime

start_date = datetime(2001,05,01)
stop_date = datetime(2010,04,15)
air.apply_temporal_subsetting(start_date, stop_date):

Working with multiple datasets

Simple arithmetic operations

.. plot:: ../../pycmbs/examples/03_data_analysis.py
  :include-source:

the map_plot wrapper needs some more docstrings

Generate acceptance tests that would emulate benchmarking run

Test installation using

a) pip
b) setup.py

Implement unittests for these installations to ensure that they always work

one example would be this one