pcmdi / pcmdi_metrics Goto Github PK

Open-source Python package for Systematic Evaluation of Climate and Earth System Models

Home Page: http://pcmdi.github.io/pcmdi_metrics/

License: BSD 3-Clause "New" or "Revised" License

Python 12.41% Shell 0.16% Jupyter Notebook 87.43%

python climate climate-data climate-model climate-models climate-model-evaluation climate-science climate-analysis climate-variability

pcmdi_metrics's Introduction

PCMDI Metrics Package (PMP)

Conda-forge (NEW, recommended):

PCMDI Conda Channel (halted):

The PCMDI Metrics Package (PMP) is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations. Results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. Among other purposes, this enables modeling groups to evaluate changes during the development cycle in the context of the structural error distribution of the multi-model ensemble. Currently, the comparisons emphasize metrics of large- to global-scale annual cycle, tropical and extra-tropical modes of variability, ENSO, MJO, regional monsoons, high frequency characteristics of simulated precipitation, and cloud feedback.

PCMDI uses the PMP to produce quick-look simulation summaries across generations of CMIP.

The metrics package consists of the following parts:

Analysis software
Observation-based reference database of global (or near global, land or ocean) time series and climatologies
Package documentation and interactive jupyter notebook demos
Database of performance metrics computed for CMIP models

The package expects model data to be CF-compliant. To successfully use the package some input data "conditioning" may be required. We provide several demo scripts within the package.

Documentation

Getting Started

Installation requirements and instructions are available on the Install page
Users will need to contact the PMP developers ([email protected]) to obtain supporting datasets and get started using the package.
An overview for using the package and template scripts are detailed on the Using-the-package page
View Demo

Reference

Lee, J., P. J. Gleckler, M.-S. Ahn, A. Ordonez, P. Ullrich, K. R. Sperber, K. E. Taylor, Y. Y. Planton, E. Guilyardi, P. Durack, C. Bonfils, M. D. Zelinka, L.-W. Chao, B. Dong, C. Doutriaux, C. Zhang, T. Vo, J. Boutte, M. F. Wehner, A. G. Pendergrass, D. Kim, Z. Xue, A. T. Wittenberg, and J. Krasting, 2024: Systematic and Objective Evaluation of Earth System Models: PCMDI Metrics Package (PMP) version 3. Geoscientific Model Development (accepted, publication in progress) [preprint]

Contact

Report Bug

Request Feature

Some installation support for CMIP participating modeling groups is available: [email protected]

Acknowledgement

Content in this repository is developed by climate and computer scientists from the Program for Climate Model Diagnosis and Intercomparison (PCMDI) at Lawrence Livermore National Laboratory (LLNL). This work is sponsored by the Regional and Global Model Analysis (RGMA) program, of the Earth and Environmental Systems Sciences Division (EESSD) in the Office of Biological and Environmental Research (BER) within the Department of Energy's Office of Science. The work is performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

License

Distributed under the BSD 3-Clause License. See LICENSE for more information.

Release Notes and History

Versions	Update summary
v3.4.1	Technical update
v3.4	Technical update: Modes of variability xCDAT conversion
v3.3.4	Technical update
v3.3.3	Technical update
v3.3.2	Technical update
v3.3.1	Technical update
v3.3	New metric added: Sea-Ice
v3.2	New metric added: Extremes
v3.1.2	Technical update
v3.1.1	Technical and documentation update
v3.1	New metric added: Precipitation Benchmarking -- distribution bimodality
v3.0.2	Minor patch and more documentation added
v3.0.1	Minor technical patch
v3.0.0	New metric added: Cloud feedback metric by @mzelinka. xCDAT implemented for mean climate metrics
v2.5.1	Technical update
v2.5.0	New metric added: Precipitation Benchmarking -- distribution. Graphics updated
v2.4.0	New metric added: AMO in variability modes
v2.3.2	CMEC interface updates
v2.3.1	Technical update
v2.3	New graphics using archived PMP results
v2.2.2	Technical update
v2.2.1	Minor update
v2.2	New metric implemented: precipitation variability across time scale
v2.1.2	Minor update
v2.1.1	Simplified dependent libraries and CI process
v2.1.0	CMEC driver interfaced added.
v2.0	New capabilities: ENSO metrics, demos, and documentations.
v1.2	Tied to CDAT 8.0. Extensive regression testing added. New metrics: Diurnal cycle and intermittency of precipitation, sample monsoon metrics.
v1.1.2	Now managed through Anaconda, and tied to UV-CDAT 2.10. Weights on bias statistic added. Extensive provenance information incorporated into json files.
v1.1	First public release, emphasizing climatological statistics, with development branches for ENSO and regional monsoon precipitation indices
v1.0	Prototype version of the PMP

Contributors

Thanks goes to these wonderful people (emoji key):

_{Jiwoo Lee} 💻 📖 👀 ⚠️ ✅ 🔬 🤔 🚇	_{Peter Gleckler} 💻 📖 🔬 👀 ⚠️ 🔣 🤔	_{Ana Ordonez} 💻 📖 👀 ⚠️ ✅ 🚇	_{Min-Seop Ahn} 💻 📖 👀 ⚠️ ✅ 🔬	_{Paul Ullrich} 🤔 🔬	_{Charles Doutriaux} 💻	_{Karl Taylor} 🔬 🤔
_{Paul J. Durack} 💻	_{Mark Zelinka} 💻	_{Celine Bonfils} 🔬	_{Curtis C. Covey} 💻 🔬	_{Zeshawn Shaheen} 💻	_{Lina Muryanto} 🚇	_{Tom Vo} 🚇
_{Jason Boutte} 🚇	_{Jeffrey Painter} 🔣 🚇 💻	_{Stephen Po-Chedley} 🔣 🚇	_{Xylar Asay-Davis} 🚇	_{John Krasting} 💻 ⚠️	_{Angeline G Pendergrass} 💻 🔬 🤔	_{Michael Wehner} 💻 🔬
_{Daehyun Kim} 💻 🔬

This project follows the all-contributors specification.

pcmdi_metrics's People

Contributors

Stargazers

Watchers

pcmdi_metrics's Issues

installer points to ESGF

test suite needsdata to test udunits pkg

GFDL tos

Add UV-CDAT version and date stamp to json and all output files

We should add UV-CDAT and date stamp to all output formats to make debugging and problem discovery easier

Install on SuSE 11.1 (SLES 11.1 - CSIRO machine)

Ok so this seems to be getting closer..?

[ 89%] Performing install step for 'Numexpr'                                                                                
CMake Error at /work/dur041/tmp/140426_metrics/WGNE/tmp/uvcdat/uvcdat_build/Numexpr-prefix/src/Numexpr-stamp/Numexpr-install.cmake:56 (message):                                                                                                        
  CTEST_FULL_OUTPUT                                                                                                         

   Command failed: 1
   'env' 'PYTHONPATH=/work/dur041/tmp/140426_metrics/WGNE/lib/python2.7/site-packages:/work/dur041/tmp/140426_metrics/WGNE/lib/python/site-packages' '/work/dur041/tmp/140426_metrics/WGNE/bin/python' 'setup.py' 'install' '--prefix=/work/dur041/tmp/140426_metrics/WGNE'                                                                                                         
   From /work/dur041/tmp/140426_metrics/WGNE/tmp/uvcdat/uvcdat_build/Numexpr-prefix/src/Numexpr-stamp/Numexpr-install-out.log                                                                                                                           

  -------------------------------------------------

  -------------------------------------------------

   From /work/dur041/tmp/140426_metrics/WGNE/tmp/uvcdat/uvcdat_build/Numexpr-prefix/src/Numexpr-stamp/Numexpr-install-err.log                                                                                                                           

  -------------------------------------------------

  Traceback (most recent call last):

    File "setup.py", line 16, in <module>
      import numpy
    File "/work/dur041/tmp/140426_metrics/WGNE/lib/python2.7/site-packages/numpy/__init__.py", line 137, in <module>
      import add_newdocs
    File "/work/dur041/tmp/140426_metrics/WGNE/lib/python2.7/site-packages/numpy/add_newdocs.py", line 9, in <module>
      from numpy.lib import add_newdoc
    File "/work/dur041/tmp/140426_metrics/WGNE/lib/python2.7/site-packages/numpy/lib/__init__.py", line 13, in <module>
      from polynomial import *
    File "/work/dur041/tmp/140426_metrics/WGNE/lib/python2.7/site-packages/numpy/lib/polynomial.py", line 17, in <module>
      from numpy.linalg import eigvals, lstsq, inv
    File "/work/dur041/tmp/140426_metrics/WGNE/lib/python2.7/site-packages/numpy/linalg/__init__.py", line 48, in <module>
      from linalg import *
    File "/work/dur041/tmp/140426_metrics/WGNE/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 23, in <module>
      from numpy.linalg import lapack_lite

  ImportError: liblapack.so: cannot open shared object file: No such file or
  directory

  -------------------------------------------------



make[2]: *** [Numexpr-prefix/src/Numexpr-stamp/Numexpr-install] Error 1
make[1]: *** [CMakeFiles/Numexpr.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
-- PYLIBXML2 build command succeeded.  See also /work/dur041/tmp/140426_metrics/WGNE/tmp/uvcdat/uvcdat_build/PYLIBXML2-prefix/src/PYLIBXML2-stamp/PYLIBXML2-build-*.log

[ 90%] Performing install step for 'PYLIBXML2'
-- PYLIBXML2 install command succeeded.  See also /work/dur041/tmp/140426_metrics/WGNE/tmp/uvcdat/uvcdat_build/PYLIBXML2-prefix/src/PYLIBXML2-stamp/PYLIBXML2-install-*.log

[ 90%] Completed 'PYLIBXML2'
[ 90%] Built target PYLIBXML2
-- ESMF build command succeeded.  See also /work/dur041/tmp/140426_metrics/WGNE/tmp/uvcdat/uvcdat_build/ESMF-prefix/src/ESMF-stamp/ESMF-build-*.log

[ 90%] Performing install step for 'ESMF'
-- ESMF install command succeeded.  See also /work/dur041/tmp/140426_metrics/WGNE/tmp/uvcdat/uvcdat_build/ESMF-prefix/src/ESMF-stamp/ESMF-install-*.log

[ 90%] Completed 'ESMF'
[ 90%] Built target ESMF
make: *** [all] Error 2
 ERROR: Could not compile (make) cdat code (4)
/work/dur041/tmp/140426_metrics/WGNE/tmp/uvcdat/uvcdat /work/dur041/tmp/140426_metrics/WGNE/tmp/metrics

Sorry...
This action did not complete successfully
Please re-run this task until successful before continuing further

Also please review the installation FAQ it may assist you
http://esgf.org/wiki/ESGFNode/FAQ

 ERROR: Could not compile (make) cdat code
/work/dur041/tmp/140426_metrics/WGNE/tmp/metrics

Sorry...
This action did not complete successfully
Please re-run this task until successful before continuing further

Also please review the installation FAQ it may assist you
http://esgf.org/wiki/ESGFNode/FAQ

double check obs have requested var and return comprehensible error message

example: huss ?

bomb on level data

All 4d data now fails (e.g., 'ta_850','ua_200') as a result of the new regions_dict in wgne_metrics_driver.py only reading root variable (without level).

package demo: incomplete filename for interpolated_model_clims

tos_GFDL-ESM2G__historical__198501-200512.interpolated.linear.2.5x2.5-clim.nc

Also in this demo interpolated file:

the variable name is "VAR"
new metrics controlled vocabulary is not included...

VCS issue with portrait plots

Ok so first test trying to plot a portrait plot:

durack1@oceanonly:[test_portrait]:[test_branch]:[81]> python portrait_test.july23.py
Traceback (most recent call last):
  File "portrait_test.july23.py", line 17, in <module>
    import portraits
  File "/export/durack1/git/wgne-wgcm_metrics/test_portrait/portraits.py", line 12, in <module>
    import vcs
  File "/usr/local/uvcdat/2014-08-13/lib/python2.7/site-packages/vcs/__init__.py", line 52, in <module>
    install_vcs._files()
  File "/usr/local/uvcdat/2014-08-13/lib/python2.7/site-packages/vcs/install_vcs.py", line 41, in _files
    shutil.copyfile(icon_file_cp, icon_file)
  File "/usr/local/uvcdat/2014-08-13/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 13] Permission denied: '/usr/local/uvcdat/2014-08-13/lib/python2.7/site-packages/vcs/../../../../bin/vcs_icon.xbm'

And yes, the only way I could get this far down the path was to use bash.. csh bombed admirably.. I'll await the updated build on oceanonly and try again either later this week or next..

wrap processing in try/except rather than bombing

It appears the code is bombing if it can't find data, rather than skipping (the preferred result):

 ref is:  all
 refs: [u'default', u'alternate', u'ref3']
 OBS SHAPE IS  (12, 72, 144)
 psl   GFDL-CM4   (12, 72, 144)   (12, 72, 144)    default
/home/Paul.Durack/140701_metrics/WGNE/bin/wgne_metrics_driver.py:241: UserWarning: Model and observation units differed, converted model (hPa) to observation unit (Pa)
  warnings.warn("Model and observation units differed, converted model (%s) to observation unit (%s)" % (dm.units,do.units))
---- shapes  (12, 72, 144)   (12, 72, 144)
psl    annual mean bias is  19.1959297534
---- shapes  (12, 24, 144)   (12, 24, 144)
psl    annual mean bias is  -199.580140365
---- shapes  (12, 24, 144)   (12, 24, 144)
psl    annual mean bias is  100.163863037
---- shapes  (12, 24, 144)   (12, 24, 144)
psl    annual mean bias is  157.004066588
Traceback (most recent call last):
  File "/home/Paul.Durack/140701_metrics/WGNE/bin/wgne_metrics_driver.py", line 171, in <module>
    metrics_dictionary["References"][ref] = obs_dic[var][obs_dic[var][ref]]
KeyError: u'rnl_ncep'

ESMF build - "metrics" UV-CDAT build flag

Ok so the most up-to-date build was tested on the GFDL system yesterday and it seems to be working fine with ocean data.
Additionally the axis_aliases also appear to pickup axes from their units attribute.

Is there a need to generate a specific "metrics" build option so that ongoing changes to the "ESGF" build flag will not affect the metrics package in the future?

Latest master build is failing

9562abe

Looks like the installer is hanging on the final build step:

[ 96%] No patch step for 'windspharm'
[ 97%] No update step for 'windspharm'
[ 97%] No configure step for 'windspharm'
[ 98%] Performing build step for 'windspharm'
-- windspharm build command succeeded.  See also /home/p1d/140828_metrics/PCMDI_METRICS/tmp/uvcdat/uvcdat_build/windspharm-prefix/src/windspharm-stamp/windspharm-build-*.log

[ 99%] Performing install step for 'windspharm'
-- windspharm install command succeeded.  See also /home/p1d/140828_metrics/PCMDI_METRICS/tmp/uvcdat/uvcdat_build/windspharm-prefix/src/windspharm-stamp/windspharm-install-*.log

[ 99%] Completed 'windspharm'
[ 99%] Built target windspharm
-- SCIPY build command succeeded.  See also /home/p1d/140828_metrics/PCMDI_METRICS/tmp/uvcdat/uvcdat_build/SCIPY-prefix/src/SCIPY-stamp/SCIPY-build-*.log

[ 99%] Performing install step for 'SCIPY'
-- SCIPY install command succeeded.  See also /home/p1d/140828_metrics/PCMDI_METRICS/tmp/uvcdat/uvcdat_build/SCIPY-prefix/src/SCIPY-stamp/SCIPY-install-*.log

[ 99%] Completed 'SCIPY'
[ 99%] Built target SCIPY

install_metrics.bash hangs due to UVCDAT logging

[ 99%] Performing install step for 'CDATLogger'
Ping succeeded.
[100%] Completed 'CDATLogger'
[100%] Built target CDATLogger
CMAKE ARGS
UVCDAT BDIR/Users/durack1/Desktop/metrics/wgne-wgcm_metrics/WGNE/tmp/uvcdat
Hangs indefinitely at this point
^C ERROR: Could not load CDMS (cdms2) module
~/Desktop/metrics/wgne-wgcm_metrics/WGNE/tmp/metrics

Sorry...
This action did not complete successfully
Please re-run this task until successful before continuing further

Also please review the installation FAQ it may assist you
http://esgf.org/wiki/ESGFNode/FAQ

Is there a reason that the "metrics" dir is found under {build_prefix}/tmp/ rather than {build_prefix}

Should the metrics dir be dropped into the master directory (so WGNE by default) rather than hidden under /tmp which it currently is..?

Trying to remove non-empty sample_data subdirectory

As below

running install_egg_info
Writing /home/p1d/140701_metrics/WGNE/lib/python2.7/site-packages/metrics-0.1.0-py2.7.egg-info
rmdir: failed to remove `/home/p1d/140701_metrics/WGNE/sample_data': Directory not empty

*******************************
UVCDAT  - master - Install Success
Metrics - devel - Install Success
*******************************
Please test as follow:
source /home/p1d/140701_metrics/WGNE/bin/setup_runtime.sh
wgne_metrics_driver.py -p /home/p1d/140701_metrics/WGNE/test/wgne/basic_test_parameters_file.py
compare: /home/p1d/140701_metrics/WGNE/test/wgne/tos_2.5x2.5_esmf_linear_metrics.json.good with wgne_install_test_results/metrics_results/installationTest/tos_2.5x2.5_esmf_linear_metrics.json
*******************************
Create your customized input_parameters.py (inspire yourself from examples in /home/p1d/140701_metrics/WGNE/doc/wgne_input_parameters_sample.py
Once you have a parameter file run:
source /home/p1d/140701_metrics/WGNE/bin/setup_runtime.sh
wgne_metrics_driver.py -p /path/to/your/edited/parameter_file.py
*******************************
Once everything is ok, you can safely remove the temporary directory: /home/p1d/140701_metrics/WGNE/tmp
*******************************

Create 'misc' subdir to contain example code

The work done on the GFDL infrastructure to best use the metrics package to create climatologies and then generate metrics from these has been done. The scripts which have been written to do this should be available somewhere so that other users don't have to start from scratch.

Standardise units of obs data

It seems to me like there is some disagreement in the units of the obs and model data:
an014:/home/Paul.Durack/140129_metrics/test> /home/Paul.Durack/140129_metrics/WGNE/bin/python /home/Paul.Durack/140129_metrics/WGNE/bin/wgne_metrics_driver.py -p /home/Paul.Durack/140129_metrics/test/wgne_input_parameters_test.py
ref is: all
refs: [u'default']
OBS SHAPE IS (12, 72, 144)
tos GFDL-ESM2G (12, 72, 144) (12, 72, 144) default
---- shapes (12, 72, 144) (12, 72, 144)
/home/Paul.Durack/140129_metrics/WGNE/lib/python2.7/site-packages/numpy/ma/core.py:936: RuntimeWarning: overflow encountered in multiply
result = self.f(da, db, _args, *_kwargs)
tos annual mean bias is 273.421894797
---- shapes (12, 24, 144) (12, 24, 144)
tos annual mean bias is 274.136384059
---- shapes (12, 24, 144) (12, 24, 144)
tos annual mean bias is 274.230836527
---- shapes (12, 24, 144) (12, 24, 144)
tos annual mean bias is 272.039584515
an014:/home/Paul.Durack/140129_metrics/test>

Either the code should be able to deal with a deg_C -> K conversion, or I need to rewrite the obs and correct the temperature units

full path needed for parameter file

need to explain this in error message or add current path to it at load time

tos ok, but not zos nor sos... more dic probs...

Traceback (most recent call last):
File "/export/gleckler1/git/wgne-wgcm_metrics/src/python/wgne/scripts/wgne_metrics_driver.py", line 57, in
dup("ok table thing",obs_dic[var][obs_dic[var]["default"]]["CMIP_CMOR_TABLE"],"---")
KeyError: u'CNES_AVISO_L4'

Better error messages for problem user (input) data

The problem with the rewritten GFDL data was a lack of CF compliance, so incorrect dimension names (zt,yh,xh) - it would be great if such problems were dealt with by software, as CMOR/CF lack of compliance with input data is likely to be a problem that appears time and time again..

Some sanity checks, along with axes checks (so ensure that 'tos' variables are 3D, rather than 4D for e.g.) would also be a good idea.

add options to installer

@taylor13 please confimr this is right:
default should be: "cdat_lean" (cdms2+cdutil+genutil")

add option for graphics (vcs but NO GUI)

add option for cmor

custom metrics from parameters file

edit driver to look at parameter file and run custom local metrics.

Metrics completed build grown to >8Gb?

Is this expected?

public1:/home/p1d> ls | xargs du -sh
4.0G    140516_metrics/
4.0G    140701_metrics/
8.3G    140828_metrics/

add a small test script with "tos"

would be nice to run a small test script on subset of "tos" for the sake of verifying everything probably went ok

Add a new diagnostic : jet latitude

We want to add a new diagnostic on the position of the subtropical jets

tweak driver to allow non-climatology

need to come up with system to tell driver which set of metrics to use (-m/-e option?)

wgne_metrics_driver not executable

/usr/bin/env python

should be

!/usr/bin/env python

should we add option for user defined/custom metrics

so that you can have "in-house" extra metrics

add output types

possibly csv and/or database

add option to not remove uvcdat_build directory

install pytz module as default metrics package

The pytz module (I thought it came free as a standard python module) doesn't appear to be installed in our current metrics package..

It's very useful to deal with timezoning stuff, and I'm using it for global attribute creation for files.

I'd advocate we drop this package back into the metrics..

JSON output: explicitly name reference obs

Rather than 'default'

            },
            "source": "UKMETOFFICE-HadISST-v1-1"
        }
    },
    "GridInfo": {
        "GridName": "2.5x2.5",
        "GridResolution": [
            72,
            144
        ],
        "RegridMethod": "linear",
        "RegridTool": "esmf"
    },
    "References": {
        "default": {
            "CMIP_CMOR_TABLE": "Omon",
            "period": "198002-200501",
            "shape": "(12, 180, 360)"
        }
    },
    "RegionalMasking": {
        "global": null,
        "ocean": 0.0,
        "terre": 100.0
    },
    "metrics_git_sha1": "b869df6dd490ce8c7abd001c874b2839f365d4d1",
    "uvcdat_version": "esgf-v1.7.0-12-gfb2c39b"
}

Controlled Vocabulary

Try to keep as much information from IPSL controlled vocabulary as possible, see:

https://github.com/PCMDI/wgne-wgcm_metrics/wiki/Reference-Syntax-indexation

obs class do not respect Peter's structure

need to add realm/fqcy

e.g:
ocn/mo

after the user defined root path to obs

not picking up existing python

if you have already a working cdat it should use this one. Or at least have an optino to tell the installer to use this one.

Not currently building (devel branch)

*******************************
Setting up CDAT - (Python + CDMS)... master
*******************************

Fetching the cdat project from GIT Repo...
git://github.com/UV-CDAT/uvcdat.git
Initialized empty Git repository in /home/p1d/140805_metrics/WGNE/tmp/uvcdat/uvcdat/.git/
remote: Counting objects: 28673, done.
remote: Total 28673 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (28673/28673), 188.30 MiB | 62.40 MiB/s, done.
Resolving deltas: 100% (17273/17273), done.
Already on 'master'
CMAKE ARGS: cmake -DCDAT_BUILD_PARALLEL=OFF -DCMAKE_INSTALL_PREFIX=/home/p1d/140805_metrics/WGNE -DZLIB_PATCH_SRC=3 -DCDAT_DOWNLOAD_SAMPLE_DATA=OFF -DCDAT_BUILD_MODE=LEAN -DCDAT_BUILD_ESMF_ESMP=ON -DGIT_PROTOCOL=git:// /home/p1d/140805_metrics/WGNE/tmp/uvcdat/uvcdat
PATH:/home/p1d/140805_metrics/WGNE/Externals/bin:/home/gfdl/bin:/usr/local/OSoverlay/bin:/bin:/usr/bin:/usr/local/x64/bin:/usr/local/bin:/usr/local/explorer/bin:/usr/local/pgi/linux86/bin:/usr/local/cleanscape:/usr/local/toolworks/totalview/bin:/usr/local/autodesk/maya/bin:.:/home/p1d/bin
PWD:/home/p1d/140805_metrics/WGNE/tmp/uvcdat/uvcdat
-- The C compiler identification is GNU 4.4.7
-- The CXX compiler identification is GNU 4.4.7
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
CMake Error at CMakeLists.txt:16 (message):
  [ERROR] CDAT requires an out of source Build.  Run 'git clean -dfx' to
  restore source dir.


-- Configuring incomplete, errors occurred!
See also "/home/p1d/140805_metrics/WGNE/tmp/uvcdat/uvcdat/CMakeFiles/CMakeOutput.log".
 ERROR: Could not configure (cmake) cdat code (1)
~/140805_metrics/WGNE/tmp/metrics

Sorry...
This action did not complete successfully
Please try to re-run this task until successful before continuing further

 ERROR: Could not compile (make) cdat code
~/140805_metrics/WGNE/tmp/metrics

Sorry...
This action did not complete successfully
Please try to re-run this task until successful before continuing further

Unmatched ".
public1:/home/p1d>

relative path problems

This is related to CDAT/cdat#449 and CDAT/cdat#448

It seems like cleaning up those issues might solve the below:

public1:/home/p1d/140418_metrics/test> source ../WGNE/bin/setup_runtime.csh
public1:/home/p1d/140418_metrics/test> which python
../WGNE/bin/python
public1:/home/p1d/140418_metrics/test> which wgne_metrics_driver.py
../WGNE/bin/wgne_metrics_driver.py
public1:/home/p1d/140418_metrics/test> wgne_metrics_driver.py -p wgne_input_parameters_test.py
Traceback (most recent call last):
File "../WGNE/bin/wgne_metrics_driver.py", line 46, in
exec("import %s as parameters" % fnm)
File "", line 1, in
ImportError: No module named wgne_input_parameters_test
public1:/home/p1d/140418_metrics/test>

Add CMOR to standard metrics build?

As we've agreed that building udunits to attempt to most accurately deal with weird variable units is a good idea.. Maybe we should also add CMOR to the build - that way modeling groups have quite literally all the tools installed and built for them so they can rewrite their data if required before pushing through the metrics meat grinder..?

hard coded +273.15 for tas

It appears that there is a hard coded unit conversion in some of the python scripts that are running with the metrics driver. I've found the following line in '/home/Paul.Durack/140701_metrics/WGNE/lib/python2.7/site-packages/metrics/wgne/ mean_climate_metrics_calculations.py':

if var == 'tas': do_glb = do_glb + 273.13

bump metrics numpy to 1.9.x for devel branch

Considering the performance improvements we've discovered jumping from 1.7.1 to 1.9.0b1, I'd be keen to get a devel branch of the metrics using the latest numpy version.. I can then pull this across to GFDL for further testing..

I'd be reluctant to commit this to the master branch until we've verified that the numbers coming out of numpy 1.9.0b1 are reasonable, that masked arrays are being dealt with properly etc etc..

As I've now got some files generating using the mean/min/max/median/std from GFDL CM2.6, I can rerun this code with a new build and check to make sure numbers are reasonable..

Expand attributes in metrics output json file: creation date, tracking id and climfilename.nc

For each variable/json file ...

Add to both models (under "SimulationDescription") and observations the following keys (both of which can be obtained from the netCDF files...

tracking_id
creation_date
the name of the .nc file climatology

need to change json dictionary keys for obs...

For clarity, we should the following keys, only for output (json and ascii files):

"default" => "defaultReference"
"alternate => "alternateReference"
no change to "ref3", ... "refn"

double check units

As part of standard checks before runnnig metrics, check units are compatible.

Need to incorporate ability to mask land or ocean data

This needs to be done with the land sea mask of the model. A version of these for CMIP5 can be found at: /work/gleckler1/processed_data/cmip5_fixed_fields/sftlf

But I guess we also need land sea masks on a series of target grids... 5x5, 2.5x2x5, 1x1... we should build these in somehow.

This will also require new switches in the parameter file or a list[] of ocean-only and land-only variables.

Directory tweaks before more documentation is written: 'wgne' build dir becomes 'pcmdi'

It might be a good idea to consider renaming wgne-wgcm_metrics to pcmdi_metrics before things begin to gain too much more momentum..? Thoughts?

add capability to append to jsn/dict file

from parameter file we should be able to have an option to point to an existing json file and extend it.

cowardly metrics..

public1:/home/p1d/140418_metrics/test> /home/p1d/140418_metrics/WGNE/bin/python /home/p1d/140418_metrics/WGNE/bin/wgne_metrics_driver.py -p /home/p1d/140418_metrics/test/wgne_input_parameters_test.py
ref is: all
refs: [u'default']
OBS SHAPE IS (12, 72, 144)
tos GFDL-ESM2G (12, 72, 144) (12, 72, 144) default
Error while processing observation tos for variable default:
Obs and Model -GFDL-ESM2G- have different units (K vs degC) cowardly refusing to proceed

mac needs edit on "share" folder

Looks for json test file at wrong place:

doutriaux1@maryam:[~]:[14]> wgne_metrics_driver.py -p /Users/doutriaux1/WGNE/test/wgne/basic_test_parameters_file.py
Traceback (most recent call last):
  File "/Users/doutriaux1/WGNE/bin/wgne_metrics_driver.py", line 21, in <module>
    obs_dic = json.loads(open(os.path.join(sys.prefix,"share","wgne","obs_info_dictionary.json")).read())
IOError: [Errno 2] No such file or directory: '/Users/doutriaux1/WGNE/Library/Frameworks/Python.framework/Versions/2.7/share/wgne/obs_info_dictionary.json'

whereas it is really at:

doutriaux1@maryam:[~]:[14]> ll /Users/doutriaux1/WGNE/share/wgne/
total 24
-rw-r--r--  1 doutriaux1  45773    11K Jul 31 17:22 obs_info_dictionary.json

attribute error: wgne_metrics_driver.py

So this testing is on the GFDL machine:

public1:/home/p1d/140129_metrics/test> /home/p1d/140129_metrics/WGNE/bin/python /home/p1d/140129_metrics/WGNE/bin/wgne_metrics_driver.py -p /home/p1d/140129_metrics/test/wgne_input_parameters_test.py
ref is: all
refs: [u'default', u'alternate']
Traceback (most recent call last):
File "/home/p1d/140129_metrics/WGNE/bin/wgne_metrics_driver.py", line 94, in
applyCustomKeys(OUT,parameters.custom_keys,var)
AttributeError: 'module' object has no attribute 'custom_keys'
public1:/home/p1d/140129_metrics/test>

Can we set up a simple script which executes the rediculously long command line?

~~/git/WGNE/bin/python~~/git/wgne-wgcm_metrics/src/python/wgne/scripts/wgne_metrics_driver.py –p ~/git/wgne-wgcm_metrics/doc/wgne_input_parameters_test.py

Problem with input data format (GFDL)?

an102:/home/Paul.Durack/140129_metrics/test> /home/Paul.Durack/140129_metrics/WGNE/bin/python /home/Paul.Durack/140129_metrics/WGNE/bin/wgne_metrics_driver.py -p /home/Paul.Durack/140129_metrics/test/wgne_input_parameters_test.py
ref is: all
refs: [u'default']
OBS SHAPE IS (12, 72, 144)
Failed to get variable tos for version: GFDL-ESM2G, error:
'NoneType' object has no attribute 'getBounds'

pcmdi / pcmdi_metrics Goto Github PK

pcmdi_metrics's Introduction

PCMDI Metrics Package (PMP)

Documentation

Contact

Acknowledgement

License

Release Notes and History

Contributors

pcmdi_metrics's People

Contributors

Stargazers

Watchers

Forkers

pcmdi_metrics's Issues

/usr/bin/env python

!/usr/bin/env python

Recommend Projects

Recommend Topics

Recommend Org