Git Product home page Git Product logo

cesm's Introduction

The Community Earth System Model

See the CESM web site for documentation and information:

http://www.cesm.ucar.edu

The CESM Quickstart Guide is available at:

http://escomp.github.io/cesm

This repository provides tools for managing the external components that make up a CESM tag - alpha, beta and release. CESM tag creation should be coordinated through CSEG at NCAR.

This repository is also connected to slack at http://cesm2.slack.com

Software requirements

Software requirements for installing, building and running CESM

Installing, building and running CESM requires:

  • a Unix-like operating system (Linux, AIX, OS X, etc.)
  • git client version 1.8 or newer
  • subversion client (we have tested with versions 1.6.11 and newer)
  • python3 version 3.6 or newer
  • perl version 5
  • build tools gmake and cmake
  • Fortran and C compilers
  • LAPACK and BLAS libraries
  • a NetCDF library version 4.3 or newer built with the same compiler you will use for CESM
    • a PnetCDF library is optional
  • a functioning MPI environment (unless you plan to run on a single core with the CIME mpi-serial library)

Details on Fortran compiler versions

The Fortran compiler must support Fortran 2003 features. However, even among mainstream Fortran compilers that claim to support Fortran 2003, we have found numerous bugs. Thus, many compiler versions do not build or run CESM properly (see https://wiki.ucar.edu/display/ccsm/Fortran+Compiler+Bug+List for more details on older Fortran compiler versions).

CESM2 is tested on several different systems with newer Fortran compilers: Please see CESM Compiler/Machine Tests for a spreadsheet of the current results.

More details on porting CESM

For more details on porting CESM to a new machine, see http://esmci.github.io/cime/users_guide/porting-cime.html

Obtaining the full model code and associated scripting infrastructure

CESM is now released via github. You will need some familiarity with git in order to modify the code and commit these changes. However, to simply checkout and run the code, no git knowledge is required other than what is documented in the following steps.

To obtain the CESM code you need to do the following:

  1. Clone the repository. :

    git clone https://github.com/escomp/cesm.git my_cesm_sandbox

    This will create a directory my_cesm_sandbox/ in your current working directory.

  2. Go into the newly created CESM repository and determine what version of CESM you want. To see what cesm tags are available, simply issue the git tag command. :

    cd my_cesm_sandbox
    git tag
  3. Do a git checkout of the tag you want. If you want to checkout release-cesm2.1.2, you would issue the following. :

    git checkout release-cesm2.1.2

    (It is normal and expected to get a message about being in 'detached HEAD' state. For now you can ignore this, but it becomes important if you want to make changes to your Externals.cfg file and commit those changes to a branch.)

  4. Run the script manage_externals/checkout_externals. :

    ./manage_externals/checkout_externals

    The checkout_externals script is a package manager that will populate the cesm directory with the relevant versions of each of the components along with the CIME infrastructure code.

At this point you have a working version of CESM.

To see full details of how to set up a case, compile and run, see the CIME documentation at http://esmci.github.io/cime/ .

More details on checkout_externals

The file Externals.cfg in your top-level CESM directory tells checkout_externals which tag/branch of each component should be brought in to generate your sandbox. (This file serves the same purpose as SVN_EXTERNAL_DIRECTORIES when CESM was in a subversion repository.)

NOTE: Just like svn externals, checkout_externals will always attempt to make the working copy exactly match the externals description. For example, if you manually modify an external without updating Externals.cfg, (e.g. switch to a different tag), then rerunning checkout_externals will automatically restore the externals described in Externals.cfg. See below documentation Customizing your CESM sandbox for more details.

You need to rerun checkout_externals whenever Externals.cfg has changed (unless you have already manually updated the relevant external(s) to have the correct branch/tag checked out). Common times when this is needed are:

  • After checking out a new CESM branch/tag
  • After merging some other CESM branch/tag into your currently checked-out branch

checkout_externals must be run from the root of the source tree. For example, if you cloned CESM with:

git clone https://github.com/escomp/cesm.git my_cesm_sandbox

then you must run checkout_externals from /path/to/my_cesm_sandbox.

To see more details of checkout_externals, issue :

./manage_externals/checkout_externals --help

Customizing your CESM sandbox

There are several use cases to consider when you want to customize or modify your CESM sandbox.

Switching to a different CESM tag

If you have already checked out a tag and HAVE NOT MADE ANY MODIFICATIONS it is simple to change your sandbox. Say that you checked out release-cesm2.1.2 but really wanted to have release-cesm2.1.3; you would simply do the following:

git checkout release-cesm2.1.3
./manage_externals/checkout_externals

You should not use this method if you have made any source code changes, or if you have any ongoing CESM cases that were created from this sandbox. In these cases, it is often easiest to do a second git clone.

Pointing to a different version of a component

Each entry in Externals.cfg has the following form (we use CAM as an example below):

[cam]
tag = trunk_tags/cam5_4_143/components/cam
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/cam1
local_path = components/cam
required = True

Each entry specifies either a tag or a branch. To point to a new tag:

  1. Modify the relevant entry/entries in Externals.cfg (e.g., changing cam5_4_143 to cam5_4_144 above)
  2. Checkout the new component(s):

    ./manage_externals/checkout_externals

Keep in mind that changing individual components from a tag may result in an invalid model (won't compile, won't run, not scientifically meaningful) and is unsupported.

Committing your change to Externals.cfg ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

After making this change, it's a good idea to commit the change in your local CESM git repository. First create a CESM branch in your local repository, then commit it. (Unlike with subversion, branches are stored locally unless you explicitly push them up to github. Feel free to create whatever local branches you'd like.) For example:

git checkout -b my_cesm_branch
git add Externals.cfg
git commit -m "Update CAM to cam5_4_144"

Modifying a component

If you'd like to modify a component via a branch and point to that branch in your CESM sandbox, use the following procedure (again, using CAM as an example):

  1. Create a CAM branch. Since CAM originates from a subversion repository, you will first need to create a branch in that repository. Let's assume you have created this branch and called it my_branch.
  2. Update Externals.cfg to point to your branch. You can replace the tag entry with a branch entry, as follows:

    [cam]
    branch = branches/my_branch/components/cam
    protocol = svn
    repo_url = https://svn-ccsm-models.cgd.ucar.edu/cam1
    local_path = components/cam
    required = True
  3. Checkout your branch:

    ./manage_externals/checkout_externals

It's a good idea to commit your Externals.cfg file changes. See the above documentation, Committing your change to Externals.cfg.

Developer setup

Developers who have not already done so should follow the recommended one-time setup directions for git. Developers may also want to set up ssh keys and switch to using the [email protected]:ESCOMP/cesm.git form of the github URLs.

cesm's People

Contributors

allibco avatar alperaltuntas avatar bandre-ucar avatar bertinia avatar billsacks avatar bjandre avatar cacraigucar avatar ekluzek avatar fischer-ncar avatar fvitt avatar jedwards4b avatar katetc avatar mnlevy1981 avatar mvertens avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cesm's Issues

release-clm5.0.01 seems to be a "private repository"

When I try to build a CAM6 from the released CESM2_0,
or update the components using
$> ./manage_externals/checkout_externals it fails when dealing with CLM
(https://github.com/ESCOMP/ctsm:release-clm5.0.01)
with the message:

Checking status of externals: clm, ERROR:root:
subprocess call to 'git' has exceeded timeout limit of 300 seconds.
Timout errors typically occur when svn or git requires
authentication to access a private repository. ...
When running case.build, I can add --skip-provenance-check
and it builds. But I'm not aware of a way to do a similar thing
when running checkout_externals. And it seems like such a
kluge shouldn't be necessary.
The directory where this is happening is
/gpfs/fs1/work/raeder/Models/cesm2_0,
which has a few untracked files, but nothing added or committed.

Add to quick start documentation: once you create a case, do not change the sandbox

A seemingly unintuitive aspect of CESM/cime is that, once you have created a case from a particular directory, it is very important not to change the code checked out in that directory until you are completely done with your case. Is that documented somewhere? I don't see it in the quick start guide, and that seems like a good place to include a warning about this. Maybe we can have a section at the end like "Important points to bear in mind"?

Need a way to determine what release of checkout_externals was used in a specific CESM tag

Summary of Issue:

External components which maintain standalone capability need to maintain the checkout_externals version. Typically for CAM's case we take a previously tested CESM tag and duplicate the externals used in it. This is now being extended to needing to also maintain the version of checkout_externals. We want to identify and use the version of checkout_externals that was used in a specific CESM tag. Being able to check this easily, becomes more important as time goes on and checkout_externals is only updated sporadically - we need an easy way to determine that it has been updated.

Expected behavior and actual behavior:

An ideal solution would be for the plans page to have a location somewhere on each tag's page which indicates the checkout_externals which was used. Perhaps this could be populated (automatically?) when checkout_externals is brought into the CESM tag. Currently, there is no good way to determine the checkout_externals tag other than cloning cesm and mining the log.

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

N/A

What is the changeset ID of the code, and the machine you are using:

N/A

have you modified the code? If so, it must be committed and available for testing:

N/A

Screen output or log file showing the error message and context:

N/A

@bjandre suggested that I bring @mvertens , @fischer-ncar , @bertinia and possibly @jedwards4b into this discussion.

Need to update two REFCASES

Summary of Issue:

nanr noticed that B1850 points to the wrong REFCASE

I set up the experiment database for the standard B1850 case, setting:
RUN_REFCASE = b.e21.B1850.f09_g17.CMIP6-piControl.001,

but my case is coming out of the box with this as my refcase: RUN_REFCASE=b.e20.B1850.f09_g17.pi_control.all.299_merge_v2

Similarly the BW1850 compset is pointing to an older REFCASE as well. All of the other compsets point to b.e21 REFCASES

Expected behavior and actual behavior:

REFCASE should be b.e21.B1850.f09_g17.CMIP6-piControl.001
but is: b.e20.B1850.f09_g17.pi_control.all.299_merge_v2

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

SMS_Ln1.f09_g17.B1850.cheyenne_intel.allactive-defaultio> ./xmlquery RUN_REFCASE,RUN_REFDATE         
Results in group run_begin_stop_restart
	RUN_REFCASE: b.e20.B1850.f09_g17.pi_control.all.299_merge_v3
	RUN_REFDATE: 0134-01-01

And for BWHIST:

scripts/SMS_Ln1.f09_g17.BW1850.cheyenne_intel.allactive-defaultio.20190322_143108_50jbax> ./xmlquery RUN_REFCASE,RUN_REFDATE        
Results in group run_begin_stop_restart
	RUN_REFCASE: b.e20.B1850.f09_g17.pi_control.all.299_merge_v3waccm
	RUN_REFDATE: 0134-01-01
### What is the changeset ID of the code, and the machine you are using: cesm2.1.1-exp13

have you modified the code? If so, it must be committed and available for testing: no

Remove allactive/default testmods directory

As @fischer-ncar and I noticed, the allactive/default testmods directory just contains a bunch of empty files. We should remove this testmods directory to avoid confusion. Then we'll need to (1) change tests that use allactive/default to instead have no testmod, and (2) remove references to this from include_user_mods files in other allactive testmods.

@fischer-ncar - #84 started with the point of doing this, but then I realized that it would change test names and so break some baseline comparisons, so I thought it would be best to leave this to you to do at a convenient time.

./manage_externals/checkout_externals script failing

Summary of Issue:

The script fails with

Checking status of externals: clm, mosart, ww3, cime, cice, pop, cism, rtm, cam,
Checking out externals: clm, mosart, ww3, ERROR:root:Command '[u'svn', u'checkout', u'--quiet', u'https://svn-ccsm-models.cgd.ucar.edu/ww3/release_tags/ww3_cesm2_0_rel_01', u'/home/mboisson/.local/easybuild/software/2017/Core/cesm/2.0.0/components/ww3']' returned non-zero exit status 1
ERROR:root:Failed with output:
    svn: E170013: Unable to connect to a repository at URL 'https://svn-ccsm-models.cgd.ucar.edu/ww3/release_tags/ww3_cesm2_0_rel_01'
    svn: E230001: Server SSL certificate verification failed: issuer is not trusted

ERROR: In directory
    /home/mboisson/.local/easybuild/software/2017/Core/cesm/2.0.0
Process did not run successfully; returned status 1:
    svn checkout --quiet https://svn-ccsm-models.cgd.ucar.edu/ww3/release_tags/ww3_cesm2_0_rel_01 /home/mboisson/.local/easybuild/software/2017/Core/cesm/2.0.0/components/ww3
See above for output from failed command.
Please check the log file manage_externals.log for more details.

ERROR: Failed with output:
    svn: E170013: Unable to connect to a repository at URL 'https://svn-ccsm-models.cgd.ucar.edu/ww3/release_tags/ww3_cesm2_0_rel_01'
    svn: E230001: Server SSL certificate verification failed: issuer is not trusted

ERROR: In directory
    /home/mboisson/.local/easybuild/software/2017/Core/cesm/2.0.0
Process did not run successfully; returned status 1:
    svn checkout --quiet https://svn-ccsm-models.cgd.ucar.edu/ww3/release_tags/ww3_cesm2_0_rel_01 /home/mboisson/.local/easybuild/software/2017/Core/cesm/2.0.0/components/ww3
See above for output from failed command.
Please check the log file manage_externals.log for more details.``` 

Add Code of Conduct to CESM

Summary of Issue: Add a code of conduct to CESM

We need to add a Code of Conduct (COC) to the CESM project. Having a COC is a github best practice. And it's important for NSF funding. A COC helps maintain diversity and inclusion in a project as well as making the project as a whole more productive.

The COC needs to go both on master and cesm2.1 release since both of those branches continue to be maintained going forward.

Bug in --status regex

Summary of Issue: The RE_TRACKING reg ex in repository_git.py appears to be too restrictive.

Expected behavior and actual behavior:I have a clm checkout that looks like

git status
On branch pio2_cleanup
Your branch and 'mydev/pio2_cleanup' have diverged,
and have 62 and 2 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   src/main/ncdio_pio.F90.in

no changes added to commit (use "git add" and/or "git commit -a")

Current behavior is:

Processing externals description file : Externals.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_POP.cfg
Processing externals description file : Externals_CISM.cfg
Checking status of externals: clm, 
ERROR: DEV_ERROR: regex to detect tracking branch failed.

If I changes the RE_TRACKING to be RE_TRACKING = re.compile(r'[(.+)]')
then I get what I think that I should expect:

Processing externals description file : Externals.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_POP.cfg
Processing externals description file : Externals_CISM.cfg
Checking status of externals: clm, fates, ptclm, mosart, ww3, cime, cice, pop, cvmix, marbl, cism, source_cism, rtm, cam, 
sM  ./cime
        modified sandbox, cesmdev/master: ahead 136 --> cime5.4.0-alpha.23
 M  ./components/cam
        modified sandbox, on cam1/trunk_tags/cam5_4_166/components/cam
s   ./components/cice
        clean sandbox, origin/master --> cice5_20180123
    ./components/cism
        clean sandbox, on cism2_1_46
    ./components/cism/glimmer-cism
        clean sandbox, on test_coupling_move_up_v2_n01
sM  ./components/clm
        modified sandbox, mydev/pio2_cleanup: ahead 62, behind 2 --> clm5.0.dev001
    ./components/clm/src/fates
        clean sandbox, on fates_s1.4.1_a3.0.0_rev3
s   ./components/clm/tools/PTCLM
        clean sandbox, PTCLM2_171216c --> PTCLM2_180214
    ./components/mosart
        clean sandbox, on mosart1_0_30
    ./components/pop
        clean sandbox, on pop2/trunk_tags/cesm_pop_2_1_20180205
    ./components/pop/externals/CVMix
        clean sandbox, on v0.72-conv_bugfix.002
    ./components/pop/externals/MARBL
        clean sandbox, on MARBL/trunk_tags/MARBL0_22_0
    ./components/rtm
        clean sandbox, on rtm1_0_65
    ./components/ww3
        clean sandbox, on ww3/trunk_tags/ww3_180115

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

What is the changeset ID of the code, and the machine you are using:

have you modified the code? If so, it must be committed and available for testing:

Screen output or log file showing the error message and context:

Document ssh vs https for git repositories

Summary of Issue:

By default cesm is pointing to https urls for git repositories. This is fine for reading public repositories, but requires the user to input their github password many times for private repos.

Expected behavior and actual behavior:

Document the recommended use and setup of ssh keys for accessing private forks.

Updates to output file, naming conventions

http://www.cesm.ucar.edu/models/cesm2/naming_conventions.html#modelOutputFilenames
says that

a new metadata field called time_period_freq is now written to every history file header with one of the corresponding values:

second_N... N ∈ {1..3599} - data are written every N seconds

...
But this is not true for the cpl.hi. files, which seem to contain
instantaneous values.

It would also be helpful to have either a description of the meaning
of the standard characters that follow .h, (a,i,0-9,...) or pointers to that information,
like
http://www.cesm.ucar.edu/models/cesm2/component_settings/cam_nml.html:fincl1
for the first CAM history file.
Similarly, it would be helpful to update the driver namelist documentation
http://www.cesm.ucar.edu/models/cesm2/component_settings/drv_nml.html:history_option
to note that the output will appear in a .hi. file.
Similarly for the cpl.ha* files.

I wouldn't mind writing some of that documentation,
but I can't find the source code for pages like these.
They are not in the CESM2_0 release.
Thanks,
Kevin

Add status writes to manage_externals

Going from a git or svn checkout where the screen is inundated with every file checked out, to manage_externals where it silently sits there for minutes can be disconcerting as you are not sure if the command is making progress or not. A simple "starting to check out XXX", "completed check out of XXX" for each of the components would be useful so the user can make sure progress is actually being made. I ended up opening a new window and seeing that something was happening just to make sure progress was being made and it wasn't simply hung.

Change CESM.cfg to be alphabetical order

Would it be possible to store the CESM.cfg file components in alphabetical order? This
would help with the testdb display when comparing the component info stored in the database
with what is in the CESM.cfg file. Here's what is displayed from the database:

[cam]
tag = trunk_tags/cam5_4_143/components/cam
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/cam1
local_path = components/cam
required = True

[cice]
tag = cice5_20170927
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/cice/trunk_tags/cice5_20170927
local_path = components/cice
required = True

[cime]
tag = cime5.4.0-alpha.11
protocol = git
repo_url = https://github.com/ESMCI/cime/releases/tag/cime5.4.0-alpha.11
local_path = cime
required = True

[cime_config]
tag = cime_config0.1.0.alpha.05
protocol = git
repo_url = https://github.com/CESM-Development/cime_config/releases/tag/cime_config0.1.0.alpha.05
local_path = cime_config
required = True

[cism]
tag = trunk_tags/cism2_1_40
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/glc
local_path = components/cism
required = True

[clm]
tag =
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/clm2
local_path = components/clm
required = True

[mosart]
tag = branch_tags/mosart1_0_26_cimeupdates_tags/n01
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/mosart
local_path = components/mosart
required = True

[pop]
tag = cesm_pop_2_1_20171117
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/pop2/trunk_tags/cesm_pop_2_1_20171117
local_path = components/pop
required = True

[rtm]
tag = branch_tags/rtm1_0_62_cimeupdate_tags/n01
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/rivrtm
local_path = components/rtm
required = True

[ww3]
tag = trunk_tags/ww3_170731
protocol = svn
repo_url = https://svn-ccsm-models.cgd.ucar.edu/ww3
local_path = components/ww3
required = True

[externals_description]
schema_version = 1.0.0

Python 2 end of life

In the README, I read that some components in CESM aren't Python 3 compliant.

In less than one month, Python 2 will reach it's end of life.
That means that even if security bugs appear, that that will not be solved.

Also, many numerical packages like Numpy and Scipy will no longer be maintained.

It is probably a good time to upgrade the Python 2 only code 👍

Extensions to 2300 for SSP5-3.4 overshoot and SSP5-8.5

Summary of Issue:

We need to have a tag with updated CTSM to run from 2100-2300 for the SSP5 scenarios (SSP5-3.4, and SSP5-8.5). This will only be running with WACCM.

This will be on the CESM2.1. release series.

The CTSM tag needed for this is documented in this issue:
ESCOMP/CTSM#944

It likely will also need a CAM tag.

Future scenario period names in compset longnames need to change

Summary of Issue:

CMIP5 future scenario names which are currently in compset longnames are based on the four standard RCP's: 8.5, 6.0, 4.5, 2.6. The CMIP6 future scenarios are SSP (Shared Socioeconomic Pathway) RCP (Representative concentration Pathway) combinations, usually expressed as SSP#-#.# for the SSP number, and RCP concentration.

Expected behavior and actual behavior:

The only recognized future scenario's are RCP[2468]. We need something like SSP[12345][123468], which is a minimum of five digits for it rather than the usual four.

The entire list is:

SSP5-RCP8.5
SSP1-RCP2.6
SSP3-RCP7.0
SSP5-RCP3.4
SSP2-RCP4.5
SSP1-RCP1.9
SSP4-RCP3.4
SSP4-RCP6.0

CESM2.2 series needs to point to an updated CMEPS/CDEPS

Summary of Issue:

cesm2_2_beta04 points to 471ac52 for CMEPS (in Externals_cime.cfg). At least some cases built with this can't even create the case.

Note that CTSM is using the CMEPS hash: 253f612acae07b2b1dc73c84f1bb30b8e1b86ddd which is functional for the following case.

Expected behavior and actual behavior: Expect the case to PASS, instead it FAILS as case create as below...

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

on cheyenne
qcmd -- ./create_test SMS_D_Ld5_Vnuopc.f10_f10_musgs.I2000Clm50BgcCropGs.cheyenne_intel.clm-default -r .

What is the changeset ID of the code, and the machine you are using: ctsm1.0.dev092 (using cime5.8.19)

have you modified the code? If so, it must be committed and available for testing: No

Screen output or log file showing the error message and context:

.
.
.

Pes setting: tasks is {'NTASKS_ATM': -2, 'NTASKS_ICE': -2, 'NTASKS_CPL': -2, 'NTASKS_LND': -2, 'NTASKS_WAV': -2, 'NTASKS_ROF': -2, 'NTASKS_OCN': -2, 'NTASKS_GLC': -2}
Pes setting: threads is {'NTHRDS_ICE': 1, 'NTHRDS_ATM': 1, 'NTHRDS_ROF': 1, 'NTHRDS_LND': 1, 'NTHRDS_WAV': 1, 'NTHRDS_OCN': 1, 'NTHRDS_CPL': 1, 'NTHRDS_GLC': 1}
Pes setting: rootpe is {'ROOTPE_OCN': 0, 'ROOTPE_LND': 0, 'ROOTPE_ATM': 0, 'ROOTPE_ICE': 0, 'ROOTPE_WAV': 0, 'ROOTPE_CPL': 0, 'ROOTPE_ROF': 0, 'ROOTPE_GLC': 0}
Pes setting: pstrid is {}
Pes other settings: {}
Pes comments: none
Compset is: 2000_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_MOSART_SGLC_SWAV_SESP
Grid is: a%10x15_l%10x15_oi%null_r%r05_g%null_w%null_z%null_m%usgs
Components in compset are: ['datm', 'clm', 'sice', 'socn', 'mosart', 'sglc', 'swav', 'sesp', 'drv', 'dart']
No charge_account info available, using value from PROJECT
cesm model version found: ctsm1.0.dev091-9-g1f5afc67
Batch_system_type is pbs
ERROR: Looks like job groups have already been created


Document required git (and svn?) versions?

@cecilehannay ran into trouble trying to checkout CESM using the instructions when using yellowstone. It turns out that the out-of-the-box git version on yellowstone is 1.7.1, but there are issues trying to fetch cime with this old git version. This is not related to manage_externals: even doing a manual git clone / git fetch leads to this issue:

First, when cloning:

$ git clone https://github.com/CESM-Development/cime
Initialized empty Git repository in /glade/p/work/sacks/temporary/cime/.git/
remote: Counting objects: 87692, done.
remote: Total 87692 (delta 0), reused 0 (delta 0), pack-reused 87692
Receiving objects: 100% (87692/87692), 33.18 MiB | 18.91 MiB/s, done.
Resolving deltas: 100% (64250/64250), done.
error: refs/tags/cime0_1_0 does not point to a valid object!
error: refs/tags/cime0.3.2 does not point to a valid object!
error: refs/tags/cime0.3.1 does not point to a valid object!
error: refs/tags/cime0.2.3 does not point to a valid object!
error: refs/tags/cime0.2.2 does not point to a valid object!
error: refs/tags/cime0.2.1 does not point to a valid object!
error: refs/tags/cime0.2.0 does not point to a valid object!

Then, when fetching:

$ git fetch origin
remote: Counting objects: 14, done.
remote: Total 14 (delta 1), reused 1 (delta 1), pack-reused 13
Unpacking objects: 100% (14/14), done.
error: unable to find 5fef7ed2dc848bda2aa3c106a77ba0a008ca858f
fatal: object 5fef7ed2dc848bda2aa3c106a77ba0a008ca858f not found

The error in fetching leads checkout_externals to abort.

yellowstone will be going away before this git-based version sees wide use. But should we document somewhere that a newer git version is needed? Version 1.8.3.1 (on hobart) works for me, so it appears that we need version 1.8 or later. Where should this documentation go?

I'm not sure if there are requirements for svn, or if any reasonable svn version should work.

checkout_externals should have include/exclude options

Summary of Issue: checkout_externals currently wants to update all of the externals in the sandbox, but often I want to update only some of the components, or there is a particular component I do not want to update. I think that this can be easily handled with --include and --exclude options to checkout_externals

Expected behavior and actual behavior: only those externals in the include list (or not in the exclude list) would be updated.

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

What is the changeset ID of the code, and the machine you are using:

have you modified the code? If so, it must be committed and available for testing:

Screen output or log file showing the error message and context:

update Quickstart with release tag clone command

The Downloading the CESM2 Code page includes
clone instruction with a generic branch name "release2.0". The idea is to have
the release2.0 branch point to the most current, officially supported code base.
For example, the initial release2.0 branch would point to CESM2.0.0 tag; when
CESM2.0.1 tag is available, the release2.0 branch would be updated in the repo BUT
the documentation could remain the same.

This will require git describe instructions be added for how to check the code version in
a users cloned sandbox.

Get rid of ">" in quick start instructions?

I feel that the ">" at the start of shell commands in the quick start instructions is more troublesome than helpful, because it means you can't copy and paste commands. At least one other person (I think it was Cecile Hannay, but I may be wrong) felt the same way. This is a low-priority cleanup fix, but would be nice to do if others agree.

@bertinia are you attached to these ">" characters?

CESM2.1.0 qualifiers too specific

Summary of Issue:

In CESM2.1.0, some refcase qualifiers (e.g., for RUN_REFCASE) were introduced which are possibly too specific about the compset match. While these items are currently working, they will not work with a proposed CIME modification (see ESMCI/cime#3055).

Expected behavior and actual behavior:

Problem: entries assume no components after WW3 (or any WAV component).
Expect: more general patterns

What is the changeset ID of the code, and the machine you are using:

cesm2.1.0

have you modified the code? If so, it must be committed and available for testing:

no

Suggested change

I think we can change these (24) entries from:

compset="1850_CAM60_CLM50%BGC-CROP.*_CICE.*_POP2%ECO.*_MOSART_CISM2%NOEVOLVE_WW3_BGC%BDRD"

to

compset="1850_CAM60_CLM50%BGC-CROP.*_CICE.*_POP2%ECO.*_MOSART_CISM2%NOEVOLVE_WW3.*_BGC%BDRD"

I ran a test with this substitution with and without my proposed CIME change (which added _SESP after WW3). I think making this simple change will allow us to then update CIME to a version with the new compset scheme.
Thoughts? @klindsay28? @jedwards4b? @billsacks? @mvertens?

Add a script that prints version information

An important piece of information that we request via the forums is the model version, from running git describe and manage_externals/checkout_externals --status --verbose. But I have been finding that users run these from the wrong place or run them incorrectly (e.g., running checkout_externals --status and then separately checkout_externals --verbose.

I'd like to make it easier for users to provide the correct information (and easier for us to ensure that users have provided the correct information) by adding a simple python script in the top level of CESM called something like describe_version. This script (which would not accept any arguments, other than --help) would:

  • change to the directory where the script resides (to ensure that the following commands work properly)
  • run git describe
  • run manage_externals/checkout_externals --status --verbose

I'd like to add this to both master and the cesm2.1 release branches.

@jedwards4b @mvertens does this make sense to you? Let me know if you have any thoughts on the details.

Remove BWSSP126extcmip6 since SSP 2100-01-01 is missing

Summary of Issue:

The 2101-01-01 restart is missing for
so the BWSSP126extcmip6 compset fails.

Expected behavior and actual behavior:

Expect compset test to run and PASS, it dies at case creation.

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

./create_test SMS_Ld1.f09_g17.BWSSP126extcmip6.cheyenne_intel.allactive-default

What is the changeset ID of the code, and the machine you are using: cesm2.1.4-rc.01

have you modified the code? If so, it must be committed and available for testing: N

Screen output or log file showing the error message and context:

    Errors were:
        Building test for SMS in directory /glade/scratch/erik/cesm2.1.4-rc.01/cime/scripts/SMS_Ld1.f09_g17.BWSSP126extcmip6.cheyenne_intel.allactive-default.20200406_134811_in66vc
        Refcase not found in /glade/p/cesmdata/cseg/inputdata/cesm2_init/b.e21.BWSSP126cmip6.f09_g17.CMIP6-SSP1-2.6-WACCM.001/2101-01-01, will attempt to download from inputdata
        Client protocol gftp not enabled
          Model refcase missing file refdir = '/glade/p/cesmdata/cseg/inputdata/cesm2_init/b.e21.BWSSP126cmip6.f09_g17.CMIP6-SSP1-2.6-WACCM.001/2101-01-01/'
        wget failed with output:  and errput --2020-04-06 13:48:18--  ftp://ftp.cgd.ucar.edu/cesm/inputdata/cesm2_init/b.e21.BWSSP126cmip6.f09_g17.CMIP6-SSP1-2.6-WACCM.001/2101-01-01/
                   => ‘.listing’
        Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.23.220
        Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.23.220|:21... connected.
        Logging in as anonymous ... Logged in!
        ==> SYST ... done.    ==> PWD ... done.
        ==> TYPE I ... done.  ==> CWD (1) /cesm/inputdata/cesm2_init/b.e21.BWSSP126cmip6.f09_g17.CMIP6-SSP1-2.6-WACCM.001/2101-01-01 ... 
        No such directory ‘cesm/inputdata/cesm2_init/b.e21.BWSSP126cmip6.f09_g17.CMIP6-SSP1-2.6-WACCM.001/2101-01-01’.
        
          Model refcase missing file refdir = '/glade/p/cesmdata/cseg/inputdata/cesm2_init/b.e21.BWSSP126cmip6.f09_g17.CMIP6-SSP1-2.6-WACCM.001/2101-01-01/'
          Model refcase missing file refdir = '/glade/p/cesmdata/cseg/inputdata/cesm2_init/b.e21.BWSSP126cmip6.f09_g17.CMIP6-SSP1-2.6-WACCM.001/2101-01-01/'
        svn export failed with output:  and errput svn: E170000: URL 'https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/cesm2_init/b.e21.BWSSP126cmip6.f09_g17.CMIP6-SSP1-2.6-WACCM.001/2101-01-01' doesn't exist
        
        Client protocol None not enabled
        Exception during build:
        ERROR: Could not download refcase from any serve

Update J1850G compset

The current J1850G compset is set to:
1850_DATM%CRU_CLM50%BGC-CROP_CICE_POP2_MOSART_CISM2%EVOLVE_SWAV

But it should be:
1850_DATM%CRU_CLM50%BGC-CROP_CICE_POP2%ECO_MOSART_CISM2%EVOLVE_WW3_BGC%BDRD

Also, running a quick ERS test on the J1850G compset with the resolution used in the glacier spin-up runs results in these errors:

    Model datm missing file file1 = '/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.cruncep_qianFill.0.5d.V4.c130305/Precip6Hrly/clmforc.cruncep.V4.c2011.0.5d.Prec.1901-01.nc'
        Cannot download file since it lives outside of the input_data_root '/glade/p/cesmdata/cseg/inputdata'
    Model datm missing file file2 = '/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.cruncep_qianFill.0.5d.V4.c130305/Precip6Hrly/clmforc.cruncep.V4.c2011.0.5d.Prec.1901-02.nc'
        Cannot download file since it lives outside of the input_data_root '/glade/p/cesmdata/cseg/inputdata'
    Model datm missing file file3 = '/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.cruncep_qianFill.0.5d.V4.c130305/Precip6Hrly/clmforc.cruncep.V4.c2011.0.5d.Prec.1901-03.nc'
        Cannot download file since it lives outside of the input_data_root '/glade/p/cesmdata/cseg/inputdata'

This error goes on 240 times. Also seems to be some errors about missing '/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.cruncep_qianFill.0.5d.V4.c130305/Solar6Hrly/clmforc.cruncep.V4.c2011.0.5d.Solr.1917-07.nc' and possibly others. So, maybe the forcing data needs to be updated for this compset as well?

Test

Please ignore this issue - testing connection to slack

New interpolated REFCASES needed

Summary of Issue:

For cesm2.1.1-rc.02 which updates clm fsurdat files the REFCASES for B compsets need to be updated.

Expected behavior and actual behavior:

Expect cases to run out of the box, however, they fail in trying to initialize CLM

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

These tests fail:

SMS_Ln1.f09_f09_mg17.FHIST_BGC.cheyenne_intel.cam-outfrq9s.20190314_115153_xlpcd1
SMS_Ln1.f09_g17.B1850.cheyenne_intel.allactive-defaultio.20190314_115153_xlpcd1
SMS_Ln1.f09_g17.B1PCTcmip6.cheyenne_intel.allactive-defaultio_min.20190314_115153_xlpcd1

What is the changeset ID of the code, and the machine you are using: cesm2.1.1-rc.01-25-ga88632e

have you modified the code? If so, it must be committed and available for testing: no

Screen output or log file showing the error message and context:

For example for SMS_Ln1.f09_f09_mg17.FHIST_BGC.cheyenne_intel.cam-outfrq9s

(GETFIL): attempting to find local file
b.e20.BHIST.f09_g17.20thC.297_01_v2.clm2.r.1979-01-01-00000.nc
(GETFIL): using b.e20.BHIST.f09_g17.20thC.297_01_v2.clm2.r.1979-01-01-00000.nc
in current working directory
Reading restart dataset
check_dim ERROR: mismatch of input dimension 62034 with expected value
62100 for variable landunit
Did you mean to set use_init_interp = .true. in user_nl_clm?
(Setting use_init_interp = .true. is needed when doing a
transient run using an initial conditions file from a non-transient run,
or a non-transient run using an initial conditions file from a transient run,
or when running a resolution or configuration that differs from the initial conditions.)
ERROR:
ERROR in /gpfs/fs1/scratch/erik/cesm2.1.1-rc.02/components/clm/src/main/ncdio_p
io.F90.in at line 368

CRU forcing for JG compset needs to use v7 version

Summary of Issue:

The J1850G is pointing to the older CRUNCEP dataset rather than the v7 version that is now being used. We've removed the V4 version.

Expected behavior and actual behavior:

J1850G points to the older CRUNCEPV4 version rather than the v7 version

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

Setup the following test:

SMS_Ld5.f09_g17_gl4.J1850G.cheyenne_gnu.allactive-cism-test_coupling

Here's the fix that needs to be done:

diff --git a/cime_config/config_compsets.xml b/cime_config/config_compsets.xml
index d8a5b70..156b0e0 100644
--- a/cime_config/config_compsets.xml
+++ b/cime_config/config_compsets.xml
@@ -145,7 +145,7 @@
 
   <compset>
      <alias>J1850G</alias>
-     <lname>1850_DATM%CRU_CLM50%BGC-CROP_CICE_POP2_MOSART_CISM2%EVOLVE_SWAV</lname>
+     <lname>1850_DATM%CRUv7_CLM50%BGC-CROP_CICE_POP2_MOSART_CISM2%EVOLVE_SWAV</lname>
   </compset>
 
   <entries
```>
### What is the changeset ID of the code, and the machine you are using: cesm2_2_alpha04g
### have you modified the code? If so, it must be committed and available for testing: No
### Screen output or log file showing the error message and context:

check input data shows data is missing.

> ./check_input_data
> .
> .
> .
>   Model datm missing file file239 = '/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.cruncep_qianFill.0.5d.V4.c130305/TPHWL6Hrly/clmforc.cruncep.V4.c2011.0.5d.TPQWL.1920-11.nc'
>   Model datm missing file file240 = '/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.cruncep_qianFill.0.5d.V4.c130305/TPHWL6Hrly/clmforc.cruncep.V4.c2011.0.5d.TPQWL.1920-12.nc'

Technically we could install the V4 data, but we are using the V7 data and want to be using the latest data rather than older versions. The older version takes a TByte of data and we don't want to waste the disk space.

Problems running manage_externals when updating along the cesm2.0 release branch

Summary of Issue:

I had a clone of the cesm repo, with all externals checked out via manage_externals (I'm not sure what version of cesm I was at originally - sorry). Then I updated to the latest version of master (2c75ae5). When I reran ./manage_externals/checkout_externals, I got errors like this:

Checking out externals: clm, mosart, ww3, cime, cice, pop, ERROR:root:Command '[u'svn', u'switch', u'--quiet', u'https://svn-ccsm-models.cgd.ucar.edu/pop2/release_tags/pop2_cesm2_0_rel_n02']' returned non-zero exit status 1
ERROR:root:Failed with output:
    svn: E195012: Path '.' does not share common version control ancestry with the requested switch location.  Use --ignore-ancestry to disable this check.
    svn: E195012: 'https://svn-ccsm-models.cgd.ucar.edu/pop2/release_tags/pop2_cesm2_0_rel_n02' shares no common ancestry with '/Users/sacks/cesm_code/cesm/components/pop'

ERROR: In directory
    /Users/sacks/cesm_code/cesm/components/pop
Process did not run successfully; returned status 1:
    svn switch --quiet https://svn-ccsm-models.cgd.ucar.edu/pop2/release_tags/pop2_cesm2_0_rel_n02
See above for output from failed command.
Please check the log file manage_externals.log for more details.

ERROR: Failed with output:
    svn: E195012: Path '.' does not share common version control ancestry with the requested switch location.  Use --ignore-ancestry to disable this check.
    svn: E195012: 'https://svn-ccsm-models.cgd.ucar.edu/pop2/release_tags/pop2_cesm2_0_rel_n02' shares no common ancestry with '/Users/sacks/cesm_code/cesm/components/pop'

ERROR: In directory
    /Users/sacks/cesm_code/cesm/components/pop
Process did not run successfully; returned status 1:
    svn switch --quiet https://svn-ccsm-models.cgd.ucar.edu/pop2/release_tags/pop2_cesm2_0_rel_n02
See above for output from failed command.
Please check the log file manage_externals.log for more details.

I worked around this by doing rm -rf components/pop then rerunning ./manage_externals/checkout_externals. This gave a similar error for cam, which I again worked around with rm -rf components/cam. Then it worked.

Should we fold cime_config into this repository?

I feel like some of us discussed this before, but I can't remember the outcome: Would it make sense to fold https://github.com/cesm-development/cime_config into this new cesm repository? My understanding of that cime_config repository is that it serves the same role for a full cesm checkout that individual components' cime_config serve for those individual components. In particular: my impression is that https://github.com/cesm-development/cime_config is only needed for all-active cases, so there shouldn't be a need for (e.g.) CTSM or CAM standalone checkouts to include it.

The advantage of folding that in is that we'd have one less repository for which we need to manage issues, etc.

@mvertens @jedwards4b @fischer-ncar or others: Any thoughts on this?

notifications to CESM developers when new tags are available

Summary of Issue:

Currently, when new tags are created on the NCAR SVN server, an automated email is sent to
a couple of different email lists; cgd-cseg for alpha tags and cesm-dev mailman account for
beta and release tags. The automated email is sent from a perl script that runs as a nightly cron job on the NCAR SVN server.

How do we want to handle tag update notifications from this github repo? Should we encourage users and developers to setup a personal github account and manage their notifications /watches or do we need a script that sends notifications similar to what we currently have for SVN?

Quick start documentation minor issues

I have given the quick start documentation a quick read. Overall, it's very nice - thank you @bertinia for all of your work on this.

I noticed a few minor issues:

  • Directory structure of the source: I don't think the source (on master) should have subdirectories for the different versions. A given version of master will only contain a single version of the documentation. e.g., the cesm2.0 release branch will just contain documentation applicable to cesm2.0, the cesm2.1 release branch just documentation for cesm2.1, etc. It's just the build for which we want to have separate directories for the separate versions on a single branch.

  • Formatting of code blocks: by default, sphinx renders code blocks in python, I believe. This leads to syntax highlighting that is not usually appropriate. You can specify the language (including shell) prior to the code block, with something like .. code-block:: shell rather than just ::

  • For CESM2 software requirements:

    • I would put python and perl on separate lines
    • Do we still require perl's LibXML? I think I ran into an issue with not having this recently.
    • Minor issue: change subversion client version (1.8 or greater) to subversion client (version 1.8 or greater)
    • For Fortran compiler: I don't think we use any Fortran 2008 features, but we do use Fortran 2003 features heavily. So I'd say: Fortran compiler with support for Fortran 2003.
    • Trilinos used to be needed for some configurations with CISM, but this is no longer the case. Is Trilinos needed for any other components?
    • I think LAPACK is now used extensively in default configurations, so I'd list this as required. Should this also list BLAS?
  • Under "CESM2 component sets", remove this sentence: "An exception to this rule is the use of “G” as a second letter to indicate use of the active glc model, CISM." -- this is no longer true

RUN_TYPE and GET_REFCASE wrong for SSP extensions

Summary of Issue:

RUN_TYPE and GET_REFCASE are wrong fro the SSP extensions

Expected behavior and actual behavior:

RUN_TYPE is startup but should be hybrid
and GET_REF_CASE is FALSE, but should be TRUE

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

Setup a case for: BWSSP585extcmip6, BWSSP534osextcmip6, or BWSSP126extcmip6

What is the changeset ID of the code, and the machine you are using:

cesm2.1.4-rc.01

on cheyenne, but machine doesn't matter.

have you modified the code? If so, it must be committed and available for testing: No

Incorporate testing documentation from google doc

We currently have a google doc with some testing documentation:

https://docs.google.com/document/d/1oDNFPzw8bFbuyY129vX6-RJG-C9ZNUs-O1x2uVjSO4M/edit

CESM's old README file (from when CESM was in svn) pointed to this with the following text:

The most current process for updating and testing CESM versions is documented in a shared
NCAR/UCAR google document at:

https://docs.google.com/a/ucar.edu/document/d/1oDNFPzw8bFbuyY129vX6-RJG-C9ZNUs-O1x2uVjSO4M/edit?usp=sharing

I haven't looked through this documentation very carefully. It's quite possible that some of it is out-dated at this point. But some of it may be worth maintaining.

We should consider moving relevant parts to either the Wiki or other documentation within this repository.

Need new REFCASES for B1850 and BHIST

Summary of Issue:

New initial conditions for B1850 and BHIST

The current initial conditions REFCASES for B1850 and BHIST are from the LENS cases from quite a while ago.

The REFCASES that need to be used

  • 297 pi control starts from 293 at yr 130
  • 297 20th century starts 297 at yr 130.

tag naming convention

Summary of Issue:

I find the tag naming convention I started using in this repo, e.g. cesm2.0.beta07, is hard to read. I'd like to suggest moving forward we use:

cesm-2.0.0-beta07

Advice wanted: DATM variables needed to force all other components

As part of an NSC proposal to generate CAM6 ensemble
re-analysis forcing for all of the other CESM2 components,
I need to figure out which variables should be written
by the CAM assimilation into cpl history files.
This will be a 10-20 year series, saving every 3 hours,
so I should pare the list to the minimum, but complete, set.
I need to run some tests before writing the proposal,
which is due in mid-September, so I need to figure this
out soon, in order to calculate the data volume.

My understanding of the DATM mode in CESM2 is that
each component might want a distinct set of variables,
parcelled out into 1 or more streams.
The variables in the streams might be derived from
variables in the cpl history files. We won't calculate
those, but want to provide the input variables.

The 2 degree, CAM4, forcing files we've been using
for CLM, POP, and CICE have a minimal set of variables
a2x_Faxa_{rain,snow,lw,sw}*
a2x_Sa_{dens,pbot,pslv,ptem,shum,tbot,topo,u,v,z}

At the other end of the spectrum,
@mvertens recommended saving instantaneous fields,
which I interpreted to mean
./xmlchange HIST_OPTION='nhours'
./xmlchange HIST_N=3
which generate a cpl.hi file every 3 hours
with ~450 variables:
a2x_*
i2x_*
l2x_*
o2x_*
r2x_*
x2a_*
x2i_*
x2l_*
x2r_*
x2oacc_*
xaoa_*
xaoo_*
dom[ailor]*
frac[aior]
*

I'm guessing and hoping that I don't need all of those in CESM2,
but I have questions about, for example, running an ocean
(+CICE?) only case, which should(?) be forced by river run-off,
which comes from CLM, which is not represented by just a2x fields.
Would I need to keep the l2x and r2x fields?
Only some of the variables?

@ekluzek says that for forcing CLM I just want
histaux_a2x3h = .true.,
which "turns on coupler history stream for 3-hour average atm
to coupler fields" according to the cesm1.2 web site.
This generated a cpl.ha2x3h file, with the variables
in the CAM4 set, above, but with 3 more variables;
a2x_Sa_{co2,topo}*
But I don't know if this is sufficient in CESM2
for all the other components.
And are averaged fields desirable for other components?

Erik also pointed me to a recent CLM spinup case,
which used files in
/glade/p/cesm/bgcwg_dev/forcing/b.e20.B1850.f09_g17.pi_control.all.297
They have the same variables as the histaux_a2x3h = .true. case,
plus 14 a2x_Faxa_{bc,dst,oc}* variables.
I haven't figured out how the additional variables were added,
or whether they're actually needed.

I dug up a POP case and see some correspondence between the
variables in its stream files and those in the CLM stream files,
but I can't tell whether all the POP variables can be derived
from the variables needed by CLM.

I haven't looked into the river, land ice, and wave models.
I hope that I can define a sufficient set of variables without
doing that.

Thanks for any (more) guidance on any of this.
Kevin

Revise documentation on how to use a branch of a component within a CESM checkout

I periodically get questions about how to point to a branch of a component (say, CAM, CTSM or CISM) within a CESM checkout (most recently from @Ivanderkelen). We have some documentation in the CESM README file, but I'm not sure that what we have is actually the method that makes the most sense. In particular, there are at least two problems with telling people to point to their branch in Externals.cfg:

  1. Support for branches with manage_externals still leaves a lot to be desired (see ESMCI/manage_externals#34)

  2. Additional problems are caused if you change an Externals.cfg file in the middle of the tree (rather than the top-level file). For example, if you want to point to a different version of FATES (an external of CTSM), you might be tempted to modify the file components/clm/Externals_CLM.cfg. However, if you do that, when you rerun manage_externals from the top level (from the root of the CESM clone), you will get an error because the components/clm external has a modified file. (@ekluzek has suggested allowing checkout_externals to proceed if there are modifications in these Externals files. This may be worth considering, although it might be hard to implement and the implications should be thought through carefully.)

So I'm inclined to recommend that people checkout their branch using regular git commands. For example, if you want to checkout your branch of CTSM in a CESM checkout, I would recommend getting CESM as normal and doing an initial run of manage_externals/checkout_externals. Then do:

cd components/clm
git remote add ...
git fetch ...
git checkout ...

However, this isn't completely straightforward if your branch is of a component that has its own sub-externals (e.g., you have a branch of CTSM, which has a FATES external). In this case, I would probably still recommend using the above procedure to get your branch, but then getting any sub-externals by running the following from within components/clm (NOT from the CESM top level):

./manage_externals/checkout_externals clm

I'd like to hear some thoughts from others. I don't feel in a rush to update this documentation, so I think it's best if we take some time to gather thoughts until we feel pretty happy with a recommendation for users.

@gold2718 @cacraigucar @nusbaume @ekluzek @Katetc @mnlevy1981 @alperaltuntas @jedwards4b @mvertens @rsdunlapiv @uturuncoglu

Update Externals.cfg to use uppercase repository names

@fischer-ncar - while not necessary, it will keep things more consistent if Externals.cfg is updated at some point to use the new, uppercase repository names in the repo URLs.

The relevant changes I made yesterday are:

  • ctsm -> CTSM
  • cism-wrapper -> CISM-wrapper
  • rtm -> RTM
  • mosart -> MOSART

want to get status from external python

Summary of Issue: I want to call the equivalent of ./checkout_externals --status --verbose from

an external python program, the current structure of checkout.py does not lend itself well to this desire.

Expected behavior and actual behavior:

Steps to reproduce the problem. Include externals description file(s) and link to public repository):

What is the changeset ID of the code, and the machine you are using:

have you modified the code? If so, it must be committed and available for testing:

Screen output or log file showing the error message and context:

Build error with 'ocn.base.tavg.csh' error

Summary of Issue: Build Error with the following messages below

Expected behavior and actual behavior: It won't build. No problem with Case.Setup .

Steps to reproduce the problem. Include externals description file(s) and link to public repository): I have no idea what the error message means. 'ocn.base.tavg.csh' is in the right directory as indicated in the error message.

What is the changeset ID of the code, and the machine you are using: Proprietary machine named Drybiscuit

have you modified the code? If so, it must be committed and available for testing: The code had not been modified

Screen output or log file showing the error message and context:

[yaman@drybiscuit C0]$ ./case.build
Building case in directory /home/yaman/Testsite/Climate_Model/my_cesm_sandbox/cime/scripts/C0
sharedlib_only is False
model_only is False
Generating component namelists as part of build
Creating component namelists
Calling /home/yaman/Testsite/Climate_Model/my_cesm_sandbox/components/cam//cime_config/buildnml
...calling cam buildcpp to set build time options
CAM namelist copy: file1 /home/yaman/Testsite/Climate_Model/my_cesm_sandbox/cime/scripts/C0/Buildconf/camconf/atm_in file2 /home/yaman/Testsite/Climate_Model/Cases/C0/run/atm_in
Calling /home/yaman/Testsite/Climate_Model/my_cesm_sandbox/components/clm//cime_config/buildnml
Calling /home/yaman/Testsite/Climate_Model/my_cesm_sandbox/components/cice//cime_config/buildnml
...calling cice buildcpp to set build time options
Calling /home/yaman/Testsite/Climate_Model/my_cesm_sandbox/components/pop//cime_config/buildnml
...calling pop buildcpp to set build time options
ERROR: Command /home/yaman/Testsite/Climate_Model/my_cesm_sandbox/components/pop/bld/build-namelist failed rc=2
out=
err=env: ‘/home/yaman/Testsite/Climate_Model/my_cesm_sandbox/components/pop/input_templates/ocn.base.tavg.csh’: No such file or directory
ERROR ocn.base.tavg.csh: env CASEBUILD=/home/yaman/Testsite/Climate_Model/my_cesm_sandbox/cime/scripts/C0/Buildconf OCN_GRID=gx1v7 /home/yaman/Testsite/Climate_Model/my_cesm_sandbox/components/pop/input_templates/ocn.base.tavg.csh failed: 32512

Remove CMIP5 future scenario compsets/tests

Summary of Issue:

Remove the CMIP5 scenario compsets and tests and only allow the CMIP6 ones.

Expected behavior and actual behavior:

The CMIP5 RCP's are available as compsets and tests. We should remove them as we add the new CMIP6 ones.

What is the changeset ID of the code, and the machine you are using: cesm2.1.1-rc.01 hobart

have you modified the code? If so, it must be committed and available for testing: no

Screen output or log file showing the error message and context:

Here are the current CMIP5 rcp's available:

[erik@hobart cime_config]$ grep RCP config_compsets.xml 
  <!-- CMIP5 RCP8.5 compsets -->
    <alias>BRCP85L45BGCR</alias>
    <lname>RCP8_CAM60_CLM45%BGC_CICE_POP2_RTM_SGLC_SWAV</lname>
    <alias>BRCP85C5L45BGC</alias>
    <lname>RCP8_CAM50_CLM45%BGC_CICE_POP2_MOSART_SGLC_SWAV</lname>
        <value compset="RCP[2468]_">2005-01-01</value>
[erik@hobart cime_config]$ grep RCP test
testlist_allactive.xml  testmods_dirs/          
[erik@hobart cime_config]$ grep RCP testlist_allactive.xml 
  <test name="IRT" grid="f09_g17" compset="BRCP85L45BGCR" testmods="allactive/defaultio">
  <test name="ERS_Ld7" grid="f09_g17" compset="BRCP85L45BGCR" testmods="allactive/defaultio">

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.