noresmhub / blom Goto Github PK

Include infor about the BLOM repo structure in the wiki.

In a BLOM-core meeting today with @matsbn @TomasTorsvik @milicak and @JorgSchwinger we discussed the need for some restructuring of the code and putting together a minimum standalone running infrastructure. Specifically, there is a need to move the idealized test cases under a specific folder (or possibly outside the BLOM git structure). Overall, these changes are needed for #80 and some sort of initial try was given in #83.

We felt that we should take a bit of time to make a good choice for the structure so I created a doc that we can use for planning. I put there an illustration of the current structure and a suggestion.

• Outline different possibilities for the structure
• Agree on a which structure to proceed with
• git-mv existing directories to the new structure
• Make a new pull request or modify #83 to bring in new changes

@matsbn @nordmoen @AleksiNummelin @JorgSchwinger @milicak
I suggest that we can use the wiki to document building and running stand alone BLOM/iHAMOCC. This is now in the README.md file, but that file is not really suitable for a long documentation text. The wiki is a better option for this because it can be structured over several pages for different sub-topics. At the moment I have just copied over the meson info from the README.md file. I was also thinking to include some description about the different test cases. What do you think, is this a way to go or do you prefer the current format with info in the README.md file?

Currently "natural tracers" are only partly implemented in the sediment code, limiting its applicability for high emission scenarios (or long integrations). This will effectively lead to a duplication of the sediment code (but for "natural" constituents), but it will extend the diagnostic capabilities of iHAMOCC greatly.

@matsbn @JorgSchwinger @annefou @mvhulten
Hi!
I just looked at the NorESM2 documentation page
https://noresm-docs.readthedocs.io/en/noresm2/
and I realized that there is not much documentation related to BLOM/iHAMOCC. With the new NorESM2 structure, the components should contain their own documentation, and there is already a "doc" folder with content provided by Marco.

Should we set up a readthedocs page for BLOM to host the documentation for BLOM/iHAMOCC? I'm assuming we want to use 'readthedocs', but perhaps this should also be a point for discussion? If you agree, I can go ahead and create a page based on the existing "doc" content.

@matsbn @JorgSchwinger
It would be nice to have a stand-alone test case that can run iHAMOCC, so I was wondering if it would be possible to extend the fuk95 case to allow for this. There are some tracers that are initialized with GLODAP and WOA data in beleg_vars.F90. Are there any other obstacles that need to be handled?
I was thinking to just extend the existing mod_fuk95 with an additional subroutine, e.g. inihamocc_fuk95, to set the necessary variables. What do you think about this?

I ported BLOM for the Intel2019 compiler and will make the corresponding pull request.

An extended nitrogen cycle in iHAMOCC

Within the EU-ESM2025 project, we are implementing an extended nitrogen cycle in the ocean biogeochemistry model iHAMOCC as part of NorESM. The aim is to explicitly represent the major and climate-relevant nitrogen species and the processes affecting these species. Namely, iHAMOCC will be enhanced by nitrite and ammonium in both, the water column and the sediment. The eventual aim is to enable, run and analyse coupled NorESM simulations with the extended nitrogen cycle.

Details on the implementation:

• The current plan is to enable switching on and off the extended nitrogen cycle via preprocessor flags (extNcycle).
• The implementation will be done in a step-wise manner - first the water column followed by the sediment.
• For questions, suggestions or discussion (also related to details on process descriptions) get in touch with [email protected]

It's an ongoing project and the new feature branch will presumably enter the master branch at some point.

@matsbn @JorgSchwinger

Hi again,
I was wondering if it could make sense to create a tag on the commit a8a7e94. This is the point on "master" where the last merge to release-v1.0.0 was made, and is the starting point for a few feature branches. I made a tag on my own fork as an example.
-Tomas

Clean up, re-organise and extend the inventory routine to give a full budget of all important constituents (taking into account all in- and out-fluxes). Create annual netCDF output for the inventory, which replaces the text-output in the ocean log-file.

@JorgSchwinger @matsbn
From mo_intfcblom.F90 header:
! Once BLOM has transitioned to free source format, the interface routines
! blom2hamocc and hamocc2blom can be incorporated into this module

I think this can be done now. Any reason why not?

@matsbn @JorgSchwinger
Would it make sense to update the release-1.2 branch with the latest commit from Jörg and the pull request I mad just now to master? Note that we have not yet created a v1.2.0 tag, or a code release for v1.2, only a release branch.

Implement carbon (and nutrient(?)) transport diagnostic output in the same way as heat and salt fluxes. This requires work in the BLOM code and should be discussed with Mats.

Currently, adding all the source files in hamocc does not lead to a valid build with Meson (checkout the meson branch).

There are quite a few errors when compiling so I'm unsure how to proceed with enabling hamocc. Could someone with more experience comment on which files are required by hamocc and if there are special flags that are needed to compile. In addition, should hamocc be placed behind a feature flag in Meson in the same vein as MPI support?

With the new Actions support added to the meson branch, any pull requests made are automatically tested. This can be seen under the Actions tab in the Github interface. So if you want to experiment, simply create a pull request against the meson branch and add hamocc support.

Hi @nordmoen and all,

I want to use an external fortran library within blom, I've tried

lib = fcc.find_library('libname', has_header='lib.mod', required=true)
deps+=lib

kinda works.
but the fortran compiler does not find the module file unless the following is added

includes+=<hard-coded library_path>/include/fortran

How to get this <hard-coded library_path> from lib?

Include link to ReadTheDocs documentation in README.md file.

Update the carbon chemistry code to using mocsy (www.geosci-model-dev.net/8/485/2015/). Details remain to be discussed.

HAMOCC initial condition files need to be written to blom.input_data_list also for runtype/=STARTUP. The model attempts to read them in any case so they need to be present/downloaded. This will resolve NorESMhub/NorESM#149

@matsbn @TomasTorsvik we need to discuss where to put these kind of changes now. Certainly into the master branch (but then master should first be synchronised with the release-1.0 branch). But probably also into release-1.0, since this will be a candidate included in future minor releases?

The new C-isotope code of NorESM has to be coupled to the sediment. Currently the code can only be run with -Dsedbypass

Explicit use statements in iHAMOCC modules/subroutines

For code readability and safety, the inclusion of explicit use statements throughout the iHAMOCC code would be nice to achieve. A first branch with all necessary coding is done (expl_use_statem) and tested to the point that i) all iHAMOCC preprocessor-flags (but BOXATM, see below) were compiled and run in single_column mode (partially with dummy input files) and ii) a regular run case for a 1yr historical simulation produces (bit) identical results in NorESM (comparing NorESM version 2.0.5, but with i) BLOM master and ii) branch expl_use_statem).

Before opening a pull request, the updated inventory calculations -now also accounting for river and nitrogen deposition fluxes (#143)- could be merged in (or the present inventory calculation changes in expl_use_statem need to be reverted back to the master branch state).

Observations on the way:

• mo_boxatm.F90 is missing so that testing of flag BOXATM wasn't possible
• in a number of subroutines, the explicit declaration of intent(in/inout) is still missing - debating, whether we should change it in this process as well
• float numbers could be attributed with an explicit working precision, e.g.9.1_wp statement (future task? - mo_kind.F90?)

Potential impact on merging other branches:
Explicit use statements might need to be enhanced, if new parameters, diagnostics, etc. are handled in the modules/subroutines. At present state, one file experienced renaming and transformation to a module (get_pi_ph.F90 -> mo_get_pi_ph.F90).

DO with shared termination was declared obsolescent in Fortran 90, but still part of the Standard.
Example:
DO 100 J = 1,NJ
DO 100 I = 1,NI
X = X + ARRAY(I,J)
100 CONTINUE

This style still appears in some places in BLOM/iHAMOCC. I would like to replace these instances with the more modern DO ... END DO format.

It would be nice to setup some basic testing for BLOM, both for continuous integration (to ensure that the output of the different compilers actually work) and for assurance when updating the code that nothing is broken. With a good test setup it would also be easier to make performance improvements since there would be ways to ensure that the new code is up to specification.

Meson has built in support for unit testing which we could leverage, however, the structure of these unit tests are quite free. Ideally the executable used for the test should be self-contained (meaning it should test a few relevant factors), deterministic (the data which is tested against should not be affected by changes to the code) and quite quick to run.

Several avenues are open to us when implementing this:

1. We could implement the executable in Fortran. The executable would then call into BLOM with predefined inputs and known valid output and compare the results.
2. We could implement the executable in a different language (e.g. Python) and use existing facilities in BLOM that output statistics during a run to compare against. This requires us to generate an initial set of known good values for different inputs, but it requires minimal changes to BLOM.
   • One challenge with this is that the test executable needs to run the new version of BLOM, something that is easier to automate with suggestion 1.
3. Other suggestions?

I would be available to help with this, especially the Meson and CI integration, but I would require some help defining test cases and figuring out how to implement it in BLOM.

@JorgSchwinger @matsbn

I found a bug when compiling with the CPP flag "cisonew" true. In beleg_param.F90 the variables iatmc13 and iatmc14 are not defined, because they are not loaded from the mo_param1_bgc module. This module is loaded with
use mo_param1_bgc, only: iatmco2,iatmnco2,iatmo2,iatmn2

The problem is that iatmc13 and iatmc14 are only defined when the "cisonew" flag is true, so it is not trivial to add these variables to the list that is loaded from the module.

• Solution 1: remove "only" condition when using the module
• Solution 2: have different variants of the module load directive, depending on CPP flags
• Solution 3: Remove the CPP flag settings, find some other method to turn these effect on and of

For now I suggest we use solution 1 in these cases, but maybe we should think about solution 3 for the long term?

Currently, if iHAMOCC is stopped/restarted, the averaging information for output files that have not yet been written is lost. E.g. if IHAMOCC is stopped end of June and restarted 1st of July, the annual output for this particular year will be incorrect (containing only the average July-December). To fix this, the fields saving the average output need to be written to restart.

I consider this relatively low priority, because it is rarely needed for ESM scenario simulations. It will be needed for data assimilation purposes, so when NorCPM upgrades to the current version of the iHAMOCC code, this will be needed.

Currently we use depth integrated light intensity to determine phytoplankton growth rate, but growth rate is a non-linear function if light. By using the formulation of Evans & Parslow (1985) we can hopefully improve the (too low) productivity for deep mixed layers and low light conditions.

It seems that writing of restart files for tracers is hard wired to only work for expcnf .eq. 'cesm' experiments, where rstfnm has a file name that contains blom(inst_suffix).r.. I get an error from aufw_bgc when I try to run single column with iHAMOCC.

Suggested workaround: Only call restart_trcwt(rstfnm) for the 'cesm' experiment.

Perhaps it would be nice to have restart files for other experiments as well, but I think it's not needed for the single column case.

For discussion:
If we should have a BLOM/iHAMOCC model that can accommodate different BGC tracer counts at run time, it is necessary to first make some changes in the BLOM/iHAMOCC interface, and the way that the tracer module works. I propose the following changes:

• make "ntr" adjustable at runtime through a new subroutine ("set_trc") and change the attribute from PARAMETER to PROTECTED.
• Make the tracer arrays allocatable: trc, trcold, uflxtr, vflxtr, trflx
• Make a new "allocate_tracers" subroutine
• Set value of "ntrbgc" based on variable settings in input namelist (e.g. "with_cfc", "with_natDIC", "with_AGG", "with_ciso" as .true. or .false.)
  • advantage: No need for "double booking" of i_base, i_iso, i_cfc, i_agg, i_nat_dic

This setup will require the following procedure:

1. Read namelist for iHAMOCC tracer variables
2. Set "ntrbgc"
3. Set "ntr"
4. Allocate tracer arrays

After that the model structure should be the same as before, so I don't think there is a need for major restructuring of the code. However, the tracer allocation will have to be included somewhere in the initialization procedure.

There are still some references to BELEG_BGC in some comments. BELEG_BGC has been replaced by BELEG_PARM and BELEG_VARS.

@mvhulten @JorgSchwinger @matsbn
I would like to merge the sediment code written by Marco, into the iHAMOCC source code. I was able to convert his original mercurial repository into git and import it to gitHub (https://github.com/TomasTorsvik-work/hamocc_sediment_legacy). I think the next step should be to create a feature branch in the BLOM repository and try to merge in incremental steps, but I'm not yet sure how I will proceed with this. Will probably tinker a bit in my personal repo before I start pushing anything. Let me know if you have any ideas/suggestions about this work.

ocprod.F90 has k-loop as the innermost loop in several loops (loop1-loop4). Ordering should normally be k,j,i in all loops except those dealing with sinking.

With OCN_NCPL=1 it is daily ocean coupling. Probably, the time management change associated with the major code restructuring (commit e2a4230) has not been properly tested with daily coupling (most CMIP6 simulations uses sub-diurnal coupling frequency). This should of course be corrected and is likely unrelated to the restart filename generation of this PR.

Originally posted by @matsbn in #124 (comment)

Related to #10, we also have to fix the remaining missing files in input data list for iHAMOCC (i.e. initial value tracer files and CFC concentration input files).

@TomasTorsvik @matsbn @MichaelSchulzMETNO Please can you create a BLOM fork with name BLOMGPU in NorESMhub? We need it for development on GPUs?

Thanks a lot
Alok

@matsbn , @JorgSchwinger
Hi,
I would like to make a fixed point for a feature branch to be used in the GPU hackathon. I don't think the GPU stuff will be merged into master any time soon, so I was thinking to got with release-v1.0.0. However, I see there have been a few more commits on the release branch. Would it be OK to make a tag v1.0.1 on the release branch, and use that as a starting point for the feature branch?
-Tomas

The OpenMP directives in iHAMOCC need to be updated to be able to run NorESM with hybrid MPI/OpenMPI parallelisation in compsets with iHAMOCC enabled.

@matsbn , @monsieuralok , @AleksiNummelin

Hi,
I made a copy of the feature_channel2 case based on the path that Alok provided us (see https://github.com/TomasTorsvik/BLOM-TTfork/tree/feature_GPU_channel2). I have included most of the files, also some that have intentionally been left out by Aleksi earlier, but I was not able to include two netCDF files from the "regions_and _indices" folder. Maybe there is some trick to include large binary files in gitHub, but perhaps there are better options around for sharing these files?

If this looks OK for you, I can create a corresponding feature branch in the BLOM repository.

-Tomas

There are a few bug fixes for C14 isotopes that have not yet been included in the main code. Anne Moree has developed and tested the code. The fixes can be found in /cluster/projects/nn2980k/Moree/NorESM2/cases/NOINYOC_tn21_18d/SourceMods/. Note that these runs are based on the BCCR-fast version of NorESM1, so care has to be taken when implementing this in NorESM2.

In connection with #124 I have updated the restart_trc[rd/wt] files from fixed-form (F) to free-form (F90) style. As this is not part of the bugfix, I would like to add these changes afterwards.

The original comment included information that should not have been made public. The content of this issue has therefore been removed.

Atmospheric CO2 mixing ratio in the gas exchange formulation is modified by sea level pressure and a water vapour correction in the gas exchange formulation, but these are not in the output, so the true Delta pCO2 used cannot be traced.

I would like to replace some preprocessor flags with namelist variables for iHAMOCC. At the moment preprocessor flags are used to turn on/off some basic functionality (e.g. cfc, natDIC, carbon isotopes), that I think would be better handled by a namelist variable. This will make it possible to change the model behavior without recompiling and creating a new executable. To start with, I want to replace FB_BGC_OCE, CFC, natDIC and CISONEW.

We have a plan to expand the nitrogen cycle representation in iHAMOCC, and perhaps we would like to include other tracers later on. I believe it will become increasingly difficult to maintain the code if we continue adding preprocessor flags, so I would like to set a new structure before adding more tracers.

@IngoBethke pointed out an issue of non-conservation when applying time smoothing for coupling intervals shorter than one day. Indeed, the smoothing is hardcoded to only work with daily coupling intervals and it is a bug that smoothing is currently allowed with different coupling intervals.

This issue should be seen as a meta issue for the discussion around a new build system for BLOM.

Possible candidates

• CMake
• SCons
• waf
• Meson
• Make

What we should discover is what are the requirements for BLOM's build system and discuss in this issue. The following is a small list compiled after a discussion with @matsbn. Feel free to add to this top post with new requirements.

Requirements

• Must support Linux/Unix and Sigma2 resources, but should also build on local machines with Linux/macOS
• Must handle dependencies (e.g. netCDF or MPI)
• Should be able to automatically generate runtime files such as dimensions.F
• Should support externally updated files for configuration

Currently (BLOM/iHAMOCC v1.2) the N-fixation by cyano bacteria is limited to the surface mixed layer. It might make more sense to instead limit the N-fixation to the euphotic zone, currently set as a parameter at 100.0 m depth. This will change the model output, so it is a question when and where to introduce this change; (a) in connection with introduction of hybrid vertical coordinates, or (b) in connection with nitrogen cycle implementation. I think it makes most sense to see this in connection with the N-cycle implementation, but it could also be done in a separate step.

The ocean mask variable "omask" is defined in mo_intfcblom, but is still passed as a variable in subroutine calls. I assume the intention is to replace these calls with something like
use mo_intfcblom, only: omask
@JorgSchwinger are you working on this, or should I go ahead and try to implement this?

I would like to very soon make a persistent release branch for the BLOM version to be used in the upcoming NorESM2 release. Then a tag of this release branch will be used in the Externals.cfg of NorESM. Since release branches and tags will persist, it can be good to choose a sensible naming convention. My suggestion is that we call release branches for "release-<major version>.<minor version>" and tags for "v<major version>.<minor version>.<patch>". Then the next release will be maintained in branch "release-1.0" and tagged "v1.0.0". Maintenance and bug fixes or documentation enhancements of BLOM1.0 will then happen in "release-1.0" branch and tagged "v1.0.1", "v1.0.2" and so on. Changes in a release branch should not change answers of existing configurations. When it is time to release a new answer-changing version or significant new functionality, I think a new release branch, say "release-1.1", should be made. I am not sure about when to increment the major version number, but that we can leave for later discussions (it could be as infrequent as to coincide with CMIP phases).

If you have some comments/suggestions to this branch and tag name convention, it would be great to discuss it in this thread.

It would be useful to clarify the license for BLOM. Assuming we are free to chose whatever license, I would suggest (having consulted with Marco) to go for either a GNU Lesser General Public License v3.0 (lgpl-3.0) or an Apache license 2.0 (apache-2.0).

Got this when trying to configure with [email protected] and [email protected] and [email protected] (and newer 0.61.2)

mpiifort binary missing from cross or native file, or env var undefined.
Trying a default mpiifort fallback at mpiifort
mpiifort found: NO
Run-time dependency MPI for fortran found: NO (tried config-tool and system)

../meson.build:130:2: ERROR: Dependency "mpi" not found, tried config-tool and system

Do you know how to debug this?
@matsbn, @monsieuralok or others

a simple project as following fails:

project('mpitest', 'fortran')
dependency('mpi', language: 'fortran')

@tjiputra @JorgSchwinger
We have a CPP flag DMSPH that determines if we calculate the field pi_ph. This is used in ocprod to calculate dms_ph which in turn is used to calculate dmsprod. At the moment pi_ph is not used and we set dms_ph = 1., so everything related to the DMSPH CPP flag is bypassed.

Should we maintain the option to set dms_ph from pi_ph? If so, can we replace the DMSPH CPP flag with a logical option that we can read from a namelist file (.e.g. with_dmsph) and make an if statement to select how to set dms_ph?

The "meson" branch has been merged with master. Is it OK with everyone if I delete this branch now?