noaa-emc / gsi Goto Github PK

LeoGeo AMVs are high-altitude winds retrieved from combined geostationary and polar-orbiting satellite imagery. They fill a data gap in AMVs global coverage, i.e. 50-70 N and S latitudinal bands.

Changes to GSI are needed in the following files to add the LeoGeo AMVs:
read_satwnd.f90
read_obs.F90
/fix/global_convinfo.txt

LeoGeo AMVs are assigned SatID 854
LeoGeo AMVs are assigned NCEP type 255

Branch used for LeoGeo evaluation: AMV_Genkova_parallel_ncio_LG

All regression tests passed:
(Hera) /scratch1/NCEPDEV/da/Iliana.Genkova/GSI/regression
Suggested reviewers: Jim Jung , Cathy Thomas, Xiujuan Su

LeoGeo_NWP_evaluation_Genkova.pptx

Add support for gpsro data into the existing format of the Conventional data monitor (ConMon). This is a reissue of Redmine issue 51902.

In setupt.f90, ges_z is used to check ges_ps and ges_th2m and ges_q2m allocation status. It causes GSI stop when ges_z is used in setupt.f90. The bug can be fixed by using ges_ps, ges_th2m and ges_q2m itself to check variable allocation status.

GSI failure case:

m_berror_stats_reg::berror_read_wgt_reg(PREWGT_REG): read error amplitudes "
berror_stats". mype,nsigstat,nlatstat = 0 42 192
setupt: ps already incorrectly alloc
STOP2 ABORTING EXECUTION w/code= 999

  if (istatus==0) then
       if(allocated(ges_z))then                                                                                                    
          write(6,*) trim(myname), ': ', trim(varname), ' already incorrectly alloc '                                               
          call stop2(999)
       endif
       allocate(ges_ps(size(rank2,1),size(rank2,2),nfldsig))
       ges_ps(:,:,1)=rank2
       do ifld=2,nfldsig
          call gsi_bundlegetpointer(gsi_metguess_bundle(ifld),trim(varname),rank2,istatus)                                          
          ges_ps(:,:,ifld)=rank2                                                                                                    
       enddo
  else     
       write(6,*) trim(myname),': ', trim(varname), ' not found in met bundle, ier= ',istatus                                       
       call stop2(999)
  endif       

The new aircraft observations from AMDAR, LATAM, TaMDAR and Canadian AMDar were assimilated in the GSI. The new VQC scheme was not applied the new aircraft observations. The O-A and O-B histograms from aircraft observations were studied. Different VQC parameters were evaluated for all aircraft observations.

Only global_convinfo.txt needs to be modified.

The current GSI can not handle the FV3LAM grid orientation W-E/S-N (SW corner is the first grid point).
The new code was added based on the current GSI FV3LAM interface to handle both grid orientation.

One new namelist option was added to make GSI work with two FV3LAM grid orientations.

   grid_type_fv3_regional = 0 : decide grid orientation based on
                               grid_lat/grid_lon
                            1 : input is E-W N-S grid
                            2 : input is W-E S-N grid

There are several existing minor fixes that the RadMon package has needed, and some tweaks to the problem reporting are now needed as well. The full list is:

• rm all old versions of gdas_radmon.v* and radmon_common.v*

• rename gdas_radmon.v3.0.0 to gdas_radmon, rename radmon_common.v3.0.0 to radmon_common

• rm known plotting inefficiencies

• clean up warning messaging strategy and make extraction and copy functions output identical

• modify criteria for low obs counts to reduce the number of warnings.

• update install_glb.sh script to use correctly identify the last plot date.

This is a re-issue of Redmine ticket #76267 ( https://vlab.ncep.noaa.gov/redmine/issues/76267 ).

Original ticket description: I thought the conversion from theia to hera had been done long ago but it's not in ProdGSI/master. Perhaps I'm confusing that with the conversion to slurm job controls. Either way, the task is to port the RadMon to theia, making the appropriate /scratchX and module changes.

so berror file can be regenerated using GFSv16 parallel output (PR #52)

This issue is to track efforts to introduce git-lfs to manage the fix files.

Forked GSI repo to my area: https://github.com/KateFriedman-NOAA/GSI
Created branch: https://github.com/KateFriedman-NOAA/GSI/tree/feature/git-lfs

This issue tracks DA development for the GFSv16 implementation.

This github issue is a continuation of VLAB issue #65376 following the transition of the ProdGSI repository from VLAB to the NOAA-EMC/GSI repository in github.

As of 6/3/2020, GFS v16 DA development moved to github branch release/gfsda.v16.0.0.

Necessary changes to compile the GSI, EnKF, and utilities on Hera will be made to https://github.com/MichaelLueken-NOAA/GSI/tree/feature/hpc-stack-update.

This new issue is to include the capability to assimilate HDOBs in the V16 based on Xingren's HDOBs parallel experiments.

GSI code changes: read_fl_hdob.f90

gsiparm change: exglobal_analysis_fv3gfs.sh.ecf

global_convinfo.txt and error table changes are also applied.

This is a re-issue of Redmine ticket #71916:

The RadMon's summary and time series plots support displaying/hiding data from a single comparison source using a checkbox. When the user clicks the checkbox the comparison source toggles between shown (checked) and hidden (unchecked). The actual comparison source (i.e. the operational GDAS) is hard coded.

Recent changes were made to the javascript used to support the l127_gps, l127_rad, and l127_rad2 experiments to use a selection list to determine the source of the comparison data, retaining the checkbox to toggle the comparison on/off. This ticket will capture those changes, generalizing them and adding creation of the comparison list to the RadMon html generation script.

The recent changes in GSI EnKF IO for GFS (like the additional prefix for names of surface ensemble files, parallel netcdf reading/writing subroutines) are need to be "mirrored" in the IO modules for the regional applications including FV3-LAM.
This issue is only for changes in the IO interface on the FV3-LAM IO modules (gridio_fv3reg.f90) to allow the GSI EnKF for FV3-LAM could run without their original setups . The actual functions like IO for surface ensembles and parallel netcdf IO are to be considered in the future.

In setupt. f90, ges_q2 and ges_th2 should be deallocated in subroutine final_vars_

https://vlab.ncep.noaa.gov/redmine/issues/75555

The existing bufr libraries will cease to work after 2020 Dec 31, so the module loads must be updated.

Current GEFS is on its own branch, so any fixes already made to develop aren't applicable.

Discovered by accident that L127 global_gsi.x hangs when run with 1020 or more tasks. Subsequent debugging identified a problem in general_read_gfsatm_nc.

When the L127 global_gsi.x is run with 1020 or more tasks, logical variable procuse is .false. for 1 or more tasks. This is problematic. Only tasks for which procuse=.true. execute function open_dataset. general_read_gfsatm_nc executes open_dataset with paraopen=.true. (ie, parallel netcdf read). By default open_dataset assumes all tasks call the function. general_read_gfsatm_nc executes open_dataset with this default. Hence, the problem. open_dataset assumes all tasks execute the function but this is not true when running the L127 global_gsi.x with 1020 or more tasks.

Function open_dataset has an optional argument for a sub-communicator. Code added to general_read_gfstam_nc to construct a communicator including only the tasks for which procuse=.true. With this addition, L127 global_gsi.x runs to completion for tasks counts of 1020 or higher. This change does not alter analysis results.

This issue is opened to document this change.

This is used to generate an internal ensemble from randomly sampling the static B.

! generate_ens - if true, then generate internal ensemble based on existing background error

It was originally added to test the ensemble Var solver, but I would like to add the capability to write out a random sample of the static B to use as additive inflation in the EnKF. Apparently it hasn't been used in a while, and changes to general_sub2grid and/or ckgcov have broken this option. The traceback is below.

forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
global_gsi         0000000001F3632D  Unknown               Unknown  Unknown
libpthread-2.17.s  00002B7A759915D0  Unknown               Unknown  Unknown
libmpi.so.12.0     00002B7A74C4C2D2  Unknown               Unknown  Unknown
libmpi.so.12.0     00002B7A74A2981B  Unknown               Unknown  Unknown
libmpi.so.12.0     00002B7A74C68810  Unknown               Unknown  Unknown
libmpi.so.12.0     00002B7A74C3DE50  Unknown               Unknown  Unknown
libmpi.so.12.0     00002B7A74B79A25  Unknown               Unknown  Unknown
libmpi.so.12       00002B7A749EB274  PMPI_Alltoallv        Unknown  Unknown
libmpifort.so.12.  00002B7A74595B21  pmpi_alltoallv_       Unknown  Unknown
global_gsi         000000000050DD19  general_sub2grid_        1627  general_sub2grid_mod.f90
global_gsi         00000000013AAEA0  ckgcov_                   157  bkgcov.f90
global_gsi         0000000000E7BFF3  hybrid_ensemble_i        1502  hybrid_ensemble_isotropic.F90
global_gsi         0000000000E7A5FA  hybrid_ensemble_i        1245  hybrid_ensemble_isotropic.F90
global_gsi         0000000000E4F9C1  glbsoi_                   269  glbsoi.f90
global_gsi         000000000062A620  gsisub_                   201  gsisub.F90
global_gsi         000000000043AF7D  gsimod_mp_gsimain        1840  gsimod.F90
global_gsi         000000000043AEB7  MAIN__                    631  gsimain.f90
global_gsi         000000000043AE5E  Unknown               Unknown  Unknown
libc-2.17.so       00002B7A79407495  __libc_start_main     Unknown  Unknown
global_gsi         000000000043AD69  Unknown               Unknown  Unknown

GFS v15.3.2 was implemented in early July 2020 to address minimization issues in the GFS v15.3 GSI. GFS v15.3.2 adjusted the error limits for CrIS water vapor channels in fix file global_satinfo.txt. This helped improve v15.3 GSI minimization but EMC monitoring continues to detect minimization problems. The most recent abnormally terminated v15.3.2 minimization was 2020090300 gfs.

Experimentation, both single case and parallel testing (one month), have identified a simple change that improves GSI minimization. The change is to turn off variational quality control for aircraft temperature observations. This is achieved by setting the varqc parameters for five aircraft temperature observation types to 0.0 in fix file global_convinfo.txt. At the same time the gross check for these aircraft temperature observations are tightened.

This issue is opened to document the updating of global_convinfo.txt in release/gfsda.v15.3.3.

This issue is documenting the research and development on evaluating Aeolus Doppler Wind Lidar wind profiles in the Global GFS model. Some aspects of this work is focusing on using Aeolus winds for TC/hurricanes characterization.
Relevant branches are:
gw.gfsv16b_aeolus_KA
and
gsi.fd_gfsda.v16.0.0_aeolus_varqc_KA_copy

https://vlab.ncep.noaa.gov/redmine/issues/71386

A new BUFR encoding sequence will be introduced in June 2020 for all EUMETSAT’s wind products. For full details and sample files, see the URL provided below. [Rev. 2]: Date changed to 7 October 2020. To facilitate user transition, AMV products encoded in the new BUFR sequence will be distributed on the GTS in parallel to the currently operational AMV products, from 15 July until 6 October 2020. For full details, see the URL provided below.

http://www.eumetsat.int/website/home/News/DAT_4593260.html

Issue found while building GSI from the forked NOAA-EMC:master

I followed Kate's e-mail regarding the model update for python and cmake, and updated modules accordingly in GSI/modulefiles/modulefiles.ProdGSI.hera:

# python
module use -a /contrib/anaconda/modulefiles
module load anaconda/2.3.0

module use -a /contrib/cmake/modulefiles
module load cmake/3.9.0

During the build process, I ran into the following error message:

Configuring utility in adderrspec.fd
Configuring utility in adjustps.fd
Configuring utility in calc_increment_ens.fd
Configuring utility in calc_increment_serial.fd
Configuring utility in getnstensmeanp.fd
Configuring utility in getsfcensmeanp.fd
Configuring utility in getsfcnstensupdp.fd
Configuring utility in getsigensmeanp_smooth.fd
Configuring utility in getsigensstatp.fd
 hey, incl dirs are /apps/intel/compilers_and_libraries_2018/linux/mpi/intel64/include/gfortran;/apps/intel/compilers_and_libraries_2018/linux/mpi/intel64/include 
Configuring utility in gribmean.fd
Configuring utility in recenternemsiop_hybgain.fd
Configuring utility in recentersigp.fd
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
FRAMEMODULE
    linked by target "global_gsi.x" in directory /scratch1/NCEPDEV/da/Emily.Liu/JEDI-GSI/GSI/src/gsi
FRAMEPACK
    linked by target "global_gsi.x" in directory /scratch1/NCEPDEV/da/Emily.Liu/JEDI-GSI/GSI/src/gsi
IOINT_LIB
    linked by target "global_gsi.x" in directory /scratch1/NCEPDEV/da/Emily.Liu/JEDI-GSI/GSI/src/gsi
WRFNETCDF_LIB
    linked by target "global_gsi.x" in directory /scratch1/NCEPDEV/da/Emily.Liu/JEDI-GSI/GSI/src/gsi

-- Configuring incomplete, errors occurred!

I also checked out GSI master from gerrit:ProdGSI and modified the modulefiles as described above. The GSI build process completed successfully on HERA.

I am wondering if I missed something when I tried to build the GSI master checked out from GitHub?

NESDIS is transitioning their Metop and AVHRR winds to Cloud computing, and they will be using a GOES-R like algorithm (similar to what is used for GOES-16&17) and the new BUFR encoding sequence 03-10-077.

Sample data for evaluation and testing is expected after late October.

We added code to reflect the new BUFR format in branch
feature/gfsda.v16.0.0_EUM_MetAVHRR_bufr

That branch also handles 1) EUMETSAT winds in new BUFR and 2) fix for VIIRS winds.
For more info on these changes, #49 and #66

In order to update delz analysis for fv3 regional da, some new code development were added in the following subroutine. Request to merge those into the GSI master.

gridmod.F90
gsimod.F90
gsi_rfv3io_mod.f90
guess_grids.F90
pcgsoi.f90
read_guess.F90

Add use_gfs_ncio namelist option to the regional application HWRF, read GFS V16 netCDF files.

This is to switch from assimilating satellite microwave antenna temperature observations to brightness temperatures.

As it is now, the bias between the observed antenna temperature and the simulated brightness temperature is being handled by the variational bias correction. For the platforms and feeds which have only antenna temperature, an accurate formula for converting antenna temperatures to brightness temperatures will be used. Doing this and assimilating brightness temperatures removes a burden of the variational bias correction.

Many (or all) other centers are assimilating brightness temperatures.

The following files have the same name and pose a problem on a case insensitive filesystem e.g. macOS.

Please consider renaming one or the other with a distinct name.

warning: the following paths have collided (e.g. case-sensitive paths
on a case-insensitive filesystem) and only one from the same
colliding group is in the working tree:

  'util/Conventional_Monitor/image_gen/ush/Transfer.sh'
  'util/Conventional_Monitor/image_gen/ush/transfer.sh'

@CoryMartin-NOAA I have cloned branch feature/files_for_jedi_aop20 from your feature/files_for_jedi.
I made some changes and additions in ffiles_for_jedi_aop20. Could you take a look at the changes in the branch? I will merge the changes back to feature/files_for_jedi once you give an OK.

The changes I made compare to feature/files_for_jedi can be found from the following link:
feature/files_for_jedi...emilyhcliu:feature/files_for_jedi_aop20

I'm using GSI (v3.7) as an observer operator in combination with ENKF (v1.3) to assimilate radiances from several sensors/satellites.

Doing some initial test I discovered that the ENKF (letkf version) wasn't incorporating any observation and traced back the error to the observation level pressure which was different from the level pressure defined at the GSI step.

I solved it turn it on the lwrite_peakwt argument to true in the GSI namelist to save the correct observation level pressure in the diag files so the letkf can use that information later.

I didn't find any mention of this requirement in the documentation or the code but would be great to add it in a future version. Is there a way to contribute to that?
Thanks!

Modify the Conventional data monitor (ConMon): * change the package level build script to use the cmake utility and the $target specific module files. * utilize the latest version of read_diag.f90 to enable support for NetCDF formatted cnvstat diagnostic files.

This issue began as VLab #63754. Development will continue in repository EdwardSafford-NOAA/GSI in branch esafford_ConMon_63754.

Hi. I'm trying to build GSI from the source contained in the release tarball https://github.com/NOAA-EMC/GSI/archive/gefs_v12.0.2.tar.gz. My build invocation looks like

cmake -DBUILD_CORELIBS=OFF -DBUILD_ENKF=OFF -DBUILD_GLOBAL=ON -DCMAKE_INSTALL_PREFIX=$PREFIX -DCMAKE_PREFIX_PATH=$PREFIX -DUSE_WRF=OFF ..

I have already built the required NCEP libraries (individually, each from their own git repo) and have them installed under $PREFIX. The build makes some progress, but then gets to this:

Found BACIO library $PREFIX/lib/libbacio_4.a
Found BUFR library BUFR_LIBRARY-NOTFOUND
Could not find BUFR library, so building from libsrc
searching for source for bufr_v in 
didn't find directory
WARNING: Did not find  of bufr, looking for alternates
CMake Error at cmake/Modules/FindBUFR.cmake:44 (add_subdirectory):
  add_subdirectory given source
  "$SRCPREFIX/gsi/libsrc/bufr"
  which is not an existing directory.
Call Stack (most recent call first):
  CMakeLists.txt:226 (find_package)


find: ‘/sigio’: No such file or directory
find: ‘/sorc’: No such file or directory
CMake Error at cmake/Modules/findHelpers.cmake:137 (string):
  string sub-command REGEX, mode MATCH needs at least 5 arguments total to
  command.
Call Stack (most recent call first):
  cmake/Modules/FindSIGIO.cmake:19 (findInc)
  CMakeLists.txt:227 (find_package)

(I replaced some long paths there with $PREFIX and $SRCPREFIX for readability -- the values are correct, but shouldn't be interesting.)

I note in the output that libbacio is found, but libbufr isn't. However:

% cd $PREFIX/lib
% ls libbacio*
libbacio_4.a  libbacio_8.a
% ls libbufr*
libbufr_4.a  libbufr_4_DA.a  libbufr_8.a  libbufr_8_DA.a  libbufr_d.a  libbufr_d_DA.a
% ls libcrtm*
libcrtm.a
% ls libnemsio*
libnemsio.a
% ls libsigio*
libsigio.a
% ls libsfcio*
libsfcio.a
% ls libsp*
libsp_4.a  libsp_8.a  libsp_d.a
% ls libw3emc*
libw3emc_4.a  libw3emc_8.a  libw3emc_d.a
% ls libw3nco*
libw3nco_4.a  libw3nco_8.a  libw3nco_d.a

I also tried setting

export COREPATH=$PREFIX/lib

export BACIO_LIB=$PREFIX/lib/libbacio_4.a
export BUFR_LIB=$PREFIX/lib/libbufr_4.a
export CRTM_LIB=$PREFIX/lib/libcrtm.a
export NEMSIO_LIB=$PREFIX/lib/libnemsio.a
export SFCIO_LIB=$PREFIX/lib/libsfcio.a
export SIGIO_LIB=$PREFIX/lib/libsigio.a
export SP_LIB=$PREFIX/lib/libsp_4.a
export W3EMC_LIB=$PREFIX/lib/libw3emc_4.a
export W3NCO_LIB=$PREFIX/lib/libw3nco_4.a

but the build fails in the same way.

Maybe interestingly, if I set

export BACIO_LIB=$PREFIX/lib/libbacio_8.a

(i.e. libbacio_8.a instead of libbacio_4.a -- I'm guessing this is a kind=8 build of the library, whether or not that's what is actually desired for the GSI build), I still see

Found BACIO library $PREFIX/lib/libbacio_4.a

in the output of a new cmake invocation, so that I'm not sure this environment variable is having any effect.

Can you offer any advice on what I might be doing wrong here? Thanks in advance!

This issue refs the HWRF GSI code changes for TDR and 88D thinning from Jason and OU.

1. added FV3GFS_NCIO_Fortran_FLAGS for GNU compilers

2. Moved gsdcloud to src/, make appropriate changes in CMakeLists.txt and a few bug fixes

3. pietc.f90 and pvqc_tables.f90

    use kinds, only: dp,dpc
--> use kinds, only: r_kind, dp, dpc

This change is to avoid compiling error

error #6580: Name in only-list does not exist or is not accessible. [DP]

Don't know the exact reason, but either (1) removing all #ifdef block in kinds.F90 or (2) use r_kind first can solve this problem.

4. updated util/EnKF/arw/src/enspreproc_regional.fd/CMakeLists.txt

5. bug fixes for GNU compilers and regional EnKF.

6. updated ush/build.comgsi for community users

To secure the functionality of aerosol radiance impacts in GSI,

1. A new regression test, fv3aerorad, is added into the package.
   Two fix files, fv3aerorad_anavinfo and fv3aerorad_satinfo.txt, are also included for the regression test.
   This regression test is based on fv3aero which is for aerosol analysis.

2. Bugfix for reading external aerosol files in read_files.f90.
   The capability is for GSI to read aerosol information from external files.
   Before the fix, GSI can't count "aerf" files properly.
   External reading for aerosol information is also utilized in "fv3aerorad" regression test.

In forked branch: https://github.com/CoryMartin-NOAA/GSI/tree/feature/gdaschgres

Currently, calcanl_gfs.py computes the full resolution analysis by interpolating the analysis increment. For the GDAS cycle, it runs enkf_chgres_recenter to regrid the GDAS forecast to the ensemble resolution for use in the recentering step. This regridding can be done after gdasfcst rather than during gdasanalcalc in order to limit the amount of time the workflow waits before the EnKF can start.

This change will also introduce changes to the existing python scripts to:

• improve readability with 4 spaces instead of 2 (pycodestyle convention)
• add print statements for linking and copying files as well as making directories

The OznMon has not kept up with changes to the oznstat file structure.

Here's the results of an ncdump -h on an oznstat file from 2019091200, which was used as test data when writing the OznMon's read_diag.f90 file:

netcdf diag_gome_metop-a_ges.2019091200 {
dimensions:
nobs = UNLIMITED ; // (2905 currently)
variables:
int MPI_Task_Number(nobs) ;
float Latitude(nobs) ;
float Longitude(nobs) ;
float Time(nobs) ;
float Reference_Pressure(nobs) ;
int Analysis_Use_Flag(nobs) ;
float Observation(nobs) ;
float Inverse_Observation_Error(nobs) ;
float Obs_Minus_Forecast_adjusted(nobs) ;
float Obs_Minus_Forecast_unadjusted(nobs) ;
float Solar_Zenith_Angle(nobs) ;
float Scan_Position(nobs) ;
float Row_Anomaly_Index(nobs) ;

// global attributes:
:_NCProperties = "version=1|netcdflibversion=4.5.0|hdf5libversion=1.10.1" ;
:date_time = 2019091200 ;
:Number_of_state_vars = 1146 ;
}

Note the global attribute list. Now a dump from 2020070912:

netcdf diag_gome_metop-a_ges.2020070912 {
dimensions:
nobs = UNLIMITED ; // (2705 currently)
variables:
int MPI_Task_Number(nobs) ;
float Latitude(nobs) ;
float Longitude(nobs) ;
float Time(nobs) ;
float Reference_Pressure(nobs) ;
int Analysis_Use_Flag(nobs) ;
float Observation(nobs) ;
float Inverse_Observation_Error(nobs) ;
float Obs_Minus_Forecast_adjusted(nobs) ;
float Obs_Minus_Forecast_unadjusted(nobs) ;
float Solar_Zenith_Angle(nobs) ;
float Scan_Position(nobs) ;
float Row_Anomaly_Index(nobs) ;

// global attributes:
:date_time = 2020070912 ;
:Satellite_Sensor = "gome_metop-a" ;
:Satellite = "metop-a" ;
:Observation_type = "gome" ;
:pobs = 0. ;
:gross = 120. ;
:tnoise = 2.236 ;
}

This ticket will include adding the ability to read all of the current global attributes, removing those global attributes that no longer exist, and will evaluate if and how those global attributes should be used in the OznMon and appropriate implementation(s). Also the OznMon's read_diag.f90 module will be renamed to oznmon_read_diag.f90 to avoid confusion with similarly named modules.

Initialized tropical cyclone Douglas in V15 GDAS was found to have stronger winds and lower center pressure depression than the observed. The only TCVITAL type of observations assimilated was SLP in GDAS. A few sensitivity tests were performed on the possible culprits; i.e. strong constraint , hybrid ensemble-var, and the position of the TC center in the first guess. It was found that the misalignment of the TC position in the first guess was the main reason for the erroneous initialization.
Two risk mitigation strategies were proposed and tested; Option 1: matching observed location to the first guess TC position, and Option 2: applying the forcing from option 1 at the observed TC location.

NCO implemented GFS.v15.3.2 on 6 July 2020 as described in the excerpt below taken from NCO RFC memo - July 2, 2020.

RFC 7036 – On WCOSS, implement GFS.v15.3.2 updates to the GSI fix file and global_satinfo.txt. This change is being made to address minimization issues in the GSI and tighten quality control for the seven CrIS water vapor channels. To be implemented on July 6 at 1400Z.

This issue is being opened to document the following:

• rename branch DA_GFS_v15.3 to release/gfsda.v15.3 to be consistent with gitflow naming convention
• update release/gfsda.v15.3 with fix associated with gfs.v15.3.2
• tag updated release/gfsda.v15.3 as gfsda.v15.3.2

#76692 on vlab (https://vlab.ncep.noaa.gov/redmine/issues/76692)

Under certain conditions, the EnKF will try to issue a diagnostic print that tries to access an unallocated array. This causes a segfault on Orion. Fix in PR #12 just removes the unallocated array from the print statement (does not change results).

It was found that the code for the new BUFR encoding sequence 03-10-077 for VIIRS AMVs (type 260, subset NC005091), had a bug in the SatID range check. This lead to not setting itype properly and assimilating the VIIRS winds with inverse error 0, i.e. effectively not assimilating them (for the time since 5 Dec 2019).

The Satellite ID range check for winds with subset NC005091 should be 200-250, instead of 250-299, in read_satwnd.f90

The big fix was implemented and successfully tested:
feature/gfsda.v16.0.0_EUM_MetAVHRR_bufr

The branch with the VIIRS AMVs fix also: 1) adds code for new BUFR for EUMETSAT winds ; 2) adds code for new BUFR for NESDIS's Metop and AVHRR winds For more info on these changes, see issue #49 and issue #67

Several small issues with the RadMon have cropped up. None are major or significant so they will be combined into this ticket. Items include:

• Update Copy operation to avoid copying only partial data sets.
• Reorganize gdas and common directories to remove the outdated version numbering scheme.
• Implement RadMon code changes requested changes from NCO.

There are several coding standard issues with the routines in the GSI. Have been going through and correcting these issues within the code. Thus far, issues corrected have included:

• Ensuring proper indentation in all routines.
• Ensuring proper precision is given for all real values.
• Ensuring that integers are given (i_kind) at variable declaration.
• Several routines are full of random capitalization. Ensure that variables, if, do, while, and other intrinsic functions are lower case through out.
• Replacing .ge., .le., .gt., .lt., and .eq. with >=, <=, >, <, and ==.
• Replacing .F90 with .f90 for routines that don't contain machine logic.

This work was started in the ProdGSI repo, but was ported to NOAA-EMC/GSI. Have cleared through:

src/gsi/gsi_4dcouplermod.f90

This work can be found in:

https://github.com/MichaelLueken-NOAA/GSI/commits/feature/code_cleanup_2020

This branch gsi_fv3reg4coldstart contains recent fixes and development for FV3SAR GSI interface, including:

1. fix for the mismatch in the treatment of the wind-rotation for the observation and background variables.
   Before this fix, the observed wind were rotated to grid-relative wind winds, while the background wind was earth-relative winds.
2. fix for the fv3sar_bg_opt !=0, when the cold start files are used to provide the background. Bugs were related to the grid mismatch between read-in and write-out steps.
3. developments to e GOES-17 update in read_abi.f90
   This branch has been used in the regional parallel runs and work by collaborators at GSL. This branch has a part of the GSI transferred from the Vlab to Github.
   This issue is opened for monitoring/updating on "pushing back" this branch to the GSI master/develop branch.

FV3LAMDA sometimes can't read NEXRAD bufr files due to checking subset status before reading data. This checking step is not necessary and should be turned off.

Before the fix:
For today's 12z LAMDA:

regional_gsianl_tm01_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
regional_gsianl_tm02_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
regional_gsianl_tm03_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
regional_gsianl_tm04_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
regional_gsianl_tm05_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"

File sizes for the 12z NAM nexrad file:

-rw-rw-r-- 1 nwprod prod 1218249536 Oct 21 13:15 nam.t12z.nexrad.tm00.bufr_d
-rw-rw-r-- 1 nwprod prod 829225392 Oct 21 12:18 nam.t12z.nexrad.tm01.bufr_d
-rw-rw-r-- 1 nwprod prod 951898336 Oct 21 11:23 nam.t12z.nexrad.tm02.bufr_d
-rw-rw-r-- 1 nwprod prod 951984000 Oct 21 10:30 nam.t12z.nexrad.tm03.bufr_d
-rw-rw-r-- 1 nwprod prod 1030995288 Oct 21 10:00 nam.t12z.nexrad.tm04.bufr_d
-rw-rw-r-- 1 nwprod prod 1072084336 Oct 21 10:00 nam.t12z.nexrad.tm05.bufr_d
-rw-rw-r-- 1 nwprod prod 1235644904 Oct 21 09:40 nam.t12z.nexrad.tm06.bufr_d

Conclusion: The smaller NAM nexrad files are the ones the code has problems reading. I recall seeing this before but this is the first time I was able to quantify it. It makes sense that the RAP files are giving the same error since they are always much smaller than the NAM ones due to the early data cutoff.

after the bug fix, gsi can read NEXRAD bufr files from both RAP and NAM dump:

[Shun.Liu@m110a2 log]$ grep RADAR_BUFR 12.log | grep num_radar
regional_gsianl_firstguess_tm06_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 107
regional_gsianl_tm00_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 78
regional_gsianl_tm01_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 92
regional_gsianl_tm02_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 91
regional_gsianl_tm03_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 93
regional_gsianl_tm04_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 98
regional_gsianl_tm05_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 107

regional_gsianl_tm**_12.log .0 is from NAM dump.
[Shun.Liu@m110a2 log]$ grep RADAR_BUFR 12.log.0 | grep num_radar
regional_gsianl_firstguess_tm06_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 106
regional_gsianl_tm00_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 86
regional_gsianl_tm01_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 92
regional_gsianl_tm02_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 95
regional_gsianl_tm03_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 106
regional_gsianl_tm04_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 107
regional_gsianl_tm05_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 109

The CMake methods for identifying both external packages (NetCDF, HDF5, etc.) and NCEPLIBS have progressed tremendously since they were first used with the GSI. The build system for the GSI needs to be updated to work with the new build system while also continuing to work with the module-based system used across NOAA, NCAR, and NASA.

I have been attempting to generate the nc_diag files containing the GSI computed forward operator H(x); previous versions of the GSI have worked fine and are backwards compatible with the previous CRTM revision files (on RDHPCS Hera, /scratch1/NCEPDEV/global/glopara/crtm/) and the current (on RDHPCS Hera, /scratch2/NCEPDEV/nwprod/NCEPLIBS/fix/crtm_v2.3.0/).

I have tested the attempts to create the nc_diag files with two different executables (both built on RDHPCS Hera) from two different revisions:

• /scratch2/BMC/gsienkf/whitaker/gsi/GSI-github-jswhit1/exec/global_gsi.x (works)

• /scratch2/BMC/gsienkf/Henry.Winterbottom/UFS-RNR/exec/ufsrnr_gsi.x (fails)

My working directory on RDHPCS Hera is: /scratch2/BMC/gsienkf/Henry.Winterbottom/work/UFSRNR_DEMO/20191203060000/gsi/cntrl_hox

Within the above directory is a log file named gsi.log which contains the stdout from the failed run. I have also attached it.

The failed GSI experiments are failing in the setuprad.f90 routine suggesting (to me) that either the satellite bias-correction file is incorrect and/or there is a mismatch with the CRTM.

GSI stdout log: gsi.log