This is part of the review of the openjournals/joss-reviews#6025

Reference Date

Firstly, I would suggest setting the reference date for Incoming Neutron Intensity to be 1 May 2011. Your current method can cause different reference dates for each site.

The 1 May 2011 date is used by all the large networks (Zreda et a., 2012, Hawdon et al., 2014, Bogena et al., 2022). It is somewhat arbitrary, but it makes comparing sites easier and keeps things to a standard (especially if a user wants to compare their sensor to another from other networks). It wouldn’t impact the outputted soil moisture, but it would make it harder to compare intensity corrections between sites processed in CRNPy and sites processed by the networks. This could be provided as a hard coded data point that you collect for each potential NMDB sensor.

Methods

Your current method of selecting NMDB with nearest cutoff rigidity was initially used by CosmOz (add the reference for this: Hawdon et al., 2014). Although note, they’ve recently changes their method to use a revised calculation (McJannet and Desilets, 2023) (https://cosmoz.csiro.au/about).

COSMOS-EUROPE use a slightly different method which adjusts the differences between the reference NMDB tower (usually JUNG) and the specific CRNS cutoff rigidity (equations 4 and 5 in the Hawdon et al., 2014 paper).

I think it would be good to add these options in your code base. Until the CRNS community agrees on the "correct" method it would be good to provide users the options to match commonly used methods.

Filling missing data

There is a step that fills the fi column with a 1 when NMDB data is missing. This should be removed. This could lead to corrected neutron quantities suddenly changing by several % which will translate to changes in soil moisture. A user wouldn't know that this is due to gap filling and could misinterpret events.

This is part of the review of the openjournals/joss-reviews#6025

There are already some python tools that process CRNS data e.g. crspy (Power et al., 2021), and corny (Schrön 2021).

The references should be included and a little more description on how this tool differs in its approach and compares to other CRNS processing tools.

Power, D., Rico-Ramirez, M. A., Desilets, S., Desilets, D., and Rosolem, R.: Cosmic-Ray neutron Sensor PYthon tool (crspy 1.2.1): an open-source tool for the processing of cosmic-ray neutron and soil moisture data, Geosci. Model Dev., 14, 7287–7307, https://doi.org/10.5194/gmd-14-7287-2021, 2021

Schrön, M.: CORNish PASDy – COsmic-Ray Neutron flavored PASDy PASDy – Processing and Analysis of Sensor Data in pYthon [data set], https://git.ufz.de/CRNS/cornish_pasdy, last access: xxx

This is part of the review of the openjournals/joss-reviews#6025

The weighting procedure currently implemented has since been revised (Schrön et al., 2017).

It is a refinement of the Köhli 2015 weighting already implemented in this code (it now takes into account the influence of pressure and humidity, as well as allowing the weighting of samples <0.5m from the sensor.

I feel this should be added as an option to allow users to match the weighting strategy of the networks currently implementing it (e.g., https://cosmoz.csiro.au/about). It involves some different weights and a convergence approach (described in section 2.3 from the paper).

Schrön, M., Köhli, M., Scheiffele, L., Iwema, J., Bogena, H. R., Lv, L., Martini, E., Baroni, G., Rosolem, R., Weimar, J., Mai, J., Cuntz, M., Rebmann, C., Oswald, S. E., Dietrich, P., Schmidt, U., and Zacharias, S.: Improving calibration and validation of cosmic-ray neutron sensors in the light of spatial sensitivity, Hydrol. Earth Syst. Sci., 21, 5009–5030, https://doi.org/10.5194/hess-21-5009-2017, 2017.

It seems that there is an error in this nan lookup:

Round time function ignores the time column name parameter

This is related to the review of openjournals/joss-reviews#6025

Add the Schrön et al., (2017) reference for the sensing depth function (it’s implemented in the code but not referenced in the paper). Double check that all the methods you've applied in the code are referenced in the paper.

Some of the references are given incorrectly, the "implementing corrections" bullet point:

Overall description (Andreasen et al., 2017)
Atmospheric pressure (Zreda et al., 2012)
Air humidity (Rosolem et al., 2013)
Incoming Neutron Flux (Zreda et al., 2012, Hawdon et al., 2014, McJannet and Desilets 2023).

I’d suggest rephrasing this to account for the specific references and how they relate to the methods described.

Schrön, M., Köhli, M., Scheiffele, L., Iwema, J., Bogena, H. R., Lv, L., Martini, E., Baroni, G., Rosolem, R., Weimar, J., Mai, J., Cuntz, M., Rebmann, C., Oswald, S. E., Dietrich, P., Schmidt, U., and Zacharias, S.: Improving calibration and validation of cosmic-ray neutron sensors in the light of spatial sensitivity, Hydrol. Earth Syst. Sci., 21, 5009–5030, https://doi.org/10.5194/hess-21-5009-2017, 2017.

Hawdon, A., McJannet, D., and Wallace, J. 2014. Calibration and correction procedures for cosmic-ray neutron soil moisture probes located across Australia. Water Resour. Res. 50: 5029–5043.

McJannet, D.L.; Desilets, D. Incoming Neutron Flux Corrections for Cosmic-Ray Soil and Snow Sensors Using the Global Neutron Monitor Network. Water Resour. Res. 2023, 59, e2022WR033889.

This issue contains my review for the JOSS submission openjournals/joss-reviews#6025 :

You can either address this as a single issue or spawn a new issue from each bullet point

Paper:
Reviews for the paper are including in the attached pdf 10.21105.joss.05814_jl.pdf. Of note:

• The statement of need could be better stated. Correction and filtering methods documented in many different papers is fairly common. I suggest reframing the statement of need with regard to reproducibility and robustness.
• References in the paper are missing DOIs

Software:

• Unversion the crnpy.egg-info folder from the repository. This is a folder that pip creates at installation on a local machine and shouldn't be needed in the repo.
• Readme.md: Include a section/link to the example problems and maybe a short description of each example problem
• Testing: It would be nice to see testing of each/most of the individual methods to make sure that the code is giving you the expected output from synthetic or literature test cases. This will also make tracking down and fixing code errors in the future much easier.
• A type checker/conversion method should be developed and used with your processing methods. This would also help with long term maintainability of the code. I noticed in multiple methods, docstrings indicate that numpy, list, or pandas could be supplied as inputs, however there were pandas specific methods in the processing that would fail with a numpy array or a list.
• total_row_counts() method has unused input parameters. These unused input parameters should be removed from the method.
• correction_pressure(): update the docstrings. Docstrings list atm_pressure as an input parameter, however the parameter name is pressure.
• correction_humidity(): update the docstrings. temp is listed as an input parameter although it is not currently one.
• get_incoming_flux(): unused date_format variable should be removed
• smooth_1d(): Need to add the proper reference for the Savitzky-Golay filtering method. "Savitzky and Golay (1964)".
• sensing_depth(): remove trailing semicolons
• find_neutron_monitor(): verbose parameter is missing from docstrings
• lat_lon_to_utm(): This method could and probably should be removed. The python package utm does the same thing, is robust, and does not have unnecessary dependencies.

Optional but highly recommended:

• Lint the code for readability and uniformity. I personally like using black for this, but there are many python linters that will do the job