vahid-sb / mitepid_sim Goto Github PK

This issue is a part of the JOSS review (openjournals/joss-reviews#2880).

Hi @vahid-sb ,
One of the tick boxes in our revision form asks if the functional claims of the software have been confirmed. Another one: "Are there automated tests or manual steps described so that the functionality of the software can be verified?"
I could not find any CI nor a walkthrough on how to carry out the simulations, however, I noticed a lot of scripts in the examples dir...

Could you please include this information? It would be really nice if you add a Continous Integration mechanisms with unit tests / simulations. Preferably, GitHub Actions, so that you keep everything related to the project in one place (however, I would not mind other CI solutions, if you prefer any).

====
EDIT:
I notice there are quite a few claims about the functionality of the package you listed in the README; all of these should be verifiable.

=====
EDIT2:

Later in the README is says:

There are a few scripts in the examples folder in the repository which can be used as examples of what the code can do. To see samples of the plots generated by the code, you can look at this manuscript or this one . All the plots in these two manuscripts are generated using MiTepid_sim.

To have such description is unfortunately, not sufficient. You need to make it transparent. When a paper passes JOSS revision it will get a digital object identifier and will essentially get "immortalised". Therefore, the repository should be as much self-contained as possible, meaning code runs, tests, examples etc. well-documented here.

This issue is a part of the JOSS review (openjournals/joss-reviews#2880).

Hi @vahid-sb,

From the reviewer's list:

Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Please add these explicite. Also, it would be really good if you add a CODE_OF_CONDUCT for the project too :)

This issue is a part of the JOSS review (openjournals/joss-reviews#2880).

Hi @vahid-sb,

I see that the installation is handled by pip, great!

However, in the README you do not provide versions for the packages your work rely on. You should, if you want to do reproducible research. You cannot guarantee that it will work with every possible combination of versions between all dependencies, right? I assume you did not test them all out. Also, you cannot forsee the future and can't tell if the next minor upgrade in numpy will not introduce a change in behaviour, thus breaking your code. However, what you can do is to specify the exact versions which you used for your development and testing. And for these you indeed can guarantee the software will work.

So that is what I would ask you to do: please specify exact versions both in the README but also in the setup.py (so that users always get the same versions of dependencies during the installation).

This issue is a part of the JOSS review (openjournals/joss-reviews#2880).

@vahid-sb

Following the reviewer's checklist:

Do the authors include examples of how to use the software (ideally to solve real-world analysis problems)?

At the moment when I am writting this post neither the functionality nor testing is documented for this project. Therefore I cannot tell what the examples from the directory examples actually do...
So, right now, my answer to the quoted qestion would be: no.

This issue is a part of the JOSS review (openjournals/joss-reviews#2880).

Hi @vahid-sb,

I compiled and looked closer at your paper.md.
In my opinion it requires major revisions before we can proceed with the revisions checks.

My checklist points out:

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?

State of the field: Do the authors describe how this software compares to other commonly-used packages?

Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?

References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

The points I could raise now:

• there is no need to describe the types of models in such great detail. You can just provide a reference to a nice review on epidemiological models, preferably in a similar context :) Instead of that detailed description please include a motivation behind the software "for a diverse, non-specialist audience".
• I notice the section "Statement of need" in the paper but it just renamed section "Introduction" from the README file, isn't it? In general, SoN and Introduction should contain distinct information. In the SoN you should put exactly: What is the problem the software solves, why is it important, who can use it, how does it approach the problem, what are the results, in what form, how is this useful etc.
• I see no information on the "state of the field". Granted, computational/statistical models are quite old and no reasonable person would ask you to recap 50 years of literature but I would like you to do some research on the current technology/tools/packages that people who carry out simulations in epidemiology use. I would like you to have a short section in the paper which presents your tool in the context of the current "standard" procedures and practices. This would be what I imagine as "state of the field".
• I would like you to improve the language in the text. The style fits a markdown file in the repository but it is a little too informal for a paper (for my taste). You should definietely rewrite phrases like: we can stratify the into various relevant groups, for example, age-groups., There are a few scripts, what the code can do, you can look and the excessive use of word quite.
• I do not check references yet since this will for sure change. You need to include the references to epidemiological models as well as software packages (state of the field).
• In general, this publication looks more to me like a copy-paste of the README.md file, which should not be the case. The publication should be more general and aimed "for a diverse, non-specialist audience".

I hope these bullet points will guide you in the right direction.
Please let me know once you push version 2.0 of the publication.

This issue is a part of the JOSS review (openjournals/joss-reviews#2880).

@vahid-sb

Following the review checklist for the repository's documentation:

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?

Could you please include this information? I see that, in principle, one could extract it from the "Introduction" section in the README but a statement of need is more general and high-level. As far as I am concerned it would be OK just to add one more short section to that file.

This issue is a part of the JOSS review (openjournals/joss-reviews#2880).

@vahid-sb
I cannot find any documentation on how to actually use mitepid... Did I skip it?
The core part of the repository (i.e. your package) should be well documented for the users to know how to use it in their reseach.

This issue is a part of the JOSS review (openjournals/joss-reviews#2880).

@vahid-sb
I am not sure if including your Copyright notice on top of the license text is the correct way to go about it.
Now it states:

Copyright (c) 2019 Vahid Samadi Bokharaie

GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007

Copyright (C) 2007 Free Software Foundation, Inc. https://fsf.org/
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

So who exactly holds the rights? Vahid Samadi Bokharaie or Free Software Foundation?
Also, please notice that it explicitly says that changing the license document is not allowed...

Could you please look deeper into the case, paste here some info on how to provide a copyright note for a GPL-3 work and adjust accordingly?