Git Product home page Git Product logo

jabba's People

Contributors

jabbamodel avatar mkapur avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jabba's Issues

MCMC diagnostics problem and proposed fix

Hello, I found a problem on line 160 of of your fit_jabba() function here: https://github.com/jabbamodel/JABBA/blob/master/R/fit_jabba.R

I found it while trying to see if I could modify your code to produce bilingual (English and French) figures and also use the plotting functions from bayesplot and shinystan on JABBA output by saving the mod object created by R2jags::jags() on line 139 in the output of fit_jabba().

The line in question is:
posteriors = mod$BUGSoutput$sims.list

This code concatenates the chains and randomizes the order of the draws. This is because of a bug in R2jags and is discussed here by, among others, Martyn Plummer, the author of rjags etc. https://sourceforge.net/p/mcmc-jags/discussion/610037/thread/cc61b820/. Essentially .$sims.list and .$sims.matrix do not maintain draw order. The author of the R2jags package subsequently modified the R2jags:::as.mcmc.jags() function to maintain the correct order of the draws.

As a result, the model convergence diagnostics from coda, coda::geweke.diag() and coda::heidel.diag(), as well as visual inspection of the mcmc chains from jbplot_mcmc() cannot be used as criteria for model convergence. However, this bug does not change parameter estimates so no worries there.

This can easily be fixed.
Line 160 can be replaced with :

mod_mcmc <- R2jags:::as.mcmc.rjags(mod)
modmcmc <- as.data.frame(as.matrix(mod_mcmc))

These objects can then be saved along with all the other fit_jabba output. For example, with these lines of code after line 432:

jabba$jags <- mod
jabba$mcmc <- mod_mcmc

From this, the model convergence diagnostics tools you provide can be used internally or externally. In addition, the saved mcmc and fitted jags model could subsequently be used by other packages such as tidybayes, bayesplot, shinystan, etc.

I also have another proposal for your code. It could eventually be included as an argument in the function that be turned on or off. The code essentially updates the jags model until Gelman and Rubin's Rhat diagnostic for all estimated parameters is below 1.1. Note that I use the pipe operator %>% from magrittr.

 # Andrew added this next bit ----
  # Effective Sample (ESS) should be as large as possible, although for most applications,
  # an effective sample size greater than 1000 is sufficient for stable estimates (Bürkner, 2017).
  # The ESS corresponds to the number of independent samples with the same estimation power as the N auto-correlated samples.
  # It is is a measure of “how much independent information there is in auto-correlated chains” (Kruschke 2015, p182-3).

  # get model summary and examine Rhat, DIC, effective sample size, and Gelman Rubin diagnostic
  cat("Initial JAGS run summary: \n")

  cat("\n DIC = ", mod$BUGSoutput$DIC, "\n")

  mod_summary <- mod$BUGSoutput$summary %>% as.data.frame()
summary(mod_summary)

  cat("\n Checking convergence via the Gelman-Rubin statistic (potential scale reduction) comparing within and between chain variance ... \n")
  if (any(mod_summary$Rhat > 1.1)) {
    message("\n Rhat values for one or more of the estimated values are above 1.1 indicating the model has not converged ... updating model with more iterations and treating initial as burn in \n")
    message("\n Mean effective sample size among parameters = ", round(mean(mod_summary$n.eff)),"\n")
    cat("\n DIC = ", mod$BUGSoutput$DIC, "\n")
    # update until model shows no signs of non convergence
    mod <- R2jags::autojags(mod, n.iter = 500000, n.thin = 50, n.update = 3)
  }else{
    message("\n Rhat values < 1.1 indicate lack of evidence for non convergence \n")
    message(" \n Mean effective sample size among parameters = ", round(mean(mod_summary$n.eff)),"\n")
    cat("\n DIC = ", mod$BUGSoutput$DIC, "\n")
    mod <- mod
  }
rm(mod_summary)

# Do a second round of n = 3 if still no convergence
  mod_summary <- mod$BUGSoutput$summary %>% as.data.frame()

  cat("\n Double checking convergence via the Gelman-Rubin statistic (potential scale reduction) comparing within and between chain variance ... \n")
  if (any(mod_summary$Rhat > 1.1)) {
    message("\n Rhat values for one or more of the estimated values are above 1.1 indicating the model has not converged ... updating model with more iterations and treating initial as burn in \n")
    message("\n Mean effective sample size among parameters = ", round(mean(mod_summary$n.eff)),"\n")
    cat("\n DIC = ", mod$BUGSoutput$DIC, "\n")
    # update until model shows no signs of non convergence
    mod <- R2jags::autojags(mod, n.iter = 500000, n.thin = 50, n.update = 3)
  }else{
    message("\n Rhat values < 1.1 indicate lack of evidence for non convergence \n")
    message(" \n Mean effective sample size among parameters = ", round(mean(mod_summary$n.eff)),"\n")
    cat("\n DIC = ", mod$BUGSoutput$DIC, "\n")
    mod <- mod
  }

# Extract mcmc chains (consider tidybayes in the future)
  mod_mcmc <- R2jags:::as.mcmc.rjags(mod)
  modmcmc <- as.data.frame(as.matrix(mod_mcmc))

End of tweak ----

Happy to discuss and hope this is of use to you.

Cheers,

Andrew

Wrong ylab max value

Hi, take a look to function plot():

plot(years,(jabba$catch),type="n",ylim=c(0,max(predC,na.rm=T)),lty=1,lwd=1.3,xlab="Year",ylab=paste0("Catch ",jabba$settings$catch.metric),main="")

calculation of ylim is wrong:

predC = jabba$est.catch
 plot(years, (jabba$catch), type = "n", ylim = c(0, max(predC, 
            na.rm = T)), lty = 1, lwd = 1.3, xlab = "Year", ylab = paste0("Catch ", 
            jabba$settings$catch.metric), main = "")

this part:

ylim = c(0, max(predC, na.rm = T)

should be:

ylim = c(0, max(predC[,-1], na.rm = T)

because predC is a dataframe and first column - year index, should be removed ... Thanks.

Height is not used when single.plots = F for jbplot_logfits

Looking at the jbplot_logfits code. When single.plots = FALSE, the input height is not used. Rather a copy of the default is used.

Current code is:
if(as.png==TRUE){png(file = paste0(output.dir,"/logFits_",jabba$assessment,"_",jabba$scenario,".png"), width = 7, height = ifelse(n.indices==1,5,ifelse(n.indices==2,3.,2.5))*round(n.indices/2+0.01,0),
res = 200, units = "in")}

Guessing it should be:
if(as.png==TRUE){png(file = paste0(output.dir,"/logFits_",jabba$assessment,"_",jabba$scenario,".png"), width = 7, height = height, res = 200, units = "in")}

Undocumented data set: iccat

R package checker found no data documentation for the iccat data object

> checking for missing documentation entries ... WARNING
  Undocumented code objects:
    'iccat'
  Undocumented data sets:
    'iccat'
  All user-level objects in a package should have documentation entries.
  See chapter 'Writing R documentation files' in the 'Writing R
  Extensions' manual.

Update 10/4/2022: checking the JABBA R package yielded iccat, catch , catch.n, index, and om as "undocumented data sets":

❯ checking for missing documentation entries ... WARNING
  Undocumented code objects:
    'catch' 'catch.n' 'iccat' 'index' 'om'
  Undocumented data sets:
    'iccat' 'catch' 'catch.n' 'index' 'om'
  All user-level objects in a package should have documentation entries.
  See chapter 'Writing R documentation files' in the 'Writing R
  Extensions' manual.

addBFrac: 'jabba' global variable

R package checker found an single instance of the jabba global variable in addBfrac.

JABBA/R/jabba_utils.R

Lines 487 to 499 in 425f1cd

#' addBfrac()
#'
#' add biomass reference to kb ouput as fraction Bmsy or B0, e.g. for Blim or MSST
#' @param jabba bfrac fraction of Bmsy or B0
#' @param base defines biomass base "bmsy" or "b0"
#' @param quantiles default is 95CIs as c(0.025,0.975)
#' @return
#' @export
addBfrac <- function(kb, bfrac=0.5, bref = c("bmsy","b0"),quantiles = c(0.025,0.975)){
if(!is.null(kb$settings)){
if(is.null(jabba$kbtrj)) stop("rerun with fit_jabba(...,save.trj = TRUE)")
kb = kb$kbtrj
}

Was jabba$kbtrj meant to be kb$kbtrj?

example code block error

jbplot_Crisk

Example roxygen code block for jbplot_Crisk is missing a reference to fit2. This will cause an error if JABBA is checked with its examples. Error checks could be bypassed via \dontrun, however it looks like jbplot_Crisk documentation was designed to have working example code.

JABBA/R/jbplot_fwcrisk.R

Lines 61 to 76 in 425f1cd

#' @examples
#' data(iccat)
#' bet = iccat$bet
#' # Fit Fox and Schaefer
#' jb1 <- build_jabba(catch=bet$catch,cpue=bet$cpue,se=bet$se,scenario = "Fox",model.type="Fox")
#' fit1 = fit_jabba(jb1,quickmcmc=TRUE,verbose=TRUE)
#' # Compare
#' jbplot_ensemble(fit1)
#' # Do forecast
#' prjc = fw_jabba(list(fit1,fit2),quant="Catch",type="abs",imp.values = seq(60,100,1)*1000)
#' jbplot_Crisk(prjc,subplots=1)
#' jbplot_Crisk(prjc,subplots=2)
#' jbplot_Crisk(prjc,subplots=3) # MSST as (1-0.5)*Bmsy with bfrac =0.5
#' jbplot_Crisk(prjc,subplots=3, bfrac=0.2,bref="b0") # Blim = 0.2 B0
#' jbplot_Crisk(prjc,subplots=3, bfrac=0.2,bref="b0",ylabs=expression(B/B[lim])) # Blim = 0.2 B0

Looking further into the example code, fit2 may be extraneous; Removing the variable fixes the error without issue.

jbplot_ensemble

Another example code error was found at jbplot_ensemble. Here is the error log:

 Running examples in 'JABBA-Ex.R' failed
  The error most likely occurred in:
  
  > base::assign(".ptime", proc.time(), pos = "CheckExEnv")
  > ### Name: jbplot_ensemble
  > ### Title: jbplot_ensemble()
  > ### Aliases: jbplot_ensemble
  > 
  > ### ** Examples
  > 
  > data(iccat)
  > bet <- iccat$bet 
  > 
  > # Fit Fox and Schaefer
  > jb1 <- build_jabba(catch=bet$catch, 
  +                    cpue=bet$cpue,
  +                    se=bet$se,
  +                    scenario = "Fox",
  +                    model.type="Fox")
  
  ><> Prepare JABBA input data <><
  
  
  
  ><> Assume Catch with error CV = 0.1 <><
  
  
  
  ><> Model type:Fox <><
  
  
  ><> Shape m =1.001
  
  
  ><> K prior mean =1079485.2368and CV =1(log.sd = 0.832554611157698)
  
  
  ><> r prior mean =0.2and CV =0.532940350027788(log.sd = 0.5)
  
  
  ><> Psi (B1/K) prior mean =0.9and CV =0.25withlnormdestribution
  
  
  
  
  ><> ALWAYS ENSURE to adjust default settings to your specific stock <><
  
  
  > jb2 <- build_jabba(catch=bet$catch,
  +                    cpue=bet$cpue,
  +                    se=bet$se,
  +                    scenario = "Schaefer",
  +                    model.type="Schaefer")
  
  ><> Prepare JABBA input data <><
  
  
  
  ><> Assume Catch with error CV = 0.1 <><
  
  
  
  ><> Model type:Schaefer <><
  
  
  ><> Shape m =2
  
  
  ><> K prior mean =1079485.2368and CV =1(log.sd = 0.832554611157698)
  
  
  ><> r prior mean =0.2and CV =0.532940350027788(log.sd = 0.5)
  
  
  ><> Psi (B1/K) prior mean =0.9and CV =0.25withlnormdestribution
  
  
  ><> ALWAYS ENSURE to adjust default settings to your specific stock <><
  
  
  > fit1 <- fit_jabba(jb1,
  +                   quickmcmc=TRUE,
  +                   verbose=TRUE)
  Compiling model graph
     Resolving undeclared variables
     Allocating nodes
  Graph information:
     Observed stochastic nodes: 199
     Unobserved stochastic nodes: 358
     Total graph size: 4305
  
  Initializing model
  
  
  ><> Produce results output of Fox model for jabba Fox <><
  
  
  
  ><> Scenario Fox_Fox completed in 0 min and 33 sec <><
  
  > fit2 <- fit_jabba(jb2,
  +                   quickmcmc=TRUE,
  +                   verbose=TRUE)
  Compiling model graph
     Resolving undeclared variables
     Allocating nodes
  Graph information:
     Observed stochastic nodes: 199
     Unobserved stochastic nodes: 358
     Total graph size: 4305
  
  Initializing model
  
  
  ><> Produce results output of Schaefer model for jabba Schaefer <><
  
  
  
  ><> Scenario Schaefer_Schaefer completed in 0 min and 33 sec <><
  
  > # Compare
  > jbplot_ensemble(list(fit1,fit2))
  Error in kb[kb$type == "prj", ] : incorrect number of dimensions
  Calls: jbplot_ensemble -> pmin
  Execution halted

In the example code block, a list with fit1 and fit2 passes trough jbplot_ensemble as kb. Error occurs where it attempts to look for columns or find data frames subset kb$type == "prj". fit1 and fit2 does not have this column in which throws the error.

JABBA/R/jbplot_ensemble.R

Lines 158 to 160 in 425f1cd

# Constraint on F/Fmsy
kb$harvest[kb$type=="prj"] = pmin(kb[kb$type=="prj",]$harvest,fmax)
kb$H[kb$type=="prj"]= pmin(fmax*median(kb[kb$type=="prj",]$H/kb[kb$type=="prj",]$harvest),kb[kb$type=="prj",]$H)

To resolve this issue Separate list and data frame centric functionality.

how to get the output results in CSV format

It is easy to get the output in terms of plots, however, when one wants to use the result for management measures, the absolute results might be required in csv / tsv format. How to get the result in numeric format?

A little bug in jbplot_retro

Hi all,

I think there is a little bug in lines 1291, 1322, and 1334 in the jabba_plots.R file (related to the function jbplot_retro, where you added a few lines to put the Mohn's rho value at the top of each panel).

The current code is
legend("top", paste0("Mohn's rho = ",round(mean(rho[i-1,k]),2)),bty="n",y.intersp=-0.2,cex=0.9)

but it should probably be
legend("top", paste0("Mohn's rho = ",round(mean(rho[,k]),2)),bty="n",y.intersp=-0.2,cex=0.9)

so that it is the average of all the peels.

Cheers,

TinyPic missing images in Model Diagnostic Vignette

JABBA's Model Diagnostics Vignette has image references linked from former image hosting site tinypic; tinypic ceased image hosting operations. Knitting the vignette as a html_document will show the "missing" tinypic resource.

Screenshot 2022-02-08 154055_small

jbplot_hcxval: Undefined global variable 'valid' (+fix)

R package checker detected global variable valid in jbplot_hcxval. Occurs to overwrite over any "valid" (or nonoverlapping?) indices from the hindcast horizon.

JABBA/R/jabba_plots.R

Lines 1825 to 1837 in 425f1cd

for(i in 1:length(indices)){
if(nrow(d.[d.$name%in%indices[i] & d.$year>styr & d.$retro.peels%in%peels[1],])>1){ # Only run if overlap
if(single.plots==TRUE){
if(is.null(width)) width = 5
if(is.null(height)) height = 3.5
Par = list(mfrow=c(1,1),mar = c(3.5, 3.5, 0.5, 0.1), mgp =c(2.,0.5,0), tck = -0.02,cex=0.8)
if(as.png==TRUE){png(file = paste0(output.dir,"/hcxval_",hc$scenario,"_",indices[i],".png"), width = width, height = height,
res = 200, units = "in")}
if(add==F){
if(as.png==TRUE | indices[i]==valid[1]) par(Par)
}
}

Update

Adding back the valid code from jbmase corrects this.

JABBA/R/jabba_utils.R

Lines 386 to 389 in 425f1cd

valid = NULL
for(i in 1:length(indices)){
if(nrow(d.[d.$name%in%indices[i] & d.$year>styr & d.$retro.peels%in%peels[1],])>1){ # Only run if overlap
valid=c(valid,paste(indices[i]))}

If this is not desirable, consider using an alternative for "valid" (or nonoverlapping?) indices from the hindcast horizon

jbplot_Crisk: 'output,.dir' and 'jbs' globals

Simlar to #12

R package detected output.dir and jbs as undefined global variables here In jbplot_Crisk. This is used to construct a PNG filepath.

if(as.png==TRUE){png(file = paste0(output.dir,"/Risk_",jbs$assessment,".png"), width = 7, height = 6,

output.dir could be substituted for plotdir.

What does jbs , or jbs$assessments needed for to create a PNG file for jbplot_Crisk plots?

Plot forward forecast wrong path

Found save image issue jbplot_ensemble(..., as.png = T) in jbplot_ensemble.R#L201 :

if(as.png==TRUE){png(file = paste0(output.dir,"/",prefix,"_",jbs$assessment,".png"), width = 7, height = 8,  res = 200, units = "in")}
par(Par)
}

as can been seen variables output.dir, jbs and prefix are not defined in function arguments and function body. Requires to rewrite save features with new argument naming: plotdir, filenameprefix.

if(as.png==TRUE){png(file = paste0(output.dir,"/",prefix,"_",jbs$assessment,".png"), width = 7, height = 8,

JABBAZ.r: Reference to rData file

Lines in undocumented JABBAZ.r refer to Research/Laurie/JabbaZ/om.75.RData. This file is not found at this repo.

JABBA/jabbaz.R

Lines 3 to 4 in 425f1cd

setwd("C:/Work/Research/Laurie/JabbaZ")
load("C:/Work/Research/Laurie/JabbaZ/om.75.RData",verbose=T)

bug? unnecessary addition NAs for conv.catch on line 79 in JABBAv1.1.R ?

In JABBAv.1.1.R, line 79

conv.catch = as.numeric(rbind(matrix(rep(NA,(styr.I-1)*n.catches),styr.I-1,n.catches),as.matrix(catch[,-1])))

this statement added extra NAs for catch matrix where in many cases may be not necessary.

Our data sets of catch and cpue series are with the same length and with above line, it doesn't work. We simply changed it as below and it works:

conv.catch = as.matrix(catch[,-1])

Regards
Roy

JABBA example code performance (Hindcasting)

The examples for jbhcxval and hindcast_jabba is significantly more intensive than JABBA's other examples. Hindcasting Performances may contribute to the issue.

✔  checking examples (15m 33.7s)
   Examples with CPU (user + system) or elapsed time > 5s
                     user system elapsed
   hindcast_jabba  264.82   6.22  271.19
   jbhcxval        263.20   6.10  269.39
   jbretro          88.49   2.36   90.94
   jbplot_retro     88.10   2.41   90.55
   jbplot_ensemble  59.08   2.69   61.79
   jbplot_Crisk     41.68   2.03   43.75
   mp_jabba         34.64   0.67   35.33
   fit_jabba        22.39   0.86   23.27
   jbrunstest       22.54   0.58   23.14
   jbplot_runstest  22.43   0.55   22.97

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.