Example of current non-informative error msg:

code <- nimbleCode({
alpha[1:K] ~ ddirch(theta[1:K])
})
K <- 5
m <- nimbleModel(code, inits = list(theta = rgamma(K, 3, 1)))
Error in getSymbolicParentNodesRecurse(x, constNames, indexNames, nimbleFunctionNames) :
Error, R function : has non-replaceable node values as arguments. Must be a nimble function.

Non-scalar replacements for node functions

Allow for non-scalar replacement quantities to become member variables (setup outputs) of node functions.

1. fix help info so that removeSamplers() and getSamplers() indicate that one can pass vectors of node or var names

2. addSampler help info is missing 'target'. Also my thought is that 'target' should be the first arg, not 'type'

3. (maybe worth discussion) I think we want getSamplers() to return an R object that contains info about the sampler that a user could then work with programmatically. Perhaps it should just be the (vector of) ref class object as is from samplerSpecs. It could return this invisibly with print=TRUE being the standard functionality or the return could be the main poitn, with print=FALSE the standard functionality.

3a) if getSamplers() returns list of sampler specs, it would be nice to have each element named based on the vertex being sampled

3b) (enhancement for down the road perhaps) we might want to allow a user to modify a spec (i.e., change a value in the control list) without having to create a totally new sampler and delete the old one just to, say, change the adaptation time

norm() in R and C nimble functions differ as the R default seems to be the "1" norm (the 'type' argument) and the C version is the Frobenius

isData will accept variables rather than node names, but the help info on the nodeNames argument states:

nodeNames: A character vector of node names. This must be
entirely node names, not model variables.

Same issue for the main argument to expandNodeNames().

I would just change the wording, but wanted to run this by Daniel first.

e.g.,

log(mu) ~ dnorm(0,1)

user will initialize mu, but log_mu will not be initialized

would need to check declInfo for 'indexExpr', 'valueExpr' and for loop indices and built up an object ConstantsUsed and then warn if any of the elements were zero.

or add nimStep as a synonym?

Hi nimble-devs,

Haven't quite wrapped my head around the expected format for the data argument to nimbleModel(). For instance, it seems that x is a valid name but not X; I get the error:

variable name not suitable for setData(): X

when using a data.frame with X as a column name (yes, I realize that data expects a list, though a data.frame appears to work, presumably by some automatic coercion. Why the default data format is a list and not a data.frame is another thing that isn't clear to me). I think I may be missing something, but this seems like a somewhat strange restriction. Thanks for the help.

Cheers,
Carl

p.s. greatly enjoying v0.3.0; thanks for the thorough release notes on the syntax changes.

current plan:

• all core C code use doubles and has some checking of inputs
• interface code is place for more checking
• only provide types in inputList when dimension > 1, and always use double
• tell user in user-defined section of manual that we assume double(0) and they should use double and not int

work on handling of NA and NaN in C code for distribution functions

• add tests of conjugacy to testing system
• have Daniel's run-time checking of numerical values against M-H be optional based on user-provided flag in nimbleOptions

I am excited about the release and can't resist the chance to submit issue #1

Here is my bug report:

I want to check out your code but can't find any of the files.

Would be nice to do, particularly as the current warning at run time doesn't say what RHS only variable is missing, but I don't see a good way to do this, given the variety of ways that a single variable can contain parameters, data (LHS), and RHS only. Furthermore, part of a variable might never get used in a model so it would be fine to have NAs for that part.

nimbleFunction() issues a warning (from reference class creation) whenever we use the assignment form of values() <-

This can be seen by creating such a NF:

nfdef <- nimbleFunction(
setup = function(model, nodes) {},
run = function(p = double(1)) {
values(model, nodes) <- p
}
)

It seems RHSonly nodes should not be assigned nodeFunctionIDs, but still should be handled correctly by getNodeNames(), e.g.

pareto, double exponential/laplace, inverse wishart, inverse gamma, generalized gamma, multivariate t

also make sure to add standard R help info for any R distribution functions

At this point, these propagate to C++ and cause compilation errors as they conflict with C++ OOP.

More generally might deal with cases like a user user 'log' as a function and as a variable/argument.

Also avoid conflict of user variable/arg names with variable names generated when lifting in nimble code.

e.g., trying to build a C MCMC via compileNimble but forgetting to compile the model, gives this error message:

code <- nimbleCode({
y ~ dnorm(mu,1)
mu ~dnorm(0,1)
})
m <- nimbleModel(code, data = list(y = 1), inits = list(mu = 0))
sp <- configureMCMC(m)
Rmcmc <- buildMCMC(sp)
Cmcmc <- compileNimble(Rmcmc)
Error in cModel$.basePtr : $ operator not defined for this S4 class

this would allow a user to catch run-time errors. nimWarn would also be nice.

We have a bunch of methods related to MCMC that take a print argument.

We might do the following:

1. have the default value be the result of getNimbleOptions('verbose'), and have the default nimbleOption value for verbose be FALSE. This mimics R's 'verbose' option

2. change 'print' to 'verbose' and in general in other cases where we want this sort of thing have the arg be 'verbose'

3. possibly have getNimbleOptions('verbose') default to getOptions('verbose')

Any thoughts? (Daniel, in particular)

Here's an example of something awry:

code <- nimbleCode({
ind <- mu < 0
})
m <- nimbleModel(code)

cm$ind
[1] NA
calculate(cm,'ind')
[1] 0
cm$ind
[1] 0

The issue here is why the NA is not being propagated to 'ind', given that 'mu' is NA at the moment.

e.g., for non-negative parameters, think about:

• sampling on log scale
• reflecting proposals back across zero (I forget the name of this)

Should we combined these two functions by allowing getNodeNames to take a 'nodes' argument?

Also, I think the help info on the nodeNames arg to expandNodeNames is not quite right. It says that it takes a vector of nodeNames, but it could also take variable names, right?

(Replacing original post here):

It would be nice to use/imitate R's scoping for nimbleFunctions and models. E.g. it would be nice to write

foo <- function() {

nf1 <- nimbleFunction(...)

nf2 <- nimbleFunction(...)  use nf1 in here

mc <- nimbleCode(...) use nf1 or nf2 in here

m <- nimbleModel(mc)

compiledStuff <- compileNimble(...) compile nf1, nf2, mc, etc. here

compiledStuff

}

This would require several things to work:

• When we process a model definition (i.e. nimbleModel), we look in the right environments for user-defined nimbleFunctions (or other objects, potentially), i.e. we find them from the environment from which nimbleModel was called.
• When we compile, we again look in the right environment(s) to find objects.
• During uncompiled (R) execution, one nimbleFunction calling another finds it via R's scoping rules.

We could accomplish the first two (I got through the first before reaching the conclusion of this note), but R's reference classes make the third apparently impossible. We build models, the nodeFunctions within models, and other nimbleFunctions by generating and evaluating code for new reference class definitions. The member functions in these classes do not use R's scoping, so we're stuck. In other words, we could arrange to build m, but then calculate(m) would attempt to use its nodeFunctions, which are reference class objects, which might contain a call to nf1 but wouldn't find it as one would expect from lexical scoping. Again, this is a limitation of R's reference class system.

As a result, users must define their models and nimbleFunctions in the global environment.

also do we want a function that tells us the dimension of a node?

handle for-loop indexes consistent with BUGS & jags. e.g., for(i in 1:3+1) {}, should give lowest precendence to the ':' operator

Anything that might take more than 15-30 seconds or so might have a progress bar.

A first step would be to have a general function to help with this. There might be some code available on the web.

Currently, my understanding is that arg matching by name works for standard nimbleFunctions but something is up with how we process nodefunctions such that in a nodefunction, the arg names are ignored.

Perry, I know we chatted briefly about this, but I'm forgetting what was said. If you can point me in the right direction, I'm happy to take this on.

We had this as an issue in the issues file. Do we want to move ahead with this and have a nimbleOption to turn this off?

also have sigma and var for dt

Currently if we have
mu.c ~ dconstraint(mu < 1)
the user needs to manually lift mu<1:
ind <- mu<1
mu.c ~ dconstraint(ind)

I believe this arises more generally when using a boolean expression as a distribution argument/parameter.

When T() is used, modify range and set truncation flag to TRUE

Also tell users to do this if add user-provided distribution

Once keyword processing is dealt with, we should update the manual to reflect the additional flexibility we allow.

Also note at that location in the manual that user defined dist functions can be used. (Note need to deal with keyword processing for user-defined too via matchFunctions analog in nimbleUserNamespace)

The following code provides a non-informative warning message, does not error out, and produces a non-functional model, with a node function for "mu[]"

Note the syntax error that the user forgot to have {} wrap the two lines intended to be part of the for loop.

code = nimbleCode({
for(i in 1:3)
y[i] ~ dnorm(mu[i],1)
mu[i] ~ dnorm(0,1)
})
m = nimbleModel(code)

e.g. exp is gamma, unif is everything, chisq is gamma, etc.
scalar N with mv N descendants is conj (might be hard)

We might want to change this to put in NA_REAL if that is possible. Might be a general change that C modelValues are initialized with NA_REAL?

Note that when returnScalarComponents = FALSE, we are returning extra "y" nodes.

code <- nimbleCode({
for(i in 1:4) {
y[i] ~ dnorm(mu[i],1)
}
mu[1:2] ~ dmnorm(z[1:2],I[1:2,1:2])
mu[3:4] ~dmnorm(z[1:2],I[1:2,1:2])
})

m <- nimbleModel(code)
m$getDependencies("mu[2:3]")
[1] "mu[1:2]" "mu[3:4]" "y[1]" "y[2]" "y[3]" "y[4]"
m$getDependencies("mu[1]")
[1] "mu[1:2]" "y[1]" "y[2]"
m$getDependencies("mu[1:2]")
[1] "mu[1:2]" "y[1]" "y[2]"
m$getDependencies("mu[2]")
[1] "mu[1:2]" "y[1]" "y[2]"

m$getDependencies("mu[2:3]", returnScalarComponents = TRUE)
[1] "mu[2]" "mu[3]" "y[2]" "y[3]"
m$getDependencies("mu[1]", returnScalarComponents = TRUE)
[1] "mu[1]" "y[1]"
m$getDependencies("mu[1:2]", returnScalarComponents = TRUE)
[1] "mu[1]" "mu[2]" "y[1]" "y[2]"
m$getDependencies("mu[2]", returnScalarComponents = TRUE)
[1] "mu[2]" "y[2]"

Potential for massive improvement in the efficiency of conjugacy checking.

In a number of cases our error msgs are non-informative when no brackets are provided for non-scalars (see below) (note we are much better when brackets are provided but the indexing is missing). One possibility would be to write a little system to check code based on knowing dimensions of distribution values and parameters and knowing expected dimensions for various DSL functions.

Examples:
code <- nimbleCode({
b[1:2] ~ dmnorm(mn[1:2], I[1:2,1:2])
z <- Z%*%b
})

m <- nimbleModel(code, inits = list(mn = rep(0, 2), I = diag(rep(0,2)), b = c(1,2), Z =matrix(rnorm(4),2)))
Error in parentExprReplaced[[iI + 2]] :
object of type 'symbol' is not subsettable

code <- nimbleCode({
b ~ dmnorm(mn, I)
})

m <- nimbleModel(code, inits = list(mn = rep(0, 2), I = diag(rep(1,2)), b = c(1,2)))
cm <- compileNimble(m)
Error: Error, sizeUnaryCwiseSquare was called with an argument that is not a matrix

code <- nimbleCode({
b ~ dmnorm(mn, I)
z[1:2] <- Z[1:2,1:2]%*%b[1:2]
})

m <- nimbleModel(code, inits = list(mn = rep(0, 2), I = diag(rep(1,2)), b = c(1,2), Z =matrix(rnorm(4),2)))
Error in if (is.numeric(v) && any(v < 0)) { :
missing value where TRUE/FALSE needed