Implement system to deal with singularities / numerical errors etc. in score functions about pints HOT 7 CLOSED

Probably best to allow nan and inf to be returned from score functions...

from pints.

See #251

from pints.

Hmm, I'd be wary about dealing with such things too nicely and allowing them to happen quietly. In my limited experience it points to bad choices within the model itself, the parameter scalings, or the prior. I'd generally advocate failing loudly with nice suggestions of what to investigate!

from pints.

from pints.

OK.
There's a couple of levels on which we need to deal with these things:

1. In the models we write (not a pints issue!)
2. In ErrorMeasure and LogLikelihood classes provided by Pints. Here we could, for example, check that the input to these measures (the model results) can never contain NaN. And perhaps we could check the returned value for NaN too, but too be honest that would only occur if the data had an inf and the simulation had an inf of the opposite sign, so seems a bit of an odd case to sacrifice performance for?
3. If users define their own error measures or loglikelihoods (or logpdfs) we can only handle things inside the individual methods. I'm inclined to ignore this level, as it'd mean extra code in every method?. We could decide to check for it in the Optimisation and MCMCSampling classes that use Optimisers etc. (so the few "controller" classes that most users will use), as a happy medium?

from pints.

For point 3: We're currently building an architecture that separates controllers (things that check stopping criteria, do parallelisation, talk nicely to the user etc.) from the individual methods (CMAES, PSO, AdaptiveCovarianceMCMC) that propose points to evaluate and then perform some kind of update step. For example see here: https://github.com/pints-team/pints/blob/master/pints/_optimisers/__init__.py#L154-L476

from pints.

In the end, this all boils down to the following convention:

1. If NaNs occur, feel free to let things crash
2. For error measures and (unnormalised) log likelihoods, +inf and @-inf are perfectly acceptable answers so don't treat them in any special way

The only exception to rule 2 is that we ask that the first point for any optimisation/inference problem has a finite error/log_pdf, so that we don't start from a garbage point

from pints.