• Put an option in for CoarseningLocalisation that can accept ensembles of functions in and then carry out CL on each but only make one function to inject to at the start of the loop so that not to waste N functions.

• Make changes in scripts, e.g. emd_kernel that use CoarseningLocalisation and make them more efficient by using list/tuple inputs

Target to do a sub-repo of verification tools for Firedrake ensemble forecasts

• Multidimensional Rank Histogram - landed in #23
• CRPS
• Reliability Diagram

Design a localisation scheme for the covariance vector Functions in the Kalman updates implemented by kernels.

• Use similar localisation on the covariance vector as we did in emd_kernel.py - possibly using the degenerate localisation scheme (so not coarsening one) in localisation.py.

• Included sparse matrix compatibility for banded covariance matrices to be efficient

• Created runtime demos with different r_locs in (remember default is None localisation) to see speed-up of Kalman update

We need to correct the localisation arguments r_loc_c and r_loc_f in coupling.py.

• Make only one argument, r_loc in this, as it's very important that same localisation is used, however let both coarse and fine be simply the same r_loc and thus use the same r_loc for both the estimator versions of each level estimators -> this is the only thing needed for consistency!

• Change the way likelihood is calculated by making a function with the constant measurement error variance on all data points and then use that in likelihood.

• Make a DG0 function with all cell centres set to R and then project this to the ensemble space and use it in weight calculation. more of a weight update procedure than an observations change.

• Change demos to match this way of finding observations.

• Need to make it so that the observations can be computed by user by just summing together evaluated function at coordinate and measurement error, not to basis coefficient.

• Just like in the cost tensor computation, place all ensemble functions, that go into dictionary for the ensemble transform kernel, into just one vector function (same with the output functions) and change string loops to fit these indices. This will create speed ups in kernels and also in preallocating the output functions.

This might meaning not using the ensembles appending to dictionary - turns out kalman doesn't use this when using all ensembles as vector functions

• Remove error for the case where user gives a mesh not from a hierarchy. Instead set r_loc = 0 and don't carry on into CoarseningLocalisation. Possibly give a warning this happened.

Warning isn't given.

• Allow a mesh, not part of a hierarchy, to be the mesh of ensemble functions.
• Design convenience function to do this for user, if it isn't part of an ensemble in the first place

• Make measurement error variance, R, an argument into initialization of Observations
• Remove argument R from weight_update
• Whenever R is referenced in weight_update, switch it with Observations.R

To Do:

• We need to make the ensemble_transform, weight_update, Observations and kalman_update functionalities compatible with Constants as to maximise what can be fed into the firedrake-mlmc package, especially EnsembleHierarchy. Basically allow one to carry out the data assimilation techniques to be carried out on Constant representing scalars.

• On Multidimensional Rank Histogram, insert citation into doc-string for Gneiting et al. 08.

Currently, any function that uses emd_kernel, such as ensemble_transform_update needs to be done inside fade/.

• Change the path of the kernel functions emd in emd_kernel.py so it can adapt to any user's path.

• Allow update_Dictionary function to be included in a separate, possibly firedrake_da/utils.py script, so that we can import it from both the new kalman_kernel.py script to be made and the emd_kernel.py script

We want to restructure the kalman code so that the ensemble assimilation is carried out using kernel based approaches, with compilmentary localisation and covariance tensors.

TODO:

• Create C kernels that can carry out Kalman transform.

• Use and create covariance tensors vectors, like in the cost tensor generation in ensemble_transform.py. Alter the covariance.py script.