I have fixed the simulator such that the proper models of source, gain, fluctuations, cross talk and rfi are simulated. The generation script will need to be refactored and comitted to the repo. Below an example of the simluator outputs can be shown:
Intermedia performance evaulation:
As discovered in a meeting this week I found that the previously simulated data was trivial for RFI detection and could be detected using a single threshold with an AUROC of above 0.9.
With the newly simulated dataset we get an AUROC of 0.78 with a single threshold, this is still well above random, but is an improvement relative to the previous result.
I wonder if it is meaningful to really try improve this result (i.e. decrease the single threshold performance) or just continue as is
As it stands the UNET achieves an AUROC of 0.97 and NLN (using a VAE) gets 0.96, however on AUROC and IOU we get lower performance.
The AOFlagger obtains a 0.87, this is not properly optimised but it is at least now better than hte simple threhsold
Furthermore this 0.97 result is inline with the previous paper that made use of this data and a UNET architecture.
TODO
Redo the OOD results
Without RFI_DTV, RFI_stations, RFI_Impulse, RFI_Scatter
Only RFI_DTV,
Tune the AOFlagger for the new HERA data
Redo the poor heuristic results
LOFAR
I am still labelling the LOFAR data, hopefully this can be completed sooner than later.