see title

Maybe redefine the add attribute of the SST object class.
If one wants to add a SST observation to a previous one (or list) (as they are hourly files).

They should be updated when changing Mini-arrays within the same instance

Allow plotData() to modify the plot display

• Make use of dask? https://docs.dask.org/en/latest/
• RAW: Cedric's code: https://gitlab.obspm.fr/cviou/nenufar_raw_tf_readers/-/blob/master/nenufar_raw.py
• dedispersed during reading of .lane file rather than reading first and dedispersing (might be more computing time efficient)
• rebin
• correct for pointing jumps
• add a backgound method to SData
• clean?
• save in FITS/HDF5...

Implement an rfi cleaning algorithm before averaging in time and frequency.

Correct undysputed data for the pointing jumps every 6 min

Hello,
I am trying to create a 'noise cube',
while doing so I tried and failed to reproduce the outputs of the SEFD and sensitivity functions.
Even if the formulas are 'simple', I can't find where the difference comes from.
Is it a bug or I forget something in the formula?

import nenupy.instru as NenuTele
from astropy import constants as const

freq = 80 * units.MHz
Tsys = NenuTele.instru.inst_temperature(freq=freq) + NenuTele.instru.sky_temperature(freq=freq)
A = NenuTele.instru.effective_area(miniarrays=0)

tint = 100 * units.s
df = 196000 * units.Hz

print( 'sefd : ', (2 * const.k_B * Tsys / A ).to('Jy') )
print( NenuTele.instru.sefd( freq, antennas=None, miniarrays=0) )

print( 'sensitivity : ', ( 2 * const.k_B * Tsys / ( A*np.sqrt( tint*df )  ) ).to('Jy')  )
print( NenuTele.instru.sensitivity(mode='beamforming', freq=freq, miniarrays=0, dt=tint, df=df) )

sefd formula : 32062.404637411935 Jy
nenupy out : 82062.88146091324 Jy
sensitivity formula : 7.242158994011914 Jy
nenupy out : 18.536115483777987 Jy

It could be nice to have it up 85 MHz, or at least a very raw estimates, so that ABeam et al. can work to frequencies >= 80 MHz.

There is an stdout redirection in Skymodel.get_sky_model(), to kill some too verbose functions I presume, that prevent print() to work as epxected in the notebook after this function is called. I would suggest to store the current stdout before the verbose function call and restore it after.

data selection keywords are passed to plot functions

Ability to call a fake BST observation

Kind of 90 deg rotation in the UV coverage. This seems to be compensated by the image computation.

SST obstart and obstop are defoned for the whole observation rather than a particular file selected by the user.

TESTANT analog pointing file altazA does not have the same structure compared to regular observations.
Make AnalogPointing able to read it.

;========================================================================================
; az cor, el cor : method 'mix'
;YYYY-MM-DD HH:NN:SS  ANA      az            el      az cor        el cor      Frq  el a pointer
2020-11-20T14:16:10Z 0000   180.0184      83.4188   180.3685      83.3130     50MHz   83.3130
2020-11-20T15:16:55Z 0000     0.0000      90.0000     0.0000      90.0000     50MHz   90.0000

Since the release of matplotlib 3.3.3, the title does not appear for HpxSky plots (it was OK with matplotlib 3.2.1).

If a SST file starts after midnight, while the obs beginning is before, the parset name can't be found

IDL:
idl "cmd_idl_J2000_liste_full.pro" -args 30 43200 40 1 2020-09-09 350.85 58.815
-->
2020-09-09T23:57:40Z 359.98987 78.448215 2020-09-09T18:07:40Z 2020-09-10T05:47:40Z 2020-09-09T11:57:40Z 2020-09-10T11:55:40Z

nenuFAR_J2000_liste_Full, dtsec, ahsec, azel_min1, azel_min2, date1, ra1_2000, dec1_2000, date2, ra2_2000, dec2_2000.....

Donc si j'identifie :
dtsec = 30 secondes de résolution
ahsec = 43200 (12 h en secondes)
azelmin1 = 40 degrés
azelmin2 = 1 degré
date1 = 2020-09-09 --> converti en 2020-09-09 00:00:00
ra1_2000 = 350.85 degrés (RA de Cas A)
dec1_2000 = 58.815 degrés (Dec de Cas A)

courbe popup:
idl "cmd_idl_J2000_Elevation_full.pro" -args 30 43200 2020-09-09 350.85 58.815

les arguments de la ligne de commande IDL pour les élévation 1° et 40° sont
60 43200 40 1 2020-09-08 350.85 58.815
le retour est un tableau de plusieurs lignes de la forme
2020-09-08T00:05:31Z 359.99842 78.448417 2020-09-07T18:15:01Z 2020-09-08T05:56:01Z 2020-09-07T12:05:31Z 2020-09-08T12:05:01Z

transit_date transit_azimuth transit_elevation date_start_40deg_cross date_end_40deg_cross date_start_1deg_cross date_end_1deg_cross

Maybe add an option to plotData() to save the plot rather than showing it..

Currently AnaBeam correct for the beam squint, but with a fixed optfreq of 30 MHz, which can be quite extreme correction when observing at higher frequency with an higher optfreq. For example for the NCP (ele 47.4 deg), the correction is 41.75 for optfreq=30 and 46.4 for optfreq=60 (which is what we use for your higher frequencies observation).

MA list should rely on the analogic setup, not on the whole instrument configuration.

Hi!
In https://github.com/AlanLoh/nenupy/blob/master/nenupy/beamlet/sstdata.py:636, only MR ON (used for observation) are read from the fits file (noMROn).
But for engineering purposes, one could want to get data from all MR (noMR) monitored by the telescope.
Is it possible to give an extra argument to SST_Data (All_MR=True) to allow advanced user to get them all, please?
Thanks,
Cedric

bst.plotBeam() method fails when ma list is generated from np.arange( N )

>>> malist=np.arange(10)
>>> malist
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> bst=BSTbeam(ma=malist)

	==== WARNING: MA rotations are set by default ===

	==== WARNING: MA positions are set by default ===

	==== WARNING: MA delays are set by default ===
>>> bst.plotBeam()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/nenupy/beam/bstbeam.py", line 247, in plotBeam
    self.getBeam(**kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/nenupy/beam/bstbeam.py", line 236, in getBeam
    mabeam.getBeam()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/nenupy/beam/sstbeam.py", line 186, in getBeam
    beam  = PhasedArrayBeam(p=self._antPos(), m=model, a=az, e=el)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/nenupy/beam/sstbeam.py", line 240, in _antPos
    if self.marotation is not None:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/nenupy/beam/sstbeam.py", line 161, in marotation
    return self._marotation[self.ma]
IndexError: invalid index to scalar variable.

• time selection
• visibility rephasing

Do not show azana and elana while printing SST

The FITS file NenuFAR_thph.fits is too large to be stored on Github. Use a mock object to simulate opening of this file (analog_pointing()) in order to test beam and simulation modules.

e.g

bst=BSTbeam(ma=[1,1,1,2,3,4,5],azdig=45,eldig=30,azana=45,elana=30,freq=50)

will compute the array beam with multiple instances of MA N°1.
==> unique()

Currently there is no warning when the user selects a bad time interval

Write a method to get Jones matrices (2, 2) diagonals as the beam is computed as Stokes I. In order to implement that calculation to DDFacet.
Basically: compute the beam (HealPIX?) and then loop over Facets centers and return the coefficients as diagonal matrices.

Add the ability to call for a fake SST observation

BSTbeam returns error when a python list is provided for parameter "ma".

>>> bst=BSTbeam(ma=range(10))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/nenupy/beam/bstbeam.py", line 40, in __init__
    self._evalkwargs(kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/nenupy/beam/bstbeam.py", line 298, in _evalkwargs
    self.ma         = kwargs.get('ma', None)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/nenupy/beam/bstbeam.py", line 114, in ma
    raise TypeError("\n\t=== Attribute 'ma' should be an array ===")
TypeError:
	=== Attribute 'ma' should be an array ===

#---------------------------------------------------------------------
#parameters = "FOLD: --TFOLD=10.737 --DEFARADAY --SRC=J0828+53"

##duration in sec
##nchan
##TFOLD in sec (default is 10.737)
##NPOL (default is 4)
##NBIN (default is 2048)

#if (--ONLYI): NPOL=1

#size = (duration/TFOLD ) * nchan * NPOL * 32 * NBIN/8. # Bytes

#---------------------------------------------------------------------

#parameters = "SINGLE: --SEARCH --DSTIME=128 --DEFARADAY --SRC=J0205+6449"
#parameters = "SINGLE: --SEARCH --SRC=B0531+21 --DSTIME=64 --ONLYI"

##NPOL (default is 4)

#if (--ONLYI): NPOL=1
##DSTIME in sample (default is 128)

#size = 32NPOLnchan*(duration/5.12e-6/DSTIME)/8 # Bytes

#---------------------------------------------------------------------

#parameters = "WAVE:"

##nchan
##duration in sec
#size = 84nchan*(duration/5.12e-6)/8 # Bytes

#---------------------------------------------------------------------

#parameters = "WAVEOLAF:"

#size = 451590 * nchan * duration # Bytes

This is in order to fill up a database with observation configurations.

Cedric's code: https://gitlab.obspm.fr/cviou/nenufar_raw_tf_readers/-/blob/master/nenufar_raw.py

Add the abeam option for data selection/plot in SST

This is not correct!

Save as FITS or HDF5, try to be consistent with LOFAR

https://www.astron.nl/lofarwiki/lib/exe/fetch.php?media=public:documents:lofar-usg-icd-003.pdf

https://www.astron.nl/lofarwiki/lib/exe/fetch.php?media=public:documents:lofar-usg-icd-006.pdf

https://support.astron.nl/LOFARBeamformedCookbook/introduction.html#beam-definitions

When using astropy 3.2 I have the following issue when calling analog_pointing:

[...]
File "/home/fmertens/nenucal-cd/lib/python3.6/site-packages/nenupy/beam/beam.py", line 376, in beam_values
az, el = analog_pointing(self.azana, el)
File "/home/fmertens/nenucal-cd/lib/python3.6/site-packages/nenupy/instru/instru.py", line 321, in analog_pointing
phi_idx = (azimuth.value/0.05 - 0.5).astype(int)
AttributeError: 'float' object has no attribute 'astype'

The same code works fine with astropy 4. The issue is that azimuth.value is a python native float object in astropy 3, which do not have a method astype, while in astropy 4 it is a numpy float object which then have a method astype.

One fix would be to change the code that check whatever azimuth/elevation is a scalar to check azimuth.value instead of azimuth.