astronomyk / pyreduce Goto Github PK

Dear all,

as promised during the meeting this morning, here are the links to the Github and Google Drive repositories:

REQUEST: Please tell me your github username so that I can add you as a collaborator to the github repo. (@hugo, I know yours already)

Raw MICADO data from Wolgang (Google Drive):

https://drive.google.com/drive/folders/1s27bdRiBYmMWMIkjC1w9k9OaWD2mleb2?usp=sharing

@wolfgang and Nadeen, please add/delete/update the data in here as you see fit

Github fork of PyReduce

https://github.com/astronomyk/PyReduce

To get the repo, simply clone it in the directory of choice on your local machine with

$ git clone https://github.com/astronomyk/PyReduce.git

The MICADO related files are found in the MICADO_info directory. These include:

• The 6 files we discussed today. For this I have simply copied the relevant files from the XSHOOTER example. Obviously we will need to change the contents of these files to suit MICADO. But for now, these can be seen as place holders for what needs to be done.
• My notes on PyReduce and the work plan (pyreduce_readme_kieran.rst)
• The Piskunov papers for REDUCE (2002) and PyReduce (2021)

Work plan

1. [Nadeen, Wolfgang] - 3. Wavecal initial guess (micado.npz), details in issue #2
2. [Wolfgang] - 2. Header keyword map (micado.json), details in #4
3. [Hugo] A. Subclassing Instrument for MICADO (micado.py), details in issue #3
4. [Nadeen] - 1. PyReduce runtime config file (settings_micado.json)
5. [Nadeen] - B. PyReduce runtime script (micado_example.py)
6. [Nadeen] - 4. Bad pixel mask FITS file (mask_micado.fits.gz)

Just to reiterate what was mentioned during the telecon, the highest priority item is the wavelength calibration initial guess (micado.npz). Everything else can(?) be "fudged" to get PyReduce up and running.
I've added the Piskunov papers to the github repo, as these will surely be needed to work out what the data and format of the NPZ file should be.

Pyreduce Documentation

Just FYI, all the pyreduce documentation is still contained in the Github repo, so you need not constantly refer to the online docs.
The doc page most relevant to us is the docs/instruments.rst page, as this contains the description of what columns need to be included in the wavecal (.npz) file

Fingers crossed this doesn't prove too difficult to implement.

Best regards, and best of luck to us all!

Kieran

Dear @nadsabha and @wkausch ,

We had a first look at micado.json. To make progress we need your input:

• please add in this git issue the list of FITS files from the googledrive that will be used as input by PyReduce for your prototyping. In other words the files that will be placed in /DATA/Pyreduce/DATASETS/MICADO/RAW . Then we can check the header keyword mapping against those files.

Cheers, @hugobuddel and Gijs

One of the files to be created to make PyReduce run on MICADO simulated data:
3-Wavelength calibration file: wavecal/micado.npz (numbering scheme following the #1 (comment))

It is a table (numpy recarray) with initial guess for wavelength calibration as an npz file (pickled numpy recarray). From https://pyreduce-astro.readthedocs.io/en/latest/instruments.html it should be as follows: (e.g. https://github.com/astronomyk/PyReduce/blob/master/pyreduce/wavecal/uves_red_860nm_2D.npz)

"get_wavecal_filename, should return the filename to the wavelength calibration first guess. That is a npz file containing a numpy recarray called cs_lines. The recarray should have the fields:

((“wlc”, “WLC”), “>f8”), # Wavelength (before fit)
((“wll”, “WLL”), “>f8”), # Wavelength (after fit)
((“posc”, “POSC”), “>f8”), # Pixel Position (before fit)
((“posm”, “POSM”), “>f8”), # Pixel Position (after fit)
((“xfirst”, “XFIRST”), “>i2”), # first pixel of the line
((“xlast”, “XLAST”), “>i2”), # last pixel of the line
((“approx”, “APPROX”), “O”), # ???
((“width”, “WIDTH”), “>f8”), # width of the line in pixels
((“height”, “HEIGHT”), “>f8”), # relative strength of the line
((“order”, “ORDER”), “>i2”), # echelle order the line is found in
(“flag”, “?”), # flag that tells us if we should use the line or not
"

The files we/I have are: (sent by Wolfgang)

i- Spectral layout folder (https://drive.google.com/drive/folders/1zBQzAZmonYXxeaQxfzc5bNrxosRMabi6?usp=sharing) with the files:

• specorders-180626.fits: what is implemented in SpecCADO
• orders_02_003_nogap.zip: the original data from Frank. A bunch of textfiles, which Oliver converted into the fits file.
• E-MCD-FPA-572089EB_tab_new.asc: gives the coordinate system on the detector array, cx and cy are the coordinates of the detector centers xhw and yhw the half width (all in [mm]).

ii- Line lists folder (https://drive.google.com/drive/folders/1l9kKF2w-6gHgOdihVu7fB3mjxmUvSEut?usp=sharing) with the files:

• /mpia/ = ArKrXe from the FTS measurements as I got them from Gabriele + supplementarly Ne from KMOS; there is also the FPI line list included, but there's a gap in the H-band, so there will be a different Fabry-Perot device
• keck_XeArKrNe_0.95-2.6mum.csv = from the Keck observatory (https://www2.keck.hawaii.edu/inst/nirspec/lines.html)
• XeArKrNe_nist_0.75-2.4mum.dat = XeArKrNe line list from the NIST database (https://physics.nist.gov/PhysRefData/ASD/lines_form.html)
• penray_nir_custom_air_ext_2012PBR.fits = line list used by XShooter; Note this only contains a list of lines, no further info on flux/species etc...

This issue is to discuss "[Hugo, Kieran?] A. Subclassing Instrument for MICADO (micado.py)" as described in parent issue #1.

We need a micado.py to place in https://github.com/astronomyk/PyReduce/tree/master/pyreduce/instruments . The draft for this file is https://github.com/astronomyk/PyReduce/blob/master/MICADO_info/micado.py

The main thing micado.py needs to do is to convert header values that cannot be mapped one-to-one. We can do that as soon as we have mapped all the header keywords that can be mapped directly. An initial draft for these keywords are in https://github.com/astronomyk/PyReduce/blob/master/MICADO_info/micado.json and needs to be placed in the above directory as well.