indiajoe / tirspec Goto Github PK

It will be easier for users to manage reduced (and intermediate) files, if they are written to a user specified separate new directory instead of contaminating the raw directory.

The photometry output should be written in some standard table format astropy can load easily. Present output is not scalable, and it is painful to make it compatible for other softwares to load.

From the apall stage onwards, reduction of cross disperse spectrum has to be implemented.

While doing spectroscopic reduction some people might not want to do flat fielding with Tungsten lamp. Make it optional.

Telluric line script has some issue while fitting certain number of points with a Gaussian model. It is repeatable but happens rarely.
My first guess is that it is raising an Index Error due to fitting going bad. Need to track this down.

We should slowly replace all iraf calls using pyraf with pure python tools as they develop.
Following iraf binaries are currently used for each functions, which need to be replaced.

• KeyboardInterrupt_handler(): > IRAF free
• Backup_subrout(): > IRAF free
• LoadDirectories(): > IRAF free
• SelectionofFrames_subrout(): > IRAF free
• Manual_InspectObj_subrout():
  display
  imexam
• Manual_InspectFlat_subrout():
  display
• CombDith_FlatCorr_subrout():
  imcombine
  imstatistics
  imarith
• FixBadPixels():
  fixpix
• AlignNcombine_subrout():
  display
  imexam
  imalign
  imcombine
• Createlist_subrout(): > IRAF free
• Star_sky_subrout():
  display
  imexam
• Sextractor_subrout(): > IRAF free
• is_number(): > IRAF free
• Photometry():
  xyxymatch
  display
  geomap
  geoxytran
  imexam
  gauss
  daofind
  phot
  psf
  allstar
  hedit
  qphot
• NoOfDaophotApertures(}: > IRAF free
• SpectralPairSubtraction_subrout():
  display
  imarith
• SpectralExtraction_subrout():
  apall
  reidentify
  hedit
  dispcor
  scombine

Convert this pipeline to a package in pypi.

Following is the plan.

• Convert scripts to user argparse for better unified terminal interface
• https://docs.python.org/2/library/argparse.html

Restructure the directories in a package format.

Add setup.py , etc.

Following links will be useful for that.

• http://www.scotttorborg.com/python-packaging/index.html
• https://gehrcke.de/2014/02/distributing-a-python-command-line-application/
• http://calvinx.com/2012/09/09/python-packaging-define-an-entry-point-for-console-commands/
• https://packaging.python.org/en/latest/installing/
• http://peterdowns.com/posts/first-time-with-pypi.html

A feature to read in J, H and Ks magnitude and flux calibrate the continuum corrected spectra will be useful for users.

Print Warnings, Change in Argon /Lamp file etc in some colour like Red. It is too difficult to notice anything important in the current b/w scheme.

This error pops up every time i try running the TIRSPECdataReduction.py script on my mac OS X el capitan version. but it seems to work fine on linux.

Steps you have already finished : 1 2 3
Enter the list : 4
Time now: Mon Jul 25 10:06:13 2016
RUNNING TASK:4 Combine images in a Dither [,subtract sky], apply Flat Correction and Bad pixel interpolation

Output directory /Users/namithaissac/Documents/EGOG12.42+0.50/G12.42+0.50/NAMITHA/20140926 already exists.
Everything inside it will be overwritten.
Working on night: 20140926
TIRSPECdataReduction.py:1128: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
statlist.append(statfunction(pyfits.getdata(img)[Ymin-1:Ymax,Xmin-1:Xmax]))
median of images: [-0.066154473, -0.22823668, -0.22276227, -0.20556127, -0.22107205, -0.21710287, -0.17962617]
Zeroshifts of images: [-0.12534779 0.03673442 0.03126001 0.01405901 0.02956979 0.02560061
-0.01187609] :: ImgAvg =-0.191502258182

Jul 25 10:06: IMCOMBINE
combine = median, scale = none, zero = @/Users/namithaissac/Documents/EGOG12.42+0.50/G12.42+0.50/NAMITHA/20140926/G1242-24_median_G1242-30.fits_zeroshifts.txt, weight = none
reject = sigclip, mclip = yes, nkeep = 1
lsigma = 3., hsigma = 3.
blank = 0.
statsec = Jul 25 10:06
Images Zero
20140926/Slope-G1242-24.fits -0.1253
20140926/Slope-G1242-25.fits 0.03673
20140926/Slope-G1242-26.fits 0.03126
20140926/Slope-G1242-27.fits 0.01406
20140926/Slope-G1242-28.fits 0.02957
20140926/Slope-G1242-29.fits 0.0256
20140926/Slope-G1242-30.fits -0.0119

Output image = /Users/namithaissac/Documents/EGOG12.42+0.50/G12.42+0.50/NAMITHA/20140926/G1242-24_median_G1242-30.fits, ncombine = 7
Image section used for normalising Flat is [200:800,200:800]

Jul 25 10:06: IMCOMBINE
combine = median, scale = median, zero = none, weight = none
reject = sigclip, mclip = yes, nkeep = 1
lsigma = 3., hsigma = 3.
blank = 0.
statsec = Jul 25 10:06
Images Median Scale
20140926/Slope-Continuum-40.fits 503.63 1.000
20140926/Slope-Continuum-39.fits 502.54 1.002
20140926/Slope-Continuum-41.fits 505.03 0.997

Output image = /Users/namithaissac/Documents/EGOG12.42+0.50/G12.42+0.50/NAMITHA/20140926/G1242-24_median_G1242-30_flat.fits, ncombine = 3
Traceback (most recent call last):
File "TIRSPECdataReduction.py", line 1928, in
main()
File "TIRSPECdataReduction.py", line 1885, in main
CombDith_FlatCorr_subrout(PC,method=PC.IMGCOMBMETHOD)
File "TIRSPECdataReduction.py", line 1230, in CombDith_FlatCorr_subrout
if (PC.TODO == 'S') and (PC.CONTINUUMGRADREMOVE == 'Y'):
AttributeError: 'PipelineConfig' object has no attribute 'CONTINUUMGRADREMOVE'

Right now, we are using re identify task in iraf to match argon template. It will be better to write a pure python template matching code for re identifying argon lines.

Right now the two dither positions are blindly averaged.
We should mask the spurious points in spectrum before averaging across. Otherwise it almost defeats the purpose of taking two dither positions to avoid bad pixels.

Right now Automated star position matching for each image is done just before photometry. It will be good to separate image coordinate matching and Photometry as two separate tasks.

Advantage: Possibility to verify things more neatly before running final photometry.
And also make Photometry more general function which can be reused somewhere else.