conornally / starbug2 Goto Github PK

View Code? Open in Web Editor NEW

0.0 1.0 2.0 2.39 MB

JWST PSF photometry for complex and crowded fields

License: GNU General Public License v3.0

Python 100.00%

starbug2's Introduction

StarBugII

PSF photometry in complex and crowded fields

• Detect deeply dust-embedded sources in NIRCam and MIRI images

• Full suite of photometry tools

• Powerful modular and simple GNU Linux standard command line interface

Installation

StarbugII requires webbpsf installed on the system to function fully.

$~ pip install starbug2
$~ starbug2 --init

Important

If you make use of StarbugII in any published or presented work, please include a citation.

StarbugII uses methods and datatypes from astropy and photutils, please acknowledge them accordingly.

Documentation

See the full documentation for the complete installation and detailed guides to using the photometric routines. Basic usage information is produced by running:

StarbugII - JWST PSF photometry
usage: starbug2 [-ABDfGhMPSv] [-b bgdfile] [-d apfile] [-n ncores] [-o ouput] [-p file.param] [-s opt=val] image.fits ...
   -A  --apphot          : run aperture photometry on a source list
   -B  --background      : run background estimation
   -b  --bgdfile         : load background (-bgd.fits) file
   -d  --apfile  ap.fits : load a source detection (-ap.fits) file to skip the source detection step
   -D  --detect          : run source detection
   -f  --find            : attempt to find associated -ap -bgd files
   -G  --geom            : calculate geometric stats on source list
   -h  --help            : display uasage information
   -M  --match           : match outputs from all input image files
   -n  --ncores      num : number of CPU cores to split process between
   -o  --output      dir : output directory
   -p  --param   a.param : load parameter file
   -P  --psf             : run psf photometry
   -s  --set      option : set value in parameter file at runtime (-s SIGSKY=3)
   -S  --subbgd          : subtract background from image
   -v  --verbose         : display verbose outputs

   --> Single run commands
       --init                     : Initialise Starbug (post install)
       --local-param              : Make a local copy of the default parameter file
       --update-param             : Update an out-of-date local parameter file
       --generate-psf             : Generate a single PSF. Set FILTER, DET_NAME, PSF_SIZE with -s
       --generate-region   a.fits : Make a ds9 region file with a detection file
       --generate-run      *.fits : Generate a simple run script
       --version                  : Print starbug2 version

       --apply-zeropint    a.fits : Apply a zeropoint (-s ZP_MAG=1.0) to a.fits
       --calc-instr-zp     a.fits : Calculate and apply an instrumental zero point onto a.fits

   --> typical runs
      $~ starbug2 -vD -p file.param image.fits      //Source detect on image with a parameter file
      $~ starbug2 -vDM -n4 images*.fits             //Source detect and match outputs of a list of images
      $~ starbug2 -vd image-ap.fits -BP image.fits  //PSF photometry on an image with a source file (image-ap.fits)

See https://starbug2.readthedocs.io for more information.

StarbugII Matching 
usage: starbug2-match [-BCGfhv] [-e column] [-o output] [-p file.param] [-s KEY=VAL] table.fits ...
    -B  --band               : match in "BAND" mode (does not preserve a column for every frame)
    -C  --cascade            : match in "CASCADE" mode (left justify columns)
    -G  --generic            : match in "GENERIC" mode

    -e  --error   column     : photometric error column ("eflux" or "stdflux")
    -f  --full               : export full catalogue
    -h  --help               : show help message
    -o  --output  file.fits  : output matched catalogue
    -p  --param   file.param : load starbug parameter file
    -s  --set     option     : set value in parameter file at runtime (-s MATCH_THRESH=1)
    -v  --verbose            : display verbose outputs

        --band-depr          : match in "old" band mode

    --> typical runs
       $~ starbug2-match -Gfo outfile.fits tab1.fits tab2.fits
       $~ starbug2-match -sMATCH_THRESH=0.2 -sBRIDGE_COL=F444W -Bo out.fits F*W.fits

starbug2's People

Contributors

Watchers

Forkers

keflavich low-sky

starbug2's Issues

PSF fitting saturated sources

We can fit the wings of a saturated star by removing the centre of the PSF model, this would allow manual recovering the lost stars.

Detecting would have to maybe be done manually or the centre of the saturated region could maybe be detected

Recursion error in photutils

On presumably bad source fits in the PSF, python breaks with a recursion error:

Traceback (most recent call last):
File "/home/conor/.local/bin/starbug2", line 243, in
if ncores==1: starbugs=[ fn(fname) for fname in args ]
File "/home/conor/.local/bin/starbug2", line 243, in
if ncores==1: starbugs=[ fn(fname) for fname in args ]
File "/home/conor/.local/bin/starbug2", line 231, in fn
sb.photometry()
File "/home/conor/dev/starbug2/starbug2/starbug.py", line 443, in photometry
_psf_cat=phot(image,init_guesses=init_guesses)
File "/home/conor/.local/lib/python3.10/site-packages/photutils/psf/photometry.py", line 290, in call
return self.do_photometry(image, mask=mask, init_guesses=init_guesses,
File "/home/conor/dev/starbug2/starbug2/routines.py", line 413, in do_photometry
cat=super().do_photometry(image, mask=mask, init_guesses=init_guesses, progress_bar=False)
File "/home/conor/.local/lib/python3.10/site-packages/photutils/psf/photometry.py", line 432, in do_photometry
output_tab, self._residual_image = self.nstar(
File "/home/conor/.local/lib/python3.10/site-packages/photutils/psf/photometry.py", line 512, in nstar
group_psf = get_grouped_psf_model(self.psf_model, group,
File "/home/conor/.local/lib/python3.10/site-packages/photutils/psf/utils.py", line 171, in get_grouped_psf_model
group_psf = group_psf + psf_to_add
File "/home/conor/.local/lib/python3.10/site-packages/astropy/modeling/core.py", line 55, in
return lambda left, right: CompoundModel(oper, left, right, **kwargs)
File "/home/conor/.local/lib/python3.10/site-packages/astropy/modeling/core.py", line 3006, in init
self._map_parameters()
File "/home/conor/.local/lib/python3.10/site-packages/astropy/modeling/core.py", line 3522, in _map_parameters
self._make_leaflist()
File "/home/conor/.local/lib/python3.10/site-packages/astropy/modeling/core.py", line 3285, in _make_leaflist
make_subtree_dict(self, '', tdict, leaflist)
File "/home/conor/.local/lib/python3.10/site-packages/astropy/modeling/core.py", line 4050, in make_subtree_dict
make_subtree_dict(tree.left, nodepath+'l', tdict, leaflist)
File "/home/conor/.local/lib/python3.10/site-packages/astropy/modeling/core.py", line 4050, in make_subtree_dict
make_subtree_dict(tree.left, nodepath+'l', tdict, leaflist)
File "/home/conor/.local/lib/python3.10/site-packages/astropy/modeling/core.py", line 4050, in make_subtree_dict
make_subtree_dict(tree.left, nodepath+'l', tdict, leaflist)
[Previous line repeated 980 more times]
File "/home/conor/.local/lib/python3.10/site-packages/astropy/modeling/core.py", line 4046, in make_subtree_dict
if not hasattr(tree, 'isleaf'):
RecursionError: maximum recursion depth exceeded

Silent killing of child process when it runs out of memory

When multiprocessing over several files, if the system runs out of memory the subprocesses get killed but they don't report back to the parent, so it sits and waits for them to finish.
Not sure what the fix is here though

source detection column order

Output goes CN, x, ra, dec, y... need to fix that ordering

PSF generate for stage3 image

Currently, the psf generation code creates a PSF file for every detector module. Starbug looks into the header file and checks DETECTOR and FILTER to load the appropriate PSF, however, a stage 3 image has DETECOR="MULTIPLE", for which there is no file. Either need to make one or get it to use one that exists

MIRI implementation

As of version 0.2.6 MIRI has not been implemented.
Areas to be looked into:

Filters FWHM
Filters Zero points
Aperture correction
Default parameters, the FWHMs will be very different

MATCHING

This is an issue in its own right

RM_MATCH quirks

Issue: RM_MATCH set through --set RM_MATCH=... doesn't affect the code, because I read the parameter file, not the "setdict".

The actual functionality of it is a bit weird. In a four-dither match, to keep number match 2,3,4 you need to set RM_MATCH=3, which is just a bit backwards

Output file locations not always where you expect them to be

Some processes that produce outputs don't always put them where you expect.

If you give -Do DIR in bin/starbug2 then the resulting -ap file goes to DIR, however, a dither matched (123..)-ap catalogue gets exported into the current directory.
--generate-region outputs in the current directory too. Regardless of input file location or -o output

There needs to be

uniform behaviour
a better-defined hierarchy of output options:

if given, -o output
default input file directory
probably never just in CWD

AB-Vega mag toggle

Add a flag to change the output magnitude

Overwriting exports

Rightly when exporting products at the end of the run, starbug overwrites any that exist there already.
However if e.g. a source file is loaded in -d global_sources.fits then this will overwrite the original -ap file, thus removing all the old useful single frame source lists.

The exporting should probably happen at the end of the routine, not at the end of the run. Also it should only export if it wasn't a preloaded file.

Re-Implement stage3 into Aperture Photometry

Just need to make sure that the current version supports pipeline stage images again.
Areas that need to be looked at:

Apphot area array correction (it doesn't need one, but it will try to do one)
matching, make sure that the catalogue matching is backwards compatible