Comments (20)
wmwv
commented on September 27, 2024
Generated coadds for 53 fields with enough images that passed successfully through processCcd.py
from wiyn_lsst.
wmwv
commented on September 27, 2024
[serenity test_dr1] ls rerun/coadd/deepCoadd/*/*.fits
rerun/coadd/deepCoadd/CSS121114:090202+101800/CSS121114:090202+101800_H_0_0,0.fits
rerun/coadd/deepCoadd/CSS121114:090202+101800/CSS121114:090202+101800_J_0_0,0.fits
rerun/coadd/deepCoadd/CSS130218:092354+385837/CSS130218:092354+385837_H_1_0,0.fits
rerun/coadd/deepCoadd/CSS130317:082848+293031/CSS130317:082848+293031_H_2_0,0.fits
rerun/coadd/deepCoadd/CSS130317:082848+293031/CSS130317:082848+293031_J_2_0,0.fits
rerun/coadd/deepCoadd/LSQ12fhs/LSQ12fhs_H_3_0,0.fits
rerun/coadd/deepCoadd/LSQ12fhs/LSQ12fhs_J_3_0,0.fits
rerun/coadd/deepCoadd/LSQ12fmx/LSQ12fmx_H_4_0,0.fits
rerun/coadd/deepCoadd/LSQ12fmx/LSQ12fmx_J_4_0,0.fits
rerun/coadd/deepCoadd/LSQ12fuk/LSQ12fuk_H_5_0,0.fits
rerun/coadd/deepCoadd/LSQ12gef/LSQ12gef_H_6_0,0.fits
rerun/coadd/deepCoadd/LSQ13cmt/LSQ13cmt_H_7_0,0.fits
rerun/coadd/deepCoadd/LSQ13cmt/LSQ13cmt_J_7_0,0.fits
rerun/coadd/deepCoadd/LSQ13crf/LSQ13crf_H_8_0,0.fits
rerun/coadd/deepCoadd/LSQ13crf/LSQ13crf_J_8_0,0.fits
rerun/coadd/deepCoadd/LSQ13cwp/LSQ13cwp_H_9_0,0.fits
rerun/coadd/deepCoadd/LSQ13cwp/LSQ13cwp_J_9_0,0.fits
rerun/coadd/deepCoadd/P161-D/P161-D_H_10_0,0.fits
rerun/coadd/deepCoadd/P161-D/P161-D_J_10_0,0.fits
rerun/coadd/deepCoadd/P212-C/P212-C_H_11_0,0.fits
rerun/coadd/deepCoadd/P212-C/P212-C_J_11_0,0.fits
rerun/coadd/deepCoadd/P525-E/P525-E_H_13_0,0.fits
rerun/coadd/deepCoadd/P525-E/P525-E_J_13_0,0.fits
rerun/coadd/deepCoadd/PS1-12bwh/PS1-12bwh_H_14_0,0.fits
rerun/coadd/deepCoadd/PS1-12bwh/PS1-12bwh_J_14_0,0.fits
rerun/coadd/deepCoadd/PS1-13dkh/PS1-13dkh_H_15_0,0.fits
rerun/coadd/deepCoadd/PS1-13dkh/PS1-13dkh_J_15_0,0.fits
rerun/coadd/deepCoadd/PSNJ07250042+2347030/PSNJ07250042+2347030_H_16_0,0.fits
rerun/coadd/deepCoadd/PSNJ07250042+2347030/PSNJ07250042+2347030_J_16_0,0.fits
rerun/coadd/deepCoadd/PTF11moy/PTF11moy_H_17_0,0.fits
rerun/coadd/deepCoadd/PTF11moy/PTF11moy_J_17_0,0.fits
rerun/coadd/deepCoadd/PTF11mty/PTF11mty_H_18_0,0.fits
rerun/coadd/deepCoadd/PTF11mty/PTF11mty_J_18_0,0.fits
rerun/coadd/deepCoadd/PTF11owc/PTF11owc_H_19_0,0.fits
rerun/coadd/deepCoadd/PTF11owc/PTF11owc_J_19_0,0.fits
rerun/coadd/deepCoadd/PTF11qmo/PTF11qmo_H_20_0,0.fits
rerun/coadd/deepCoadd/PTF11qmo/PTF11qmo_J_20_0,0.fits
rerun/coadd/deepCoadd/PTF11qpc/PTF11qpc_H_21_0,0.fits
rerun/coadd/deepCoadd/PTF11qri/PTF11qri_H_22_0,0.fits
rerun/coadd/deepCoadd/PTF11qri/PTF11qri_J_22_0,0.fits
rerun/coadd/deepCoadd/PTF11qzq/PTF11qzq_H_23_0,0.fits
rerun/coadd/deepCoadd/PTF13dad/PTF13dad_J_24_0,0.fits
rerun/coadd/deepCoadd/SN2011gf/SN2011gf_J_25_0,0.fits
rerun/coadd/deepCoadd/SN2011gy/SN2011gy_H_26_0,0.fits
rerun/coadd/deepCoadd/SN2011gy/SN2011gy_J_26_0,0.fits
rerun/coadd/deepCoadd/SN2011ha/SN2011ha_H_27_0,0.fits
rerun/coadd/deepCoadd/SN2011ha/SN2011ha_J_27_0,0.fits
rerun/coadd/deepCoadd/SN2011hb/SN2011hb_H_28_0,0.fits
rerun/coadd/deepCoadd/SN2011hb/SN2011hb_J_28_0,0.fits
rerun/coadd/deepCoadd/SN2011hk/SN2011hk_H_29_0,0.fits
rerun/coadd/deepCoadd/SN2011hk/SN2011hk_J_29_0,0.fits
rerun/coadd/deepCoadd/SN2011ho/SN2011ho_H_30_0,0.fits
rerun/coadd/deepCoadd/SN2011hr/SN2011hr_H_31_0,0.fits
rerun/coadd/deepCoadd/SN2011hr/SN2011hr_J_31_0,0.fits
rerun/coadd/deepCoadd/SN2011io/SN2011io_H_32_0,0.fits
rerun/coadd/deepCoadd/SN2011iu/SN2011iu_H_33_0,0.fits
rerun/coadd/deepCoadd/SN2011iu/SN2011iu_J_33_0,0.fits
rerun/coadd/deepCoadd/SN2011iy/SN2011iy_H_34_0,0.fits
rerun/coadd/deepCoadd/SN2011jh/SN2011jh_H_35_0,0.fits
rerun/coadd/deepCoadd/SN2011jh/SN2011jh_J_35_0,0.fits
rerun/coadd/deepCoadd/SN2012bh/SN2012bh_H_36_0,0.fits
rerun/coadd/deepCoadd/SN2012bp/SN2012bp_H_37_0,0.fits
rerun/coadd/deepCoadd/SN2012cg/SN2012cg_H_38_0,0.fits
rerun/coadd/deepCoadd/SN2012cg/SN2012cg_J_38_0,0.fits
rerun/coadd/deepCoadd/SN2012em/SN2012em_H_39_0,0.fits
rerun/coadd/deepCoadd/SN2012em/SN2012em_J_39_0,0.fits
rerun/coadd/deepCoadd/SN2012et/SN2012et_H_40_0,0.fits
rerun/coadd/deepCoadd/SN2012et/SN2012et_J_40_0,0.fits
rerun/coadd/deepCoadd/SN2012fb/SN2012fb_H_41_0,0.fits
rerun/coadd/deepCoadd/SN2012fb/SN2012fb_J_41_0,0.fits
rerun/coadd/deepCoadd/SN2012fk/SN2012fk_H_42_0,0.fits
rerun/coadd/deepCoadd/SN2012fk/SN2012fk_J_42_0,0.fits
rerun/coadd/deepCoadd/SN2012fm/SN2012fm_H_43_0,0.fits
rerun/coadd/deepCoadd/SN2012fm/SN2012fm_J_43_0,0.fits
rerun/coadd/deepCoadd/SN2012fr/SN2012fr_H_44_0,0.fits
rerun/coadd/deepCoadd/SN2012fr/SN2012fr_J_44_0,0.fits
rerun/coadd/deepCoadd/SN2012gm/SN2012gm_J_45_0,0.fits
rerun/coadd/deepCoadd/SN2012go/SN2012go_H_46_0,0.fits
rerun/coadd/deepCoadd/SN2012go/SN2012go_J_46_0,0.fits
rerun/coadd/deepCoadd/SN2013ar/SN2013ar_H_47_0,0.fits
rerun/coadd/deepCoadd/SN2013be/SN2013be_H_48_0,0.fits
rerun/coadd/deepCoadd/SN2013bo/SN2013bo_H_49_0,0.fits
rerun/coadd/deepCoadd/SN2013bo/SN2013bo_J_49_0,0.fits
rerun/coadd/deepCoadd/SN2013bs/SN2013bs_H_50_0,0.fits
rerun/coadd/deepCoadd/SN2013bs/SN2013bs_J_50_0,0.fits
rerun/coadd/deepCoadd/SN2013bt/SN2013bt_H_51_0,0.fits
rerun/coadd/deepCoadd/SN2013bt/SN2013bt_J_51_0,0.fits
rerun/coadd/deepCoadd/SN2013fw/SN2013fw_H_52_0,0.fits
rerun/coadd/deepCoadd/SN2013fw/SN2013fw_J_52_0,0.fits
rerun/coadd/deepCoadd/iPTF13dkj/iPTF13dkj_H_54_0,0.fits
from wiyn_lsst.
wmwv
commented on September 27, 2024
90 coadds generated from 53 fields.
90 calexps from these coadds generated.
33 merged reference tables generated (out of 53 fields)
45 forced-photometry catalogs generated.
from wiyn_lsst.
wmwv
commented on September 27, 2024
I re-ran the coadds with d_2018_04_09, and got
58 coadds generated from 53 fields.
I don't immediately know why there is the difference.
from wiyn_lsst.
wmwv
commented on September 27, 2024
The images that didn't succeed under the recent d_2018_04_09 were
[serenity wiyn_lsst] ls /Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/*/*.fits | xargs basename > w_2018_14.coadds
[serenity wiyn_lsst] ls /Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coaddNew/deepCoadd/*/*.fits | xargs basename > d_2018_04_09.coadds
[serenity wiyn_lsst] diff w_2018_14.coadds d_2018_04_09.coadds
59,90d58
< SN2011jh_H_35_0,0.fits
< SN2011jh_J_35_0,0.fits
< SN2012bh_H_36_0,0.fits
< SN2012bp_H_37_0,0.fits
< SN2012cg_H_38_0,0.fits
< SN2012cg_J_38_0,0.fits
< SN2012em_H_39_0,0.fits
< SN2012em_J_39_0,0.fits
< SN2012et_H_40_0,0.fits
< SN2012et_J_40_0,0.fits
< SN2012fb_H_41_0,0.fits
< SN2012fb_J_41_0,0.fits
< SN2012fk_H_42_0,0.fits
< SN2012fk_J_42_0,0.fits
< SN2012fm_H_43_0,0.fits
< SN2012fm_J_43_0,0.fits
< SN2012fr_H_44_0,0.fits
< SN2012fr_J_44_0,0.fits
< SN2012gm_J_45_0,0.fits
< SN2012go_H_46_0,0.fits
< SN2012go_J_46_0,0.fits
< SN2013ar_H_47_0,0.fits
< SN2013be_H_48_0,0.fits
< SN2013bo_H_49_0,0.fits
< SN2013bo_J_49_0,0.fits
< SN2013bs_H_50_0,0.fits
< SN2013bs_J_50_0,0.fits
< SN2013bt_H_51_0,0.fits
< SN2013bt_J_51_0,0.fits
< SN2013fw_H_52_0,0.fits
< SN2013fw_J_52_0,0.fits
< iPTF13dkj_H_54_0,0.fits
from wiyn_lsst.
wmwv
commented on September 27, 2024
It looks like measureCoaddSources.py just hung at SN2011jh. The all_coadds_new.log file ends with SN2011jh, having just done the makeCoaddTempExp step.
[serenity wiyn_lsst] ps x | grep lsstsw
85711 ?? R 5984:30.80 /Users/wmwv/lsstsw/python/miniconda3-4.3.21/bin/python /Users/wmwv/lsstsw/stack/miniconda3-4.3.21-10a4fa6/DarwinX86/pipe_tasks/15.0-3-ga659d1f3/bin/measureCoaddSources.py /Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1 --rerun coaddPhot --id field=P212-C filter=H^J --configfile config/measureCoaddSources.py --clobber-versions
11762 s002 S+ 0:00.00 grep -d skip lsstsw
91735 s002 T 0:07.05 /Users/wmwv/lsstsw/python/miniconda3-4.3.21/bin/python /Users/wmwv/lsstsw/stack/miniconda3-4.3.21-10a4fa6/DarwinX86/pipe_tasks/15.0-3-ga659d1f3/bin/detectCoaddSources.py /Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1 --rerun coadd:coaddPhot --id field=SN2012bp tract=37 filter=H --id field=SN2012bp tract=37 filter=J --clobber-versions
5984 minutes is a long time.
from wiyn_lsst.
wmwv
commented on September 27, 2024
Hmmm... The 5984 is from a different run and is processing P212-C.
There's also the detectCoaddSources.py processing SN2012bp, which several down from SN2011jh in the dr1_coadd.list
from wiyn_lsst.
wmwv
commented on September 27, 2024
I killed both of the jobs. I had to kill -9 85711. The P212-C processing continued and then failed with a stream of message about being unable to find run_photometry.sh in the all_photometry.log file. This indicates that this was an older run that was relying on the older names of the scripts.
This also reminds me that someday I may have to update to have an explicit production directory.
But this was all a distraction. Let me try to manually run the SN2011jh processing.
from wiyn_lsst.
wmwv
commented on September 27, 2024
I manually re-ran SN2011jh with:
[serenity wiyn_lsst] . run_coadd_sn.sh SN2011jh
and then re-ran the rest with
[serenity wiyn_lsst] while read -r line; do source run_coadd.sh "${REPO}" ${line}; done < dr1_coadd_actual_makeup.list > all_coadds_new_makeup.log 2>&1 &
where dr1_coadd_actual_makeup.list was a list of all of the SNe in dr1_coadd_actual.list after SN2011jh (which was successful above).
from wiyn_lsst.
wmwv
commented on September 27, 2024
The fields that didn't generate coadds are: P330-E and iPTF13dge.
[serenity wiyn_lsst] while read -r sn _; do ls ${REPO}/rerun/coadd/deepCoadd/${sn}/; done < dr1_coadd_actual.list
CSS121114:090202+101800_H_0_0,0.fits CSS121114:090202+101800_J_0_0,0.fits
CSS130218:092354+385837_H_1_0,0.fits
CSS130317:082848+293031_H_2_0,0.fits CSS130317:082848+293031_J_2_0,0.fits
LSQ12fhs_H_3_0,0.fits LSQ12fhs_J_3_0,0.fits
LSQ12fmx_H_4_0,0.fits LSQ12fmx_J_4_0,0.fits
LSQ12fuk_H_5_0,0.fits
LSQ12gef_H_6_0,0.fits
LSQ13cmt_H_7_0,0.fits LSQ13cmt_J_7_0,0.fits
LSQ13crf_H_8_0,0.fits LSQ13crf_J_8_0,0.fits
LSQ13cwp_H_9_0,0.fits LSQ13cwp_J_9_0,0.fits
P161-D_H_10_0,0.fits P161-D_J_10_0,0.fits
P212-C_H_11_0,0.fits P212-C_J_11_0,0.fits
ls: /Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P330-E/: No such file or directory
P525-E_H_13_0,0.fits P525-E_J_13_0,0.fits
PS1-12bwh_H_14_0,0.fits PS1-12bwh_J_14_0,0.fits
PS1-13dkh_H_15_0,0.fits PS1-13dkh_J_15_0,0.fits
PSNJ07250042+2347030_H_16_0,0.fits PSNJ07250042+2347030_J_16_0,0.fits
PTF11moy_H_17_0,0.fits PTF11moy_J_17_0,0.fits
PTF11mty_H_18_0,0.fits PTF11mty_J_18_0,0.fits
PTF11owc_H_19_0,0.fits PTF11owc_J_19_0,0.fits
PTF11qmo_H_20_0,0.fits PTF11qmo_J_20_0,0.fits
PTF11qpc_H_21_0,0.fits
PTF11qri_H_22_0,0.fits PTF11qri_J_22_0,0.fits
PTF11qzq_H_23_0,0.fits
PTF13dad_J_24_0,0.fits
SN2011gf_J_25_0,0.fits
SN2011gy_H_26_0,0.fits SN2011gy_J_26_0,0.fits
SN2011ha_H_27_0,0.fits SN2011ha_J_27_0,0.fits
SN2011hb_H_28_0,0.fits SN2011hb_J_28_0,0.fits
SN2011hk_H_29_0,0.fits SN2011hk_J_29_0,0.fits
SN2011ho_H_30_0,0.fits
SN2011hr_H_31_0,0.fits SN2011hr_J_31_0,0.fits
SN2011io_H_32_0,0.fits
SN2011iu_H_33_0,0.fits SN2011iu_J_33_0,0.fits
SN2011iy_H_34_0,0.fits
SN2011jh_H_35_0,0.fits SN2011jh_J_35_0,0.fits
SN2012bh_H_36_0,0.fits
SN2012bp_H_37_0,0.fits
SN2012cg_H_38_0,0.fits SN2012cg_J_38_0,0.fits
SN2012em_H_39_0,0.fits SN2012em_J_39_0,0.fits
SN2012et_H_40_0,0.fits SN2012et_J_40_0,0.fits
SN2012fb_H_41_0,0.fits SN2012fb_J_41_0,0.fits
SN2012fk_H_42_0,0.fits SN2012fk_J_42_0,0.fits
SN2012fm_H_43_0,0.fits SN2012fm_J_43_0,0.fits
SN2012fr_H_44_0,0.fits SN2012fr_J_44_0,0.fits
SN2012gm_J_45_0,0.fits
SN2012go_H_46_0,0.fits SN2012go_J_46_0,0.fits
SN2013ar_H_47_0,0.fits
SN2013be_H_48_0,0.fits
SN2013bo_H_49_0,0.fits SN2013bo_J_49_0,0.fits
SN2013bs_H_50_0,0.fits SN2013bs_J_50_0,0.fits
SN2013bt_H_51_0,0.fits SN2013bt_J_51_0,0.fits
SN2013fw_H_52_0,0.fits SN2013fw_J_52_0,0.fits
ls: /Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/iPTF13dge/: No such file or directory
iPTF13dkj_H_54_0,0.fits
from wiyn_lsst.
wmwv
commented on September 27, 2024
Ah, and that's merely because the P330-E and iPTF13dge fields only have Ks images, and I'm only constructing J, H coadds for now.
from wiyn_lsst.
wmwv
commented on September 27, 2024
There are 99 calexp field-filter pairs for J, H band:
[serenity wiyn_lsst] ls ${REPO}/rerun/processCcd/calexp/bkgd*| cut -d / -f 11 | sed -e 's/bkgd-\(.*\)_\([A-F]\)_\([JHKS]*\)_\([0-9]\{8\}\)_\([0-9][0-9]*\)\.fits/\2 \3 \4 \5 \1/' | awk '{print $3, $1, $4, $2, $5}' | uniq -f 3 | grep -v KS | wc
99 495 2648
[serenity wiyn_lsst] ls ${REPO}/rerun/processCcd/calexp/bkgd*| cut -d / -f 11 | sed -e 's/bkgd-\(.*\)_\([A-F]\)_\([JHKS]*\)_\([0-9]\{8\}\)_\([0-9][0-9]*\)\.fits/\2 \3 \4 \5 \1/' | awk '{print $3, $1, $4, $2, $5}' | uniq -f 3 | grep -v KS
20121125 A 556 H CSS121114:090202+101800
20121125 A 569 J CSS121114:090202+101800
20130420 A 272 H CSS130218:092354+385837
20130326 A 204 H CSS130317:082848+293031
20130326 A 221 J CSS130317:082848+293031
20121028 A 210 H LSQ12fhs
20121028 A 228 J LSQ12fhs
20121028 A 356 H LSQ12fmx
20121028 A 406 J LSQ12fmx
20121102 A 386 H LSQ12fuk
20121122 A 286 H LSQ12gef
20131015 A 314 H LSQ13cmt
20131015 A 339 J LSQ13cmt
20131111 A 390 H LSQ13crf
20131111 A 431 J LSQ13crf
20131111 A 521 H LSQ13cwp
20131111 A 562 J LSQ13cwp
20121125 A 401 H P161-D
20121102 A 565 J P161-D
20121125 B 499 H P161-D
20130927 B 206 J P161-D
20121125 C 597 H P161-D
20121125 C 606 J P161-D
20130326 A 092 J P212-C
20130420 B 245 H P212-C
20130420 C 317 H P212-C
20121122 A 158 H P525-E
20121125 A 097 J P525-E
20121122 B 336 H P525-E
20121122 B 327 J P525-E
20121125 C 320 H P525-E
20121125 C 329 J P525-E
20121028 A 514 H PS1-12bwh
20121028 A 505 J PS1-12bwh
20131018 A 136 H PS1-13dkh
20131018 A 145 J PS1-13dkh
20121028 A 542 H PSNJ07250042+2347030
20121028 A 533 J PSNJ07250042+2347030
20111025 A 127 H PTF11moy
20111025 A 109 J PTF11moy
20111025 A 218 H PTF11mty
20111025 A 200 J PTF11mty
20111115 A 313 H PTF11owc
20111115 A 371 J PTF11owc
20111208 A 477 H PTF11qmo
20111208 A 495 J PTF11qmo
20120108 A 508 H PTF11qpc
20111208 A 514 H PTF11qri
20111208 A 532 J PTF11qri
20121007 A 408 H PTF11qzq
20130925 A 395 J PTF13dad
20111115 A 092 J SN2011gf
20111115 A 323 H SN2011gy
20111115 A 342 J SN2011gy
20111115 A 273 H SN2011ha
20111115 A 290 J SN2011ha
20111115 A 197 H SN2011hb
20111115 A 215 J SN2011hb
20111115 A 234 H SN2011hk
20111115 A 252 J SN2011hk
20120108 A 546 H SN2011ho
20111121 A 196 H SN2011hr
20111121 A 205 J SN2011hr
20111208 A 161 H SN2011io
20111208 A 234 H SN2011iu
20111208 A 252 J SN2011iu
20120402 A 334 H SN2011iy
20120402 A 302 H SN2011jh
20120402 A 320 J SN2011jh
20120402 A 244 H SN2012bh
20120402 A 473 H SN2012bp
20130425 A 247 H SN2012cg
20130425 A 260 J SN2012cg
20120925 A 156 H SN2012em
20120925 A 184 J SN2012em
20121001 A 167 H SN2012et
20121122 A 179 J SN2012et
20120925 A 391 H SN2012fb
20120925 A 409 J SN2012fb
20121102 A 353 H SN2012fk
20121102 A 370 J SN2012fk
20121028 A 493 H SN2012fm
20121028 A 484 J SN2012fm
20121102 A 332 H SN2012fr
20121102 A 344 J SN2012fr
20121125 A 177 J SN2012gm
20121125 A 121 H SN2012go
20121125 A 125 J SN2012go
20130326 A 110 H SN2013ar
20130429 A 186 H SN2013be
20130429 A 308 H SN2013bo
20130420 A 415 J SN2013bo
20130420 A 457 H SN2013bs
20130420 A 478 J SN2013bs
20130420 A 424 H SN2013bt
20130425 A 453 J SN2013bt
20131209 A 025 H SN2013fw
20131111 A 150 J SN2013fw
20130927 A 033 H iPTF13dkj
from wiyn_lsst.
wmwv
commented on September 27, 2024
Apparently my counting above wasn't quite complete. There are indeed 99 coadds generated from this set up 55 fields.
[serenity wiyn_lsst] ls ${REPO}/rerun/processCcd/calexp/bkgd*| cut -d / -f 11 | sed -e 's/bkgd-\(.*\)_\([A-F]\)_\([JHKS]*\)_\([0-9]\{8\}\)_\([0-9][0-9]*\)\.fits/\2 \3 \4 \5 \1/' | awk '{print $3, $1, $4, $2, $5}' | uniq -f 3 | grep -v KS | awk '{print $5, $4}' > dr1_coadd_possible.list
[serenity wiyn_lsst] while read -r sn filt; do ls ${REPO}/rerun/coadd/deepCoadd/${sn}/${sn}_${filt}_*; done < dr1_coadd_possible.list | wc
99 99 10031
from wiyn_lsst.
wmwv
commented on September 27, 2024
[serenity wiyn_lsst] while read -r sn filt; do ls ${REPO}/rerun/coadd/deepCoadd/${sn}/${sn}_${filt}_*; done < dr1_coadd_possible.list
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/CSS121114:090202+101800/CSS121114:090202+101800_H_0_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/CSS121114:090202+101800/CSS121114:090202+101800_J_0_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/CSS130218:092354+385837/CSS130218:092354+385837_H_1_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/CSS130317:082848+293031/CSS130317:082848+293031_H_2_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/CSS130317:082848+293031/CSS130317:082848+293031_J_2_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ12fhs/LSQ12fhs_H_3_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ12fhs/LSQ12fhs_J_3_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ12fmx/LSQ12fmx_H_4_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ12fmx/LSQ12fmx_J_4_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ12fuk/LSQ12fuk_H_5_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ12gef/LSQ12gef_H_6_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ13cmt/LSQ13cmt_H_7_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ13cmt/LSQ13cmt_J_7_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ13crf/LSQ13crf_H_8_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ13crf/LSQ13crf_J_8_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ13cwp/LSQ13cwp_H_9_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/LSQ13cwp/LSQ13cwp_J_9_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P161-D/P161-D_H_10_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P161-D/P161-D_J_10_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P161-D/P161-D_H_10_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P161-D/P161-D_J_10_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P161-D/P161-D_H_10_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P161-D/P161-D_J_10_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P212-C/P212-C_J_11_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P212-C/P212-C_H_11_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P212-C/P212-C_H_11_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P525-E/P525-E_H_13_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P525-E/P525-E_J_13_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P525-E/P525-E_H_13_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P525-E/P525-E_J_13_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P525-E/P525-E_H_13_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/P525-E/P525-E_J_13_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PS1-12bwh/PS1-12bwh_H_14_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PS1-12bwh/PS1-12bwh_J_14_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PS1-13dkh/PS1-13dkh_H_15_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PS1-13dkh/PS1-13dkh_J_15_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PSNJ07250042+2347030/PSNJ07250042+2347030_H_16_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PSNJ07250042+2347030/PSNJ07250042+2347030_J_16_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11moy/PTF11moy_H_17_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11moy/PTF11moy_J_17_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11mty/PTF11mty_H_18_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11mty/PTF11mty_J_18_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11owc/PTF11owc_H_19_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11owc/PTF11owc_J_19_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11qmo/PTF11qmo_H_20_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11qmo/PTF11qmo_J_20_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11qpc/PTF11qpc_H_21_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11qri/PTF11qri_H_22_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11qri/PTF11qri_J_22_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF11qzq/PTF11qzq_H_23_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/PTF13dad/PTF13dad_J_24_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011gf/SN2011gf_J_25_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011gy/SN2011gy_H_26_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011gy/SN2011gy_J_26_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011ha/SN2011ha_H_27_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011ha/SN2011ha_J_27_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011hb/SN2011hb_H_28_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011hb/SN2011hb_J_28_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011hk/SN2011hk_H_29_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011hk/SN2011hk_J_29_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011ho/SN2011ho_H_30_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011hr/SN2011hr_H_31_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011hr/SN2011hr_J_31_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011io/SN2011io_H_32_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011iu/SN2011iu_H_33_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011iu/SN2011iu_J_33_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011iy/SN2011iy_H_34_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011jh/SN2011jh_H_35_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2011jh/SN2011jh_J_35_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012bh/SN2012bh_H_36_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012bp/SN2012bp_H_37_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012cg/SN2012cg_H_38_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012cg/SN2012cg_J_38_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012em/SN2012em_H_39_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012em/SN2012em_J_39_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012et/SN2012et_H_40_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012et/SN2012et_J_40_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012fb/SN2012fb_H_41_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012fb/SN2012fb_J_41_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012fk/SN2012fk_H_42_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012fk/SN2012fk_J_42_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012fm/SN2012fm_H_43_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012fm/SN2012fm_J_43_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012fr/SN2012fr_H_44_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012fr/SN2012fr_J_44_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012gm/SN2012gm_J_45_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012go/SN2012go_H_46_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2012go/SN2012go_J_46_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013ar/SN2013ar_H_47_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013be/SN2013be_H_48_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013bo/SN2013bo_H_49_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013bo/SN2013bo_J_49_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013bs/SN2013bs_H_50_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013bs/SN2013bs_J_50_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013bt/SN2013bt_H_51_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013bt/SN2013bt_J_51_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013fw/SN2013fw_H_52_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/SN2013fw/SN2013fw_J_52_0,0.fits
/Volumes/PS1/SweetSpot/DR1_data/repo/test_dr1/rerun/coadd/deepCoadd/iPTF13dkj/iPTF13dkj_H_54_0,0.fits
from wiyn_lsst.
wmwv
commented on September 27, 2024
All 99 coadds now successfully generated under d_2018_04_09 in coaddNew.
from wiyn_lsst.
wmwv
commented on September 27, 2024
Note: I believe the daily tag I had been trying out for coaddnew was d_2018_04_12.
I'm now using w_2018_15 (which is after d_2018_04_12).
from wiyn_lsst.
wmwv
commented on September 27, 2024
The problem with calexp images being all NaN due to just one bad pixel has been resolved and mergeCoaddDetections->measureCoaddSources->mergeCoaddMeasurements->forcedPhotCoadd is working for LSQ13cwp.
from wiyn_lsst.
wmwv
commented on September 27, 2024
I've been having trouble with image processing of a coadd failing on my desktop, Serenity (Mac Os X 10.13.4), but not on lsst-dev01.ncsa.illinois.edu. Both using lsst_distrib -t w_2018_15 plus obs_wiyn.
I can't debug with gdb on Serenity because of SIP, and it doesn't fail on lsst-dev01.
from wiyn_lsst.
wmwv
commented on September 27, 2024
Running on lsst-dev01, got a total of 131 coadds for 72 out of 74 fields.
The missing fields are P330-E and S791-C.
from wiyn_lsst.
wmwv
commented on September 27, 2024
Ran photometry on coadds. 131 calexp files generated from the 72 fields.
from wiyn_lsst.
Related Issues (20)
- Adapt to fully use `obs_wiyn` HOT 1
- Create HTM catalogs for SweetSpot calibration HOT 1
- Solve "Too Many CR pixels" errors; presumably by setting smaller initial PSF guess HOT 1
- Create per field/filter coadds for all DR1 fields HOT 23
- Do forced-photometry on stacks based on coadd detections for all objects HOT 1
- Run subtractions for DR1 fields with references. HOT 1
- Run processCcd for all DR1 stacks HOT 5
- Run subtraction for one field. HOT 1
- Extract lightcurve for one field: LSQ13cwp HOT 4
- Extract lightcurve from subtractions for isolated SN
- Create individual-image lightcurves for 33 DR1 supernova HOT 12
- Switch to including `expnum` in filenames HOT 2
- Figure out why I need to specify `expnum` in subtractions.
- Make sn_lightcurve.py use same Butler object HOT 1
- Identify why some fields have multiple patches in the tract.
- Investigate LSQ13cwp, LSQ13cmt, iPTF13dkl calexp forced-photometry lightcurves HOT 2
- Run calexp forced photometry for non-host-contaminated SNe HOT 2
- Calibrate all DR2 images. HOT 3
- The filename mapper is not figuring out what to call the file in subtraction routine HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wiyn_lsst.