# IRC_SPEC_TOOLKIT Change Log ====================================================================== [20181203] (Phase 1&2; Phase 3) ---------------------------------------------------------------------- --- Release Notes on IRC_SPECRED v20181203 for phase 1 & 2 data reduction Corrections made in this version 1. Revision of the wavelength and spectral response calibrations of NIR/grism spectroscopy by considering the refractive index of the grism as a function of the wavelength and removing contamination of the second-order light. The new calibration extends the available spectral coverage of the NIR/grism mode from 4.9 micron up to 5.0 micron. Details are given in Baba et al. 2016 (https://doi.org/10.1093/pasj/psw013). Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/RESPONSE_NG.lst IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/response_NG_2nd_P2.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/response_NG_1st_P2.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/WAVEPAR/WAVETABLE_NG.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/init_specred.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/irc_senscalib.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/irc_specred.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/plot_spec_with_image.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/make_env_var.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/neighbor_id.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/ng_make_dullness_matrix.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/ng_make_response_matrix.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/ng_waveconv.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/ng_waveget.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/read_wavetable_ng.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/set_response.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/set_wavetable_ng.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/wavelength_func.pro 2. Spectral response curves of the MIR-S spectroscopy (SG1, SG2) are revised using the latest spectroscopic toolkit. The new spectral response curve of SG2 is 10 % larger than the old curve, whereas the old and new spectral response curves of SG1 are roughly the same. Details are given in Yamagishi et al. 2019 (https://doi.org/10.1093/pasj/psy132). Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/RESPONSE_SG1.lst IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/RESPONSE_SG1_HD42525_20171022_adjust.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/RESPONSE_SG2.lst IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/RESPONSE_SG2_HD42525_20171022_adjust.dat 3. Wavelength dependence is newly considered in the aperture correction of the MIR-S spectroscopy (SG1, SG2). Details are given in Yamagishi et al. 2019 (https://doi.org/10.1093/pasj/psy132). Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/plot_spec_with_image.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/read_apcor.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/set_apcor.pro 4. Correction of the estimation of the flux uncertainties by considering both the statistical and systematic errors. The error propagation method throughout the toolkit has also been revised. Details are given in Yano, K. 2017 (Ph.D. Thesis, The University of Tokyo; https://doi.org/10.15083/00075557). Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/irc_senscalib.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/make_spec1d.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/plot_spec_with_image.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/calc_noise_for_plot.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/do_smooth_spec_for_plot.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/make_offspec_for_plot.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/write_spec.pro 5. Minor revision of displaying extracted images with plot_spec_with_image. Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/draw_tv.pro No changes in the usage of the toolkit. ====================================================================== --- Release Notes on IRC_SPECRED v20181203 for phase 3 data reduction Corrections made in this version 1. Revision of the wavelength and spectral response calibrations of NIR/grism spectroscopy by considering the refractive index of the grism as a function of the wavelength and removing contamination of the second-order light to construct the spectral responses of the system. The new calibration extends the available spectral coverage of the NIR/grism mode from 4.9 micron up to 5.0 micron. Details are given in Baba et al. 2019 (https://doi.org/10.1093/pasj/psy131). Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/RESPONSE_NG.lst IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/response_NG_1st_P3.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/response_NG_2nd_P3.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/WAVEPAR/WAVETABLE_NG.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/init_specred.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/irc_senscalib.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/irc_specred.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/ng_2ndcalib.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/plot_spec_with_image.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/make_env_var.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/neighbor_id.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/ng_make_dullness_matrix.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/ng_make_response_matrix.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/ng_waveconv.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/ng_waveget.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/read_wavetable_ng.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/set_response.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/set_wavetable_ng.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/wavelength_func.pro 2. Correction of the estimation of the flux uncertainties by considering both the statistical and systematic errors. The error propagation method throughout the toolkit has also been revised. Details are given in Yano, K. 2017 (Ph.D. Thesis, The University of Tokyo; https://doi.org/10.15083/00075557). Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/irc_senscalib.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/make_spec1d.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/plot_spec_with_image.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/calc_noise_for_plot.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/do_bin_spec_for_plot.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/do_smooth_spec_for_plot.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/make_offspec_for_plot.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/write_spec.pro 3. Correction of the sensitivity due to increasing of the detector temperature. Details are given in Baba et al. 2019 (https://doi.org/10.1093/pasj/psy131). Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/RESPONSE/TEMPPAR_NG.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/irc_specred.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/correct_response_tempdepen.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/get_temperature.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/make_env_var.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/set_response.pro 4. Applying subpixel stacking of spectral images. Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/fshift2d.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/irc_shift_image_cube.pro 5. Cosmic ray removal before spectral image stacking in order to avoid failure of image stacking. Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/calc_image_shift.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/la_cosmic.pro 6. Revision of the distortion correction of the spectral images. Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/CALIBDIR/DISTPAR/IRCDISTPAR_NG.dat IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/irc_distcalib.pro 7. Minor revision of displaying extracted images with plot_spec_with_image. Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/draw_tv.pro No changes in the usage of the toolkit. ====================================================================== [20150331] (Phase 1&2; Phase 3) ---------------------------------------------------------------------- Corrections made in this version << for Phase 1&2 toolkit >> 1. Correction of the problem on Nh slit spectroscopy. The width of the Nh slit (3 arcsec) was not properly considered in the flux scaling of the plotting tool and the output flux was unreasonably decreased by 3/5. Revised programs: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/write_spec.pro IRC_SPEC_TOOLKIT/ASTRO-F/IRC_UTIL/plot_spec.pro 2. Correction of the problem on displaying extracted images with plot_spec_with_image. Previously bg_nsum option was not properly reflected on the display. The resulted spectra have been correct. Revised program: IRC_SPEC_TOOLKIT/ASTRO-F/IRC_SPECRED/draw_tv.pro No changes in the usage of the toolkit. << for Phase 3 toolkit >> 1. Correction of the problem on Nh slit spectroscopy. The width of the Nh slit (3 arcsec) was not properly considered in the flux scaling of the plotting tool and the output flux was unreasonably decreased by 3/5. Revised programs: IRC_SPEC_TOOLKIT_P3/ASTRO-F/IRC_UTIL/write_spec.pro IRC_SPEC_TOOLKIT_P3/ASTRO-F/IRC_UTIL/plot_spec.pro 2. make_spec1d.pro is revised to work with the slit width up to 30 pixels. Revised program: IRC_SPEC_TOOLKIT_P3/ASTRO-F/IRC_SPECRED/make_spec1d.pro 3. Correction of the problem on displaying extracted images with plot_spec_with_image. Previously bg_nsum option was not properly reflected on the display. The resulted spectra have been correct. Revised program: IRC_SPEC_TOOLKIT_P3/ASTRO-F/IRC_SPECRED/draw_tv.pro No changes in the usage of the toolkit. ====================================================================== [20130823] (Phase 1&2) ---------------------------------------------------------------------- This release includes two revisions: 1. The spectral response for NP and NG spectroscopy are revised. New response data are derived from observations of the following standard stars: For NG; KF09T1 (5020032.1, Nc) for 2.5--4.15 and 4.55--5.0 micron KF06T2 (5020019.1, Nc) for 3.0--4.15 micron Bp+66_1073 (5124003.1, Nc) for 3.0--4.55 micron For NP; KF06T2 (5020028.1, Nc) for 2.3 -- 5.0 micron HD1757132 (5124039.1, Nc) for 2.3 -- 4.0 micron KF09T1 (5124055.1, Nc) for 4.5 -- 5.5 micron The response data are named as newRC_NG_P2.dat and newRC_NP_P2.dat, respectively. They are smoothed in the spectral direction to match with the resolution determined by the image size to reduce the noises in the small spectral scale. For the details of the response construction for NP, please consult Shimonishi et al. (2013) AJ, 145, 32. 2. make_spec1d.pro is revised to work with the slit width up to 30 pixels. ====================================================================== [20111121] (Phase 3) ---------------------------------------------------------------------- Following routines are replaced. IRC_SPEC_TOOLKIT_P3/ASTRO-F/IRC_SPECRED/calc_shift_specbox_y_ng2.pro IRC_SPEC_TOOLKIT_P3/ASTRO-F/IRC_SPECRED/do_wavecalib.pro IRC_SPEC_TOOLKIT_P3/ASTRO-F/IRC_UTIL/calc_image_shift.pro The revised program will (1) fix the bug that the program stops when only one source is detected under some conditions, (2) and return (0, 0) when the image matching fails. Usage of the program will not be changed. ====================================================================== [20110301] (Phase 3) ---------------------------------------------------------------------- Response curve data are replaced with the corrected ones. A serious bug was found that the uncertainties in the response were erroneously estimated by 30% lower than it should be. This affects both NG and NP spectroscopy. ====================================================================== [20110117] (Phase 1&2 and Phase 3) ---------------------------------------------------------------------- They fix the following bugs in previous versions. They affect only split spectroscopy. 1. A typo in IRC_UTIL/calc_mir_refimage_shift.pro has been corrected. It fixes an abrupt halt when you try to work on MIR slit spectroscopy. 2. IRC_SPECRED/plot_spec_with_image.pro has been revised. It fixes a bug that gives infinite numbers for the errors in the ASCII file for the slit spectroscopy only when you specify median_width=0. 3. IRC_SPECRED/preset_wave_offset_pix.pro has bee updated. The wavelength calibration for slit spectroscopy in phase 2 has been adjusted. It now provides the same calibration as in phase 3. ====================================================================== [20101025] (Phase 1&2 and Phase 3) ---------------------------------------------------------------------- For IRC_SPEC_TOOLKIT, the following problems have been corrected: 1. overestimate in the error calculation for all IRC slit spectroscopy data 2. bad pixel propagation 3. treatment of Nan data in the split spectroscopy For IRC_SPEC_TOOLKIT_P3, in addition to the above three, 4. New response functions for NG and NP, which have already been announced in 2009 December, have been included. Note that the items 1 & 3 only affected slit spectroscopy. ====================================================================== [20090211] ---------------------------------------------------------------------- The followings are from the Release Notes. Please see the document as well as IDUM. + Robustness - The program was modified to be more robust for various types of data. + Suppressing warning messages in the batch mode - Some messages are unnecessarily in the batch mode. + Suppressing too much warning output. - Under some circumstances too many warning messages disturb process logging. They are omitted. ====================================================================== [20080528] ---------------------------------------------------------------------- The followings are from the Release Notes. Please see the document as well as IDUM. + TOOLKIT bug fixes - X-shift value of the spectroscopy apertures has been corrected - Removing some sub-frames for stacking on ds9 works properly - Appropriate flat selection according to the observation date works - Mask for sky subtraction adjusted for each sub-frames - Bitshift (wraparound) correction properly working - Short-exposure frames are properly processed - Wrong flux scale of the slit spectra with /diffuse option corrected + New tips: - Flat fielding for slit spectroscopy data. - Removing effect of column pull-down in NIR images. + Improvement of calibration/processing parameters - Source mask sizes improved. - Sky fitting over stacked images improved - Dark subtraction for slit spectra at NIR improved + New features - New option for masking column-pulled-down columns for NIR spectroscopy (NP/NG) implemented. - Spectroscopy WCS attached in output FITS images. - Saturation masking implemented. + Changes in command specifications - Changes of 'change_offset' command format + Calibration data updates - NP/SG1/2 response curves updated - Aperture correction tables updated ====================================================================== [20070913] ---------------------------------------------------------------------- There are many major changes in the program. See IDUM for details. ====================================================================== [20070306] ---------------------------------------------------------------------- Title: A bug in writing target tables for MIR-S and MIR-L was found, and is corrected. Problems: Coordinate conversion that is only required for NIR (that is rotated by 90 degree) works also for MIR-S and MIR-L. Effect: Wrong coordinate conversion was applied for MIR-S and MIR-L, and the results were recorded on *.target_table.tbl Extent of the impact: For MIR-S and MIR-L. No problems for NIR. The program was for writing pixel coordinates of the sources onto disk files, just before finishing the main program (after calibrating everything). Modified file(s): IRC_UTIL/write_target_table.pro, called from IRC_UTIL/write_all_output.pro ---------------------------------------------------------------------- Title: A bug in checking input/database files, before start actual processing the data, was found, and is corrected. Problems: Not well known. not really a problem? Effect: Nothing? (because the problem was found within the IF condition that will be never permitted...?) Extent of the impact: None? Modified file(s): IRC_SPECRED/do_param_initialize.pro, called from IRC_SPECRED/init_specred.pro ---------------------------------------------------------------------- Title: The same program exists in the Spitzer's SMART package, and the software may confuse to find the appropriate procedure. Problems: depending on the path settings, the IRC_SPECRED package may read the get_aot_info.pro of the SMART package. Effect: Error stop within the procedure (variable undefined). Extent of the impact: Error stop before finishing up the irc_specred. Modified files(s): rename IRC_UTIL/get_aot_info.pro to IRC_UTIL/get_aot_info_irc.pro, and modified IRC_SPECRED/do_param_initialize.pro that calls the get_aot_info.pro ====================================================================== [20070303] ---------------------------------------------------------------------- Title: A bug in plotting the NP spectra was found, and is corrected Problems: The 'nsum' parameter in 'plot_spec_with_image' procedure does not work correctly for the NP data. Effect: Reduced S/N in NP spectra Extent of the impact: For NP only, i.e., no problems for other grisms. The program is for extracting the one-dimensional spectra, after calibrating everything in the extracted spectroscopy images in 2D. Modified file(s): IRC_SPECRED/make_spec1d.pro, called from plot_spec_with_image.pro