The Cambridge Infrared Survey Instrument (CIRSI), a near-IR mosaic imager containing a 2x2 array of Rockwell Hawaii I 1024x1024 detectors (Beckett et al. 1996; Mackay et al. 2000), has been in operation for about two years, obtaining almost 1TB of imaging data. The uniquely wide field accessible by CIRSI on a 2-4m class telescope makes it ideal for moderate depth large-area surveys. Preliminary results from two such current surveys include the measurement of galaxy clustering at intermediate redshift (McCarthy et al. 2001), and the demonstration of a reddening-independent quasar selection technique based on combined deep optical and near-IR color diagrams (Sabbey et al. 2001).
However, the CIRSI data reduction poses several challenges. With the
large data rate (5-10GB of data taken per night, 100 nights
per year, and a significant data backlog currently) the software has
to be very efficient and completely automated. Also, the software
should handle diverse data sets, from Galactic center observations to
very sparse fields at high Galactic latitude. From thousands of
individual images taken over many nights, wide-field, deep mosaic
images are generated. Since the gaps between detectors comparable to
detector size, filled mosaic images are made using the co-added dither
sets from different chips and telescope pointings. Thus weight maps
and accurate astrometry are crucial and the common simplification of
clipping the dither sets to their intersection region is not
appropriate.
Although existing software packages are used when possible (see below), the decision was made to write core image processing routines to satisfy certain design requirements. For example, a two pass reduction provides: object masks derived from first pass co-added dither sets, subpixel image registration/co-addition using full-weight maps without image clipping, optimizations for CPU and disk efficiency, customized artifact cleaning (destriping and defringing), and reusable tools (from having stand-alone tasks to be glued together with a high level scripting language, to making C library calls, or even extracting portions of source code). The basic processing steps, described below, are: flatfield correction, running sky frame subtraction, dither offsets measurement, dither set co-addition, and mosaic image creation.
The image stacking is done using cubemean.c, which calculates the median, robust standard deviation, or robust mean plane with a choice of weights (none, scalar, or maps) and image scaling or zero offsets using the image modes. This is not a general purpose tool like IRAF's imcombine, but for the specific task of calculating the median plane was found to be 2.5times faster (for a stack of 50data frames). The flatfield image is converted to a gain map and bad pixels identified using gainmap.c. The data are flatfield corrected using flat.c.
Running sky frame subtraction is normally a significant bottleneck in
processing infrared imaging, so optimizations are important. To do
running sky frame subtraction for a stack of N images, the program
skyfilter.c uses a sliding window (circular buffer of image pointers) to
require only N image reads and N image mode calculations. In contrast,
putting this logic into a script normally involves N M image
reads and mode calculations, where M is the width of the sky filter
in frames. Also, the disk I/O (and storage) for non-co-added data uses
short integers (2bytes deep), even though most calculations are done
in floating point (4bytes). Some calculations work with short integer
data however to allow optimizations. For example, the almost trivial
distribution sort can be used to obtain an image histogram, sorted
image array, and median value in
time (
5times faster
than running an optimized median routine on typical data images).
The cross-correlation technique uses coordinate, magnitude, and shape
information and is more reliable than matching object coordinate lists
(improvement was noted in extreme cases, e.g., Galactic center images
and nearly empty fields with an extended galaxy). Subpixel offset
measurement accuracy of pixel is obtained by fitting a
parabola to the peak of the cross-correlation image. This
cross-correlation method was found to be
times faster
(for typical survey data and a relatively large search box of
100pixels) than IRAF STSDAS crosscor. Although the success
rate is
100%, offset measurements corresponding exactly to
the search area border, or a small fraction of object pixels
overlapping in the aligned data images, would indicate failure.
With dithercubemean.c, dither frames are combined by calculating
the weighted mean pixel value at each position of the dither
stack, with pixel values
from the median at each
position rejected. Images borders are added during registration to
avoid clipping the data to the intersection of the dither frames. The
standard deviation (
) at each position is calculated from
, where MAD is the median absolute
deviation from the median (simpler methods such as minmax rejection
are inappropriate when taking averages of small numbers of values,
e.g., 5-9frames per dither set). The weight maps are combined by
calculating the sum at each
position of the stack of weight
maps (for pixels not clipped during co-addition).
The current astrometry pipeline, a small SExtractor Perl script that
produces an object catalog for each co-added dither set, runs
APMCAT (a stand-alone C
program)
to download over the
network the APM sky coordinates of objects in each field of view and
then runs IMWCS from WCSTools (Mink 1999) to calculate the astrometry
fit and update FITS header WCS information. Co-added dither sets and
weight maps from different chips, telescope pointings, and nights are
then drizzled onto a wide-field mosaic image using
EIS Drizzle.
Astrometry residuals between the mosaic image and the APM catalogue
show a random error of
0.3
without significant
systematic effects.
Beckett, M. G., et al. 1996, SPIE, 2871, 1152
Bertin, E., & Arnouts, S. 1996, A&AS, 117, 393
Devillard, N. 2000, The Messenger, 100, 48
Mackay, C. D., et al. 2000, SPIE, 4008, 1317
McCarthy, P., et al. 2001, in ESO ``Deep Field'' Workshop, held in Garching, Germany, October 2000, astro-ph/0011499
Mink, D., 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, ed. David M. Mehringer, Raymond L. Plante, & Douglas A. Roberts (San Francisco: ASP), 498
Sabbey, C. N., et al. 2001, in New Era of Wide Field Astronomy, held in Preston, England, August 2000, astro-ph/0012294