1   Overview¶

Ultimately, LSST Alert Production (AP) will need to perform all steps of the Level 1 pipeline automatically on each new raw image from the telescope. Details are in the LSST Data Management Science Pipelines Design (LDM-151 [2]) and the LSST Data Products Definition Document (DPDD [1]). Briefly summarized below and in Figure 1, the first portion of these steps are:

1. Instrument Signature Removal (ISR), which includes crosstalk and linearity corrections, bad pixel masking, bias subtraction, and flat field division,
2. Point Source Function (PSF) and background characterization, which includes source detection and measurement,
3. Photometric and World Coordinate System (WCS) calibration, and
4. Image Differencing with an appropriate template (which may need constructing) to create Difference Image Analysis (DIA) source catalogs.

Figure 1 The first portion of the Nightly Processing Pipeline overview, from Figure 1 of [2]. This encompasses all of “Single Frame Processing” and part of “Alert Detection.”

A number of packages in the LSST Software Stack have been developed to perform portions of one or more of these four tasks, but they are not yet linked together in an automated way. This Prototype AP Pipeline is the first step toward the goal of an automatic Level 1 Pipeline that can process a set of raw images through each of these steps and verify that the output is what we expect.

To achieve this, there are two main python modules under development: ap_pipe and ap_verify. The former, described here, is responsible for running the Prototype Pipeline. The latter uses ap_pipe to verify the output.

2   Dataset¶

We use images from the High Cadence Transient Survey (HiTS) [3], a three-year survey from 2013–2015 which used the Dark Energy Camera (DECam) on the 4 m Blanco telescope at Cerro Tololo Interamerican Observatory (CTIO). The primary science goal of this survey is to detect and follow up optical transients that have characteristic timescales from hours to days, with a particular interest in catching early supernovae. The distribution of the HiTS survey fields in the sky is shown in Figure 2.

Figure 2 Spatial distribution of the fields observed in the 2013–2015 HiTS campaign (Figure 4 from [3]). The fields selected for this dataset are three of the pink ones, which indicate they were visited in both 2014 and 2015. Specifically, they are the pair of partially-overlapping pink fields near the bottom center of the 2014 and 2015 field region and the upper-right-most pink field.

We select HiTS fields Blind15A_26, Blind15A_40, and Blind15A_42, which are repeat observations of the same region of sky as Blind14A_04, Blind14A_10, and Blind14A_09 fields, respectively. Each field has 34–36 individual visits over about a month, and each DECam image consists of 62 CCDs (however, two of these are non-functional). Most of the visits are in the g-band, some are in the r-band, and a few are in i.

All of these data are in the git-lfs repo ap_verify_hits2015 in raw format, along with the corresponding DECam Master Calibration files and camera defect files. The data also exist on the lsst-dev server at /datasets/decam/_internal/raw/hits with the calibration files at /dataset/decam/_internal/calib/cpHits and the camera defects at /dataset/decam/_internal/calib/bpmDes. The raw images and calibration files were originally obtained from the NOAO Science Archive by searching for PI Förster.

3   Tutorial¶

This tutorial walks a user through using ap_pipe to run four main processing steps with the LSST Stack on a portion of the Dataset described above:

1. ingestImagesDecam.py, from the obs_decam package but drawing heavily on pipe_tasks,
2. ingestCalibs.py, from the pipe_tasks package,
3. processCcd.py, from the pipe_tasks package, and
4. imageDifference.py, also from the pipe_tasks package.

The prerequisites for running ap_pipe are:

• The LSST Science Pipelines stack with the lsst_apps packages (a subset of lsst_distrib)
• The obs_decam and ap_pipe packages
• A clone of the ap_verify_hits2015 dataset

You may set all of this up as follows:

setup lsst_apps
# If using the lsst-dev server, lsst_apps may be setup using a centrally
# available weekly build, but the following packages are not included
git clone https://github.com/lsst/obs_decam.git
git clone https://github.com/lsst-dm/ap_pipe.git
setup -k -r obs_decam
setup -k -r ap_pipe
git clone https://github.com/lsst/ap_verify_hits2015.git

Once you are ready, run ap_pipe from the command line. You must point to the dataset with the -d flag, a desired output location on disk with the -o flag, and provide a valid visit and ccdnum dataId string with the -i flag.

python ap_pipe/bin.src/ap_pipe.py -d ap_verify_hits2015/ -o output_dir -i "visit=410985 ccdnum=25"

This will run all four steps of the pipeline in order. Each step will create a new subdirectory in output_dir: one for ingested images, one for ingested calibration products, one for processed images, and one for difference imaging.

4   Results¶

The difference image and DIA Source catalog for a test image are available on the lsst-dev server at /project/mrawls/prototype_ap/zomg7/diffim/deepDiff/v410985/. A screenshot of the calibrated exposure (calexp) and difference image is shown in Figure 3.

Figure 3 Calibrated science exposure (top) and difference image (bottom) for CCD 25 of HiTS field Blind15A_40. This example uses visit 410929 as the template image and visit 410985 as the science image.

Future work will use coadded 2014 visits of the three HiTS fields described in Dataset as templates and the corresponding 2015 visits as science. This pipeline is a core component of the AP Minimum Viable System (note this link is private to LSST team members only). The goal is to run and verify each component of LSST image processing as we build a fully functional ap_verify system. The ap_pipe pipeline will also be available to users who wish to process any DECam dataset independently.

5   References¶