Pipelines
Different pipelines exist for different instruments, based on their performance characteristics and what we want to do with the data in terms of archiving.
RATCam pipeline [go to RATCam page]
Basic instrumental reductions are applied to all RATCam images before the data are passed to users. This includes bias subtraction, trimming of the overscan regions and flat fielding. A library of the current calibration frames is maintained as part of the data archive and updated daily so that images are always reduced using the latest available flat-field image available at the time. Each of the operations are performed as described below.
Bias Subtraction
There is insufficient repeatable structure for bias frames to be useful. Bias subtraction is therefore based purely on analysis of the underscan region. Linear regression is used to determine a fit to the bias counts as a function of pixel row number and values deducted across the image according to this smooth function. Experience shows that RATCam does have a small ramp in the bias down each column and this first order fit is required. No attempt is made to remove any bias gradient in rows across the image.
Overscan Trimming
The overscan regions are trimmed off the image leaving a 2048x2048 (assuming on-chip binning was not used) pixel image.
Dark Substraction
This is not currently performed though the facility exists in the reduction pipeline. Experience has shown that when the camera is at normal operating temperature, dark current is not significant. If the camera temperature fluctuates for any reason, the dark current is sufficiently variable that a dark would need to be obtained with every frame and library dark frames are no use.
Flat Fielding
The appropriate master flat field is selected from the library to match the filter and binning configuration of the current exposure. In fact the library holds reciprocal flat-fields normalised to unity because of the computational efficiency of multiplying rather than dividing. The image data are therefore multiplied by the library flat.
Each twilight the instrument control software (ICS) attempts to update the oldest master flats in its library, by taking 3-5 raw sky flats for each filter/binning combination, giving preference to the most used. There usually isn't enough time to take sky flats for all combinations in one twilight session, so the next oldest in the list is attempted in following nights and so on. Usually the update is complete after 2-3 nights and the process starts again.
The master flat is derived from the median of each sky flat after each has been normalised to the common mean count level. New master flats, and their corresponding ratio image of new/old masters, are inspected for any corruption, inclusion of stars, etc.
Bad Pixel Mask
No cosmic ray rejection or bad pixel mask is applied since it is important for users performing accurate photometry to know exactly what masking has been applied. However, bad pixel masks have been generated and kindly made available by the Angstrom project, a gravitational lensing programme underway on the LT and RoboNet:
Vignetting
The filter wheel slightly vignettes the optical beam to a different extent for each filter. In the extreme corners of the worst affected bands (i' and g') the flux is reduced by up to 15% compared to the unobstructed beam. In the other filters, obscuration is about 5% in the very corner of the observed field, falling to negligible values between 10 - 20 arcsec from the field edge.
The vignetting generally flat fields out very well and is rarely obvious in the reduced data, sometimes leaving distortions only in the range 3 - 5 arcsec from the field corner. Even where the data is well flat fielded though, noise characteristics of photon counting statistics could be affected in these regions of the frame.
Fringe Frames
We currently do not perform any automated defringing on CCD data before it is loaded into the archive. In order to help you defringe your own data, linked below are prepared master fringe frames created by stacking multiple deep integrations of blank fields. The master fringe frames are updated infrequently because the fringes on the CCD have been found to vary only on timescales of months. If you need access to the individual integrations which go into building these master frames, they are publicly available from the data archive. Simply select RATFringe from the Proposal ID drop-down list. You can therefore extract the most recent fringe frame from the archive at any time.
|
|
FITS Header Error Codes
Error codes are written into the FITS headers for specific error
flags brought up during processing. These error flags are stored in the
header keywords
L1STATOV, L1STATZE, L1STATTR, L1STATFL and L1STATDA and
follow this convention:
Tables of failure and warning states are given below - but please note these are not exhaustive lists.
Code # |
Process Name |
Error String |
-12 |
DpRT_startup |
Input is not a valid LT filename. |
-14 |
DpRT_startup |
According to filename flags, input has already been processed. |
-15 |
DpRT_startup |
Run_mode is not valid (%d). See dprt.h for valid values. |
-18 |
DpRT_startup |
Invalid exposure type flag. |
-35 |
DpRT_zero |
Cannot find zero (bias) frame file. |
-37 |
DpRT_dark |
Cannot find dark frame file. |
-38 |
DpRT_flat |
Mean counts in flat <= 0. Processing abandoned. |
-40 |
DpRT_flat |
Cannot find flatfield file. |
-60 |
DpRT_fringe |
Cannot find fringe file. |
-61 |
DpRT_fringe |
Fringe and data frame sizes do not match. |
-63 |
DpRT_fringe |
Correlation scaling failed. No defringing performed. |
-73 |
DpRT_dark |
Correlation scaling failed. No dedarking performed. |
-261 |
Error opening FITS. |
|
-263 |
Error getting header keywords. |
|
-264 |
Non-square binning. |
|
-268 |
Error reading image array. |
|
-354 |
DpRT_make_bias |
Could not open working directory. |
-356 |
DpRT_make_bias |
Failure to allocate memory for **fits_pointers. |
-357 |
DpRT_make_bias |
Failed to open FITS file. |
-358 |
DpRT_make_bias |
Failed to allocate memory to median_array or new_bias_array. |
-359 |
DpRT_make_bias |
Failed to allocate memory to temp_array. |
-360 |
DpRT_make_bias |
Failed to allocate memory to temp_array[%d]. |
-361 |
DpRT_make_bias |
Failed to read image data from FITS file number %d. |
-363 |
DpRT_make_bias |
Poor stats in new_bias. |
-453 |
DpRT_make_flat |
Could not read file (outer loop). |
-454 |
DpRT_make_flat |
Could not open directory %s (inner loop). |
-462 |
DpRT_make_flat |
Fewer than %d good flats from which to make master flat. |
-452 |
DpRT_make_flat |
Could not read existing flat frame. |
-356 |
DpRT_make_flat |
Failure to allocate memory for **fits_pointers. |
-364 |
DpRT_make_bias |
New bias is highly deviant from the old one: mean abs dev = %. |
-454 |
DpRT_make_flat |
Could not open directory. |
-457 |
DpRT_make_flat |
Failed to open FITS file. |
-458 |
DpRT_make_flat |
Failed to allocate memory to median_array or [mean|median]_flat_array. |
-459 |
DpRT_make_flat |
Failed to allocate memory to temp_array. |
-460 |
DpRT_make_flat |
Failed to allocate memory to temp_array[%d]. |
-461 |
DpRT_make_flat |
Failed to read image data from FITS file number %d. |
Code # |
Process Name |
Error String |
32 |
DpRT_startup |
CCD not at thermal set point. |
52 |
DpRT_output |
No filter calibration data for filter. |
53 |
DpRT_output |
No filter calibration data. |
62 |
DpRT_fringe |
No correlation between data and fringe. Defringing will use simple exposure time scaing. |
64 |
DpRT_fringe |
Correlation scaling failed. Simple exposure time scaing will be used. |
72 |
DpRT_dark |
No correlation between data and dark. Dedarking will use simple exposure time scaing. |
220 |
DpRT_init |
No '.' character in flatfield filename. |
225 |
DpRT_init |
Divide-by-zeroes were safely trapped. |
226 |
DpRT_init |
Failed to trim the flat field. |
230 |
DpRT_init |
Failed to open FLATLIB directory. |
232 |
filter_params |
Failed to open filter parameter lookup table. |
233 |
filter_params |
Failed to parse filter config line. |
231 |
filter_params |
Failed to allocate name memory for filter. |
234 |
filter_params |
Filter config line has a name > 49 chars. |
235 |
filter_params |
Filter scale factor or ZP read from file is not valid for filter. |
236 |
filter_params |
Failed to allocate filter memory. |
251 |
DpRT_init |
Failed to allocate memory for *photstar_data. Calibration data will not be available. |
252 |
DpRT_init |
Failed to open photstar parameter lookup table. |
351 |
DpRT_make_bias |
Could not open log file. |
352 |
DpRT_make_bias |
Could not read existing bias frame. |
353 |
DpRT_make_bias |
Could not read file. |
355 |
DpRT_make_bias |
Too many good bias frames were found. |
362 |
DpRT_make_bias |
New bias is highly discrepant from library bias. |
366 |
DpRT_make_bias |
More bias frames were discarded than kept. |
367 |
DpRT_make_bias |
Fewer than %d good bias frames. |
363 |
DpRT_make_flat |
Poor stats in new [mean|median] flat. |
364 |
DpRT_make_flat |
New flat is highly deviant from the old one. |
365 |
DpRT_make_flat |
No suitable flat frames found in directory. %d were read but discarded. |
453 |
DpRT_make_flat |
Could not read file (inner loop). |
500 - 507 |
DpRT_init |
NULL pointer in dprt_*_lib. |