7.5.3: Specifications for DEIMOS Output Data Products
The tangible products with scientific value from an astronomical
instrument are the data. These data return home with the investigator
for further analysis. In keeping with astronomical convention, all
data products from DEIMOS shall be stored within FITS files.
188.8.131.52: Required Data Products
The primary data product from DEIMOS is a CCD image containing
either a direct image or spectra from the slitlets.
This image is stored in FITS format. DEIMOS
uses the newer Keck Data Acquisition System (DAS) rather than the
older one long in use at Lick Observatory. The Keck DAS permits such
FITS files to be stored on disk and/or tape.
The identities of the objects in each spectrum on the CCD image cannot
be documented without including auxiliary information. This
information is based upon the observer's original input catalog and
upon the slitmask configuration which was used for the observation. It
provides a complete description of the mapping between spectra,
slitlets, objects and their celestial coordinate locations. This
auxiliary information shall be stored in a FITS ASCII table extension.
The archivally correct scheme to keep the FITS table extension
properly associated with the FITS image would be to append the table
extension to the FITS header and data unit (HDU) for the primary
image. Unfortunately most current FITS image readers ignore anything
beyond the primary HDU (PHDU); when reprocessing such an image in place on
disk they will typically destroy any extension HDUs. Also, the Keck
DAS is not currently designed to append arbitrary FITS extensions to
the image HDUs. For these reasons we choose to create the table
extension for DEIMOS in a separate FITS file.
In order to document the relationships between these separate files
FITS Grouping Convention shall be used.
The grouping convention requires that an additional FITS table be
'GROUPING' table contains a list of all
the related HDUs. In the case of the current DEIMOS data the list of related
HDUs would be
Other tables could also be included or referenced. For example:
- The primary HDU(s) containing the components of the CCD mosaic image
- The overall FITS
GROUPING table (lists all others)
- The image FITS
GROUPING table (lists image HDUs)
- The ASCII table containing input star/galaxy catalog
- The ASCII table containing the slitmask description
- The ASCII table containing the CCD defects
- The ASCII table(s) containing status & event logs for the observation
- ASCII tables pointing to calibration frames for pipeline reduction
- pointers to calibration frames (dark, flat, bias, comparison lamps)
as part of a scheme of pipeline data reduction
- pointers to other frames taken of the same field (as designated
during the mask design process)
184.108.40.206: Existing Practice
During an observing session the Keck DAS produces FITS files on disk with
names such as /full/path/prefixNNNN.fits.
The full directory path (/full/path/) can be modified and is
stored in the FITS header as the string value of the keyword
The file name (prefix) can also be modified and is stored in the
FITS header as the string value of the keyword
The sequence number (NNNN) is always zero-padded to 4
digits and is stored in the FITS header as the integer value of the
FRAMENO (and for compatibility with the Lick DAS
The sequence number typically increments by one after each saved CCD
readout, but the observer is allowed to modify this sequence number
such that more than one FITS file has the same value of
There is thus no guarantee of a single unique identifier for each
image, but in practice it is extremely uncommon for an observer so to
modify the sequence number.
220.127.116.11.1: A point of difficulty
The FITS Grouping Convention works well when all HDUs are stored
within the same FITS file. It also works well when HDUs are stored
in separate named files on a directory-structured file system with
Uniform Resource Identifier (URI)-like characteristics.
Because FITS tapes lack a universal scheme for associating files
with file names the grouping convention can only work well if the
tape reader uses the
OUTDIR keywords to reconstruct a file name.
One possible solution would be to adopt the NOAO convention
of storing the file system name in the PHDU using the FITS keyword
IRAFNAME. Other solutions are in use by other projects.
For DEIMOS the FITS file containing the grouping
table and other table extensions should be prepended to the
image data by the FITS Grouper tool.
In the long run the lack of a single FITS keyword
which can serve as a unique, file system-oriented identifier in the
Keck DAS needs to be addressed.
18.104.22.168: Other Requirements
While these images are stored on disk their file names
can also be used to indicate their association. The tables can be
stored in a file whose name is clearly associated with the image. As
an example, for image
d0001 the files containing the
n image sections shall be
d0001_n.fits grouping table and the other
tables shall be
22.214.171.124.1: Grouping keywords in the CCD image HDU
The guidelines of the FITS grouping convention do not require
the FITS file containing the CCD image to contain
indications that it is a member of a group.
However, the grouping convention strongly recommends that two
additional FITS keywords be added to the image header. These keywords
provide the indication that the primary HDU is a member of a group by
pointing to the grouping table that contains the PHDU.
GRPID1 = -1
This FITS grouping keyword requires an integer value.
This indicates that the HDU is a member of a group.
The negative sign indicates that the grouping table
resides in a different FITS file; therefore there must also be
GRPLC1 keyword. Taking
indicates that the grouping table to which this HDU belongs
EXTVER = 1.
GRPLC1 = 'prefixNNNN.dei'
This FITS grouping keyword gives a partial URI (in this case
the name of another file in the same directory) in which
the grouping table resides.
a FITS HDU may indicate its membership in as many as 999 groups.
In this current specification the DEIMOS PHDU belongs to only one.
As we develop the ideas of the automated data reduction pipeline we
me see further relationships between the image frames and, e.g.,
calibration frames. These could be documented by membership in
and pointing to other groups.
126.96.36.199.2: Structure of the FITS file with the tables
The FITS grouping file must consist of several different HDUs.
The detailed structure of each HDU is indicated within the links.
Section 188.8.131.52 contains a data dictionary
that describes the DEIMOS-specific keywords in the above HDUs. It is
essential reading for understanding of the DEIMOS and its data
A primary HDU with zero-size image
FITS requires that a PHDU be at the beginning of all files.
An overall grouping table
Documents the relationship of the CCD image and the
other FITS HDU(s).
The input catalog table
This reiterates the total content of all input catalogs
provided by the user. If the assignment procedure creates
apertures at positions not contained in the input catalogs
(e.g., ``sky slitlets'') then the celestial coordinates of
those positions are also included.
The slitmask configuration table
This contains a description of the apertures (to be) cut into
the slitmask which was in use during a spectral observation.
Model information which was used to plan the observation and cut
the slitlets. It should contain slit identifiers, object
identifiers and much more. In particular, the WCS information
used by the quick look tool to report wavelength, object
ID and position should be stored here.
[It has not yet been defined]
The DEIMOS hardware table(s)
Documents the DEIMOS model used in the software.
Specifically, such things as the refraction coefficients
used in the design of the mask, the distortion coefficients
describing the WCS transformations, etc.
[These have not yet been defined]
Back to the DEIMOS home page.
Steve Allen <firstname.lastname@example.org>
$Date: 1996/03/18 20:01:26 $