Preliminary specs for MOS Output Data Products

The tangible products with scientific value from an astronomical instrument are the data. These data return home with the investigator for further analysis. In keeping with astronomical convention, all data products from the MOS shall be stored within FITS files.

Required Data Products

The primary data product from the MOS is a CCD image containing spectra from the fibers. This image is stored in FITS format. MOS uses the newer Keck Data Acquisition System (DAS) rather than the older one long in use at Lick Observatory. The Keck DAS permits such FITS files to be stored on disk and/or tape.

The identities of the object in each spectrum on the CCD image cannot be documented without including auxiliary information. This information is based upon the observer's original input catalog and upon the fiber configuration which was used for the observation. It shall provide a complete description of the mapping between spectra, fibers, objects and their celestial coordinate locations. This auxiliary information shall be stored in a FITS ASCII table extension.

The archivally correct scheme to keep the FITS table extension properly associated with the FITS image would be to append the table extension to the FITS header and data unit (HDU) for the primary image. Unfortunately most current FITS image readers ignore anything beyond the primary HDU (PHDU); when reprocessing such an image in place on disk they will typically destroy any extension HDUs. Also, the Keck DAS is not currently designed to append arbitrary FITS extensions to the image HDUs. For these reasons we choose to create the table extension for MOS in a separate FITS file.

In order to document the relationships between these separate files the FITS Grouping Convention shall be used. The grouping convention requires that an additional FITS table be created. This 'GROUPING' table contains a list of all the related HDUs. In the case of the current MOS data the list of related HDUs would be

The FITS GROUPING table (which lists the others)
The primary HDU containing the CCD image
The ASCII table containing fiber/object configuration

Other tables could also be included; e.g., it could be useful to include a table that documents approximately where on the CCD each fiber spectrum falls.

Existing Practice

During an observing session the Keck DAS produces FITS files on disk with names such as /full/path/prefixNNNN.fits. The full directory path (/full/path/) can be modified and is stored in the FITS header as the string value of the keyword OUTDIR. The file name (prefix) can also be modified and is stored in the FITS header as the string value of the keyword OUTFILE. The sequence number (NNNN) is always zero-padded to 4 digits and is stored in the FITS header as the integer value of the keyword FRAMENO (and for compatibility with the Lick DAS also as OBSNUM).

The sequence number typically increments by one after each saved CCD readout, but the observer is allowed to modify this sequence number such that more than one FITS file has the same value of FRAMENO. There is thus no guarantee of a single unique identifier for each image, but in practice it is extremely uncommon for an observer to so modify the sequence number.

A point of difficulty

The FITS Grouping Convention works well when all HDUs are stored within the same FITS file. It also works well when HDUs are stored in separate named files on a directory-structured file system with Uniform Resource Identifier (URI)-like characteristics. Because FITS tapes lack a universal scheme for associating files with file names the grouping convention can only work well if the tape reader uses the FRAMENO, OUTFILE, and OUTDIR keywords to reconstruct a file name. One possible solution would be to adopt the NOAO convention of storing the file system name in the PHDU using the FITS keyword IRAFNAME. Other solutions are in use by other projects.

For MOS it shall be that the FITS file containing the grouping table and the fiber/object table extensions will be written to tape immediately following the FITS file with the CCD image PHDU. This seems to be the best solution for the initial implementation. In the long run the lack of a single FITS keyword which can serve as a unique, file system-oriented identifier in the Keck DAS needs to be addressed.

New Requirements

The DAS must acquire the fiber/object configuration information from the LLNL SCRAP code. The SCRAP code is responsible for configuring the MOS fibers in accordance with observer requests; it is best suited to provide the information to the DAS.

During the time that these images are stored on disk the file names can also be used to indicate their association. The tables can be stored in a file whose name is clearly associated with the image. E.g., for image d0001.fits the file containing the grouping table and the fiber/object table shall be d0001.mos.

Changes to the CCD image HDU

In accordance with the guidelines of the FITS grouping convention the FITS file containing the CCD image is not required to change. It is strongly recommended, however, that two additional FITS keywords be added to the image header. These keywords provide an indication that the primary HDU is a member of a group by pointing to the grouping table that contains the PHDU.

GRPID1 = -1: This FITS grouping keyword requires an integer value. This indicates that the HDU is a member of a group. The negative sign indicates that the grouping table resides in a different FITS file; there must therefore also be a GRPLC1 keyword. Taking ABS(-1) indicates that the grouping table to which this HDU belongs has EXTVER = 1.
GRPLC1 = 'prefixNNNN.mos': This FITS grouping keyword gives a partial URI (in this case the name of another file in the same directory) in which the grouping table resides.

A FITS HDU may indicate its membership in as many as 999 groups. The MOS PHDU belongs to only one. There is no need for GRPID2 through GRPID999.

Structure of the FITS file with the tables

The newly-added FITS file must consist of five different HDUs. The detailed structure of each HDU is indicated within the links.

A primary HDU with zero-size image: FITS requires that a PHDU be at the beginning of all files.
A grouping table: Documents the relationship of the CCD image and the other FITS HDU(s).
The input catalog table: This reiterates the total content of all input catalogs provided by the user. If the assignment procedure deploys fibers to positions not contained in the input catalogs (e.g., ``sky fibers'') then the celestial coordinates of those positions are also included. Assigned objects point into the fiber table to identify their fiber.
The fiber configuration table: For each object in the input catalog table this contains the refracted position in celestial coordinates and the distorted MOS micro-positioner coordinates. The complete list of distorted MOS coordinates can be used to drive the sky-viewing camera to all objects in a cluster for the purposes of refining the distortion map. The refracted celestial coordinates are provided in order to permit post-run analysis of modeling errors in the astrometric software. (``Sky fiber'' positions are originally specified in one of these two coordinate systems; they are then transformed back into the catalog frame.)
The MOS hardware table: Documents the relationship of fibers, pivots, and ``slits''. Documents the MOS model used in the software including the locations of the pivots and the parking spots for the fibers. Assigned fibers point into the catalog tables to identify their object.

There is a data dictionary that describes the MOS-specific keywords in the above HDUs. It is essential reading for understanding of the MOS and its data products.

Back to the MOS home page.

Steve Allen <sla@ucolick.org>