Mosaic Keywords for Multi-HDU FITS files

UCO/Lick Observatory built the DEIMOS instrument which was commissioned at Keck Observatory in the spring of 2002.

The detector for DEIMOS is a mosaic of 8 CCDs, each with 2048x4096 imaging pixels (see drawing as PDF or PS or a subset in a web page). Its layout is extremely similar to that of the KPNO Mosaic detector which has been in use for several years.

There is a need to standardize the FITS keywords which are used for reporting the world coordinate systems (WCSs) of mosaic images. Without standard keywords different image display programs are unable to produce consistent views of the mosaic data.

The KPNO Mosaic

The KPNO mosaic writes its data into a multi-HDU FITS file. Typically the data are stored such that there is a separate IMAGE extension for the pixels from each amplifier used to read out the CCDs. FITS files such as these are processed and displayed by IRAF-based software.

Note that the KPNO mosaic data structures define a set of WCSs which relate FITS pixels to locations on the entire detector and on individual CCDs. These WCSs are used to display the KPNO mosaic data both in real-time during readout and subsequently during processing.

The WCS keywords used in the KPNO mosaic images make use of "image sections" to describe relevant subsets of pixels. Some image section keywords describe various subsets of pixels in the FITS image array. Other keywords describe subsets of pixels on the CCDs or subsets of pixels in the focal plane of the entire mosaic.

These image section keywords are evolved from prior usage at KPNO and within IRAF. They do not make use of the formalisms proposed in the FITS WCS draft papers by Greisen and Calabretta.

The DEIMOS Mosaic

DEIMOS has also chosen to write its mosaic data into multi-HDU FITS files with one IMAGE extension per amplifier. The layout of the amplifiers in the DEIMOS detector provides for readout through the eight ``A'' amplifiers, through the eight ``B'' amplifiers, or through all 16 A & B amplifiers (see drawing as PDF or PS or a subset in a web page).

Interoperability, or lack thereof

For DEIMOS we adopted a set of WCS keywords which are derived from Greisen & Calabretta's FITS WCS drafts. We believe the images to be viewable in both the NOAO/IRAF mosaic image display as well as the SAO/HEAD ds9 image display.

Early versions of the ds9 display program attempts to make use of the KPNO mosaic WCS keywords as defined by Frank Valdes. Unfortunately the definition of these keywords is slightly ambiguous. The result was that ds9 did not assemble a multi-HDU mosaic image in the same fashion as the IRAF tools. After much collaboration, and a tremendous amount of effort on the part of the its development team, ds9 now displays DEIMOS images in a sensible fashion.

A typical raw DEIMOS or KPNO mosaic image consists of about 140 Mbytes of data. Images of this size are too large for casual interchange and too large for some computers to display easily. Probably for this reason there has been relatively little interchange of mosaic data between tools from different sites. In an attempt to remedy this situation we present a multi-HDU FITS file containing a typical spectral image from DEIMOS. This file is 140 Mbytes big.

Documentation of UCO/Lick Mosaic WCS keywords

The meanings of the WCS keywords adopted for use in IMAGE extensions produced by mosaics built at UCO/Lick Observatory are explained here.

The meanings of the mosaic data structure keywords adopted for use in IMAGE extensions produced by mosaic built at UCO/Lick Observatory are explained in these XML files.

A sample DEIMOS frame

In an attempt to remedy this situation we present a multi-HDU FITS file containing a typical spectral image from DEIMOS. This file is 140 Mbytes big and uses 8 of the 16 amplifiers to read out the entire array. We also offer a PNG file showing how the ds9 viewer displays the DEIMOS mosaic from this multi-HDU FITS file.

Those who are familiar will see that DEIMOS has followed the basic structure of the NOAO images. The image from each amplifier is stored as a separate FITS IMAGE extension. In the details of the keywords, however, the differences become evident. DEIMOS has omitted many of the NOAO mosaic keywords in favor of ones that conform to the WCS standard.

Sample HIRES mosaic frames

The HIRES spectrograph at Keck Observatory will be upgraded in early 2004 by replacing the current single CCD with a mosaic of 3 CCDs. The dewar and an engineering version of the mosaic are already operational and being used for testing of the hardware and software. Other than the number of CCDs, the FITS files being produced by HIRES share all of characteristics of the DEIMOS fits files.

We present three early FITS files produced by readouts of the new HIRES science grade mosaic on 2003-10-28 just after it was installed in the dewar.

Because these FITS files were produced immediately after switching from the engineering mosaic to the science mosaic, the identifying metadata in the IMAGE extensions was still set to indicate that the engineering mosaic CCDs were in use. Also, the gains of the amplifiers in these images have not been adjusted for linearity. These images should be used solely for testing whether applications can understand the mosaic layout of the new HIRES detector.

We have a PNG file showing how the ds9 viewer displays a HIRES mosaic from such multi-HDU fits files. The IRAF mosaic image display should produce similar results. The images above differ in that they were taken with the X-ray calibration bar obscuring the central portion of each CCD.

The images demonstrate that the HIRES mosaic can be read out using either the three A amplifiers, or the three B amplifiers, or all six. Therefore the number and size of the FITS IMAGE extensions will vary. The FITS keywords supplied with each IMAGE extension unambiguously identify which FITS pixel came from which physical pixel through which amplifier. This PDF file contains the engineering diagram that gives the vocabulary for the names and coordinates.

PANEs with the HIRES mosaic

We have recently implemented new code which permits us to specify PANEs (known at some observatories as Regions of Interest -- ROIs) of the entire mosaic which are to be saved to the output FITS file. De Clarke has created a new GUI permits the PANEs to be specified quickly by drawing on the screen. When PANEs overlap the pixel values from the mosaic are duplicated in a separate HDU for each PANE in which that pixel occurs.

As an example of this new capability we offer

A little example of the file layout

I have constructed a multi-HDU FITS file which contains a small image whose content should be familiar to anyone. I have split these data into 8 IMAGE extensions and added prescan and overscan pixels and rows akin to those found in typical raw images. The result is a file directly akin to that which would be produced by a readout of the ``B'' amplifiers of DEIMOS -- but almost 1000 times smaller.

The source data are from the 1-degree RAND/SIO global topography map (360x180 pixels). For completeness (and ease of viewing with old image displays) the mosaic image is also available split into single-HDU FITS files as would have been obtained from the ``B'' amplifiers 1, 3, 5, 7, 10, 12, 14, 16.

Note that I have included keywords for multiple WCS in the fashion described by Greisen and Calabretta. This should display correctly with the current version of ds9, but I have not checked recently.

For the sake of initial interoperability tests I have included several mosaic WCS keywords which are peculiar to the KPNO mosaic; namely these are DETSIZE and DETSEC. These are currently required for ds9 to interpret the file as a mosaic image. However I believe these two need not be in the final, standard set of FITS mosaic WCS keywords. (I had originally included CTYPE cards with names like 'xxxx-LIN' in the expectation that this is needed for Doug Mink's WCS libraries.)

The ds9 image display correctly interprets the NOAO-inspired DATASEC DETSEC, and DETSIZE keywords. It also makes use of the multiple different WCS that are described in these files.

Initial discussion

It is very clear that a keyword conveying the information in DATASEC is required for proper display of these images. I argue that keywords such as DETSIZE and DETSEC should not be required, but there are tradeoffs.

The DETSIZE and DETSEC keywords are useful in the case where there are no transpositions or significant rotations involved in the layout of a CCD mosaic. They permit easy display using old software which is presuming that all pixels fall onto an integer grid of pixels. Without DETSIZE an image display cannot know the ultimate extents of the complete display until it has processed the DATASEC and WCS keywords from all of the IMAGE extensions in a multi-HDU mosaic FITS file. Nevertheless, I do not consider these reasons to be sufficient justification for the necessity of DETSIZE.

The NOAO image structure definitions from Frank Valdes contain ATM/ATV (and/or LTM/LTV or DTM/DTV) keywords along with the DETSEC keywords. These do provide the information necessary in the case where there are transpositions or rotations, but they do not provide any information which is not contained in the DATASEC and standard FITS WCS keywords. (None of this is a surprise because the NOAO definitions originated before the language for multiple WCSs in the current drafts.) The NOAO image structure definitions also contain hard-coded notions of CCD, amplifier, image, and detector coordinates. I think it preferable to use the standard keywords along with WCSNAME cards identify a particular WCS as 'CCD', 'amplifier', or 'detector' (WCSNAME is a feature added to the WCS drafts at my behest in anticipation of application to this situation).

Community input

Our particular application of non-overlapping CCD mosaic is only one type of multi-HDU FITS file. Others also need consideration: It would probably be good to produce some simple, small examples of the above images for interchange among the community of viewer developers.

In the general case this becomes a user-interface issue which will require significantly more research into the possibilities before we dare to standardize keywords. What kinds of keywords might be used to indicate these intentions to an image display application? Is it even appropriate to consider the standardization of keywords which are intended as verbs to be executed as opposed to the original FITS attribute/value pairs?

The issue of multi-HDU FITS files containing mosaics needs to be discussed by the entire FITS community. I hope this serves as fodder to get things started. Some consensus on keywords for the simpler cases is needed as soon as possible.


Steve Allen <sla@ucolick.org>
Initial deployment 2000-11-16T22:33:19
$Date: 2004/05/20 23:07:41 $