PANE example: PHDU from HIRES Mosaic

The first few cards are the mandatory keywords as inserted by the CFITSIO library.
SIMPLE  =                    T / file does conform to FITS standard
BITPIX  =                   16 / number of bits per data pixel
NAXIS   =                    0 / number of data axes
EXTEND  =                    T / FITS dataset may contain extensions
COMMENT   FITS (Flexible Image Transport System) format is defined in 'Astronomy
COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H
Note above that BITPIX=16, which indicates that the FITS file contains big-endian signed 16-bit two's complement integers (range -32768 to +32767). It is the case for all of the UCO/Lick CCD hardware that the analog-to-digital conversion produces unsigned 16-bit integers (range 0 to 65535). The structure of two's complement arithmetic allows these data values to be written into a FITS file simply by flipping the high order (sign) bit and setting the next two FITS keywords to these values.
BZERO   =                32768 / offset data range to that of unsigned short
BSCALE  =                    1 / default scaling factor
This is the date that CFITSIO obtained by querying the system clock for the POSIX time_t value. It is the time at which the lickserv2 process requested the creation of this HDU as part of the skeletal MEF file into which the pixel stream was descrambled. It does not directly indicate anything about the time of the exposure or the time of the movements of the camera shutter.
DATE    = '2004-05-20T22:22:47' / file creation date (YYYY-MM-DDThh:mm:ss UT)
In the mosaic data acquisition system created for Keck by UCO/Lick the multi-extension FITS file is constructed by the lickserv2 process as it captures the incoming pixel stream from the CCD crate. At that point the only FITS keywords are the ones which describe the pixel structure of each of the HDUs, but there is a large amount of extra room in the PHDU consisting of blank FITS card images. After the pixel stream is captured the FITS file is handed to the write_image process, and it inserts the following comment record to make it clear that write_image is responsible for the subsequent sequence of FITS header records.
COMMENT BEGIN observation-specific keywords written by write_image
Starting with the initiation of an exposure and proceeding until the exposure is complete the watch_ccd process has been gathering various KTL keywords and storing them. Most of the KTL keywords that watch_ccd gathers are specified in lists read from a file conventionally named header_info.
When write_image gets the FITS file it queries watch_ccd for the KTL keywords that apply to this exposure, and then it inserts sequentially into FITS keywords in the PHDU.
For the overall description of the keywords relevant to various detectors built by UCO/Lick Observatory see the sections of links regarding "bundles" of keywords on this page.
In particular, see the generated bundle of keywords generated by lickserv/write_image/watch_ccd.
COMMENT BEGIN keywords that came from KTL via watch_ccd
For the keywords that are particular to the HIRES mosaic see this generated bundle of keywords.
AUTOSHUT=                    F
ERASECNT=                    1
ANTIBLM = 'off     '
ABFREQ  =                   20
NSUBINT =                    0
ROWSHFT =                    0
INSTRUME= '        '
DETECTOR= '        '
At a temperature of 14 Celsius it is plainly evident this was an engineering exposure.
TEMPDET =          14.61323547
UTB30VEN= 'enabled '
UTB15VEN= 'enabled '
For the HIRES mosaic a new feature was introduced to give fine control over the exposures of the CCDs. The ERAMODE keyword indicates which of the CCDs are to be erased at the start of a (sub)exposure, and the MOSMODE keyword indicates which of the CCDs are read at the end of a (sub)exposure. (The VIDINBEG and VIDINEND keywords document this more finely by acting as a bitmask indicating which video inputs, i.e. amplifiers, participate in erase and readout.) Usually all CCDs will be erased and read, but if it were desired it is possible to close the shutter, read out some of the CCDs, erase some of the CCDs, and re-open the shutter. This allows different amounts of exposure on different CCDs in the HIRES mosaic by selecting which ones are erased and which ones are read out. This feature might be quite valuable if there is an object whose flux produces a tendency to have double the amount of counts on the "red" CCD as it does on the "blue" one, or vice versa. The tricky part of using this is that each shutter closing produces a multi-extension FITS file containing only the pixels from the amplifiers which were read, and the exposure time keyword in that image indicates the most recent (sub)exposure, not the total exposure which is applicable to the data from amplifiers which were not erased at the beginning of the most recent (sub)exposure.
ERAMODE = 'B, G, R '
VIDINBEG=                   63
ERASLINE=                    F
FRAMENO =                    0
KEEPPREP= 'true    '
CCDGAIN = 'low     '
CCDSPEED= 'fast    '
CCDPSIZE= '1,1,2048,4096'
NVIDINP =                    6
AMPLIST = '3,3,0,0 '
MOSMODE = 'B, G, R '
VIDINEND=                   63
In the earliest interfaces for reading out CCDs the parameters that described the windowing were expressed in binned pixels. This meant that it was necessary to re-define the readout window any time that the binning was changed. This interface became even more confusing with the advent of CCDs having more than one amplifier.
The mosaic of CCDs in DEIMOS prompted a new way of describing the pixels in the focal plane -- the PANExxxx keywords. The PANExxxx keywords employ a single coordinate system which is unique, valid everywhere on the mosaic, expressed in unbinned pixels, and unconcerned with the constraints of the physical boundaries between different CCDs and amplifiers. Furthermore, we allowed for as many as 16 different PANEs, thus allowing the specification of that many different regions of interest during a CCD mosaic readout. The PANExxxx keywords were initially created for the DEIMOS mosaic, but their operation was not fully implemented until the HIRES mosaic.
Recall that this PHDU is extracted from this multi-extension FITS file obtained during engineering checkout of the HIRES mosaic. As seen in the PANELIST keyword, this image used all 16 PANEs. (Really there are 17, for the default PANE keyword -- effectively PANE0 -- remains defined as all pixels in the mosaic, and is not used in this readout.) The PANEn keywords below correspond to the diagram below

This was a dual-amplifier readout of HIRES, and a careful count of the number of different PANE+amplifier regions in the above picture gives 41. The view of the resulting MEF file, with its 41 different FITS IMAGE extensions, as seen in ds9 is below
The idea of the PANEn keywords was to allow the definition of multiple, different kinds of regions of interest. E.g., for DEIMOS spectral exposures some PANEs might be defined to indicate spectral regions of interest, and/or slitlets of interest. For DEIMOS direct images other PANEs might be re-defined to match the alignment holes on upcoming slitmasks. While leaving the PANEn keywords unchanged, the PANELIST keyword could then be redefined suitably for the regions of interest on each different exposure.
One supposed advatage of PANEs was to reduce the amount of disk space needed to store images, but the explosion in size of disk storage media since the deployment of these instruments at Keck makes that moot. Furthermore, the advent of archives means that the regions not of interest to the initial observer may still contain serendipitous discoveries to be found during later data mining.
Nevertheless, depending on the CCD controller, the use of PANEs may reduce the amount of pixels in the readout and data transfer. For instruments which have that capability this may be advantageous in reducing the readout time for the mosaic.
PANEFITS= 'normal  '
PANELIST= '1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16'
PANE    = '0,0,6144,4096'
PANE1   = '408,711,2040,568'
PANE2   = '4936,2983,328,344'
PANE3   = '688,3519,3856,344'
PANE4   = '4872,231,488,288'
PANE5   = '3520,687,1832,424'
PANE6   = '2504,1295,1968,384'
PANE7   = '824,2775,3464,296'
PANE8   = '1720,2087,2528,392'
PANE9   = '4904,1415,376,968'
PANE10  = '1800,359,464,856'
PANE11  = '2736,151,392,584'
PANE12  = '856,255,384,640'
PANE13  = '864,1799,336,336'
PANE14  = '1424,119,216,1888'
PANE15  = '528,199,176,2984'
PANE16  = '2880,1039,496,2968'
The following keywords indicate where the FITS file was written and what its base name was. We encourage users to review the generated lists of CCD keywords for DEIMOS and HIRES with special attention to the DISKLIST keyword which allows specification of a colon-separated list of pathnames which will be tried in sequence if there is some failure writing an image to the path given by OUTDIR. Also see the DISKFREE, DISKFREP, and DISKIMGS keywords to see how much disk space is left.
OUTDIR  = '/local/u/kics/data/2004may19'
OUTFILE = 'foobar  '
OVRFLUSH=                    0
PREFLUSH=                  300
The UCO/Lick detectors have numerous keywords related to the duration of the exposure and shutter operation. When precise values are desired for the reduction of data involving dark current and exposure time it is imperative to review the definitions of this generated bundle of keywords.
TTIME   =                   24
ELAPTIME=                   24
EXPTIME =           0.00000000
DARKTIME=          24.11120033
SHBLOPTM=           0.00000000
SHBLCLTM=           0.00000000
SHLATOPN=           0.00000000
SHLATCLO=           0.00000000
SHOPNCNT=                    0
SHCLOCNT=                    0
As noted above, internally the data values for each pixel were unsigned 16-bit integers. That means the data value must never fall below zero. The CCD controller allows a "video offset" (a data value) to be applied in order to ensure that variations in voltages and noise during readout never produce a negative number. There is one video offset value for every amplifier in use during the readout.
What is most relevant here is to discuss the meaning of the index value on each of these offsets. The index here represents the "video input" on the CCD controller, not any particular amplifier. There is one video input for each amplifier in use, but their numbering need not be identical nor constant. For the CCD hardware in most instruments it is possible to permute the coaxial cables which run between the pre-amps on the dewar and the video inputs on the CCD controller. (Indeed, during engineering and debugging it is commonplace to swap cables to verify which parts of the signal pathways may not be working.)
For a diagram of this wiring with the HIRES mosaic see this image (or as PDF). Note that the HIRES diagram indicates video inputs numbered 1 thru 12 (with only the odd one in use). For the moment take this as an apology that despite the best efforts to allow for a complete description of the real world in software, sometimes engineering diagrams and code still do not use a self-consistent vocabulary.
The CCD crate which constructs the pixel stream cannot know anything about permutations of cable connections at the CCD controller. The information on the cable connectivity is stored as part of the data available to the lickserv2 process. When it creates the skeletal FITS file the lickserv2 process places its notion of the current wiring configuration into each of the individual IMAGE HDUs.
It is, therefore, incumbent on the engineers to ensure that the configuration used by lickserv2 gets changed whenever the cables are permuted. If not, then the resulting documentary and WCS keywords will be misleading. This will probably, but not necessarily, be immediately obvious when ds9 displays the mosaic image with mismatched alignment.
VIDOFF1 =                 2000
VIDOFF2 =                 2000
VIDOFF3 =                 2000
VIDOFF4 =                 2000
VIDOFF5 =                 2000
VIDOFF6 =                 2000
The CCD subsystem includes a number of string-valued KTL keywords which are traditionally stored in the infoman or infopatcher processes.
The GUI for the HIRES mosaic sets the value of the OBSTYPE keyword, uses it to streamline various configurations of the CCD subsystem, and uses it to cross-check whether other aspects of the exposure are consistent with particular kinds of observation.
The SYNOPSIS keyword is generated on-the-fly in the write_image process by substituting keyword values specified by the SYNOPFMT keyword.
OBSERVER= 'Clarke  '
OBJECT  = '        '
OBSTYPE = 'unknown '
SYNOPSIS= 'B, G, R 24 s, unknown pane 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16'
Note that we are still in the section of KTL keywords obtained from watch_ccd by write_image. In the Keck CCD subsystems it was traditional for some of the parameters which describe and/or control the structural aspects of the images to be accessible via KTL keywords. As seen farther below, it was also traditional for the Keck CCD subsystems to include records indicating the values of the structural parameters which were transmitted along with the pixel stream. This provides the possibility that there can be two separate instances of FITS keywords detailing such structural parameters, even though that was never a good idea for a FITS HDU. The following records have the values of the KTL keywords, not the structural parameters sent along with the pixel stream.
POSTLINE=                    0
POSTPIX =                   80
PRELINE =                    0
PRECOL  =                   12
None of the designs for the CCD systems in the optical instruments at Keck included any specifications for an ability to obtain a precise time stamp of observation begin or end. As a result, all such timing information is obtained via interactions between various subsystems. In actual use on the telescope the list of KTL keywords gathered by watch_ccd conventionally includes a timestamp which is obtained from the Keck DCS via the KTL keyword named UTC.
The CCD crates at Keck conventionally broadcast messages that indicate the opening and closing of the shutter, and the transit times for these messages is usually quite small. In typical use the list of keywords for the watch_ccd process indicates that the DCS UTC keyword should be obtained as part of the keywords gathered in response to the message indicating the beginning of the observation. Unfortunately, there is no mechanism for guaranteeing how the value of the DCS UTC keyword is related to the timing of the operation of the instrument shutter.
As the watch_ccd process receives the exposure sequence broadcasts (i.e., erase, shutter open, shutter close) from the CCD crate it runs through its corresponding list of KTL keywords to be collected. For each keyword watch_ccd issues a KTL read to the appropriate subsystem and then awaits the response from that subsystem. The response time depends on the load of the system being queried, and it is possible that a subsystem will not answer within a timeout of many seconds, thus delaying the query for the subsequent keywords by that much time.
In order to provide a cross-check on the exposure time keyword obtained from Keck DCS, watch_ccd collects the POSIX time_t values when it receives the broadcast messages about the first shutter opening event after the CCD is erased, and the final shutter closing event before the CCD readout. These values can, of course, only be as accurate as the Unix system clock, and they do not include sub-second information.
Many steps have been taken in the design of watch_ccd and the crafting of the lists of keywords for it to collect in order to reduce the likelihood of trouble when collecting any keywords, including the time stamps. Nevertheless, under adverse circumstances we have seen the value of the DCS UTC keyword differ from the DATE_BEG keyword by as much as 30 seconds. Observers who need precise times for the observations should compare the various time stamps and use whatever seems to make the most sense.
Note that these keywords differ from other FITS DATExxxx keywords because they are connected with underscore rather than hyphen. This choice was made in anticipation of a FITS WCS paper V, in which it is supposed that the FITS standard will designate new "reserved" keywords with hyphens which may have similar, but perhaps not identical, meanings.
DATE_BEG= '2004-05-20T22:22:20'
DATE_END= '2004-05-20T22:22:44'
COMMENT END   keywords that came from KTL via watch_ccd
As part of any sending any pixel stream to lickserv2 the CCD crates always include the metadata which describe the structural properties of the image. This is necessary so that even in the absence of any of the KTL keywords (from the various subsystems of the Keck telescope and instrument) it is still possible to descramble the image and insert it into the FITS file. The metadata are contained in a structure called imparm. The lickserv2 process hands this structure to write_image along with the skeletal FITS file, and the following keywords are generated and inserted from the data in that structure.
As noted above, some of these structural parameters have meanings similar, or identical to, KTL keywords used to control or document the CCD readout. Since the time of the original Keck CCD systems these structural parameters have traditionally been written into the FITS header with keywords that have the same names as the KTL keywords.
It has always been poor form for a FITS HDU to have more than one instance of a FITS keyword with a value. In some cases the values of the keywords duplicated in Keck HDUs have had different values, or even different data types. This ambiguity was the inspiration for, as a minimal remedy, enhancing the write_image code to add the COMMENT records. At least to a human these will hopefully indicate the origin of the two different instances and permit the possibility of making a rational choice about which value is relevant.
In the original single-HDU FITS files produced by the Keck CCD systems these keywords were the only means of identifying which regions of pixels in the image were prescan and overscan, particularly in the case of the CCDs which were read using more than one amplifier. The conventions for using those keywords were, however, not well documented and completely peculiar to Keck instruments.
With the advent of MEF files, where the pixel stream from each amplifier goes to one (or more) separate IMAGE HDUs, the lickserv2 process ensures that each HDU contains both the WCS keywords and the NOAO/IRAF image section keywords. These keywords provide unambiguous and community-standard ways of identifying which pixel regions are which. It may be possible that subsequent CCD instruments will omit the old and duplicate keywords.
COMMENT BEGIN keywords that came from CCD crate imparm structure
PRECOL  =                   12
PREROW  =                    4
PRELINE =                    0
POSTLINE=                    0
ERASLINE=                    0
KEEPPREP=                    1
PREFLUSH=                  300
OVRFLUSH=                    0
DETCNFID=                 1002
VIDINACT=                   63
BINNING = '1,1     '
CCDSUM  = '1 1     '
AMPPSIZE= '[1:1024,1:4096]'
DETLSIZE= '[1:6144,1:4096]'
COMMENT END   keywords that came from CCD crate imparm structure
COMMENT END   observation-specific keywords written by write_image
The last thing that write_image does before flushing the FITS file to disk is to ask CFITSIO to perform the FITS checksum operation on each of the HDUs. In the case of the PHDU it is possible to compare the value of the DATE keyword with the comment of the checksum keywords and see how much time elapsed between the creation of the skeletal FITS file by lickserv2 and the completion of keyword insertion by write_image.
CHECKSUM= '9VUAAVT39VT9AVT9'   / HDU checksum updated 2004-05-20T22:23:22
DATASUM = '         0'         / data unit checksum updated 2004-05-20T22:23:22

Steve Allen <>
Initial deployment 2000-11-16T22:33:19
$Date: 2010/04/16 21:15:03 $