Comments on the Internet Draft to register MIME media types for FITS

This file contains commentary on the reasoning behind the choices that were made for the text of the Internet Draft that registers MIME media types for FITS.

Despite significant effort from the FITS MIME working group, this draft still deserves much discussion and evolution. The Virtual Observatory communities may not have had sufficient chance to review it.

Choice of MIME media types to be registered

The motivating force for the fitsmime effort was Bill Joye's discovery of many unofficial MIME media types already in use for various forms of FITS files. Among these were the media types suggested by ASU which include a non-existent top level media type of binary and some media types that include some kind of compression into the MIME media type. For example:

binary/x-fits: binary is not one of the top level MIME media types defined by RFCs 2046 and 2077.
image/x-gfits: A means of compression such as gzip should not be part of a media type. It can, however, be a HTTP content-coding or a HTTP transfer-coding.

The unofficial MIME media types such as these are not suitable for registration with the IETF and IANA, and the draft RFC cannot endorse or address them. Parties employing them may continue to do so, but their continued use is fodder for the argument that some new reserved keywords which communicate the same notions should be added to the FITS standard.

(Note that there is no standard means for indicating that a FITS file being transferred by e-mail has been compressed. This is a known omission in the MIME standard. The issue has been adequately addressed by the standard for HTTP, but for e-mail it must await an RFC which revises MIME.)

When Don Wells originally suggested MIME for FITS he suggested registering four media types: image/fits, application/fits-image, application/fits-table, and application/fits-group. After some discussion on the fitsmime mailing list it was evident that the FITS community and FITS file authoring software simply do not distinguish FITS files into such subsets. It was decided to register only application/fits and image/fits.

The large list of unofficial MIME media types already in use suggests that there are other sorts of classification of FITS files that could be useful, but the discussion of the mailing list indicates that such classifications should be done within the context of FITS itself by adding new reserved keywords and values to the standard.

application/fits

The available options for the top level MIME media type are in RFCs 2046 and 2077. The only one broad enough to handle a fully general FITS file is application. Therefore this will be the catch-all MIME media type, and any FITS file may use it.

image/fits

The reason for permitting image/fits to have additional HDUs containing table and image information is to accommodate the WCS notions embodied in the drafts of WCS papers III (the -TAB non-linear algorithm) and IV (distortions described by interpolation). While such FITS files are more complex than the original PHDU described by Wells et al., the intent of such files is still to communicate a single image and its characteristics.

Permitting image/fits to be used for files of dimension other than two is somewhat dogmatic. Nevertheless, it is arbitrary to restrict image/fits to truly two-dimensional PHDUs. Even for such files a FITS-viewing application has many more responsibilities than, e.g., a GIF viewer. Among these are checking for single-pixel axes in a PHDU with NAXIS greater than 2 and supplying a color map.

Elements of the IANA Registrations

In accordance with RFC 2048, the IANA requires many fields of data in the registration of a MIME media type. Several of the draft sections for these deserve commentary.

Optional parameters: none

It remains possible that there is some utility in specifying that there can be Optional parameters that modify a MIME media type for FITS. However, there are no clear candidates for what such a parameter would be called. There is also no clear mechanism for how a webserver would be able to ascertain what values would be given. Nevertheless, the rules in RFC 2048 admit that the contents of the IANA registrations can be changed after their initial publication.

Here is a half-baked idea about new reserved keywords and their conventional values that might or might not be useful.

MIMEPAR1 through MIMEPAR9: These might be string-valued keywords whose values give MIME optional parameters and/or their values. Note, however, that transmitting these via the webserver implies that the webserver is required, at the least, to read the keywords from the PHDU and search for them. This is a bit strange, so their utility is dubious.

Published specification:

All of the existing webservers which actually host the NOST standard are inside the United States. As FITS is an international standard, it really ought to be available from webservers elsewhere. The WCS papers are currently available only from the web pages of their authors, and these are not widely linked. It would be good if they were collected onto a few mirrored sites, but until the papers are several years older this will require explicit permission from Astronomy & Astrophysics. All of the older FITS papers are available from the many NASA ADS mirror sites, and that is a Good Thing.

File extension(s): fits

Given that the Internet Draft is requesting only two different MIME media types for all of FITS, it should be relatively easy for web publishers and web masters to determine which of the two media types is best to use for the FITS files in a particular location. In the absence of a clear decision, it is always safe to choose application/fits.

The original typing mechanism in most webservers was that a single configuration file (typically named mime.types) maintained by the webmaster provided pairs of MIME media types and file extensions. Use of this old mechanism could mean that all files ending in .fits would become either image/fits or application/fits. The choice would depend on the preferences of the vendor of the webserver, possibly as modified by webmaster.

Given that the Internet Draft proposes two different media types, the MIME media type of FITS files might vary on a file-by-file basis. For that reason it would likely be necessary to use the mechanisms provided by newer webservers which permit the individual user who publishes a file to indicate the MIME media type of each file. In the case of the Apache webserver this would be done using AddType or ForceType directives issued in the .htaccess file of a served directory. Note, however, that Apache does not provide a means for giving individual MIME media types to individual files. Other webservers do provide such means, but that is at odds with the widespread presumption that MIME media type and extension correspond directly.

Security Considerations

The capabilities of FITS are effectively identical to the capabilities of TIFF as described in RFC 3302. TIFF permits tagged regions within a TIFF file to describe and contain arbitrary content.

There is an existing example of FITS being used as a wrapper to transport hierarchical directories containing and describing files of arbitrary MIME media type. This is FITSUTIL by Zarate, Tody and Seaman which is available as an external IRAF package.

Clearly it would be possible to use the FITSUTIL fgread and fgwrite functions in a system which would automatically unpack and execute files of any suitable MIME type. Clearly this would be a bad idea, but it is not impossible.

FITS as an acronym

The word FITS is not expanded as an acronym in the text of the Internet Draft. This is intentional. The intent is inspired by the proposal from Bill Pence that the acronym behind FITS be changed. I believe that the risk of creating an RFC which has an obsolete definition of FITS is greater than the benefit provided by expanding the acronym. The history of the meaning of FITS can always be sought in the defining papers.

Credit where credit is due

The list of names mentioned in the following sections is not closed. I may have missed significant efforts that are already present in the draft, and there may be others added before it becomes final.

Contributors

Anyone else with significant input to this document but who does not desire the responsibility of signing off with the RFC editor.
In accord with IESG guidelines this is the middle level in the hierarchy of author/contributor/acknowledgment.

Acknowledgments

Anyone else?
In accord with IESG guidelines this is the lowest level in the hierarchy of author/contributor/acknowledgment.

Authors

Anyone listed here will have to sign off with the RFC editor in order for the Internet-Draft to be elevated to RFC. (The IESG does enforce a limit of 5 authors.)

Things that would have been nice

It would be kind of nice to add another paragraph which points at various documents that describe particular usages of FITS files which are commonly exchanged among various sub-disciplines of astronomy. For example, the binary table format used for photon event lists by high energy astronomy. From my own experience I should be able to point at a document that describes various usages of putting CCD mosaic data into FITS files.

Partly these would be examples of the unsolved problems of FITS semantics, but moreso they could be guides for implementors who might want to teach their applications to display more kinds of data. There are undoubtedly other documents pertaining to various sorts of FITS usage which would deserve to be mentioned. Alas, the problem with all of them is that they are not solved problems. Their writeups are not in locations that are citeable in any permanent sense. As such, I do not believe that it is appropriate for the RFC to include references to them.

Steve Allen <sla@ucolick.org>

$Id: comments.html,v 1.13 2004/10/22 20:23:17 sla Exp $