4.1 Introduction

4.1.1 What is it and how does it fit the "Big Picture"?

The spectrograph control software connects commands and data requests from DEIMOS UI's and other processes with the underlying mechanical hardware that moves optics, turns on lamps, controls temperature, etc.

This document defines the interface needs of the spectrograph control software, control requirements, anticipated safety issues and basic development procedures. The document also covers preliminary design of known system components and the considerable inheritance of common system components from past control systems or systems being developed concurrently with DEIMOS.

The piezo control software for the flexure control system won't be covered by this document. It will be treated as a separate system with some common components inherited from the spectrograph control system. The piezo control software will be reviewed at a later date.

4.1.2 Control hardware overview

4.1.2.1 Control computer

The control computer is connected to the mountain network and a private instrument network for DEIMOS. The control computer might have several other private network connections, including one to the ATM or fast ethernet used for image transport from the CCD VME controllers. The control computer runs tasks required to communicate to the DEIMOS instrument control hardware. Command logs and instrument related configuration data will be maintained on its disks.

4.1.2.2 Terminal Servers

Terminal servers handle the hardware handshaking needs between the instrument computer and devices requiring a serial I/O connection. These devices include Galil motor controllers, piezo controllers, bar code readers, UPS's and possibly the console ports on the CCD VME controller chassis. A Lantronics four port terminal server has already been tested with a Galil DMC-1500 motor controller in a cold box simulating conditions on Mauna Kea. Eight or sixteen port Lantronics terminal servers are expected to be used in DEIMOS.

One terminal server is required by the systems on the instrument carriage. The terminal server will handle I/O to the Galil controller used for instrument rotation and environment monitoring. It will also serve the serial ports, if any exist, on the UPS's.

One terminal server with either eight or sixteen ports, depending on the number of bar code readers, is needed for the spectrograph systems of each beam.

4.1.2.3 Galil controllers

Galil DMC-1500 controllers will be used for motor and digital and analog device control on DEIMOS. They are third generation Galil controllers. First generation DMC-330 controllers were used by HIRES. Second generation DMC-700 controllers were used by MOS. The control language and serial interface is nearly identical between second and third generation Galil controllers. Much of the firmware and Galil knowledge gained through MOS development is directly transferable to DEIMOS.

A Galil DMC-1500 motor controller has been tested at Mauna Kea temperatures.

4.1.2.4 Piezo controllers

A piezo actuator controller from Physik Instrumente maintains alignment of the tent mirror and the camera through two piezo translators, controlling both the "X" and "Y" axes of the image.

4.1.2.5 Bar code readers

Fixed bar code readers will be used to identify the slit masks in the instrument. Additional bar code readers are likely for the gratings and the filters. Up to four bar code readers may be used per beam, one for each of the following: slit masks, gratings, CCD camera filters, TV guider filters.

4.1.2.6 Uninterruptable power supplies

At least two UPS's will be included in the AC power system: one for the CCD control systems and one for the instrument control systems.

4.1.2.7 CCD VME controllers and Leach controllers

One CCD VME controller is required per beam. One CCD VME controller can handle multiple Leach controllers by having multiple fiber optic interface boards. The number of Leach controllers is unknown and is a function of the number of amplifiers. See Chapter 6 CCD Control.

4.2 Figures

4.2.1 DEIMOS instrument control functions (Engineering drawing D0200.H)

4.2.2 Instrument control component and interface schematic

4.2.3 Keyword hierarchy

4.3 Glossary/acronyms

   cvs		- A rcs based source code control system.

   DCS		- Drive and control system for Keck I or Keck II.

   DEEP		-

   DEIMOS	- Deep imaging multi-object spectrograph.

   dispatcher	- Process that translates instrument control requests into
		  motor control commands and passes the commands to a motor
		  controller.  In use now on Kast & MOS.  Will be used on
		  ESI, PFCAM & DEIMOS.

   dual-encoder stage	- A stage with two encoders providing position
			  feedback for control purposes.  One encoder is
			  on the motor, the other provides more accurate
			  stage position feedback via a more direct
			  connection to stage.

   ESI		- Echellette spectrograph & imager.  Cassegrain instrument
		  under development at Lick.  Scheduled for first light on
		  Keck II August, 1997.

   EPICS	- Experimental Physics Industrial Control System used for
		  Keck II telescope drive and control system.

   EPICS/CA	- EPICS channel access protocol.

   fiord	- FITS input/output routine database.  A library of
		  routines used to make instrument control and data
		  requests via KTL messages.

   HIRES	- High resolution echelle spectrograph built by Lick and
		  currently operational on Keck I.

   infoman	- The database manager for MUSIC.  It maintains a database
		  of keywords and data.

   LRIS		- Low resolution imaging spectrograph.  A multi-object
		  spectrograph with direct imaging capability built by CIT
		  for Keck I.  It will be used on Keck II until the first
		  operational instruments arrive for Keck II.  It uses a
		  multiple slit mask concept similar to the one used in
		  DEIMOS.

   Kast		- Cassagrain two-beam spectrograph developed for the
		  3 meter Shane telescope at Lick.

   KICS		- Keck instrument control system.  The software used to
		  control Keck I and its instruments.

   KTL		- Keck tasking library.

   modify	- Shell level command to change a KICS keyword value for
		  a Keck instrument or control system.  The keywords can
		  change device states.

   MOS		- Multi-object spectrograph.  A multi-object fiber-fed prime
		  focus spectrograph system for the 3 meter Shane telescope
		  at Lick.

   MUSIC	- Multi-user system for instrument control.  An
		  inter-process communication protocol developed at Lick
		  for transporting data between user interfaces and
		  instruments.  In use at Lick and on Keck I.

   PFCAM	- Prime focus camera.  A direct imaging camera under
		  development for the 3 meter Shane telescope at Lick.  It
		  uses the telescope prime focus top-end developed for MOS.

   rcs		- Revision control system available on most Unix platforms.

   sccs		- Source code control system available on some Unix systems.

   show		- Shell level command to display a snapshot of KICS
		  keyword values from a Keck instrument or control
		  system.  The keywords are entered as the parameters
		  of the command.

   single-encoder stage	- A stage with a single encoder on the motor
			  providing position feedback.

   terminal server	- A device that interfaces serial ports 
			  to Ethernet connections.

   traffic	- The central process that directs MUSIC messages to their
		  desired destination processes.

   UI		- User interface.

   waitfor	- A shell command to wait for a Keck instrument or control
		  system keyword to equal a value.

   watch_irot	- A HIRES control process that handles rotator position
		  updates.

   wflman	- A program that generates a LaTex file from commands
		  embedded in the comments of a source code file.  It
		  creates formatted function documentation from standard
		  function headers.

   xdeimos	- X window based graphical user interface for DEIMOS.

   xhires	- X window based graphical user interface for HIRES.

   xlris	- X window based graphical user interface for LRIS.

   xshow	- Shell level command which creates a simple X window
		  display of KICS keyword values from a Keck instrument
		  or control system.  The keywords are entered as the
		  parameters of the command.

   UPS		- Uninterruptable power supply.

4.4 Functional requirements (and how they will be satisfied)

4.4.1 General requirements

The instrument control software which runs on the control computer will be written in C and designed to run on one of several Unix systems (e.g. Solaris or Digital Unix).

4.4.2 Interface requirements

4.4.2.1 User interface

A graphical user interface, similar to xhires and xlris, and a keyword interface using the same show, xshow, modify and waitfor commands used on Keck I will be provided to control DEIMOS.

4.4.2.2 Database interface

Information on following will require some kind of database solution:

  1. Scale and offsets values to convert between stage encoder positions and real world coordinate and measurement systems.
  2. Configuration data.
  3. System states requiring independence from the control system processes and hardware.
The method of storage and retrieval for each of these items is TBD. More than one database solution may be used to solve the information storage and retrieval needs of the items listed above.

4.4.2.3 KTL interface

The instrument control software will communicate with the outside world via KTL messages. The KTL interface is described in section 4.5 Preliminary design.

4.4.2.4 Ethernet terminal server interface

The instrument control software running on the control computer will communicate with the serial ports on the motor controllers via terminal servers connected directly to the instrument ethernet. The terminal severs will handle the underlying serial I/O hardware handshaking. Communication with the terminal servers will be via socket connections such that the instrument control software is insulated from the low-level RS-232 serial interface.

The design described above avoids software serial interface problems due to differences in RS-232 serial interface implementation between Unix vendors. The terminal server hardware is configured to accommodate implementation differences.

4.4.2.5 Motor controller interface

The motor controller interface will be written to minimize the number of messages passed via the serial I/O link to the motor controllers since the I/O port speed is the main bottle neck in the instrument control data path. Firmware on the motor controllers will be expected to have many of the complicated command sequences bundled as firmware routines to avoid sending the entire command sequence down the serial line.

4.4.3 Control requirements

4.4.3.1 Command audit trail

A time-stamped log of the command requests sent to the instrument control system and their results (success or error) will be maintained.

4.4.3.2 Simultaneous motion of stages

The control system shall provide the capability to move stages individually or simultaneously. To minimize the time required to change setups, it shall be possible to move all stages simultaneously on both beams of the spectrograph. This capability shall be supported at all levels of the software and from all versions of the user interface.

A description of collision avoidance and motion interlocks can be found in section 4.5 Preliminary design.

4.4.3.3 Stage descriptions and individual control requirements

See engineering drawing D0200.H in figure 4.2.1.

Each beam of the spectrograph has its own slitmask selection system, grating system, filter wheel, CCD focus and TV guide system. Components common to both beams of the spectrograph include the hatch, lamp, rotation and some of the environment control systems.

4.4.3.3.1 Hatch

The hatch is driven by air powered cylinders. The cylinders are controlled by a single latching solenoid. Once the solenoid is activated, the stage drives to the latched position. The solenoid is activated with a short electrical pulse. The hatch position is detected via hall effect sensors attached to the air cylinders.

4.4.3.3.2 Lamps

DEIMOS uses both internal and external lamps. The internal lamps are controlled via digital I/O ports on a motor controller by the DEIMOS instrument control system. The external dome lamps may be controlled by DCS or another Keck II system TBD.

4.4.3.3.3 Instrument rotation

A friction drive system located in the DEIMOS carriage rotates the instrument through 840 degrees. A dual encoder system is controls the stage motion. The second encoder is mounted separate from the drive roller on a second friction roller. Limit switches and hard stops prevent DEIMOS from rotating far enough to damage its cable wrap.

4.4.3.3.4 Slitmask systems

The slitmask selection and insertion is handled by one motor for mask selection and one motor for mask insertion. Three air actuators lock the mask into position. Side B of the spectrograph uses two motors as fine position mask adjustors.

Ten slitmasks can be loaded into slitmask holders on a caterpillar drive mechanism on each side of the spectrograph. The caterpillar drive handles the slitmask selection task. To load or unload a slitmask holder the caterpillar rotates the slitmask holder into a "load" position. To insert or remove a slitmask from the light path of a beam of the spectrograph, the caterpillar must be rotated to the slitmask holder's "insertion" position. The caterpillar rotates in one direction only.

The mask insertion stage moves the mask from its holder on the catapiller into the observing position. Once in the observing position, the air actuators lock the mask down onto a kinematic mount. After a mask is locked to the kinematic mount on the B side of the spectrograph, the fine position actuators on the B side can be used to align the B side slitmask.

The slitmask caterpillar and insertion drive must be interlocked. The caterpillar must be in a slitmask "insert" position for the insertion drive to operate. The caterpillar should not be moved while a slitmask is inserted into the beam.

Limit switches and hard stops protect the slit-mask insertion stage end of travel.

4.4.3.3.5 Grating systems

The grating system consists of a slide mechanism to select a grating from a choice of four options and rotators to control the angles of the gratings. The slide has three rotatable grating locations and a fixed mirror position for direct imaging. Each grating has a second encoder for use by dual-encoder control loop for maximum positioning accuracy. The grating select stage will have some kind of clamping mechanism to lock a cell in place prior to rotating the grating in the cell.

The grating select and grating rotation stages must be interlocked. The grating select stage can't move with the gratings rotated to some positions. The safest solution to this problem is to only allow rotation of the selected grating and to rotate the selected grating flat with the mirror prior to moving the grating slide.

Limit switches and hard stops protect the grating slide and grating rotation stages end of travel. Limit switches will likely be used to determine grating clearance from slit masks and other obstacles.

4.4.3.3.6 CCD filter wheels

The filter wheels are continuous rotary stages with seven filter locations. There is one filter wheel for each CCD camera. Each filter location on the wheel requires definition of a use position, and possibly a definition of a filter load position.

4.4.3.3.7 CCD focus

Each CCD camera has a linear CCD focus stage. Limit switches and hard stops protect the focus stage end of travel.

4.4.3.3.8 TV guide optics system

The TV guide optics system consists of four independent sub-systems: the TV focus stage, the TV aperture stage, the TV filter wheel stage, and the slit viewing optics stage.

The TV filter wheel stage is functionally identical to the CCD filter wheel stage, with the exception of the number of filters on each stage. The TV filter stage has eight filter locations. See the CCD filter wheel description above for details.

The TV focus mechanism is functionally similar to the CCD camera focus.

The TV aperture control mechanism is likely to be similar to the TV and CCD camera focus mechanisms.

The slit-viewing optics control mechanism is undefined at this time. It most likely is a linear stage with control requirements similar to the TV focus, TV aperture, etc.

4.4.3.3.9 Environment control systems

The carriage controller handles environment monitoring and control for DEIMOS carriage systems. These functions include power monitoring and control, instrument temperature control, and automatic LN2 filling.

4.4.3.4 Positioning accuracy

The accuracy requirements are assumed to be identical between like stages of the two beams of the spectrograph.

The control system shall meet the following positioning accuracy requirements for each stage:

	 Stage                               Accuracy requirement
         --------------                      --------------------
         TV focus                            TBD degrees of focus mechanism
         TV filter wheel                     TBD degrees of filter wheel
         TV aperture wheel                   TBD degrees of aperture wheel
         TV slit-viewing optics              TBD mm (or microns?)
         CCD dewar focus                     TBD mm (or microns?)
         CCD filter wheel                    TBD mm (or microns?)
         Grating slide                       TBD mm (or microns?)
         Grating tilts for trays 1-3         TBD degrees
         Slit mask caterpillar               TBD mm
         Slit mask insertion                 TBD mm
         Instrument rotator                  TBD degrees
         Tent mirror theta x                 TBD (??units??)
         Camera/detector theta y             TBD (??units??)

NOTE: The accuracy requirements for the stages are not formally defined for the mechanical systems at this time.

Accuracy measurement assumptions:

To meet these requirements, it is assumed that each stage can be re-initialized to its home position once at the start of each night, but that further re-initialization should not be required during the remainder of that night. These requirements establish the worst-case error that will be tolerated during that entire night.

4.4.3.5 Maximum move completion times and command response times

For unconstrained rotary stages, it is assumed that minimum-path routing will be used for all moves, so that the worst-case move will be 180-degrees. Currently, the filter wheels are the only unconstrained rotary stages.

Each stage must complete its worst-case motion (e.g., motion between the forward and reverse limits for a linear stage) within the following time intervals:

	 Stage                               Completion time
	 --------------                      --------------------
         TV focus                            30 seconds
         TV filter wheel                     10 seconds for 180-degrees
         TV aperture wheel                   20 seconds
         TV slit-viewing optics              TBD seconds
         CCD dewar focus                     20 seconds
         CCD filter wheel                    30 seconds
         Grating slide                       30 seconds
         Grating tilts for trays 1-3         20 seconds
         Slit mask caterpillar               10 seconds per position
                                                (90 seconds max since the
                                                 caterpillar rotates one
                                                 direction only)
         Slit mask insertion                 10 seconds ????
         Instrument rotator                  30 seconds for 360 degrees
         Tent mirror theta x                 TBD seconds
         Camera/detector theta y             TBD seconds
The instrument rotation stage must be able to react to frequent velocity updates from the watch_irot process or its replacement. The updates are expected at least one per second, and more frequent updates may be needed as observing fields traverse the zenith.

Some of the monitor processes may have tight control loops with minimum required response times, but none have been designed yet.

4.4.3.6 Local/manual control

The control system shall provide for local manual control of each stage via a set of five local control paddles, one for each Galil motor controller. A given paddle need only provide manual control of a single stage at a time, but may allow simultaneous manual operation of more than one stage if feasible. Such manual control shall not require the availability of any of the DEIMOS computers or networks, but will require the availability of the stand-alone motor controller chassis (e.g., Galil DMC-1500). A back up method of controlling the motors via a direct DC voltage connection will also be provided, such that a motor controller failure won't prevent controlling stages for maintenance.

The local control paddles will not directly provide a continuous readout of stage motion. However, the current state of locally- commanded manual motions shall be displayed continuously on any active instrument user interface displays, so that such motions can be accurately monitored both locally and remotely. To support local monitoring of such motions, a portable X-terminal or a portable laptop computer running X will be provided near the instrument to allow local access to such user interface displays.

When a given stage is selected via its paddle for local manual control, it will not be accessible for remote operation. Any remote motions currently in progress on a stage will be aborted if that stage is locally selected for manual control. User interface displays will clearly indicate if any stage has been locally selected for manual control.

4.4.3.7 Remote/automatic control

The control system shall provide for remote automatic control of all stages via a single network segment. Such control will be interlocked with the local manual control described in 4.4.3.6, with local control taking precedence. The control system shall provide the capability to simultaneously move all stages that are not locally selected for manual control.

The control system shall provide for remote display of all relevant parameters for each stage, including but not limited to current stage position, auto/manual status, forward/reverse limit status, and brake status, if applicable. The current stage position shall be updated at least once every 5 seconds, regardless of whether or not the stage has been commanded to move, in order that uncommanded changes in position (e.g., due to thermal drift, flexure, or telescope vibration) will be visible to the user. If a given stage has not been commanded to move to a new position, and its current stage position is observed to differ from its last commanded position by more than a specified dead band, an alarm condition will be triggered for that stage, resulting in a prominent visual cue on the instrument status display. Alarm conditions will also be generated for commanded motions which fail to complete successfully.

4.4.3.8 Multi-user operation (local and remote)

All DEIMOS user interfaces shall be designed to support multi-user operation such that an observing team split between multiple sites (e.g., Mauna Kea, Waimea, and/or California) can cooperatively operate the instrument. Observers at all such cooperating sites should receive a consistent view of the instrument status and should thus be able to observe each others actions. The control system shall provide mechanisms for restricting control to a given site or group of sites, while providing "read-only" status display to a wider set of sites.

Delays in transmitting data to remote sites, such as Santa Cruz, shouldn't produce delays in data display at local sites, such as the mountain top. The current Keck I guider system has problems when used with remote observing; local displays update at the same rate as remote displays. Since the remote displays via the ethernet link with the mainland are very slow, the display updates on the mountain are very slow. This is unacceptable for most guiding situations. User interfaces should act independently at whatever data rate they can establish with the data service routines.

4.4.3.9 Safety interlocks

The design of the general purpose mechanical & electrical safety interlocks for DEIMOS are not yet complete. However, some kind of system disable switch and/or an emergency stop button will be provided. Activating either switch will result in automatically stopping all stage motion, with the exception of the air cylinder powered stages; these continue moving until they reach their commanded destination.

Audible warnings such as buzzers, alarm bells, etc. and visible warnings such as big colorful flashing lights will be used where needed to notify people working around the spectrograph as to hazardous conditions (i.e. doors open, emergency stop active, etc.). All light sources will be interlocked to prevent lights activating during observing.

4.4.3.9.1 Stage motion/device state changes

Devices will only move, apply power, or change device I/O states on command either via automatic or manual control.

4.4.3.9.2 Emergency stop buttons

Emergency-stop buttons will drop out all AC power to the servo amplifiers and possibly some or all of the following systems; the exact systems affected are TBD:

  1. Galil controllers
  2. Terminal servers
  3. Piezo controllers
  4. CCD controllers
  5. TV guider camera/controller
  6. or Kill power to everything within DEIMOS
Killing power to DEIMOS will also close the LN2 valves and the hatch.

The location of the emergency stop buttons, if any are added to the design are TBD.

4.4.3.9.3 Instrument rotator

Rotation of the instrument while someone is working inside an access panel is extremely hazardous.

The rotator will be mechanically locked in place while it is in an unbalanced condition. An unbalanced condition can only occur while adding or removing large components from DEIMOS, such as gratings. The drive mechanism prevents instrument rotation for most differences in the instrument balance. Cutting power to the rotator motor is sufficient to hold the rotator in place in such cases.

Rotation of the instrument while access doors are open or the rotator position is locked by a mechanical means, such as a peg, will be prohibited by the control software. They may also be prevented by electrical interlocks.

4.4.3.9.4 Power up safety interlocks

The default device power up states will prevent stage movement or other unsafe conditions at instrument power up.

4.4.3.9.5 Other safety interlocks

Other safety interlocks will be added as needed.

4.4.3.10 Control interlocks

4.4.3.10.1 Lights

All warning light sources will be interlocked with some system status keywords to prevent the warning lights from activating automatically during observing. The interlock method is TBD.

4.4.3.10.2 Gratings & Slit-masks

Collisions are possible with the gratings and slit-masks during insertion and removal of each from the light path. The control software for each of these stages will prevent moves that could result in collisions, either by waiting for the other stage to complete its movement or moving the other stage out of the way before performing a move. A priority algorithm will be designed to resolve questions regarding which stage moves first, once the mechanical design for the slit-mask insertion mechanism is completed (spring 1996).

4.4.3.10.3 Access doors & instrument rotation

Collisions may be possible between access doors and the carriage or floor when the instrument rotates. Instrument rotation will be prevented when these access doors are open.

4.4.3.10.4 Other control interlocks

Other control interlocks will be added as needed.

4.4.4 Exception handling requirements

Appropriate error signals and messages will be generated when the following exceptions are encountered.

4.4.4.1 Detection and handling of motion limits

All stages whose motion is constrained shall be protected by both software limits and by primary and secondary hardware limits. The primary hardware limit switch signal is passed to both the servo amplifier and the Galil motor controller. The secondary hardware limit switch cuts power to the motor windings directly and passes a limit signal back to the motor controller.

If the software and the Galil motor controller is functioning, the software limits should prevent reaching either the primary or secondary hardware limits. If a software limit fails and the primary hardware limit is reached by the stage, the servo amplifier cuts the power to the stage and prevents any further motion in the direction of the triggered limit. The servo amplifier still allows the stage to move away from the triggered limit. If the primary hardware limit fails to stop the stage before it reaches the secondary hardware limit, the secondary hardware limit cuts the connection between the motor windings and the servo amplifier output, while shorting the motor windings to effect an immediate stop. A push button override used with the manual stage controls allow motors to be backed out of secondary limits.

The software limits shall be set within the primary hardware limits and the primary hardware limits shall be set within the range of the secondary hardware limits. The secondary limits shall be set inside any physical limits (e.g., hard stops, cable limits, etc.).

Two software methods will be used for DC stage motion limit protection on the Galil controllers:

  1. 1) The Galil 1500 series controllers provide programmable forward and reverse limits. The controllers will not accept a command to position a stage beyond its defined limits. Programmable limit positions will be the primary software method of limit protection.
  2. 2) The Galil controllers provide automatic limit switch detection and handling via a user defined limit handling routine. This routine will be executed when a primary hardware limit switch input on the controller is triggered. A limit switch handler routine will be provided which stops execution of the motion control loop of a stage, and signals a limit error to external processes, when the stage triggers a primary hardware limit switch. Motion of the stage beyond the primary hardware limit is prevented by the servo amplifier as described above.

4.4.4.2 Detection and handling of AC power failure and recovery

The signals from the UPS's will be monitored for AC power failure, and low UPS battery power states. Upon AC power failure, stage motions will be inhibited to save power for the ion pumps (and CCD controller, if applicable). Observers will be notified upon AC power failure. Notification will include the approximate time the UPS batteries will last. Upon receiving a UPS battery low signal, the hatch will be closed and lamps shut off.

4.4.4.3 Detection and handling of loss of compressed air and recovery

Operation of air cylinder powered stages with inadequate air pressure can result in unpredictable actions.

When air pressure is detected below a threshold required for operation of the air cylinder controlled devices, the air cylinder device control software will prevent further operation of the devices.

4.4.4.4 Detection and handling of emergency stop signals, open enclosure hatches, etc.

The device control software will test for hardware and software emergency stop signals and prevent operation of the devices if the an e-stop signal is true. Some devices may require other signals set to specific states prior to operation. The requirements will be determined for each device as the hardware and electronics design for the devices are finished.

4.4.4.5 Detection and handling of faults

4.4.4.5.1 Servo runaway due to encoder signal failure

The Galil controllers execute a user defined start up routine when they power up or are reset either by a software reset command or via the toggling of their reset button. During this start up procedure appropriate commands will issued, as discussed below, to prevent servo runaway.

Runaway servo motors conditions will be prevented via the same method as used on all Galil controllers at Lick and on Galil controllers for HIRES at Keck. The off-on-error command will be used with a TBD maximum allowed servo position error value to trigger an error when a runaway condition would occur. The stage will move a maximum of the position error value before the error condition is triggered. The commands to set maximum position errors and the off-on-error trigger will be included in the controller start up routine.

When a position error occurs, a position error handling routine will be called. The routine will report the position error condition and perform any other necessary clean up duties TBD.

4.4.4.5.2 Power supply failures

DC stage control loops will exit reporting a voltage error if their motor power supply voltages are found outside acceptable ranges (ranges TBD). Power to the servo amplifiers will be inhibited.

Commands to devices controlled by external logic circuitry will fail if the external logic power supplies voltages (+-15v, +5v) are found to be outside acceptable voltage ranges (ranges TBD).

Monitoring tasks will provide notification of failure of any of these power supplies via KICS keyword data.

4.4.4.5.3 Communication link failures

Communication between processes will be via KICS keyword data passed via KTL read and write operations. In the past Lick and Keck systems numerous start up and shut down problems occurred when these processes failed to reconnect properly to their respective KTL services. DEIMOS processes must be able to start up and shutdown independently without causing side effects in the other tasks. Their start up and shutdown order must be completely independent of each other.

Dispatcher tasks will handle the interface between the KICS keyword interface and the motor controllers. The serial I/O interface between dispatcher tasks and their motor controllers must be able detect the disconnection and reconnection of the serial lines and handle updating data at either end of the serial connection to properly reflect the state of the system upon reconnection.

4.4.4.5.4 Motor connection failures

A test method will be developed to check the motor controller wiring to the motor and encoders before and during motor moves.

4.4.4.5.5 Coolant failures

Coolant and other external environmental inputs will be monitored. If the coolant flow drops too low or the coolant temperature gets too high, temperature control tasks will provide notification of cooling failure. The temperature control tasks will minimize temperature increase inside the enclosures, including powering down electronic systems, if necessary.

4.4.5 Reliability requirements

4.4.5.1 Process independence

Software processes will be independent of each other and all hardware external to their own server. Losing a process or external device on another system will not compromise local processes. They will continue to run and will re-establish communication with failed processes or hardware when the failed components return to service.

Processes and hardware should be able to start in any order.

4.4.5.2 Process sub-system independence

Internal subsystems within individual processes will be made as independent of each other as possible. Failure of one subsystem, such as logging, within a single process will not compromise the ability of the process to complete its other tasks, such as control or monitoring of the instrument internal environment.

4.4.6 Maintenance requirements

4.4.6.1 Instrument control logging

A command audit trail will be provided to aid in debugging. The detail level of the logging will be keyword controlled with verbose and quiet modes. In the quiet mode only the instrument command requests and their results are logged. In the verbose mode all commands sent to the motor controllers and their results are logged. The log file name(s) will be keyword controlled.

Instrument status messages will be broadcast. User interfaces and other processes are free to use these broadcast log messages as they wish. The level of detail and content of the message are TBD. They will not contain the same detail as the messages sent to the log file(s).

Logging failures will not inhibit instrument control in any way.

4.4.6.2 Diagnostics

Diagnostic routines come in two flavors: independent routines to easily and repetitively test hardware and electronics, and routines executed during the start up of process to check the function of associated hardware, electronics and software.

Diagnostic routines will individually test the functionality of the following:

  1. Cabling
  2. Digital & analog ports
  3. Switches (limits, fiducials)
  4. Encoders
  5. Stage sub-systems
  6. Other device sub-systems (covers, slides)
  7. Connections/communications with other processes
Other diagnostics are TBD.

A diagnostic test suite for complete system validation will be provided.

4.4.6.3 Release/version control

Release/version control of software source code, libraries, executables, makefiles and documentation will be provided along with version control, archiving, and an audit trail of the instrument control system configuration data.

4.4.6.3.1 Software, firmware & configuration upgrades

Software and firmware will be maintained via rcs or an rcs based system such as CVS. The make facility will be used to create libraries, executables and control release versions. The make facility may also be used to down load firmware where possible.

Configuration changes will be handled via a interface that performs data entry error checking and automatically archives configuration versions. An easy to follow audit trail will be provided to track changes. Previous configuration versions will be able to be recalled and reinstalled within moments.

4.4.6.3.2 Test vs. release versions

A mechanism will be defined for maintaining separate test and release versions of software for the DEIMOS instrument control system. The version information will be obvious with some means of warning users that a test version is running. A test version of the software will not be able to be permanently installed. A reboot or normal restart of a process will always use the current release version of the software, not a more recent test version.

An install mechanism in the software makefiles will make a release software version from a test version and install the release.

The above description applies to software and firmware. Configuration versions will be maintained in a similar manner by a TBD mechanism.

4.4.6.4 Portable code

Modular subsystems will be used to minimize software changes if a hardware change occurs. The software components will be written in C using standard Unix libraries to allow easy transport of the software between different brands of hardware. The make and rcs facilities will be used to aid porting the code and to support multiple hardware architectures (e.g. sparc and DEC-Alpha) and operating systems (e.g. solaris and Digital Unix). The firmware will use the 3rd generation Galil controller command language. This code should be portable to other 3rd generation controllers.

4.4.6.5 Documentation

Documentation falls into two categories: the software manuals and the documentation within the source code and related data files.

Online web pages will be used for the preliminary and critical design review documents, and the final Programmer's and user's manuals.

4.4.6.5.1 Manuals

The programmer's manual will be written as the software is developed to help keep a record of the software development and design changes. It will also provide an up-to-date document for reviewing software subsystems between major review dates.

4.4.6.5.2 Code documentation

Function and module headers will be similar to MOS headers. These will be completed before starting the actual code within the module or function. The headers will contain wflman constructs to allow constructing LaTex documents from the headers. See header examples in appendix 4.10.1.

In-line code documentation will define all variables used in the function and outline the purpose of individual sections of the function. The information in the function description should be used whenever possible.

4.4.6.6 Test suite

A suite of validation tests will be produced. The test suite will verify the correct functionality of the instrument hardware and the instrument control system software.

4.5 Preliminary design

4.5.1 KTL interface & keywords

Most interprocess communications takes place via KTL messages. The KTL messages handle requests, responses and broadcasts. The underlying protocol is TBD. It will most likely be the MUSIC system used by the Keck I instruments and telescope drive and control system or it may be an EPICS channel access mechanism.

4.5.2 Dispatcher software

Dispatcher processes handle communication between user interface processes such as xdeimos, show, modify and xshow and the Galil motor controllers. There is one dispatcher process per motor controller. The dispatcher processes receive KTL messages from the user interface processes, process requests for data or state changes, and pass the requests on to the motor controllers as motor controller commands. Simple commands and requests are passed to the motor controllers. Responses to the requests are returned back via the same path. Broadcast messages are also created by the dispatchers as states of the motor controllers and their devices change. The dispatchers communicate to their associated motor controller via a socket connection to a serial port on one of the terminal servers. All serial I/O handshaking is handled by the terminal server, not the dispatcher software.

Dispatcher processes are also likely for piezo controller communications. The piezo dispatchers will be based on the Galil controller dispatchers, since they will use a common interface to the terminal server ports and have similar needs in other interface and control requirements. The major difference between the piezo and Galil dispatchers will be the command language interface to the controllers.

If communication is possible with the UPS's via a serial port, the UPS's will also have a dispatcher designed for I/O with them.

4.5.5 Motor controller software

Firmware on the motor controller consists of Galil command routines burned into program memory and burned in servo parameters and variables. Program memory on the Galil controllers is limited to 1000 program lines up to 80 characters long.

The dispatcher DEIMOS control routines will be simpler in many ways than the original routines required by MOS. More work will be handled by routines in firmware on the controllers than was possible with MOS, since MOS used an older version of Galil motor controller which had less memory. The firmware on DEIMOS motor controllers will include the following:

  1. Routines to handle manual device control signals from a portable manual control box. The control box can select a stage and drive the stage by push button control to either pre-defined positions, or in a continuous motion while the "go" control button is held down.
  2. Low-level motion routines for homing, and servoing stages. Routines will be resident for stages using one motor encoder for position feedback, and stages using a second encoder for more accurate position feed back directly from the stage. The dual-encoder servo loop
  3. Any low level monitor tasks that would profit by being added to the control code on the motor controllers, and will fit on the controllers.
  4. Any diagnostics that would profit by being present in the motor controller firmware as space and priority allows.
Most of the motor controller software has been in use in more primative forms on MOS at Lick. The dual-encoder servo control loop has been particularly successful in controlling grating tilts.

4.6 Inherited software/prototypes/tools

There is considerable inheritance from Keck I and from MOS at Lick including the KTL interface from Keck I, parts of the dispatcher and some motor controller firmware from MOS, and experience with common maintenance tools from each project.

Many of the DEIMOS instrument control system software needs are identical to those of the Lick PFCAM or Keck II ESI. Both of these instruments will be completed well in advance of DEIMOS providing platforms to develop and test the common software sub-systems. Software effort on these common systems will be shared between the instruments.

4.6.1 KTL interface/MUSIC

The KTL interface has been in use at Keck I since before the commissioning of HIRES and LRIS. If the MUSIC message protocol is used in the KTL interface of DEIMOS, the KTL interface in DEIMOS would be identical to the interfaces in HIRES, LRIS, MOS, PFCAM, and ESI.

The existing KTL interface for Keck I instruments can be used unchanged with the DEIMOS instrument control software. This saves considerable effort in developing an interprocess communication library. The only new code required by DEIMOS is a small number of message handler routines and two unique tables identifying the DEIMOS servers and DEIMOS messages.

If MUSIC is used as the KTL interface message protocol, the MUSIC message handler in the MOS dispatcher can be used directly in the DEIMOS dispatchers; PFCAM and ESI are expected to use the same message handler.

4.6.2 Dispatcher

The dispatcher software for PFCAM, ESI and DEIMOS is expected to be identical or nearly identical. The dispatcher software can be easily configured to satisfy the needs of these independent instruments.

The Lick PFCAM and Keck ESI will both be completed before DEIMOS. Only a few additions to the dispatcher code existing at the time of their completion are expected to be needed by DEIMOS. The new code would handle the needs of devices and control requirements unique to DEIMOS. Common routines would exist for servo control, solenoid control, low-level digital control and low-level analog control. Some monitor control loops, such those that control temperature will already be developed for ESI.

The PFCAM, ESI and DEIMOS dispatcher will be built from the existing MOS dispatcher.

4.6.3 Motor controller firmware

Like the dispatcher software, the firmware in DEIMOS, PFCAM, and ESI are expected to be nearly identical. The differences would handled by configuration data, except in cases of unique device requirements for DEIMOS.

The PFCAM, ESI and DEIMOS motor controller firmware will be built from the existing MOS firmware.

4.6.4 Maintenance tools (makefiles, etc.)

Considerable experience has been gained in the use of the make facility and sccs, and other release control methods and mechanisms during the development of HIRES, MOS and common components of KICS. While not identical, experience with sccs should transfer to rcs.

4.6.5 Stage test rig

A test rig simulating a simple servo stage with a solenoid controlled cover and solenoid controlled slide was developed for testing of the HIRES rotator software in Santa Cruz. It will be used to test ESI, PFCAM and DEIMOS control algorithms before their hardware is available. Other test rigs may be produced, if needed.

4.7 Additional resources needed to complete software

  1. Hardware and electronics to test final control loops and configurations. This may be in the form of more complex test rigs than the simple one mentioned in section 6.5 above, stages for PFCAM or ESI, or the actual DEIMOS stages.
  2. EPICS/CA based KTL libraries and required utilities (including EPICS libraries, etc.), if KTL message protocol is switched from MUSIC to EPICS/CA.

4.8 Dependencies with other components

The KTL command interface for DEIMOS depends on the underlying message protocol. If the message protocol is switched from MUSIC to EPICS/CA, additional time will be needed to develop a new dispatcher interface. If some common message templates are provided by CARA the additional time would be smaller.

The database systems for the DEIMOS instrument control system and the other DEIMOS and KECK II systems are interrelated. The relationship is minimally defined at this time.

4.9 Outstanding issues

  1. Definition of the remaining hardware requirements, including safety interlocks and mechanical stage parameters. In particular, requirements for the specialized needs of the instrument rotator and other unique devices.
  2. Who does xdeimos? A collaborative effort between Lick and CARA is anticipated, but a detailed division of effort hasn't yet been agreed on.
  3. EPICS or MUSIC messages for Keck II KTL libraries? Affects message format of KTL interface.
  4. Database requirements.

4.10 Appendices

4.10.1 Module & function headers/templates

The following headers do not include any wflman processing commands. wflman commands will be added to them before they are used.

4.10.1.1 Include module header

/*********************************************************************
 Module:	xxx.h

 ---------------------------------------------------------------------
 Revisions:

 ---------------------------------------------------------------------
 Description:

 *********************************************************************/

#ifndef __XXX_DEF__

#define __XXX_DEF__

/*********************************************************************/
/* Includes required */
/*********************************************************************/

/* System */

/* KTL/MUSIC */

/* Instrument */

/* Engineering */

/*********************************************************************/
/* Constants and Macros */
/*********************************************************************/


/*********************************************************************/
/* Structure and special typedefs */
/*********************************************************************/


/*********************************************************************/
/* Prototypes */
/*********************************************************************/


#endif /* __XXX_DEF__ */

4.10.1.2 Source module header

/*********************************************************************
 Module:	xxx.c

 ---------------------------------------------------------------------
 Revisions:

 ---------------------------------------------------------------------
 Description:

 *********************************************************************/

/*********************************************************************/
/* Includes */
/*********************************************************************/

/* System */

/* KTL/MUSIC */

/* Instrument */

/* Engineering */

/*********************************************************************/
/* Constants and Macros */
/*********************************************************************/


/*********************************************************************/
/* Structure and special typedefs */
/*********************************************************************/


/*********************************************************************/
/* Prototypes */
/*********************************************************************/


/*********************************************************************/
/* Globals */
/*********************************************************************/

4.10.1.3 Function header

/*********************************************************************
 Function:	

 ---------------------------------------------------------------------
 Revisions:

 ---------------------------------------------------------------------
 Author:	Dean Tucker, Lick Observatory

 Date:		

 Abstract:

 Assumes:

 Description:

 ---------------------------------------------------------------------
 Globals:

	type	name		description

 Parameters:

	type	name		description

 Returns:

	NOERROR (0)		Function succeeded.
	Errno < 0		Function failed, error number.

 *********************************************************************/