Database Support for Keck Spectrographs: Master Database Copy


There are two copies of the metadata (in RDBMS) which support Lick-built spectrographs (and soon mask milling) at Keck. After lengthy efforts to make the two database servers peers which update each other based on local activity, we gave up and made the Keck summit database server (KSUMMIT) a slave copy of the Santa Cruz server (YAKUZA).

The mechanism by which the contents of YAKUZA are written onto KSUMMIT is somewhat complex (for the sake of robustness). It involves three hosts: oyabun (host for master database server YAKUZA); waiaha (host for Keck copy KSUMMIT); and penguin (the Lick CVS server).

OYABUN

# OYABUN crontab for sybase
#
30 10,11,12,13,14,15,16,6,10,14,18,19,20,21 * * * /u/sybase/MetaMaintLite
#
0 6 * * * /u/sybase/MetaMaint
Every hour or so during the working day, oyabun checks for updates from waiaha only for the tables Mask and MaskBlu ( details here). Once a day at 6AM California time, it extracts all the metabase tables to flat files and checks them in to CVS. NOTE: due to the critical/confidential nature of some of these data we do not use the regular "cvs for source code" CVSROOT, but a different CVSROOT with more limited access.)

WAIAHA

# check for updates from/for Santa Cruz
#
# extract Mask and MaskBlu, update 'em and commit
0 8,9,10,12,13,14,15,16,17,18 * * * /u/de/MetaMaint
#
# do MetaMaint, then update all and stomp metabase with new data,
# then count the spoons
0 11 * * * /u/de/MetaStomp
#
####################################################
# NOTA BENE (this is very imoortant!)
# if you change this timing then you MUST edit the
# procedure checkLockout in the dremel Tlib.
# see cvs/kroot/kss/deimos/dremel/Tlib/checkLockout
# on keamano.
####################################################
Every hour or so throughout the working day, waiaha checks for local mask milling/scanning activity and does a cvs commit to record any that it finds. (Details here.) At 11AM Hawai'i time it does a cvs update of all the extracted flat files, then copies them to a working directory. In that directory are scripts which will destroy the metabase database and re-create it from the updated flat files.

This major update is called "Stomp" for a reason. The destruction and recreation of all the metabase tables is (of course) disruptive and must not be allowed to derange dremeling or observing activities; this presents a minor problem. Apps (and users) in HI have to know to avoid database access attempts at or near the Witching Hour (currently 11 AM).

What really happens during MetaStomp? Click to see for yourself. You can also read the source for MetaMaint Important Note: As briefly mentioned above, because of the deranged state of the database during the Stomp, database-dependent applications will not start up properly, or (in the case of dremel) fail in some of their operations during the xerox. To work around this I wrote a "checkLockout" function which is hardwired to the times of the xerox attempts on waiaha. Therefore this piece of code (used by dremel and possibly other apps) is wired to the crontab on waiaha, and the crontab should not be changed unless this code is altered as well. It would be better to have some client/serverish way to determine that xerox is in progress (a KTL keyword?) but not being sure how to approach an implementation for multiple instruments, multiple hosts etc., I chickened out and just fixed it for dremel (DEIMOS).

Important files and directories:

OYABUN
~sybase/cvs/metabase working directory where all the action takes place
~sybase/cvs/metabase/YAKUZA code specific to oyabun/YAKUZA stored here
WAIAHA
~/de/cvs/metabase working directory where the updating and committing happen
~/de/cvs/metabase/WAIAHA waiaha-specific code lives here
~/de/data/xerox directory where the Stomp work is done.
~/de/UpdateMe.log Log file from UpdateMe invoked by MetaStomp cron job.
~/de/SanityCount.log Logfile from "CountTheSpoons" invoked by MetaStomp.
~/de/LastUpdate.waiaha File containing date of last update operation.
~/de/data/xerox/Update.out Output chitchat from the database rebuilding operation.


de@ucolick.org
webmaster@ucolick.org
De Clarke
UCO/Lick Observatory
University of California
Santa Cruz, CA 95064
Tel +1 408 459 2630
Fax +1 408 454 9863