Policies for Using Online Directories and Accounts
In this note we present policies concerning allocation of directories on the online file server, as well as guidelines for using the various accounts available on the online computer system. These policies are based long-term on experience from CLAS, experience from the Hall D offline software effort, GlueX-doc-1892 developed as part of the 12GEV planning process concerning this topic, and discussions with Serguei P and others.
The next section states the policies, subsequent sections present some history and rational for the policies.
- THIS IS A DRAFT OF VERSION 2. IT HAS NOT BEEN OFFICIALLY ADOPTED YET. SEE HISTORY OF THIS PAGE TO LOCATE VERSION 1
- CHANGES ARE MARKED TO INDICATE WHETHER THEY ARE ADDITIONS or
Version 1: These policies was officially adopted by the Hall D Online group on 4-Dec-2013. Present at the meeting were: Elliott W, Bryan M, Curtis M, Dave L, Mark I, Elton S, Graham H, Sean D, Justin S, Eugene C, Yi Q, Simon T, Hovanes E, Serguei P, Mark D, Beni Z, Dave A, Vanik K.
- Elliott Wolin, Dave Lawrence, Mark Dalton
Version 2: (modifications by David Lawrence pending)
- Add statements describing detector specific hdXXXops accounts
- Make explicit the mechanism used for maintaining login scripts for hdops
- Remove policy regarding /gluex/Subsystems (not being fully honored and difficult to police)
- Remove policy regarding use of standard scons build system (too many critical systems don't use this and will be difficult to convert)
- Revised statement regarding CODA installation to bring it up to date
- Remove language specifying use of COOL_HOME (it contradicts deployed system)
Policies for Use of Online Accounts and Directories
- Operators should work exclusively from the hdops account, which has write access only to selected logging and backup directories.
- the hdops account will be maintained to use a standard environment for all gluon machines via the following files, both of which are maintained in the code repository and are not writable by hdops:
- Operators should only use code and configuration files in production/release directories (/gluex/builds,
/gluex/cool~hdops/CDAQ, /gluex/coda, and some others).
- Releases can only be made from the hdsys account, which is the sole purpose of this account. Production/release directories are only writeable from the hdsys account.
- Only selected online system experts are allowed to install code via the hdsys account, contact the Online group if you need to have code installed.
- Developers should work from their own personal accounts or a detector-specific hdXXXops account (see below), NOT from the hdsys account.
Developers should use /gluex/Subsystems when code or configuration files must be shared (same as is done in /group/halld for the offline, use umask 002). This is NOT a production/release area so you can do whatever you want here.
- Detector groups may use detector system specfic accounts for operations that might otherwise interfere with standard operations. e.g. use hdstops for running a ST specific DAQ system during long down times
- All development efforts should use code management tools (currently the online effort uses svn, same repository as the offline effort).
Use the online scons-based online build system except in special cases (e.g. EPICS, which comes with its own build system, as does our offline software).
Goals for Account and Directory Policies
The single most important goal is to protect the production software deployment directories from accidental overwrite. Note that the worst case is not simply causing the system to hang or crash, rather it is to cause the system to appear to work but in fact not work properly. Many days of data taking could be lost if this happens.
Another important goal is to provide a suitable environment for developers. Such an environment should allow for rapid development, testing and installation of new code without compromising production code, and should also allow developers to easily collaborate.
Finally, for those of us who will be on-call for online systems, another goal is to minimize the number of calls we get at e.g. 3am. This can only be accomplished if there is very strict control over what gets released for production use.
General Remarks Concerning Directory and Account Strategy
Almost all directories of interest reside on the gluonfs1 file server in /gluex. This area is visible on all computers and ROCs in the online cluster, and can be seen from all accounts (but NOT from the JLab CUE system). It is critical to get the protections in this area correct so that code can be developed and tested by individuals (or groups thereof), installed by system installers (via the hdsys account), and used but not overwritten by the hdops account. Of course the hdops account will need to have write access to some directories for logging, backup and the like.
In analogy with the offline, the main code deployment/production/release directory is /gluex/builds, with many named builds appearing underneath. This area must be read-only to hdops, and writeable only from the hdsys installation account (same as for the gluex account in the offline system). Operators and developers must be able to always rely on finding working code in the build areas, with the exception of the devel build, which by its very nature is unstable. A small number of additional directories need to be treated like the build directory. These will hold production configuration information needed by CODA and other packages, and again must be writeable only by the hdsys account.
Note that development work should NOT be done in the directories described above. /gluex/Subsystems (similar to /group/halld/Subsystems) is available for developers to use in any way they wish, and production running should never access or rely on code or configuration files in this area. Developers should work from their own accounts and should not use the hdsys account (which exists solely for making releases into the production areas). Other areas developers can work in are /gluex/Users, /home/<your-username> and /group/halld-online (available to the CUE system), where the same remarks about not being used for production running apply.
Finally, /gluex/etc/hdonline.cshrc sets up the standard production environment, and is sourced by the hdops and hdsys accounts. Feel free to source this file to set up your own environment, but never modify this file (if needed, source this file first in your personal .cshrc file and override defaults there). Contact the Hall D Online group if you want something added to the production environment.
The CODA toolkit supplies the foundation for data acquisition in the hall and it is critical that a working version exists at all times. The CODA distributions in /gluex/coda will be primarily maintained by the CODA group. The most recent version will be used for CODA development as they often need a deployed version in the gluon cluster for debugging. The CODA group will maintain responsibility for backing up the files to the CODA repository. As new releases begin development, older releases will be kept in place in the /gluex/coda directory tree so that we may easily switch back to them.
Until now /gluex/coda contained a test version of CODA maintained by Dave Abbott that was mostly used the past year or so for his private testing in the counting house. Starting very soon /gluex/coda will contain stable releases (and an unstable devel version) of CODA maintained by the Hall D DAQ group and owned by the hdsys account (first stable version will be installed just as soon as the JLab DAQ group creates such a "stable" release). This area may not contain the latest bug fixes to CODA, but developers should be able to rely on finding a working version there. Updates to this area will be carefully controlled (via the hdsys account). Note that Dave Abbott's test version of CODA will move into /gluex/Subsystems.
The official production version of COOL_HOME resides in /gluex/cool and is owned by the hdsys account. Developers should not modify files in this area, and the usual rules concerning production code/configurations apply. Contact the Online group if you wish to add configurations or scripts to this official release area. Similarly, the official release area for readout lists is in /gluex/builds, and again contact the Hall D Online group if you want to install new readout lists for production use.