Difference between revisions of "Automatic Builds of GlueX Software"

From GlueXWiki
Jump to: navigation, search
Line 1: Line 1:
 
Every night a complete build of the source directory is done on several platforms at the lab.
 
Every night a complete build of the source directory is done on several platforms at the lab.
  
* The builds are located in the directory /scratch/gluex/halld_builds on the JLab CUE. Everyday a new directory, named with the current date, is created in this directory, for example, /scratch/gluex/halld_builds/2010-10-03 .
+
* The builds are located in the directory /scratch/gluex/halld_builds on the JLab CUE. Everyday a new directory, named by date, is created in this directory, for example, /scratch/gluex/halld_builds/2010-10-03 .
 
* Both the hdds and sim-recon packages are built. Latest versions of each are used.
 
* Both the hdds and sim-recon packages are built. Latest versions of each are used.
 
* The script run is /home/gluex/bin/nightly.sh. It is scheduled as a cron job for the [[GlueX shared account on the JLab CUE|"gluex" account]] on ifarml3.jlab.org. The job runs at 3:15 am daily.
 
* The script run is /home/gluex/bin/nightly.sh. It is scheduled as a cron job for the [[GlueX shared account on the JLab CUE|"gluex" account]] on ifarml3.jlab.org. The job runs at 3:15 am daily.
* One cron job runs the builds on the various platforms, serially. The current platforms are:
+
* One cron job on ifarml3 runs the builds on the various platforms, as username gluex, serially. The current platforms are:
 
** ifarml5.jlab.org (Fedora 8)
 
** ifarml5.jlab.org (Fedora 8)
 
** jlabl3.jlab.org (RedHat Enterprise Linux 5)
 
** jlabl3.jlab.org (RedHat Enterprise Linux 5)
Line 16: Line 16:
 
  setenv HALLD_HOME /scratch/gluex/halld_builds/2010-10-03/sim-recon
 
  setenv HALLD_HOME /scratch/gluex/halld_builds/2010-10-03/sim-recon
 
  source /group/halld/Software/scripts/build_scripts/gluex_env_jlab.csh
 
  source /group/halld/Software/scripts/build_scripts/gluex_env_jlab.csh
 +
 +
=Note on ssh scheme=
 +
 +
As mentioned above although the cron job runs on ifarml3, the builds are all actually done on other nodes. To do this without having to supply a passphrase the cron job uses a special ssh private/public key pair that only allows the target script on the remote node (and no other command) to run only if the ssh connection comes from ifarml3 and if the target account holds the appropriate public key. This key has no passphrase associated with it <ref>If it did, then that passphrase would have to somehow be incorporated into scripts, a practice which is generally discouraged for security reasons.</ref> and thus can be used from a cron job. The remote target script is only mentioned in the authorized_keys file of the remote account. Only the ssh invocation is seen in the script (/home/gluex/bin/nightly.sh) on the local host (ifarml3).
 +
 +
Note that this special key pair is not the one used for standard ssh connections to gluex account on the CUE. The standard pair has a passphrase. This passphrase-less technique is described in a [http://www.linuxjournal.com/article/8257 2005 Linux Journal article].
 +
 +
----
 +
 +
<references/>

Revision as of 16:24, 5 August 2010

Every night a complete build of the source directory is done on several platforms at the lab.

  • The builds are located in the directory /scratch/gluex/halld_builds on the JLab CUE. Everyday a new directory, named by date, is created in this directory, for example, /scratch/gluex/halld_builds/2010-10-03 .
  • Both the hdds and sim-recon packages are built. Latest versions of each are used.
  • The script run is /home/gluex/bin/nightly.sh. It is scheduled as a cron job for the "gluex" account on ifarml3.jlab.org. The job runs at 3:15 am daily.
  • One cron job on ifarml3 runs the builds on the various platforms, as username gluex, serially. The current platforms are:
    • ifarml5.jlab.org (Fedora 8)
    • jlabl3.jlab.org (RedHat Enterprise Linux 5)
    • ifarml6.jlab.org (CentOS 5).
  • Both non-debug and debug builds are done on all platforms.
  • Log files of the builds are created in the daily directory, for example, /scratch/gluex/halld_builds/2010-02-28/halld_jlabl3.log . For a particular platform, the output of both non-debug and debug builds are in a single log file.
  • On Monday and Thursday mornings a summary of errors and warnings is sent to the halld-offline@jlab.org email list. The script to do this is /home/gluex/bin/nightly_report.sh and is run as a cron job (gluex, ifarml3).
  • To use one of the nightly builds, you can set up the environment as follows (assume you want to use October 3, 2010):
setenv HDDS_HOME /scratch/gluex/halld_builds/2010-10-03/hdds
setenv HALLD_HOME /scratch/gluex/halld_builds/2010-10-03/sim-recon
source /group/halld/Software/scripts/build_scripts/gluex_env_jlab.csh

Note on ssh scheme

As mentioned above although the cron job runs on ifarml3, the builds are all actually done on other nodes. To do this without having to supply a passphrase the cron job uses a special ssh private/public key pair that only allows the target script on the remote node (and no other command) to run only if the ssh connection comes from ifarml3 and if the target account holds the appropriate public key. This key has no passphrase associated with it [1] and thus can be used from a cron job. The remote target script is only mentioned in the authorized_keys file of the remote account. Only the ssh invocation is seen in the script (/home/gluex/bin/nightly.sh) on the local host (ifarml3).

Note that this special key pair is not the one used for standard ssh connections to gluex account on the CUE. The standard pair has a passphrase. This passphrase-less technique is described in a 2005 Linux Journal article.


  1. If it did, then that passphrase would have to somehow be incorporated into scripts, a practice which is generally discouraged for security reasons.