Updating Oasis for the GlueX VO

From GlueXWiki
Revision as of 19:33, 19 May 2021 by Marki (Talk | contribs) (a few more things)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

The cronjob that updates Oasis is run at 1 am, 7 am, 1 pm, and 7 pm every day by the gluex account on sandd1.jlab.org. The script is

 /home/gluex/bin/oasis_update.sh

The script performs an rsync from several areas on /group/halld at JLab to the central Oasis server maintained by the OSG. It uses a user account on that that server assigned to the GlueX VO which contains a public key allowing login from the gluex account on sandd1.jlab.org. The authorized_keys file on the OSG server identifies sandd1.jlab.org by IP number, not IP name.

The areas rsync'ed are:

Description Directory
dist directory /group/halld/www/halldweb/html/dist/*.sqlite
resources directory /group/halld/www/halldweb/html/resources/
build_scripts directory /group/halld/Software/build_scripts/
container build directory /group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/
container recent build directory /group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/recent/
halld_versions directory /group/halld/www/halldweb/html/halld_versions/
CentOS 8 container build directory /group/halld/Software/builds/Linux_CentOS8-x86_64-gcc8.3.1-cntr/

These areas on the group disk get rsync'ed to

/home/login/ouser.gluex/stage/group/halld

on the Oasis server. They appear under the Oasis CVMFS share as

 /cvmfs/oasis.opensciencegrid.org/gluex/group/halld

Note that some of these directories have "exclude" directives in their respective rsync commands; not every file at JLab will appear on Oasis.

After all directories have been rsync'ed, the osg-oasis-update command is executed on the Oasis server, but only if files have been transferred during the runs of rsync.