GlueX and Containers

From GlueXWiki
Revision as of 15:31, 10 November 2022 by Aaustreg (Talk | contribs) (Creating a Container for the OSG)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Terms to Describe Containers

The container world is not monolithic, so we have to define some terms.

Categories of Containers

The container type, installed software level, and /group/halld binding style (described below) are independent degrees of freedom when categorizing containers.

Container Types

There are three types: Docker, Singularity, and Shifter

For Singularity there are two sub-types: Sandbox and Singularity Image Format (SIF)

Installed Software Levels

There are three levels: raw, gluex-prereqs, and gluex-software

  • raw. This a vanilla OS direct (or nearly so) from the distributer, with only the basic, default set of packages.
  • gluex-prereqs-only. Starting from raw, packages needed to build and run the GlueX software stack are added. The GlueX software itself is not included and needs to be provided external to the container, usually with a "bind" directive.
  • gluex-with-software. Starting from gluex-prereqs, the GlueX software stack itself is built into the container.

Group Binding Styles

There are two styles for binding /group/halld: system-binds-group (SBG) and user-binds-group (UBG)

Distribution Systems

There are three methods we use for distributing containers: DockerHub, Web, CVMFS.

Container Use at Various Sites

Site Category (type/level/style) Distribution System
NERSC Shifter/gluex-prereqs-only/SBG DockerHub
OSG Singularity(Sandbox)/gluex-prereqs-only/SBG CVMFS
PSC Singularity(Sandbox)/gluex-prereqs-only/SBG CVMFS
Big Red (IU) Singularity(Sandbox)/gluex-prereqs-only/SBG simg image

At this writing, there is no standard container-use practice at JLab.

Creating a Singularity image for BigRed3 (IU)

1. module load singularity

2. singularity build gluex_prod_v1.simg docker:jeffersonlab/gluex_prod:v1

The simg file should be put in /group/halld/www/halldweb/html/dist

Creating a Container for the OSG

Let us assume that we want to make version 2 of the production container.

Steps 1-6 create the new Docker container on DockerHub.

1. clone the repo with the Dockerfile

git clone

2. modify the Dockerfile

cd gluex_docker_prod
vi Dockerfile

3. build the new Docker container

docker build -t gluex_prod:v2 .

4. login into DockerHub

docker login -u my_dockerhub_username

5. tag the new local container with a name on DockerHub

docker tag gluex_prod:v2 jeffersonlab/gluex_prod:v2

6. push the container to DockerHub

docker push jeffersonlab/gluex_prod:v2

Steps 7-9 record the changes to the Dockerfile back at the GitHub repository.

7. commit the new Dockerfile to your local git repository

git commit -a -m "this is version 2"

8. push to GitHub

git push

9. create a release on the GitHub site tagging this version as "v2" for documentation purposes

Steps 10-16 publish the new container as a Singularity container on CVMFS.

10. fork the OSG Singularity container list repository on GitHub () to your personal GitHub account. The OSG repository is opensciencegrid/cvmfs-singularity-sync.

11. clone your fork

cd /dir/to/contain/git/repo
git clone

12. edit the list of docker containers adding a line with "jeffersonlab/gluex_prod:v2"

cd cvmfs-singularity-sync
vi docker_images.txt

13. commit the change

git commit -a "adding new container, gluex_prod:v2"

14. push the change back to your fork

git push

15. create a pull request to the opensciencegrid parent repository from the mygithubusername/cvmfs-singularity-sync page on GitHub.

16. After the OSG folks merge the pull request, the new container should appear on CVMFS as /cvmfs/

Historical Note

This procedure is not the one employed prior to March 2022. Before that Docker supported automatic builds from Dockerfiles hosted at GitHub for free accounts and that feature was used.