Difference between revisions of "GlueX Containers Meeting, February 16, 2018"

From GlueXWiki
Jump to: navigation, search
(expand description of OSG deployment of singularity containers)
 
Line 15: Line 15:
 
* We will continue to pursue the scheme whereby the container has only system files (mainly those available from the target Linux distribution) and the GlueX software stack is built in an external directory that is maintained separately from the container. This is how things have been done on the OSG for some time now. The next step for the OSG is to update Oasis with the latest software versions. Mark has a candidate build for this.
 
* We will continue to pursue the scheme whereby the container has only system files (mainly those available from the target Linux distribution) and the GlueX software stack is built in an external directory that is maintained separately from the container. This is how things have been done on the OSG for some time now. The next step for the OSG is to update Oasis with the latest software versions. Mark has a candidate build for this.
 
* Docker containers are used as the starting point for both OSG and NERSC efforts. Each container can be represented by a GitHub repository containing only a few configuration files. We would like to have a few such standard containers represented under the Jefferson Lab organization on GitHub. Mark will clone one of Richard's repositories as a start.
 
* Docker containers are used as the starting point for both OSG and NERSC efforts. Each container can be represented by a GitHub repository containing only a few configuration files. We would like to have a few such standard containers represented under the Jefferson Lab organization on GitHub. Mark will clone one of Richard's repositories as a start.
 +
* Richard described the OSG system for generating and deploying Singularity containers for use on compute nodes. They maintain a [https://github.com/opensciencegrid/cvmfs-singularity-sync/blob/master/docker_images.txt list on GitHub]. These containers are available at run-time on all Singularity-enabled sites via CVMFS (on a partition separate from Oasis). Each entry on the list is the name of a GitHub repository. Each of these repositories is suitable for generating a Docker (not Singularity) container via a [https://docs.docker.com/docker-hub/github/ standard mechanism]. The OSG will create the Docker container and convert it to a Singularity container and put that up on the CVMFS partition. Richard showed an [https://github.com/rjones30/Gluexprod example of a GitHub repository] corresponding to a Singularity container available on the OSG.
 
* Mark will report on some of this discussion at the upcoming collaboration meeting.
 
* Mark will report on some of this discussion at the upcoming collaboration meeting.
 
* Next meeting: we will meet again in two weeks, same time.
 
* Next meeting: we will meet again in two weeks, same time.

Latest revision as of 10:57, 21 February 2018

GlueX Containers Meeting
Friday, February 16, 2018
11:30 am EST
JLab: CEBAF Center A110
BlueJeans: 968 592 007

Notes

Present: Richard Jones (UConn); Thomas Britton, Mark Ito, David Lawrence (JLab), Zisis Papandreou (Regina)

  • Oasis summary (Richard)
    • Richard went through the current directory structure on Oasis
  • Notes on recent work (Mark)
    • Mark reviewed recent work building and using Singularity containers.
  • We will continue to pursue the scheme whereby the container has only system files (mainly those available from the target Linux distribution) and the GlueX software stack is built in an external directory that is maintained separately from the container. This is how things have been done on the OSG for some time now. The next step for the OSG is to update Oasis with the latest software versions. Mark has a candidate build for this.
  • Docker containers are used as the starting point for both OSG and NERSC efforts. Each container can be represented by a GitHub repository containing only a few configuration files. We would like to have a few such standard containers represented under the Jefferson Lab organization on GitHub. Mark will clone one of Richard's repositories as a start.
  • Richard described the OSG system for generating and deploying Singularity containers for use on compute nodes. They maintain a list on GitHub. These containers are available at run-time on all Singularity-enabled sites via CVMFS (on a partition separate from Oasis). Each entry on the list is the name of a GitHub repository. Each of these repositories is suitable for generating a Docker (not Singularity) container via a standard mechanism. The OSG will create the Docker container and convert it to a Singularity container and put that up on the CVMFS partition. Richard showed an example of a GitHub repository corresponding to a Singularity container available on the OSG.
  • Mark will report on some of this discussion at the upcoming collaboration meeting.
  • Next meeting: we will meet again in two weeks, same time.