Difference between revisions of "GlueX Software Meeting, July 21, 2020"

From GlueXWiki
Jump to: navigation, search
(complete draft)
m (Compiler upgrade discussion)
 
(One intermediate revision by the same user not shown)
Line 26: Line 26:
 
# Review of [https://groups.google.com/forum/#!forum/gluex-software recent discussion on the GlueX Software Help List] (all)
 
# Review of [https://groups.google.com/forum/#!forum/gluex-software recent discussion on the GlueX Software Help List] (all)
 
# Action Item Review (all)
 
# Action Item Review (all)
 +
 +
== Minutes ==
 +
 +
Present: Alex Austregesilo, Thomas Britton, Sean Dobbs, Mark Ito (chair), Richard Jones, Naomi Jarvis, David Lawrence, Susan Schadmand, Simon Taylor, Nilanga Wickramaarachchi, Beni Zihlmann
 +
 +
There is [https://bluejeans.com/s/0rIPo9NbCrd/ a recording of his meeting] on the BlueJeans site. Use your JLab credentials to authenticate.
 +
 +
=== Announcements ===
 +
 +
[https://mailman.jlab.org/pipermail/halld-offline/2020-July/008264.html New version set, version_4.23.1.xml]. The latest version set came out last Wednesday.
 +
 +
=== Review of Minutes from the Last Software Meeting ===
 +
 +
We went over [[GlueX Software Meeting, July 7, 2020#Minutes|the minutes from the meeting on July 7]].
 +
 +
==== NERSC Status ====
 +
 +
David gave us the run-down on the preparations for the next reconstruction launch at NERSC.
 +
 +
* He has gone to a roster of plugins intermediate between the original 57, but more that the minimum used in REST production.
 +
* Igal Jaegle has looked at the latest round of monitoring histograms. They look good. Any missing plots were due to dropped plugin-ins.
 +
* Overall, the production system is ready to go.
 +
* David submitted 1,000 jobs recently. These represent a complete run. There were some SWIF2 issues that needed attention from Chris Larrieu.
 +
* The main outstanding issue is creating a complete set of fiducial times in the CCDB.
 +
** The slope of event time vs. the 250 MHz clock seems fine taking the nominal 250 MHz. The offsets need to be filled in.
 +
** Beam trip information is also missing from the CCDB. About 1 in 7 runs is missing presently.
 +
** There is an issue with SWIF2 where only the first of the ten jobs associated with a single raw data file succeeds. The following nine require re-submission. We could live with this, but not happily. Chris is working on this.
 +
 +
==== Developer-Friendly Container Build ====
 +
 +
Mark made small adjustments to the rsync of our container software to Oasis that allows building of halld_recon against Oasis. See [https://mailman.jlab.org/pipermail/halld-offline/2020-July/008257.html his recent email] for more details.
 +
 +
==== Corrupt CCDB SQLite Files ====
 +
 +
Mark changed the limit on the output file-size check when SQLite versions of the CCDB are produced. It is unlikely that the "Lost connection to MySQL server" will corrupt the file on Oasis anytime soon. If the file is not big enough, the previous version will not be replaced.
 +
 +
=== Report from the Last HDGeant4 Meeting ===
 +
 +
# We went over [[HDGeant4_Meeting, July 14, 2020#Minutes|the minutes from the HDGeant4 Meeting on July 7]]. Thomas reported that he has seen the missing normal-error that Richard reported at that meeting. It seems to only occur for certain runs (simulations of specific real runs), but not always on the same event.
 +
 +
=== Compiler upgrade discussion ===
 +
 +
Mark described the issues and possible paths forward for the problem of needing to adopt more advanced, non-default compiler in order to bring in recent versions of third-party-provided software, such as Geant4 and ROOT. See [https://docs.google.com/presentation/d/1e1UpDpI0zc4pUe-lUsnij_10kZ2ItKv6ILvRrChv7GQ/edit?usp=sharing his slides] for the details (three main slides, large font, no plots).
 +
 +
The proximate cause of the discussion is the possibility of upgrading to this year's version of Geant4, which requires GCC 4.9.3, more recent than the default 4.8.5 shipped with CentOS7. We decided on two concrete projects that move us in the right direction:
 +
 +
# Richard mentioned a new package, and has already written [[HOWTO_use_prebuilt_GlueX_software_from_any_linux_user_account_using_cvmfsexec|a HOWTO on cvmfsexec]], that will allow access via CVMFS to the Oasis share of our pre-built software stack from user space. This could greatly simplify the distribution of container-ready software.
 +
# Mark volunteered to build a container for CentOS8, which will use an advanced version of GCC natively. By using such a container, we are guaranteed that all system-supplied software is compatible with the new compiler.
 +
 +
Mark also showed a fourth slide with musings on how we might automate and improve tests of our software.
 +
 +
=== Review of recent issues and pull requests ===
 +
 +
We ran down [https://github.com/JeffersonLab/halld_recon/issues?q=is%3Aopen+is%3Aissue the list of halld_recon issues] without significant comment.
 +
 +
=== Review of recent discussion on the GlueX Software Help List ===
 +
 +
Naomi reminded us that if we see problems posting plots to the logbook, we should send a bug report to Mark Dalton with [https://groups.google.com/g/gluex-software/c/tDLG5qcStjA the info he has requested].

Latest revision as of 19:38, 21 July 2020

GlueX Software Meeting
Tuesday, July 21, 2020
3:00 pm EDT
BlueJeans: 968 592 007

Agenda

  1. Announcements
    1. New version set, version_4.23.1.xml
  2. Review of Minutes from the Last Software Meeting (all)
  3. Report from the Last HDGeant4 Meeting (all)
  4. Compiler upgrade discussion (all)
  5. Review of recent issues and pull requests:
    1. halld_recon
    2. halld_sim
    3. CCDB
    4. RCDB
  6. Review of recent discussion on the GlueX Software Help List (all)
  7. Action Item Review (all)

Minutes

Present: Alex Austregesilo, Thomas Britton, Sean Dobbs, Mark Ito (chair), Richard Jones, Naomi Jarvis, David Lawrence, Susan Schadmand, Simon Taylor, Nilanga Wickramaarachchi, Beni Zihlmann

There is a recording of his meeting on the BlueJeans site. Use your JLab credentials to authenticate.

Announcements

New version set, version_4.23.1.xml. The latest version set came out last Wednesday.

Review of Minutes from the Last Software Meeting

We went over the minutes from the meeting on July 7.

NERSC Status

David gave us the run-down on the preparations for the next reconstruction launch at NERSC.

  • He has gone to a roster of plugins intermediate between the original 57, but more that the minimum used in REST production.
  • Igal Jaegle has looked at the latest round of monitoring histograms. They look good. Any missing plots were due to dropped plugin-ins.
  • Overall, the production system is ready to go.
  • David submitted 1,000 jobs recently. These represent a complete run. There were some SWIF2 issues that needed attention from Chris Larrieu.
  • The main outstanding issue is creating a complete set of fiducial times in the CCDB.
    • The slope of event time vs. the 250 MHz clock seems fine taking the nominal 250 MHz. The offsets need to be filled in.
    • Beam trip information is also missing from the CCDB. About 1 in 7 runs is missing presently.
    • There is an issue with SWIF2 where only the first of the ten jobs associated with a single raw data file succeeds. The following nine require re-submission. We could live with this, but not happily. Chris is working on this.

Developer-Friendly Container Build

Mark made small adjustments to the rsync of our container software to Oasis that allows building of halld_recon against Oasis. See his recent email for more details.

Corrupt CCDB SQLite Files

Mark changed the limit on the output file-size check when SQLite versions of the CCDB are produced. It is unlikely that the "Lost connection to MySQL server" will corrupt the file on Oasis anytime soon. If the file is not big enough, the previous version will not be replaced.

Report from the Last HDGeant4 Meeting

  1. We went over the minutes from the HDGeant4 Meeting on July 7. Thomas reported that he has seen the missing normal-error that Richard reported at that meeting. It seems to only occur for certain runs (simulations of specific real runs), but not always on the same event.

Compiler upgrade discussion

Mark described the issues and possible paths forward for the problem of needing to adopt more advanced, non-default compiler in order to bring in recent versions of third-party-provided software, such as Geant4 and ROOT. See his slides for the details (three main slides, large font, no plots).

The proximate cause of the discussion is the possibility of upgrading to this year's version of Geant4, which requires GCC 4.9.3, more recent than the default 4.8.5 shipped with CentOS7. We decided on two concrete projects that move us in the right direction:

  1. Richard mentioned a new package, and has already written a HOWTO on cvmfsexec, that will allow access via CVMFS to the Oasis share of our pre-built software stack from user space. This could greatly simplify the distribution of container-ready software.
  2. Mark volunteered to build a container for CentOS8, which will use an advanced version of GCC natively. By using such a container, we are guaranteed that all system-supplied software is compatible with the new compiler.

Mark also showed a fourth slide with musings on how we might automate and improve tests of our software.

Review of recent issues and pull requests

We ran down the list of halld_recon issues without significant comment.

Review of recent discussion on the GlueX Software Help List

Naomi reminded us that if we see problems posting plots to the logbook, we should send a bug report to Mark Dalton with the info he has requested.