Difference between revisions of "GlueX Software Meeting, July 21, 2020"

From GlueXWiki
Jump to: navigation, search
(complete draft)
Line 1: Line 1:
GlueX Software Meeting<br>
GlueX Software Meeting<br>
Tuesday, July 7, 2020<br>
Tuesday, July 21, 2020<br>
3:00 pm EDT<br>
3:00 pm EDT<br>
BlueJeans: [http://www.bluejeans.com/968592007 968 592 007]
BlueJeans: [http://www.bluejeans.com/968592007 968 592 007]
Line 7: Line 7:
# Announcements
# Announcements
## [https://mailman.jlab.org/pipermail/halld-offline/2020-June/004104.html New version set with upgrade to Geant4: version_4.21.3.xml] (Mark I.)
## [https://mailman.jlab.org/pipermail/halld-offline/2020-July/008264.html New version set, version_4.23.1.xml]
## [https://mailman.jlab.org/pipermail/halld-offline/2020-July/004107.html multi-threaded hdgeant4 now working] (Richard)
# Review of [[GlueX Software Meeting, July 7, 2020#Minutes|Minutes from the Last Software Meeting]] (all)
## [https://mailman.jlab.org/pipermail/halld-offline/2020-July/004108.html New release of Build Scripts: version 1.58] (Mark)
# [[HDGeant4_Meeting, July 14, 2020#Minutes|Report from the Last HDGeant4 Meeting]] (all)
## [https://mailman.jlab.org/pipermail/halld-offline/2020-July/004109.html New version set: version_4.22.0.xml] (Mark)
## [https://halldweb.jlab.org/talks/2020/crc_md5.pdf Checksum changes for files written to tape] (Mark)
# Review of [[GlueX Software Meeting, June 9, 2020#Minutes|Minutes from the Last Software Meeting]] (all)
# Report from Recent HDGeant4 Meetings (all)
## [[HDGeant4_Meeting, June 16, 2020#Minutes|June 16 Meeting]]
## [[HDGeant4_Meeting, June 30, 2020#Minutes|June 30 Meeting]]
#  [https://docs.google.com/presentation/d/1bmy-IYamSmEtv4HsOeXHgceD66vVPstLZXymLNf4tE0/edit?usp=sharing Report from the SciComp Meeting] (Mark)
# [https://docs.google.com/presentation/d/1BGYUvKztfzSGDGwcfTrs6DoS_EUUvaeTHReeYFAEKvc/edit?usp=sharing NERSC status](David)
# OSG Jobs and mcsmear (Thomas)
# [https://docs.google.com/presentation/d/1e1UpDpI0zc4pUe-lUsnij_10kZ2ItKv6ILvRrChv7GQ/edit?usp=sharing Compiler upgrade discussion] (all)
# [https://docs.google.com/presentation/d/1e1UpDpI0zc4pUe-lUsnij_10kZ2ItKv6ILvRrChv7GQ/edit?usp=sharing Compiler upgrade discussion] (all)
# Review of recent issues and pull requests:
# Review of recent issues and pull requests:
Line 35: Line 26:
# Review of [https://groups.google.com/forum/#!forum/gluex-software recent discussion on the GlueX Software Help List] (all)
# Review of [https://groups.google.com/forum/#!forum/gluex-software recent discussion on the GlueX Software Help List] (all)
# Action Item Review (all)
# Action Item Review (all)
== Minutes ==
Present: Alex Austregesilo, Thomas Britton, Sean Dobbs, Mark Ito (chair), Igal Jaegle, Richard Jones, Naomi Jarvis, David Lawrence, Keigo Mizutani, Susan Schadmand, Simon Taylor, Nilanga Wickramaarachchi, Beni Zihlmann
There is [https://bluejeans.com/s/MR_xjcUQ0sR/ a recording of his meeting] on the BlueJeans site. Use your JLab credentials to authenticate.
=== Announcements ===
# [https://mailman.jlab.org/pipermail/halld-offline/2020-June/004104.html New version set with upgrade to Geant4: version_4.21.3.xml] Ready for testers of an updated version of Geant4.
# [https://mailman.jlab.org/pipermail/halld-offline/2020-July/004107.html multi-threaded hdgeant4 now working] Richard implemented patches to Geant4 that fixed issues that prevented multi-threaded running from giving sensible results.
#* To address thread safety in the HDGeant4 code proper, he made a change to provide each thread with its own copy of the magnetic field map at the cost of 300 MB of memory per additional thread. This was done out of an abundance of caution; if not necessary he will back out the change to recover the memory used.
#* Richard proposed changing the CCDB to so that the magnetic field is no longer suppressed inside selected volumes, in particular in the BCAL. Other volumes that would get changed are the FCAL, TOF, DIRC, and CCAL where the effect was not manifest because the field is so much lower there. He discovered that the suppression resulted field discontinuities that caused crashes in particle propagation reported by Naomi. The suppression had originally been introduced to speed up shower development in GEANT, but Geant4 seems not to be affected by having the field on. We endorsed the proposal.
# [https://mailman.jlab.org/pipermail/halld-offline/2020-July/004108.html New release of Build Scripts: version 1.58] This new release will accommodate our current default as well as subsequent versions of Geant4.
# [https://mailman.jlab.org/pipermail/halld-offline/2020-July/004109.html New version set: version_4.22.0.xml]. This version set incorporates the fix to multi-threaded running of HDGeant4 mentioned above.
# [https://halldweb.jlab.org/talks/2020/crc_md5.pdf Checksum changes for files written to tape] The Computer Center is dropping MD5 checksum and will rely on CRC32 sums for tape data validation.
=== Review of Minutes from the Last Software Meeting ===
We went over [[GlueX Software Meeting, June 9, 2020#Minutes|the minutes from June 9]].
==== New Release of JANA: version 0.8.2 ====
David has addressed the request for suppressing of warnings when geometry paths are requested but not present. This is often a normal situation, e.g., when probing the geometry to see how the data should be analyzed. His fix is in a new release of JANA, version 0.8.2.
==== Jana 2.0 ====
Nathan Brei continues work on porting halld_recon to use JANA 2.0. We will get a report at a future meeting on this new major release.
=== Report from Recent HDGeant4 Meetings ===
We went over [[HDGeant4_Meeting, June 16, 2020#Minutes|the minutes from the June 16 meeting]] and [[HDGeant4_Meeting, June 30, 2020#Minutes|those from the June 30 meeting]] without much comment.
=== Report from the SciComp Meeting ===
Please see [https://docs.google.com/presentation/d/1bmy-IYamSmEtv4HsOeXHgceD66vVPstLZXymLNf4tE0/edit?usp=sharing Mark's slides] for Scientific Computing news from the Computer Center.
=== NERSC status ===
David brought us up-to-date on processing at NERSC. Please see [https://docs.google.com/presentation/d/1BGYUvKztfzSGDGwcfTrs6DoS_EUUvaeTHReeYFAEKvc/edit?usp=sharing his slides] for all of the details. Some broad points:
* We have moved to two-hour jobs rather than the 6 to 8 hour jobs run in past campaigns. This gives us much better access to the backfill mechanism at NERSC.
* The move involved significant development to our workflow to (a) process only selected parts of our 20 GB raw data files in a single job and (b) recombine the resulting output files to correspond to the original 20 GB file.
* One significant challenge has been to run monitoring launches, with their 57 plugins, to give complete ROOT output files for each selection of the raw data file.
* David pointed out that the current version of the Oasis image of our software does not support development (i.e., building new versions of software), only running. He has put up a Docker container that remedies this. Richard has also run into this issue; he has put the missing pieces in an undisclosed location on Oasis.
** Mark remarked that having a developer-friendly version of Oasis would involve very little work, only real estate on Oasis. He will look into this.
* Alex suggested that one way forward is to abandon processing of monitoring launches at NERSC and concentrate on REST file production which uses a smaller set of plugins, plugins that have had better past records of success.
=== OSG Jobs and mcsmear ===
Today, Thomas noticed that many jobs were crashing in mcsmear when accessing the SQLite form of the CCDB on the OSG. Two issues:
# What is wrong with the SQLite file?
# Why is it that mcsmear exits with status code = 0 (i.e., success) after bombing?
We were only able to address the first issue. David noticed that today's SQLite file was a bit smaller than usual, making it suspect. Mark promised to recreate the SQLite file and ship it out via Oasis.
[Added in press: (a) Mark [https://mailman.jlab.org/pipermail/halld-offline/2020-July/004115.html made good on his promise] and regenerated the SQLite file. (b) David had reported this issue via email to Mark early this morning. Suffice it to say that Mark is behind on his email.]

Revision as of 20:59, 20 July 2020

GlueX Software Meeting
Tuesday, July 21, 2020
3:00 pm EDT
BlueJeans: 968 592 007


  1. Announcements
    1. New version set, version_4.23.1.xml
  2. Review of Minutes from the Last Software Meeting (all)
  3. Report from the Last HDGeant4 Meeting (all)
  4. Compiler upgrade discussion (all)
  5. Review of recent issues and pull requests:
    1. halld_recon
    2. halld_sim
    3. CCDB
    4. RCDB
  6. Review of recent discussion on the GlueX Software Help List (all)
  7. Action Item Review (all)