GlueX Software Meeting, May 11, 2021
GlueX Software Meeting
Tuesday, May 11, 2021
3:00 pm EDT
BlueJeans: 968 592 007
- 1 Agenda
- 2 Minutes
- Review of Minutes from the Last Software Meeting (all)
- Minutes from the Last HDGeant4 Meeting (all)
- Report from the May 6th SciComp Meeting (Mark)
- Review of recent issues and pull requests:
- Review of recent discussion on the GlueX Software Help List (all)
- Action Item Review (all)
Present: Alexander Austregesilo, Thomas Britton, Mark Dalton, Sean Dobbs, Mark Ito (chair), Igal Jaegle, David Lawrence, Simon Taylor, Nilanga Wickramaarachchi, Jon Zarling
- HOWTO update a condition on the RCDB. Sean described his new wiki page. He stressed that changes to the RCDB should be coordinated; don't change it and then fail to mention it to anybody.
- David asked about the particular example in the HOWTO, changing the number of data files per run. Igal responded that the number of files in a run needed to match those submitted NERSC for jobs to succeed. The RCDB thus sometimes needed modification for cases where not all of the files recorded to disk in the Counting House (original file count) actually made it to tape in the Computer Center (modified file count).
- Mark reminded us that we could develop a system where RCDB entries could be copied to the CCDB automatically. From there they could be changed in a way that left a historical record and a method for dropping back to previous versions.
- Build Scripts 2.10. Mark I. pointed out that users can now set BUILD_SCRIPTS_CONSISTENCY_CHECK=false in their environment to suppress warnings about inconsistent software versions. He also pointed out that they do so at their own risk.
- New version sets: 4.38.0 and related recon-launch-compatible sets. Mark I. led us through the release notes for the new packages.
- scan-build revived. Mark I. explained the latest scheme for generating and accessing error/warning reports for several of our software packages.
Review of Minutes from the Last Software Meeting
We went over the Minutes from the meeting on April 28.
- Mark I. continues to ping collaborators about their usage of the work disk.
- Mark I. did some checking on the values reported on the online plots are accurate but more investigation is needed.
- Mark I., Sean, and Chris Larrieu discussed setting up volume set in the tape library to speed retrieval of skim data in the future. This requires very little effort, just planning.
- Jon has not received feedback on the ROOT-related items he brought up, but progress has been made nonetheless.
- He has built ROOT 6.24 and has looked at exploiting its features. Among them a new compression scheme, based on LZ4, although increasing file size by 10-20%, decreases reading time by a factor of two.
- Jon has also test driven uproot. In a chain that starts with a ROOT tree and ends in PyROOT-friendly files[?] he sees a factor of three in speed.
- Simon mentioned that in addition to TLorentzVector, TVector2 and Tvector3 have been deprecated. All of these classes are used a lot in our charged track reconstruction. Going to the new, recommended classes seems at least worth a try.
- Sean reminded us that a complete set of random trigger files for Fall 18 have been produced.
Minutes from the Last HDGeant4 Meeting
We went over the Minutes from the meeting on May 4th without significant comment.
Report from the May 6th SciComp Meeting
Mark I. gave the report. Alex volunteered to compile a list of files that we should prioritize for recovery, files that are holding up analysis launches for Spring 2020.
Review of recent issues and pull requests
- halld_recon Issue #512: hd_root memory leak? Sean has an idea on what may be causing this.
- Thomas will review outstanding issues in MCwrapper. He thinks many of them can be closed.
Review of recent discussion on the GlueX Software Help List
We went over the list. Mark I. noted that recently a lot of software Q&A has been conducted on Slack, which is fine, but those exchanges are not archived. Because Slack has proved popular within the JLab community, the record only goes back to March at present. Many of these conversations, in hindsight, should appear on the Help List. Although it is understandable how a thread might originate on Slack, Mark encouraged us to encourage our collaborators to migrate the conversation to the Help List where a search-able record of problems and solutions can be maintained.
Action Item Review
- Do a build against ROOT 6.24. (Mark I.)
- Decide on needed tape volume sets. (Sean, Mark I.)
- Review disk usage when re-repatriating recon launch data. (Alex, Mark)
- Check validity of data reported in volatile usage plots. (Mark I.)
- Review outstanding issues with MCwrapper (Thomas)
- Take another look at halld_recon issue #512. (Sean)
- Make list of high-priority files for recovery from tape errors. (Alex)
- [Added in press: Alex generated a list and sent it to Mark]
- Think about Jon's list of improvements. (all)
- Automatically make CCDB entries for selected RCDB items. (Mark I.)
- Modify the documentation on how to copy files from JLab. (Mark I.)
- Look at replacements for TLorentzVector, TVector2, and Tvector3 (all)