GlueX Software Meeting, April 28, 2020

From GlueXWiki
Jump to: navigation, search

GlueX Software Meeting
Tuesday, April 28, 2020
3:30 pm EDT
BlueJeans: 968 592 007


  1. Announcements
  2. Review of Minutes from the Last Software Meeting (all)
  3. Report from the Last HDGeant4 Meeting (all)
  4. Tighter rules on GitHub Repositories? (Naomi)
  5. Coming Soon from MCwrapper (Thomas)
  6. Review of recent issues and pull requests:
    1. halld_recon
    2. halld_sim
    3. CCDB
    4. RCDB
  7. Review of recent discussion on the GlueX Software Help List (all)
  8. Action Item Review (all)


Present: Alex Austregesilo, Thomas Britton, Sean Dobbs, Mark Ito (chair), Igal Jaegle, Naomi Jarvis, David Lawrence, Susan Schadmand, Justin Stevens, Simon Taylor, Beni Zihlmann

There is a recording of his meeting on the BlueJeans site. Click on the image to access it. Use your JLab credentials to authenticate.

Software 2020-04-28.png


To accommodate an abbreviated agenda for the collaboration meeting, there will not be a software talk at the upcoming collaboration meeting.

Review of Minutes from the Last Software Meeting

We went over the minutes from April 14.

  • Mark still needs to get going with deployment of CCDB 2.0.
  • Alex reported that Mark Dalton has added random number seed initialization to the Bethe-Heitler generator (on a private branch). The move of the random noise generation to mcsmear still needs to be discussed.
  • Python 2 vs. Python 3.
    • Mark has not submitted a pull request for the Python-related changes he described last time. He is trying to get a successful build and execution of hdgeant4 on his Fedora 31 box first, as a test of the entire software chain.
    • Beni called our attention to a recent post on Slashdot titled Python 2's Core Devs Say 'Fond Farewell' While Releasing Its Final Version talking about the very last production release of Python 2.7. No more security updates, no more bug fixes. David remarked that the real edge of the cliff comes when Linux distributions stop shipping Python-2-based packages.
  • Hao Li's issue, hd_root not filling the skims was traced back to the HDDM input code assigning the wrong flavor of HDDM to his input file. The determination of "r" or "s" (REST or simulation) was keying off the file name, Hao's choice of "input.hddm" was unfortunate for a REST file. Richard submitted a couple of pull requests that use the first line of the input file to check if the flavor was determined correctly. Flavor information is coded there explicitly. In addition he added checks for empty event buffers, a symptom of the flavor-mis-assignment scenario.

Report from the Last HDGeant4 Meeting

We went over the minutes from the meeting on April 21. We discussed the fact that upgrading to the latest version of Geant4 will require an upgrade of GCC, from 4.8.5 to 4.9.x. The last time we did this, i.e., go to a compiler that does not come with the commonly used distribution (now that is CentOS7), it was a major disruption.

Tighter rules on GitHub Repositories?

Naomi introduced the idea of using features of the GitHub site to tighten access rules to our Git repositories there in order to avoid accidental commits to critical packages. We walked through the options on the GitHub site in real time and before anyone knew what was happening, we went ahead and changed the settings on halld_recon to require reviews of pull requests before they could be merged on to the master branch. We decided to do the same for halld_sim. This has the beneficial side-effect that pushes to the master branch are disallowed. The thinking was that this was a minimal change to our procedures and had the aforementioned desired side-effect.

MCwrapper Future Feature

Thomas presented a peak at a new feature of MCwrapper. It will be able to produce charts and graphs to display statistics from successes and failures of jobs. See his slides for the details and charts.

  • The fate of MCwrapper jobs can be used as a code health monitor.
    • All software packages are used.
  • Two pieces of data used at present: number of attempts per job before it succeeds and the stage at which jobs bomb, e.g., mcsmear, hd_root.
  • One view: average number of attempts per job vs. version set.
  • For a single project: histogram of number of attempts.
  • Pie charts can be made of stages where a job bombs (failure pies).
    • A failure pie browser is in the plan.
  • Another: average number of attempts as a function of username.

If people have questions or ideas on useful directions for future development, please see Thomas ([]).

Naomi asked about whether we could have a tool that would build and test a series of software versions to find where a bug was introduced. We had a short discussion of how hard it would be, however no volunteers appeared.

Review of recent issues and pull requests

We discussed Issue #355: ReactionFilter crash on simulated REST file. This problem seems to be single greatest contributor to job failures in MCwrapper at present. The problem is not reproducible; if one tries enough times (with one known exception), one can get a job to succeed. Sean thinks that there is some sort of memory overwrite problem involved. And that it appears to happen only in Monte Carlo, which is a hint.

[Added in press: Sean has found an anomalous event numbering scheme in the REST input file.]

Action Item Review

  1. Get CCDB 2.0 going. (Mark)
  2. Schedule a discussion of moving the random number generator in the trigger factory to mcsmear.
  3. Send Thomas your ideas for MCwrapper job statistics views.
  4. Fix the event number in Monte Carlo event files.