GlueX Offline Meeting, August 17, 2016

From GlueXWiki
Revision as of 17:45, 17 August 2016 by Marki (Talk | contribs) (add the minutes)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

GlueX Offline Software Meeting
Wednesday, August 17, 2016
1:30 pm EDT
JLab: CEBAF Center F326/327

Agenda

  1. Announcements
    1. New release: sim-recon 2.3.0 (Mark)
    2. Analysis Actions in ROOT DSelector (Paul)
  2. Review of minutes from the last meeting (all)
  3. Analysis Launch (Alex A.)
  4. Next Monitoring Launch (Paul/Alex A./Sean)
  5. sim1.1 (Sean, Mark)
  6. HDDM multithreaded i/o benchmarks (Richard)
  7. Review of recent pull requests (all)
  8. Review of recent discussion on the Gluex Software Help List.
  9. Action Item Review

Communication Information

Remote Connection

Slides

Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2016 on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2016/ .

Minutes

Present:

  • CMU: Naomi Jarvis, Curtis Meyer
  • FIU: Mahmoud Kamel
  • JLab: Alexander Austregesilo, Amber Boehnlein, Brad Cannon, Mark Ito (chair), David Lawrence, Paul Mattione, Nathan Sparks, Simon Taylor
  • NU: Sean Dobbs

You can view a recording of this meeting on the BlueJeans site.

Announcements

  1. New release: sim-recon 2.3.0. This release came out a week ago. With recent changes to tracking and tagger reconstruction, it looks like it is time for another one.
  2. Analysis Actions in ROOT DSelector. Paul led us through his email describing the new features, added so that you do not have to do everything yourself, for example calculating the beam asymmetry angle for pseudoscalers.
    • Nathan asked about custom analysis actions. Paul remarked that they can be added but for proof to work, you have to package your code.

Review of minutes from the last meeting

We went over the minutes from August 3.

  • No work has been done on managing the growing number of plugins in sim-recon.
  • The new farm nodes have arrived at the Lab.
  • The last release of sim-recon was built against both ROOT5 and ROOT6. Sean and Alex Barnes have been updating macros to work with the ROOT6 version.
    • Alex A. discovered a strange problem when building the code for the monitoring launch. There is a conflict between the definition of CPLUS_INCLUDE_PATH, as done by the RCDB environment set-up, and rootcling from ROOT6, and only under the bash shell (tcsh is OK). The work-around is to simply unset CPLUS_INCLUDE_PATH.

Analysis Launch

Alex went over his recent email describing the completion of the analysis launch. In total the size of the output is 10 TB. He will make a table of how much space is used by each of the channels.

Six files did not get processed completely; for most of these the REST files were 0 bytes. The files are pinned on the cache disk for 30 days now. Batch 2 had a few more plugins and authors than Batch 1, with 56 trees produced.

Monitoring Launch

Alex started this again this morning. The code chosen uses ROOT6, the new EVIO parser from David, and yesterday's fixes to the tracking code from Simon. There was a problem with the end-of-file handling with the new EVIO parser that David fixed earlier in the week.

Because the input files were already cached from previous attempts at start-up, this time the jobs have started executing quickly. The launch is 10% done after three hours.

sim1.1

All 8 k jobs have completed successfully, save for 20 where the output REST files were missing or of zero length. Mark has resubmitted these jobs. There was a problem last week with jobs corresponding to runs that did not have a definition for the collimator size in the RCDB. Since the job parameters depend on this definition, the jobs failed. Mark found the problem; Sean fixed the values in the RCDB.

Information on conditions used and location of output files can be found on the sim1.1 conditions page.

Alex A. volunteered to do an analysis launch on the sim1.1 REST files once all of them are done.

Sean has been working on putting dead FDC channels into the CCDB for use in a future iteration of the simulation. He is getting the list from Alex A. The next iteration will likely be just before the DNP meeting in Vancouver.

HDDM multi-threaded I/O benchmarks

Richard reported on recent work with multi-threading both input and output routines in HDDM. For jobs that do not do a lot of event processing, the compression/decompression available in the I/O stages can be limiting. See his slides for the details. He looked at compression schemes (bzip2, gzip, none), input and output, streaming access and random access, as well as input of compressed output produced by a multi-threaded task in different combinations for a total of 24 runs of the benchmark software. As an example, with four threads (on a quad-core machine) he gets a factor of three in input speed for gzipped data when compared to a single-threaded run. In many cases he saturated the raw disk bandwidth with the tests.

Exploiting this feature in JANA turns out not to be completely trivial and volunteers for the effort have not come forward. For CPU intensive tasks, like reconstruction, the gain is not significant. In other contexts it can be important. For now we know that in principle we can crank up the bandwidth is we need to in the future.

Retrieved from "https://halldweb1.jlab.org/wiki/index.php?title=GlueX_Offline_Meeting,_August_17,_2016&oldid=76909"