Difference between revisions of "GlueX Offline Meeting, March 30, 2016"

From GlueXWiki
Jump to: navigation, search
(Agenda)
m (Text replacement - "http://argus.phys.uregina.ca/cgi-bin/private" to "https://halldweb.jlab.org/doc-private")
 
(One intermediate revision by the same user not shown)
Line 10: Line 10:
 
## [https://mailman.jlab.org/pipermail/jlab-scicomp-briefs/2016q1/000121.html Change to Farm Priority Scheme]
 
## [https://mailman.jlab.org/pipermail/jlab-scicomp-briefs/2016q1/000121.html Change to Farm Priority Scheme]
 
# Review of [[GlueX Offline Meeting, March 2, 2016#Minutes|minutes from March 2]] (all)
 
# Review of [[GlueX Offline Meeting, March 2, 2016#Minutes|minutes from March 2]] (all)
# [http://argus.phys.uregina.ca/cgi-bin/private/DocDB/ShowDocument?docid=2984 Reconstruction Scaling] (Paul)
+
# [https://halldweb.jlab.org/doc-private/DocDB/ShowDocument?docid=2984 Reconstruction Scaling] (Paul)
 
# Calibration Challenge/Processing (Sean) - [https://halldweb.jlab.org/wiki/images/f/f8/Sdobbs_OfflineMtg_30Mar16.pdf slides]
 
# Calibration Challenge/Processing (Sean) - [https://halldweb.jlab.org/wiki/images/f/f8/Sdobbs_OfflineMtg_30Mar16.pdf slides]
 
# NSB not quite zero (Mike S.)
 
# NSB not quite zero (Mike S.)
Line 32: Line 32:
  
 
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2016</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2016/ .
 
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2016</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2016/ .
 +
 +
== Minutes ==
 +
 +
{|
 +
| There is a [https://bluejeans.com/s/9n7o/ recording of this meeting] on the BlueJeans site.
 +
 +
Present:
 +
* '''CMU''': Mike Staib
 +
* '''FIU''': Mahmoud Kamel
 +
* '''FSU''': Brad Cannon
 +
* '''JLab''': Alex Barnes, Mark Ito (chair), David Lawrence, Paul Mattione, Sandy Philpott, Dmitry Romanov, Nathan Sparks, Justin Stevens, Simon Taylor
 +
* '''NU''': Sean Dobbs
 +
|| [[File:offline_2016-03-30.png|200px|thumb|left|Software Meeting]]
 +
|}
 +
 +
=== Announcements ===
 +
 +
# [https://github.com/JeffersonLab/sim-recon/releases/tag/1.10.0 sim-recon version 1.10.0 was released] on March 8.
 +
# [https://mailman.jlab.org/pipermail/jlab-scicomp-briefs/2016q1/000121.html Change to Farm Priority Scheme]. The "exclusive" queue is now a thing of the past. Exclusive use of a node is still available by requesting the requisite number of cores in the Auger request. The change was made on March 10.
 +
 +
===Review of [[GlueX Offline Meeting, March 2, 2016#Minutes|minutes from March 2]]===
 +
 +
The [https://github.com/JeffersonLab/rcdb RCDB source code repository] has migrated to GitHub. The Subversion repository is no longer the authoritative source of the code. Previously, only the wiki for the GitHub repository was being used.
 +
 +
===[https://halldweb.jlab.org/doc-private/DocDB/ShowDocument?docid=2984 Reconstruction Scaling]===
 +
 +
Paul presented slides on his study of the multi-thread scaling behavior of the offline jobs. He looked at two cases: the loaded-with-32-plugin offline monitoring jobs and a 2-plugin-rest-producing jobs. Each case was tried with a range of threads. Clear memory saving is seen with multiple threads. Scaling is generally good, but the curves turn over when then number of threads gets comparable to the number of cores, especially with the histogram-heavy offline monitoring jobs.
 +
 +
Please see [https://halldweb.jlab.org/doc-private/DocDB/ShowDocument?docid=2984 his slides] for details and plots.
 +
 +
=== Calibration Challenge/Processing ===
 +
 +
Sean described his experience with the March 18 Calibration Train Journey. For the final pass, he is creating raw data skims for calibration purposes. For this pass he sees a sweet spot at eight threads. For details, please see [https://halldweb.jlab.org/wiki/images/f/f8/Sdobbs_OfflineMtg_30Mar16.pdf his slides].
 +
 +
=== NSB not quite zero ===
 +
 +
Mike described the "feature" of the F250 firmware whereby a requested value of the number of samples before the pulse (NSB) of zero was not accepted. Instead a value of one is used. This was causing an error in the pedestal subtraction; the number of samples was always one more than expected. The change is explained in [https://mailman.jlab.org/pipermail/halld-cal/2016-March/001241.html his recent email] to the Calibration Group.
 +
 +
A pull request will be submitted soon.
 +
 +
The configuration files have been changed to request NSB = 1, a request that the firmware will honor.
 +
 +
=== CCDB Improvements ===
 +
 +
Dmitry previewed changes coming with version 1.07 of the CCDB. See [https://docs.google.com/presentation/d/1wk7SYDhmOdZM0smBDGda4QeonT90Q4SWlhD-7ZA0pME/edit?usp=sharing his slides] for all of the details.
 +
 +
There was some discussion of a new scheme that allows a one-to-many relation between calibration constants (the one) to run number assignments (the many). In other words, one set of calibration constants can be applied to different run ranges; the constants themselves need not be duplicated in the database. The advantage of this feature is that the equivalence of constants for one run range with those of another can be tested by comparing the constant set references without h having to compare the constants themselves. This feature also simplifies copying constants from one range to another; one simply copies the reference. This idea was described in the original design document of the CCDB.
 +
 +
=== Future Trends in Nuclear Physics Computing Workshop ===
 +
 +
We reviewed the [https://www.jlab.org/conferences/trends2016/program.html agenda] from the workshop held two weeks ago. David gave a talk; Mark appeared on a panel.
 +
 +
There was a discussion with Amber and Frank Wuerthwein of the OSG on
 +
exploring JLab enhancing our facility with the use of OSG
 +
resources. This will be pursued initially through the GlueX VO with
 +
Richard, our contact person.
 +
 +
=== Access to Fairshare Information from JLab Farm ===
 +
 +
We agreed to ask SciComp for access to the priority information that heretofore has only been available from occasional emails.
 +
 +
=== HDPM Update ===
 +
 +
Nathan described the latest. See [https://halldweb.jlab.org/talks/2016/hdpm_03-30-16.pdf his slides] for the details.
 +
 +
His second slide on recent changes:
 +
# Setup file can be sourced from any directory
 +
# Use hdpm commands from any directory
 +
# Option added for passing subdirectory of sim-recon to "build" command
 +
# Added "hdpm run" command
 +
 +
The last item allows users to run an arbitrary command in a target environment set-up by HDPM.

Latest revision as of 17:10, 24 February 2017

GlueX Offline Software Meeting
Wednesday, March 30, 2016
3:00 pm EDT
JLab: CEBAF Center L207

Agenda

  1. Announcements
    1. sim-recon version 1.10.0 released
    2. Change to Farm Priority Scheme
  2. Review of minutes from March 2 (all)
  3. Reconstruction Scaling (Paul)
  4. Calibration Challenge/Processing (Sean) - slides
  5. NSB not quite zero (Mike S.)
  6. CCDB Improvements (Slides)(Sean, Dmitry)
  7. Future Trends in Nuclear Physics Computing Workshop (all)
  8. Access to Fairshare Information from JLab Farm (all)
  9. Review of recent pull requests (all)
  10. Action Item Review
  11. hdpm update

Communication Information

Remote Connection

Slides

Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2016 on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2016/ .

Minutes

There is a recording of this meeting on the BlueJeans site.

Present:

  • CMU: Mike Staib
  • FIU: Mahmoud Kamel
  • FSU: Brad Cannon
  • JLab: Alex Barnes, Mark Ito (chair), David Lawrence, Paul Mattione, Sandy Philpott, Dmitry Romanov, Nathan Sparks, Justin Stevens, Simon Taylor
  • NU: Sean Dobbs
Software Meeting

Announcements

  1. sim-recon version 1.10.0 was released on March 8.
  2. Change to Farm Priority Scheme. The "exclusive" queue is now a thing of the past. Exclusive use of a node is still available by requesting the requisite number of cores in the Auger request. The change was made on March 10.

Review of minutes from March 2

The RCDB source code repository has migrated to GitHub. The Subversion repository is no longer the authoritative source of the code. Previously, only the wiki for the GitHub repository was being used.

Reconstruction Scaling

Paul presented slides on his study of the multi-thread scaling behavior of the offline jobs. He looked at two cases: the loaded-with-32-plugin offline monitoring jobs and a 2-plugin-rest-producing jobs. Each case was tried with a range of threads. Clear memory saving is seen with multiple threads. Scaling is generally good, but the curves turn over when then number of threads gets comparable to the number of cores, especially with the histogram-heavy offline monitoring jobs.

Please see his slides for details and plots.

Calibration Challenge/Processing

Sean described his experience with the March 18 Calibration Train Journey. For the final pass, he is creating raw data skims for calibration purposes. For this pass he sees a sweet spot at eight threads. For details, please see his slides.

NSB not quite zero

Mike described the "feature" of the F250 firmware whereby a requested value of the number of samples before the pulse (NSB) of zero was not accepted. Instead a value of one is used. This was causing an error in the pedestal subtraction; the number of samples was always one more than expected. The change is explained in his recent email to the Calibration Group.

A pull request will be submitted soon.

The configuration files have been changed to request NSB = 1, a request that the firmware will honor.

CCDB Improvements

Dmitry previewed changes coming with version 1.07 of the CCDB. See his slides for all of the details.

There was some discussion of a new scheme that allows a one-to-many relation between calibration constants (the one) to run number assignments (the many). In other words, one set of calibration constants can be applied to different run ranges; the constants themselves need not be duplicated in the database. The advantage of this feature is that the equivalence of constants for one run range with those of another can be tested by comparing the constant set references without h having to compare the constants themselves. This feature also simplifies copying constants from one range to another; one simply copies the reference. This idea was described in the original design document of the CCDB.

Future Trends in Nuclear Physics Computing Workshop

We reviewed the agenda from the workshop held two weeks ago. David gave a talk; Mark appeared on a panel.

There was a discussion with Amber and Frank Wuerthwein of the OSG on exploring JLab enhancing our facility with the use of OSG resources. This will be pursued initially through the GlueX VO with Richard, our contact person.

Access to Fairshare Information from JLab Farm

We agreed to ask SciComp for access to the priority information that heretofore has only been available from occasional emails.

HDPM Update

Nathan described the latest. See his slides for the details.

His second slide on recent changes:

  1. Setup file can be sourced from any directory
  2. Use hdpm commands from any directory
  3. Option added for passing subdirectory of sim-recon to "build" command
  4. Added "hdpm run" command

The last item allows users to run an arbitrary command in a target environment set-up by HDPM.