GlueX Offline Meeting, October 4, 2017
GlueX Offline Software Meeting
Wednesday, October 4, 2017
11:00 am EDT
JLab: CEBAF Center F326/327
- 1 Agenda
- 2 Communication Information
- 3 Minutes
- Review of minutes from the last meeting (all)
- HDvis update 
- Reducing work disk usage (Mark)
- Preliminary Tape and Disk Usage Projections (Mark)
- Review of recent pull requests (all)
- Review of recent discussion on the GlueX Software Help List (all)
- Action Item Review (all)
- The BlueJeans meeting number is 968 592 007 .
- Join the Meeting via BlueJeans
Talks can be deposited in the directory
/group/halld/www/halldweb/html/talks/2017 on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2017/ .
- CMU: : Naomi Jarvis
- FIU : Joerg Reinhold
- FSU : Sean Dobbs
- JLab: : Alex Austregesilo, Amber Boehnlein, Thomas Britton, Eugene Chudakov, Sebastian Cole, Mark Ito (chair), David Lawrence, Sascha Somov, Simon Taylor, Beni Zihlmann
There is a recording of this meeting on the BlueJeans site. Use your JLab credentials to access it.
- new tag of mcsmear branch of sim-recon, version 3. Sean released a new tag of the mcsmear development branch of sim-recon. Mark has built it at JLab.
- old, empty volatile directories now being deleted Directories on the volatile disk that are (a) empty and (b) remain so for three months are now being deleted.
- Farm jobs should use SQLite form of the CCDB If hundreds of farm jobs access the MySQL database server for calibration constants, the server gets overwhelmed. Use the disk-resident SQLite version of the database when running on the farm.
- Thomas has added an option to MCwrapper for selecting SQLite files for CCDB.
Review of minutes from the last meeting
We went over the minutes from September 20.
- The automatic updates of the Oasis file system are still not functioning fully automatically. Mark has a ticket into the Grid folks asking for guidance on how to proceed.
- The first monthly Analysis Launch is now finished.
Thomas guided us through the latest version of the new event display. See the demo starting at 10:00 in the recording.
- The orthographic view is now in.
- Thomas has been talking to groups about what they want to see in the display of their detector system.
- He will give a demo at the collaboration meeting.
- David requested that an axis indicator appear.
- Naomi suggested fixed, standard views for re-orienting oneself. Thomas said that that is in the plan.
- Added in press: there was a glitch with mouse-over in the demo. Subsequent to the meeting, Thomas found and fixed it. It appeared only with the Firefox browser.
Reducing work disk usage
Mark went over his recent email. Some points of information:
- We are getting a new work disk. It with be ZFS and not Lustre.
- The new disk is significantly smaller than the work space we are using now. We will have to reduce our usage from 77 TB down to 45.
- Some reduction (about 10 TB) has already occurred in response to Mark's email.
- We looked at the Work Disk Usage Leader-board.
- Mark reported that it seems that SciComp has been slowing reducing our cache and volatile reservations in response to ever increasing usage on work.
- Work is not formally managed. The work allocation is done by UNIX group is really total group Lustre usage minus volatile group usage minus cache group usage. Volatile and cache however are managed by directory location so there are corner cases. As an unmanaged work grows, cache and volatile feel the squeeze.
- Mark will be contacting individuals and consulting on reducing their work disk footprint.
If you have old, unused files under /work/halld, now would be a great time to delete them or archive them to tape.
Preliminary Tape and Disk Usage Projections
Mark went over a spreadsheet summarizing the files produced by the various launches for the Spring 16 and Spring 17 runs. The input was an interview with Paul Mattione and Alex on the previous day. He outlines two strategies for what we might want to keep on disk, high-usage and low-usage. The high usage strategy requires 260 TB spinning including both Spring 16 and Sprint 17 running. The low-usage only 86 TB. He noted that our current usage is close to the middle of this range.
This is a work in progress. Beni pointed out that Monte Carlo data is not counted. Calibrations skims were left out also; Mark has not gotten to doing the research on those. At this point, these numbers are lower limits.
Review of recent pull requests
Paul checked in a major change to the analysis library. He will describe the new stuff at the collaboration meeting next week.
A couple of the recent requests (one from David, one from Will McGinley) showed large numbers of files differing on the proposed branch from the version on the master branch, nearly 100, when the number of changes being proposed were only a few. These were due to Paul's analysis library changes getting merged before the requests in question got merged. Since there were no conflicts on the two branches, these merges can go ahead without causing any problem.
Review of recent discussion on the GlueX Software Help List
We went over the recent threads.
As the result of one discussion Mark built a version of the code from the last reconstruction launch on Spring 16 data. In the future, we may want to do builds of certain launches on the group disk as a matter of course.