Difference between revisions of "GlueX Containers Meeting, March 30, 2018"

From GlueXWiki
Jump to: navigation, search
(→‎Agenda)
(→‎Notes)
 
Line 14: Line 14:
 
Present: Sean Dobbs, Mark Ito, Thomas Britton, Beni Zihlmann, Zisis Papandreau, David Lawrence, Richard Jones
 
Present: Sean Dobbs, Mark Ito, Thomas Britton, Beni Zihlmann, Zisis Papandreau, David Lawrence, Richard Jones
  
−
*
+
* Atlas and CMS and OSG et al. designed a grid to do raw data reconstruction.
 +
** MC was not considered a problem
 +
** The tiered system has as a component the raw data distribution system
 +
** software and databases were staged as "image files"
 +
* XROOTD
 +
** originally developed at SLAC
 +
** redirector: directory service, contains meta data (file locations)
 +
** on the local host open a file just like a local file except file name is replaced with a URL like root://<name of catalog><IP name of redirector><file name>, triggers an open over network
 +
** redirector refers opener to site, a data connection is opened on site, data streamed over connection
 +
** our bandwidth requirements should be OK
 +
** David: at NERSC, 64 threads on a single node, reconstruction, 4 MB/s read rate
 +
** there is a caching system, if everything is mostly working cache, is not heavily used
 +
* UConn demo system
 +
** redirector service started
 +
** 50 servers, each serving a few files
 +
** 10 Gb pipe
 +
* Client side responsibilities
 +
** download and build XROOTD
 +
** set-up ld_preload[?]
 +
** open calls get intercepted and redirected to XROOTD system
 +
* Sean has also messed with setting up an XROOTD server
 +
* David mentioned the XROOTD idea to Amber Boehnlein, she seems supportive
 +
** Consensus that it would be best if infrastructure supported at JLab on the submit host
 +
** Could be used for both raw data and random trigger files
 +
** Richard and Thomas will work on staging random triggers on UConn demo system
 +
* David: Chip may allow Physics to use 5% of future LQCD KNL cluster
 +
* David: planning for summer campaign at NERSC, reconstruction
 +
** Sean: candidates for off-site resource campaigns:
 +
*** &rho; and &omega; yield measurements with 10 or 20 files per run
 +
*** 2017 data: 2 periods, low intensity and high intensity
 +
*** 2018 data: 4 periods

Latest revision as of 20:52, 7 April 2018

GlueX Containers Meeting
Friday, March 30, 2018
11:30 am EDT
JLab: CEBAF Center L207
BlueJeans: 968 592 007

Agenda

Notes

Present: Sean Dobbs, Mark Ito, Thomas Britton, Beni Zihlmann, Zisis Papandreau, David Lawrence, Richard Jones

  • Atlas and CMS and OSG et al. designed a grid to do raw data reconstruction.
    • MC was not considered a problem
    • The tiered system has as a component the raw data distribution system
    • software and databases were staged as "image files"
  • XROOTD
    • originally developed at SLAC
    • redirector: directory service, contains meta data (file locations)
    • on the local host open a file just like a local file except file name is replaced with a URL like root://<name of catalog><IP name of redirector><file name>, triggers an open over network
    • redirector refers opener to site, a data connection is opened on site, data streamed over connection
    • our bandwidth requirements should be OK
    • David: at NERSC, 64 threads on a single node, reconstruction, 4 MB/s read rate
    • there is a caching system, if everything is mostly working cache, is not heavily used
  • UConn demo system
    • redirector service started
    • 50 servers, each serving a few files
    • 10 Gb pipe
  • Client side responsibilities
    • download and build XROOTD
    • set-up ld_preload[?]
    • open calls get intercepted and redirected to XROOTD system
  • Sean has also messed with setting up an XROOTD server
  • David mentioned the XROOTD idea to Amber Boehnlein, she seems supportive
    • Consensus that it would be best if infrastructure supported at JLab on the submit host
    • Could be used for both raw data and random trigger files
    • Richard and Thomas will work on staging random triggers on UConn demo system
  • David: Chip may allow Physics to use 5% of future LQCD KNL cluster
  • David: planning for summer campaign at NERSC, reconstruction
    • Sean: candidates for off-site resource campaigns:
      • ρ and ω yield measurements with 10 or 20 files per run
      • 2017 data: 2 periods, low intensity and high intensity
      • 2018 data: 4 periods
Retrieved from "https://halldweb1.jlab.org/wiki/index.php?title=GlueX_Containers_Meeting,_March_30,_2018&oldid=86291"