Difference between revisions of "Online farm testing 5-dec-2012"

From GlueXWiki
Jump to: navigation, search
m
m
Line 1: Line 1:
 
'''Current status of online farm tests in the counting house'''
 
'''Current status of online farm tests in the counting house'''
  
−
* Main GBit Ethernet switch working (has 10 GBit capability).
+
* Infrastructure
−
* Big UPS working.
+
** Main GBit Ethernet switch working (has 10 GBit capability).
−
* Borrowed nodes gluon02-gluon05 working (64-bit RHEL6, 16 cores each, lots of memory).
+
** Big UPS working.
−
* Dave A will install borrowed 40 GBit/sec InfiniBand switch (will eventually purchase 60 GBit).
+
** Borrowed nodes gluon02-gluon05 working (64-bit RHEL6, 16 cores each, lots of memory).
−
* gluon02-05 have both GBit and IB interfaces.
+
** Dave A will install borrowed 40 GBit/sec InfiniBand switch (will eventually purchase 60 GBit).
−
* DAQ software installed and working (EVIO, cMsg, ET).
+
** gluon02-05 have both GBit and IB interfaces.
−
* Used rawevent plugin to convert Dave L's hddm file to EVIO format using mc2coda package from Dave A.
+
 
−
* Generated compressed EVIO file using my mc2coda_compress program (to 8 GB/evt vs 24 GB/evt).
+
* Software
−
* Created ET system on gluon02.
+
** DAQ software installed and working (EVIO, cMsg, ET).
−
* Fake event builder (ET producer) montest injects EVIO events from a file into ET system.
+
** rawevent plugin converts hddm file to EVIO format using mc2coda package from Dave A, 24 kB/event.
−
* Local injection rate about 110 kHz per producer for uncompressed (25 kB) events.
+
** mc2coda_compress compresses raw EVIO file 8 GB/evt.
−
** 5 producers gives 550 kHz.
+
** montest reads EVIO files and inserts events into ET system
−
** 350 kHz per producer for (8 kB) compressed events.
+
 
−
** each producer takes 100% of one core.
+
 
−
* Minimal ET system overhead, maybe 5% of a core.
+
* Results for local transfers
−
* Local consumer can keep up with producer rate.
+
** Injection rate about 110 kHz per producer for uncompressed events, 350 kHz for compressed events.
−
* Remote consumer saturates GBit Ethernet at 95% level!
+
** Injection rate scales with the number of producers.
 +
** Each producer takes 100% of one core.
 +
** ET system overhead about 5% of a core.
 +
** Local consumer can keep up with producer rate.
 +
 
 +
 
 +
* Results for network transfers
 +
** Remote consumer saturates GBit Ethernet at 95% level
 
** 5 kHz for uncompressed events, 15 kHz for compressed events.
 
** 5 kHz for uncompressed events, 15 kHz for compressed events.
−
* Further tests when InfiniBand working, expect factor 10 increase, not factor 40.
+
** Consumers use about 25% of a core.
 +
** ET system about 20% of a core.
 +
 
 +
 
 +
* InfiniBand testing
 +
** Expect factor 10 increase, not factor 40.
 
** DAQ tests show you get 25% of 40 GBit using TCP over IB.
 
** DAQ tests show you get 25% of 40 GBit using TCP over IB.
 
** we will purchase 60 GBit IB.
 
** we will purchase 60 GBit IB.
−
* Remote event rate 5 kHz for uncompressed 25 kB events, expect 20 kHz for 8 kB compressed events.
+
 
−
* Another step is to introduce JANA analysis process. Dave L:
+
 
−
** is waiting for development of evioETchannel class (me, soon).
+
* JANA analysis processes
−
** has updated JANA to read uncompressed EVIO files.
+
** Waiting for development of evioETchannel class (me, soon).
−
** is working on reading compressed EVIO data.
+
** Dave L has updated JANA to read uncompressed EVIO files.
−
* Vardan and I will develop farm manager CODA component.
+
** Dave L is working on reading compressed EVIO data.
 +
 
 +
 
 +
* Farm manager
 +
** Vardan and I will develop this.

Revision as of 14:55, 4 December 2012

Current status of online farm tests in the counting house

  • Infrastructure
    • Main GBit Ethernet switch working (has 10 GBit capability).
    • Big UPS working.
    • Borrowed nodes gluon02-gluon05 working (64-bit RHEL6, 16 cores each, lots of memory).
    • Dave A will install borrowed 40 GBit/sec InfiniBand switch (will eventually purchase 60 GBit).
    • gluon02-05 have both GBit and IB interfaces.
  • Software
    • DAQ software installed and working (EVIO, cMsg, ET).
    • rawevent plugin converts hddm file to EVIO format using mc2coda package from Dave A, 24 kB/event.
    • mc2coda_compress compresses raw EVIO file 8 GB/evt.
    • montest reads EVIO files and inserts events into ET system


  • Results for local transfers
    • Injection rate about 110 kHz per producer for uncompressed events, 350 kHz for compressed events.
    • Injection rate scales with the number of producers.
    • Each producer takes 100% of one core.
    • ET system overhead about 5% of a core.
    • Local consumer can keep up with producer rate.


  • Results for network transfers
    • Remote consumer saturates GBit Ethernet at 95% level
    • 5 kHz for uncompressed events, 15 kHz for compressed events.
    • Consumers use about 25% of a core.
    • ET system about 20% of a core.


  • InfiniBand testing
    • Expect factor 10 increase, not factor 40.
    • DAQ tests show you get 25% of 40 GBit using TCP over IB.
    • we will purchase 60 GBit IB.


  • JANA analysis processes
    • Waiting for development of evioETchannel class (me, soon).
    • Dave L has updated JANA to read uncompressed EVIO files.
    • Dave L is working on reading compressed EVIO data.


  • Farm manager
    • Vardan and I will develop this.
Retrieved from "https://halldweb1.jlab.org/wiki/index.php?title=Online_farm_testing_5-dec-2012&oldid=42703"