Micro Data Challenge, July 30, 2012
From GlueXWiki
Status
- Two "projects"
- bggen: generate, simulate, and smear HDGeant events, write events to tape
- surrogate for data taking with GlueX
- hd_root: read events from bggen project from tape, do reconstruction, write root trees to tape
- surrogate for reconstruction, producing DSTs
- bggen: generate, simulate, and smear HDGeant events, write events to tape
- 1000 events per file
- all work on JLab batch farm
- all actions initiated and recorded by jproj.pl (perl script)
ifarm1102:marki:792> mysql -hhalldweb1 -ufarmer farming Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 225647 Server version: 5.1.36-log MySQL Community Server (GPL) Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> select count(file), sum(submitted), sum(output), sum(jput_submitted), sum(silo) from bggen; +-------------+----------------+-------------+---------------------+-----------+ | count(file) | sum(submitted) | sum(output) | sum(jput_submitted) | sum(silo) | +-------------+----------------+-------------+---------------------+-----------+ | 1000 | 1000 | 436 | 436 | 211 | +-------------+----------------+-------------+---------------------+-----------+ 1 row in set (0.01 sec) mysql> select count(file), sum(submitted), sum(output), sum(jput_submitted), sum(silo) from hd_root; +-------------+----------------+-------------+---------------------+-----------+ | count(file) | sum(submitted) | sum(output) | sum(jput_submitted) | sum(silo) | +-------------+----------------+-------------+---------------------+-----------+ | 196 | 196 | 8 | 8 | 0 | +-------------+----------------+-------------+---------------------+-----------+ 1 row in set (0.00 sec) mysql> quit Bye
Notes
- output written to new tape volume set: /mss/halld/halld-scratch
- random number seeds varied for event generation (bggen)
Early lessons learned
- JLab farm batch priority scheme
- opaque
- wrong-headed
- meeting scheduled for tomorrow with SciComp
- database schema improvements:
- multiple output files
- capture of AUGER job ID
- script improvements:
- configuration file for input/output/tape locations
Scope Expansion
- create output in REST format
- longer runs
- analysis system for results