Difference between revisions of "Micro Data Challenge, July 30, 2012"
From GlueXWiki
Line 1: | Line 1: | ||
+ | =Status= | ||
+ | |||
* Two "projects" | * Two "projects" | ||
*# '''bggen''': generate, simulate, and smear HDGeant events, write events to tape | *# '''bggen''': generate, simulate, and smear HDGeant events, write events to tape | ||
Line 39: | Line 41: | ||
</pre> | </pre> | ||
− | Notes | + | =Notes= |
* output written to new tape volume set: /mss/halld/halld-scratch | * output written to new tape volume set: /mss/halld/halld-scratch | ||
* random number seeds varied for event generation (bggen) | * random number seeds varied for event generation (bggen) | ||
− | Early lessons learned | + | =Early lessons learned= |
* JLab farm batch priority scheme | * JLab farm batch priority scheme | ||
** opaque | ** opaque | ||
** wrong-headed | ** wrong-headed | ||
+ | ** meeting scheduled for tomorrow with SciComp | ||
* database schema improvements: | * database schema improvements: | ||
** multiple output files | ** multiple output files | ||
Line 54: | Line 57: | ||
* script improvements: | * script improvements: | ||
** configuration file for input/output/tape locations | ** configuration file for input/output/tape locations | ||
+ | |||
+ | =Scope Expansion= | ||
+ | |||
+ | * create output in REST format | ||
+ | * longer runs | ||
+ | * analysis system for results |
Latest revision as of 13:00, 30 July 2012
Status
- Two "projects"
- bggen: generate, simulate, and smear HDGeant events, write events to tape
- surrogate for data taking with GlueX
- hd_root: read events from bggen project from tape, do reconstruction, write root trees to tape
- surrogate for reconstruction, producing DSTs
- bggen: generate, simulate, and smear HDGeant events, write events to tape
- 1000 events per file
- all work on JLab batch farm
- all actions initiated and recorded by jproj.pl (perl script)
ifarm1102:marki:792> mysql -hhalldweb1 -ufarmer farming Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 225647 Server version: 5.1.36-log MySQL Community Server (GPL) Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> select count(file), sum(submitted), sum(output), sum(jput_submitted), sum(silo) from bggen; +-------------+----------------+-------------+---------------------+-----------+ | count(file) | sum(submitted) | sum(output) | sum(jput_submitted) | sum(silo) | +-------------+----------------+-------------+---------------------+-----------+ | 1000 | 1000 | 436 | 436 | 211 | +-------------+----------------+-------------+---------------------+-----------+ 1 row in set (0.01 sec) mysql> select count(file), sum(submitted), sum(output), sum(jput_submitted), sum(silo) from hd_root; +-------------+----------------+-------------+---------------------+-----------+ | count(file) | sum(submitted) | sum(output) | sum(jput_submitted) | sum(silo) | +-------------+----------------+-------------+---------------------+-----------+ | 196 | 196 | 8 | 8 | 0 | +-------------+----------------+-------------+---------------------+-----------+ 1 row in set (0.00 sec) mysql> quit Bye
Notes
- output written to new tape volume set: /mss/halld/halld-scratch
- random number seeds varied for event generation (bggen)
Early lessons learned
- JLab farm batch priority scheme
- opaque
- wrong-headed
- meeting scheduled for tomorrow with SciComp
- database schema improvements:
- multiple output files
- capture of AUGER job ID
- script improvements:
- configuration file for input/output/tape locations
Scope Expansion
- create output in REST format
- longer runs
- analysis system for results