Old Calibration Train
Spring 2016 Processing
|SWIF workflow||Run Range||Total # Jobs||Notes||Total Skim Size (TB)||FCAL Skim (TB)||BCAL Skim (TB)||PS Skim (TB)|
|GlueX-CalibRun-2016-02-25||10457 - 10529||first pass, abandoned|
|GlueX-CalibRun-2016-02-29||10531 - 10647||first pass, abandoned|
|GlueX-CalibRun-2016-03-04||10649 - 10724||first pass, abandoned|
|GlueX-CalibRun-2016-03-18||10331 - 10913||6570||second pass||23||8.1||0.94||14|
|GlueX-CalibRun-2016-04-04||11048 - 11145||2993||first pass||9.4||4.7||1.0||3.5|
|GlueX-CalibRun-2016-04-11||11150 - 11312||3573||first pass||9.7||6.2||1.4||2.1|
|GlueX-CalibRun-2016-04-27||11366 - 11668||10599||first pass||25||16||2.8||5.5|
Currently 3 "passes" through the data are performed. The first pass tries to do as many calibrations as possible with one file of data. The second pass tries to do calibrations that need a larger data set. The final pass runs through a full run to use the full statistics in a run and to generate outputs for other calibration procedures that can't be done automatically yet (e.g. pi0 calibrations).
The plugins that are run on each pass are:
- Pass 1
- Step 1 - RF_online
- Step 2 - HLDetectorTiming (coarse timing+ADC/TDC alignment),TOF_TDC_shift
- Step 3 - HLDetectorTiming (track-based timing),BCAL_TDC_Timing (timewalks)
- Pass 2
- Step 1 - TAGH_timewalk,BCAL_attenlength_gainratio,BCAL_TDC_Timing (full)
- Step 2 - st_tw_corr_auto
- Pass 3
The output of the calibration jobs can be found at this webpage or at /volatile/halld/home/gxproj3/calib_jobs . There is a subdirectory for each calibration launch. Each launch has a directory for each run that is processed. For each run, several types of outputs are kept
- The results for the processing of each file are kept in a subdirectory with the same number as the file, in the format NNN
- The summed results for each pass through the data are kept in ROOT files with names of the form" hd_calib_passN_RunRRRRRR.root"
- The processed results for each pass (e.g. constants files and figures) are kept in subdirectories with names of the form "passN/"
The relevant directories are:
- ROOT files & calibration constants: /volatile/halld/home/gxproj3/calib_jobs/[workflow]/output
- EVIO skims: /volatile/halld/home/gxproj3/calib_jobs/[workflow]/skims
The software from the calibration train can be obtained from
git clone https://github.com/sdobbs/calibration_train
Job submission is controlled by the following configuration file:
# data.config - example configuration file jobname = GlueX-CalibRun-2016-04-27 # name of the SWIF workflow # data/memory sizes in GB mem_requested = 9 # max vmem for jobs disk_space = 30 # max disk space for jobs nthreads = 8 # number of threads requested # time in hours time_limit = 36 # max run time for jobs # this file is used for debugging ccdb_table_file = configs/calib_tables
Building a SWIF workflow is controlled by the job_manager.py command
ifarm1102> ./job_manager.py usage: job_manager.py init [config_file] job_manager.py build [-z] [config_file] [run file] job_manager.py run [-L] [config_file]
Some descriptions of the sub-commands:
- init - This takes the configuration file as an input, does the basic setup for the job: creates directories, saves configurations, builds CCDB SQLite file
- build - This takes the configuration file and a file with one run number per line as an input, and creates the SWIF workflow
- run - This just starts the SWIF workflow. Standard SWIF commands can be used to deal with the workflow from here.
Currently, there are two major commands to run
- The jobs will be submitted every Tuesday at noon, JLab time.
- The jobs will be run from the gxproj3 account [parallel use with EventStore jobs]
- The output of the jobs will be stored in ...
The following runs will be processed:
- 2931, 3079, 3179, 3180, 3183, 3185
Each calibration process should include the following:
- Plugin stored in standard location
- ROOT/other scripts stored in standard location
- https://halldsvn.jlab.org/repos/trunk/sim-recon/src/scripts/calibrations ?
- Individual git repos?
- Output to files
- QA routines
What is Being Run
The following plugins are currently being run:
- RF_online (RF signal)
Working on adding:
- BCAL gains - /work/halld/home/wmcginle/Gain_Calib
- TOF calibrations - https://halldsvn.jlab.org/repos/trunk/home/zihlmann/TOF_calib/