September 19, 2014 Data Monitoring
- 1 Agenda
- 2 Online Data Monitoring
- 3 Offline Data Monitoring
- 4 Example Links
- 5 Miscellaneous Useful Information
- Discuss what we want to know, and what we want to do.
- Online monitoring.
- Offline monitoring.
- Assign tasks.
Online Data Monitoring
What we want to know
- Are the detectors working?
- Detector Systems: TAGH, TAGM, SC, CDC, FDC, TOF, BCAL, FCAL, PS (Pair Spectrometer)
- Do all of the channels have hits?
- What are the hit counts/rates per channel?
- Are the energies & times OK, garbage, or out of range? (by channel)
- Is the reconstruction working?
- Run full reconstruction over a small % of data, view tracking/shower-reconstruction results online (hdview2, monitoring_hists plugins).
- Mike will point us to collected EVIO data (at least for the CDC) as it becomes available.
- David(?) will make sure everything necesssary is viewable in hdview2.
- David is updating RootSpy and the documentation.
- When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.
- We (Paul?, David?, Sean?) will stay in contact with the detector groups, and make sure that they:
- Write the plugins
- Are histogramming the data we want (see above).
- Integrate them into RootSpy, and determine which are the primary plots, the ones that are the most important for shift-takers.
- David will make sure that the RF time is included in the translation table.
- Paul will make sure that the monitoring_hists plugin plots (for reconstruction quality) are integrated and viewable in RootSpy.
Offline Data Monitoring
What we want to know
- Can we reproduce the online histograms offline:
- Can we read data from tape?
- Are all the channels present?
- Are all the recorded values within the expected ranges?
- How do the online histograms change as a function of time (e.g. hits/event vs run number)?
- Calibration status/quality histograms (individual runs & a function of time)
- Reconstruction status/quality histograms
- Num tracks, showers, hits, etc.
- Reconstructed track kinematics
- Track / detector-hit matching
- PID Quality
- Analysis histograms
- Can we see pi0 peaks
- Can we see simple final states
- γ p → p π+ π-
- γ p → p π+ π- π0
- γ p → p π+ π- η
Action Items - Now (With what is available)
- Paul: Will contact David & Mark and find locations to host the websites, mysql database, and ROOT files.
- Paul: Will contact David and make sure there is an easy way to access the configuration used for a given run (which detectors were connected/on/off, trigger configuration, did the DAQ crash or did the data get written to disk).
- Paul, Sean: Make sure that the detector groups make plugins for doing the calibrations and viewing the quality/results.
- Kei: Make cron-job scripts that automatically read EVIO data from tape, run the online/monitoring plugins, and save the ROOT files to disk.
- Raw data (hit occupancies) need to only be read once per run, calibration/reconstruction data need to be rerun and updated periodically.
- Sean: Make cron-job scripts that read the data from the ROOT files and:
- Store appropriate data in an sqlite database (e.g. hit rates, # events, # reconstructed tracks/showers, calibration resolutions) as a function of run number.
- Make png(?) files of important histograms on disk (hit occupancies, calibration plots, reconstructed track p vs. θ, etc.)
- Kei/Justin: Make cron-job scripts that create a version-0, "just get something working" webpage for viewing:
- Important histograms for each detector (png's created by Sean)
- (w/ Sean's help for database access): Values in the mysql database as a function of run number.
Action Items - Near Future
- Kei, Sean: As more detector plugins become available, expand the system for viewing their data.
- Justin: Make a more sophisticated/elaborate series of scripts & webpages for viewing:
- Today's & yesterday's online data
- Primary plots from any given run (each detector system: hit occupancies, calibration quality, reconstruction counts)
- Performance as a function of time (each detector system: hit occupancies, calibration quality, reconstruction counts)
- Links to run configuration information
Miscellaneous Useful Information
- How to: Link
- If you are running over EVIO files created from simulated data, you need to run with the environment variable:
Running with B-field = 0
- For the simulation, in the control.in file, set:
- For the reconstruction, set B=(0,0,0) everywhere by running with the command line argument: