Difference between revisions of "September 19, 2014 Data Monitoring"

From GlueXWiki
Jump to: navigation, search
(Commissioning Code Information)
m (Text replacement - "/halldweb1.jlab.org/" to "/halldweb.jlab.org/")
 
(9 intermediate revisions by 2 users not shown)
Line 10: Line 10:
 
=== What we want to know ===
 
=== What we want to know ===
 
* Are the detectors working?  
 
* Are the detectors working?  
 +
** Detector Systems: TAGH, TAGM, SC, CDC, FDC, TOF, BCAL, FCAL, PS (Pair Spectrometer)
 
** Do all of the channels have hits?  
 
** Do all of the channels have hits?  
 
** What are the hit counts/rates per channel?
 
** What are the hit counts/rates per channel?
 
** Are the energies & times OK, garbage, or out of range? (by channel)
 
** Are the energies & times OK, garbage, or out of range? (by channel)
** Is the reconstruction working?: Run full reconstruction over a small % of data, view tracking/shower-reconstruction results online (hdview2, monitoring_hists plugins).
+
* Is the reconstruction working?
 +
** Run full reconstruction over a small % of data, view tracking/shower-reconstruction results online (hdview2, monitoring_hists plugins).
  
 
=== Example ===
 
=== Example ===
Line 21: Line 23:
 
=== Action Items ===
 
=== Action Items ===
  
* Mike will point us to collected EVIO data for the different detector systems as it becomes available.  
+
* Mike will point us to collected EVIO data (at least for the CDC) as it becomes available.  
 
* David(?) will make sure everything necesssary is viewable in hdview2.  
 
* David(?) will make sure everything necesssary is viewable in hdview2.  
 
* David is updating RootSpy and the documentation.
 
* David is updating RootSpy and the documentation.
Line 56: Line 58:
 
=== Action Items - Now (With what is available) ===
 
=== Action Items - Now (With what is available) ===
  
 +
* Paul: Will contact David & Mark and find locations to host the websites, mysql database, and ROOT files.
 
* Paul: Will contact David and make sure there is an easy way to access the configuration used for a given run (which detectors were connected/on/off, trigger configuration, did the DAQ crash or did the data get written to disk).  
 
* Paul: Will contact David and make sure there is an easy way to access the configuration used for a given run (which detectors were connected/on/off, trigger configuration, did the DAQ crash or did the data get written to disk).  
 
* Paul, Sean: Make sure that the detector groups make plugins for doing the calibrations and viewing the quality/results.  
 
* Paul, Sean: Make sure that the detector groups make plugins for doing the calibrations and viewing the quality/results.  
Line 61: Line 64:
 
** Raw data (hit occupancies) need to only be read once per run, calibration/reconstruction data need to be rerun and updated periodically.  
 
** Raw data (hit occupancies) need to only be read once per run, calibration/reconstruction data need to be rerun and updated periodically.  
 
* Sean: Make cron-job scripts that read the data from the ROOT files and:  
 
* Sean: Make cron-job scripts that read the data from the ROOT files and:  
** Store appropriate data in a mysql database (e.g. hit rates, # events, # reconstructed tracks/showers, calibration resolutions) as a function of run number.  
+
** Store appropriate data in an sqlite database (e.g. hit rates, # events, # reconstructed tracks/showers, calibration resolutions) as a function of run number.  
 
** Make png(?) files of important histograms on disk (hit occupancies, calibration plots, reconstructed track p vs. θ, etc.)
 
** Make png(?) files of important histograms on disk (hit occupancies, calibration plots, reconstructed track p vs. θ, etc.)
* Kei: Make cron-job scripts that create a version-0, "just get something working" webpage for viewing:
+
* Kei/Justin: Make cron-job scripts that create a version-0, "just get something working" webpage for viewing:
 
** Important histograms for each detector (png's created by Sean)
 
** Important histograms for each detector (png's created by Sean)
 
** (w/ Sean's help for database access): Values in the mysql database as a function of run number.
 
** (w/ Sean's help for database access): Values in the mysql database as a function of run number.
Line 81: Line 84:
  
 
== Miscellaneous Useful Information ==
 
== Miscellaneous Useful Information ==
 +
=== Commissioning Software ===
 
* HDDS Commissioning Branch [https://halldsvn.jlab.org/repos/branches/hdds-commissioning Link]
 
* HDDS Commissioning Branch [https://halldsvn.jlab.org/repos/branches/hdds-commissioning Link]
 
* sim-recon Commissioning Branch [https://halldsvn.jlab.org/repos/branches/sim-recon-commissioning Link]
 
* sim-recon Commissioning Branch [https://halldsvn.jlab.org/repos/branches/sim-recon-commissioning Link]
* MC-to-EVIO-to-HDDM [https://halldweb1.jlab.org/wiki/index.php/HOWTO_convert_MC_data_to_EVIO_and_replay_it Link]
+
 
 +
=== MC-to-EVIO ===
 +
* How to: [https://halldweb.jlab.org/wiki/index.php/HOWTO_convert_MC_data_to_EVIO_and_replay_it Link]
 
** If you are running over EVIO files created from simulated data, you need to run with the environment variable:
 
** If you are running over EVIO files created from simulated data, you need to run with the environment variable:
 
<syntaxhighlight>
 
<syntaxhighlight>
 
JANA_CALIB_CONTEXT="variation=mc"
 
JANA_CALIB_CONTEXT="variation=mc"
 
</syntaxhighlight>
 
</syntaxhighlight>
* Accessing Onsite Webpages from Offsite [https://hdops.jlab.org/wiki/index.php/Accessing_Onsite_Webpages_From_Offsite Link]
+
 
* RootSpy [http://www.jlab.org/RootSpy/ Website]
+
=== Running with B-field = 0 ===
* RootSpy Source Code [https://halldsvn.jlab.org/repos/trunk/online/packages/RootSpy/ Link]
+
* For the simulation, in the control.in file, set:
* Running with B-field = 0:
+
** For the simulation, in the control.in file, set:
+
 
<syntaxhighlight>
 
<syntaxhighlight>
 
BFIELDTYPE 'NoField'
 
BFIELDTYPE 'NoField'
 
</syntaxhighlight>
 
</syntaxhighlight>
** For the reconstruction, set B=(0,0,0) everywhere by running with the command line argument:
+
* For the reconstruction, set B=(0,0,0) everywhere by running with the command line argument:
 
<syntaxhighlight>
 
<syntaxhighlight>
 
-PBFIELD_TYPE=NoField.
 
-PBFIELD_TYPE=NoField.
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
=== Miscellaneous ===
 +
* Accessing Onsite Webpages from Offsite [https://halldweb.jlab.org/hdops/wiki/index.php/Accessing_Onsite_Webpages_From_Offsite Link]
 +
* RootSpy [http://www.jlab.org/RootSpy/ Website]
 +
* RootSpy Source Code [https://halldsvn.jlab.org/repos/trunk/online/packages/RootSpy/ Link]

Latest revision as of 06:22, 1 April 2015

Agenda

  1. Discuss what we want to know, and what we want to do.
    1. Online monitoring.
    2. Offline monitoring.
  2. Assign tasks.

Online Data Monitoring

What we want to know

  • Are the detectors working?
    • Detector Systems: TAGH, TAGM, SC, CDC, FDC, TOF, BCAL, FCAL, PS (Pair Spectrometer)
    • Do all of the channels have hits?
    • What are the hit counts/rates per channel?
    • Are the energies & times OK, garbage, or out of range? (by channel)
  • Is the reconstruction working?
    • Run full reconstruction over a small % of data, view tracking/shower-reconstruction results online (hdview2, monitoring_hists plugins).

Example

Action Items

  • Mike will point us to collected EVIO data (at least for the CDC) as it becomes available.
  • David(?) will make sure everything necesssary is viewable in hdview2.
  • David is updating RootSpy and the documentation.
    • When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.
  • We (Paul?, David?, Sean?) will stay in contact with the detector groups, and make sure that they:
    • Write the plugins
    • Are histogramming the data we want (see above).
    • Integrate them into RootSpy, and determine which are the primary plots, the ones that are the most important for shift-takers.
  • David will make sure that the RF time is included in the translation table.
  • Paul will make sure that the monitoring_hists plugin plots (for reconstruction quality) are integrated and viewable in RootSpy.

Offline Data Monitoring

What we want to know

  • Can we reproduce the online histograms offline:
    • Can we read data from tape?
    • Are all the channels present?
    • Are all the recorded values within the expected ranges?
  • How do the online histograms change as a function of time (e.g. hits/event vs run number)?
  • Calibration status/quality histograms (individual runs & a function of time)
  • Reconstruction status/quality histograms
    • Num tracks, showers, hits, etc.
    • Reconstructed track kinematics
    • Track / detector-hit matching
    • PID Quality
  • Analysis histograms
    • Can we see pi0 peaks
    • Can we see simple final states
      • γ p → p π+ π-
      • γ p → p π+ π- π0
      • γ p → p π+ π- η

Action Items - Now (With what is available)

  • Paul: Will contact David & Mark and find locations to host the websites, mysql database, and ROOT files.
  • Paul: Will contact David and make sure there is an easy way to access the configuration used for a given run (which detectors were connected/on/off, trigger configuration, did the DAQ crash or did the data get written to disk).
  • Paul, Sean: Make sure that the detector groups make plugins for doing the calibrations and viewing the quality/results.
  • Kei: Make cron-job scripts that automatically read EVIO data from tape, run the online/monitoring plugins, and save the ROOT files to disk.
    • Raw data (hit occupancies) need to only be read once per run, calibration/reconstruction data need to be rerun and updated periodically.
  • Sean: Make cron-job scripts that read the data from the ROOT files and:
    • Store appropriate data in an sqlite database (e.g. hit rates, # events, # reconstructed tracks/showers, calibration resolutions) as a function of run number.
    • Make png(?) files of important histograms on disk (hit occupancies, calibration plots, reconstructed track p vs. θ, etc.)
  • Kei/Justin: Make cron-job scripts that create a version-0, "just get something working" webpage for viewing:
    • Important histograms for each detector (png's created by Sean)
    • (w/ Sean's help for database access): Values in the mysql database as a function of run number.

Action Items - Near Future

  • Kei, Sean: As more detector plugins become available, expand the system for viewing their data.
  • Justin: Make a more sophisticated/elaborate series of scripts & webpages for viewing:
    • Today's & yesterday's online data
    • Primary plots from any given run (each detector system: hit occupancies, calibration quality, reconstruction counts)
    • Performance as a function of time (each detector system: hit occupancies, calibration quality, reconstruction counts)
    • Links to run configuration information

Example Links

Miscellaneous Useful Information

Commissioning Software

  • HDDS Commissioning Branch Link
  • sim-recon Commissioning Branch Link

MC-to-EVIO

  • How to: Link
    • If you are running over EVIO files created from simulated data, you need to run with the environment variable:
JANA_CALIB_CONTEXT="variation=mc"

Running with B-field = 0

  • For the simulation, in the control.in file, set:
BFIELDTYPE 'NoField'
  • For the reconstruction, set B=(0,0,0) everywhere by running with the command line argument:
-PBFIELD_TYPE=NoField.

Miscellaneous

  • Accessing Onsite Webpages from Offsite Link
  • RootSpy Website
  • RootSpy Source Code Link