Difference between revisions of "Run Planning Meeting Notes, October 20 - October 26, 2022"

From GlueXWiki
Jump to: navigation, search
(→‎Shift Summary)
(→‎Past 24 hours)
 
(15 intermediate revisions by the same user not shown)
Line 9: Line 9:
 
Back to [[Run Coordination Meetings:Fall2022]]
 
Back to [[Run Coordination Meetings:Fall2022]]
  
−
=Thursday, October 21=
+
=Wednesday, October 26=
 
=== Past 24 hours ===
 
=== Past 24 hours ===
−
*Continued taking data at 200nA with EMPTY Target Cell till ~14:00, collected ~0.65B events, overall collected data for past period 1.533B events
+
*~0.42B out of ~2B triggers collected on EMPTY target at 200nA CW since yesterday morning ~9:30am
 +
*At morning beam was unstable, then afternoon it become more stabile
 +
*At around 3:20am VTP5H00SPD vacuum pump tripped, on call expert contacted, ~4:00am entered to Tagger Vault with the MCC/ARM, rebooted the pump
 +
*The main trigger was not recorded on EPICS, Sergey contacted, to fix it, but Sasha needs to take a look on it
 +
*Thursday's RF recovery for ~4hrs (MCC will try to keep it strictly for 4hrs)
 +
*Justin Stevens will take over at 1:30pm
 +
 
 +
=== Shift Summary ===
 +
*[https://logbooks.jlab.org/entry/4068594 Day Oct-24]
 +
*[https://logbooks.jlab.org/entry/4068905 Swing Oct-24]
 +
*[https://logbooks.jlab.org/entry/4069310 Owl Oct-25/26]
 +
*[https://logbooks.jlab.org/entry/4069558 Early Riser Oct-25]
 +
 
 +
=== Issues ===
 +
*Numerous issues with RootSpy crashes/freezes
 +
*Alex killed all the running RootSpy process https://logbooks.jlab.org/entry/4068653
 +
*Issues with trigger scalers, Sergey needed to reboot DAQ couple of times
 +
*There is one FCAL channel dropping HV -> https://logbooks.jlab.org/entry/4069105
 +
*CSS workspace was not starting with 'css_new' command, Sergey and Hovanes contacted
 +
*VTP5H00SPD vacuum pump tripped, fixed ~4:00am 
 +
 
 +
=== Plan ===
 +
*Continue taking production data at 200nA on EMPTY target till "RF recovery day"
 +
 
 +
=Tuesday, October 25=
 +
=== Past 24 hours ===
 +
*~2B triggers collected on He target at 200nA CW since yesterday update, overall after filling target collected ~13.7B events
 +
*Mostly beam was very stabile
 +
*Tim found some correlation between atmospheric pressure and NMR signal https://logbooks.jlab.org/entry/4067493
 +
*Sergey did the BCAL baseline calibration
 +
*~2hrs beam was down this morning due to blown fuse in injector site
 +
*Thursday During Day shift planned RF recovery for ~4hrs
 +
 
 +
=== Shift Summary ===
 +
*[https://logbooks.jlab.org/entry/4067515 Day Oct-24]
 +
*[https://logbooks.jlab.org/entry/4067740 Swing Oct-24]
 +
*[https://logbooks.jlab.org/entry/4068147 Owl Oct-24/25]
 +
*[https://logbooks.jlab.org/entry/4068334 Early Riser Oct-25]
 +
 
 +
=== Issues ===
 +
*ROCBCAL10 was unresponsive, Needed to restart DAQ (during Swing)
 +
*RootSpy crashed couple times
 +
 
 +
=== Plan ===
 +
*Empty the Target and take production data at 200nA
 +
 
 +
=Monday, October 24=
 +
=== Past 24 hours ===
 +
*~1.7B triggers collected on He target at 200nA CW since yesterday update, overall after filling target collected ~11B events
 +
*Mostly beam was very stabile over the weekend
 +
*Monday early morning ~2:30am beam goes OFF, the reason was injector slit water flow interlock issue, at around 7am it was recovered and MCC started beam delivery
 +
*There was one FCAL base which is locked-up, shift crew reset it -> https://logbooks.jlab.org/entry/4067100
 +
*BCAL FADC occupancy plots -> https://logbooks.jlab.org/entry/4067264
 +
*No planned "Maintenance day" yet
 +
 
 +
=== Shift Summary ===
 +
 
 +
*[https://logbooks.jlab.org/entry/4066750 Day Oct-23]
 +
*[https://logbooks.jlab.org/entry/4066883 Swing Oct-23]
 +
*[https://logbooks.jlab.org/entry/4067176 Owl Oct-23/23]
 +
*[https://logbooks.jlab.org/entry/4067369 Early Riser Oct-24]
 +
 
 +
=== Issues ===
 +
*https://cebaf.jlab.org/dtm/open-events?event_id=17195
 +
 
 +
=== Plan ===
 +
*Continue taking production data at 200nA with FULL TGT.
 +
 
 +
 
 +
=Sunday, October 23=
 +
=== Past 24 hours ===
 +
*~2.8B triggers collected on He target at 200nA CW since yesterday update (~11am)
 +
*Asked MCC for Rapid Access to change the broken FCal fADC (~1hrs down time)
 +
*Mostly beam was very stabile(Hall-A is not in business), except starting at ~5:30am-6:00am MCC is having some instabilities with 1L12 HV which is causing the instabilities for Halls D and C, but it is not a major issue for now, and MCC will continue beam delivery to halls
 +
 
 +
=== Shift Summary ===
 +
 
 +
*[https://logbooks.jlab.org/entry/4066120 Day Oct-22]
 +
*[https://logbooks.jlab.org/entry/4066343 Swing Oct-22]
 +
*[https://logbooks.jlab.org/entry/4066506 Owl Oct-22/23]
 +
*[https://logbooks.jlab.org/entry/4066646 Early Riser Oct-23]
 +
*[https://logbooks.jlab.org/entry/4066750 Day Oct-23]
 +
 
 +
=== Issues ===
 +
*https://logbooks.jlab.org/entry/4066264 -> FCal fADC replacement
 +
 
 +
=== Plan ===
 +
*Continue taking production data at 200nA with FULL TGT.
 +
 
 +
=Saturday, October 22=
 +
=== Past 24 hours ===
 +
* past 24 hours we collected ~3.43B total triggers on the He4 target with 200nA beam.
 +
* Starting from run_num 111140 one of FCal fADC is broken
 +
* Beam was OFF from 16:37 till 19:58 issues with RF 1L10 HV
 +
* After Beam is back continued taking data at 200nA FULL target
 +
* ~00:15 DAQ rate dropped to 0kHz, Contacted to Sergey, 1st power cycled rocTOF2, which brings trigger rate to 12.5kHz, but still way far from nominal 50kHz
 +
* It turns out one of FCAL outer ring fADC is broken (trying to reach expert to replace it), will continue running with this condition until it's been replaced
 +
 
 +
=== Shift Summarys ===
 +
* [https://logbooks.jlab.org/entry/4064773 Early Riser Oct-21]
 +
* [https://logbooks.jlab.org/entry/4065074 Day Oct-21]
 +
* [https://logbooks.jlab.org/entry/4065456 Swing Oct-21]
 +
* [https://logbooks.jlab.org/entry/4065653 Owl Oct-21/22]
 +
* [https://logbooks.jlab.org/entry/4065892 Early Riser Oct-22]
 +
 
 +
=== Issues ===
 +
* https://logbooks.jlab.org/entry/4065801 ->FCAL fADC broken
 +
* https://logbooks.jlab.org/entry/4066109 ->Gloun-03 reboot
 +
 
 +
=== Plan ===
 +
*Continue taking production data at 200nA with FULL LHe target over the weekend.
 +
 
 +
 
 +
=Friday, October 21=
 +
=== Past 24 hours ===
 +
*Continued taking data at 200nA with EMPTY Target Cell till ~15:00, collected ~0.65B events, overall collected data for past period 1.533B events
 
*15:00 Beam was OFF for ~2.5 hours for Injector Restoration
 
*15:00 Beam was OFF for ~2.5 hours for Injector Restoration
 
*At 16:10 started filling the target with LHe   
 
*At 16:10 started filling the target with LHe   
Line 19: Line 134:
 
*Same Target Issue happened at 20:50 again, switched back ON the PTR, and in both cases contacted to Chris Carlin, he will keep an eye on it
 
*Same Target Issue happened at 20:50 again, switched back ON the PTR, and in both cases contacted to Chris Carlin, he will keep an eye on it
 
*22:10 asked for Rapid Access to let Chris Carlin take a look on the heater/cooler control box
 
*22:10 asked for Rapid Access to let Chris Carlin take a look on the heater/cooler control box
 +
*The CDC Pressure Alarm is coming ON more frequently than it was
 +
*Collected data for LHe ~1.9B events
  
 
=== Shift Summary ===
 
=== Shift Summary ===
Line 30: Line 147:
 
*https://logbooks.jlab.org/entry/4064492
 
*https://logbooks.jlab.org/entry/4064492
 
*https://logbooks.jlab.org/entry/4064536
 
*https://logbooks.jlab.org/entry/4064536
 +
*https://logbooks.jlab.org/entry/4064580 (Chris comment)
 +
*https://logbooks.jlab.org/entry/4064899 (Alexander comment)
 +
 
=== Plan ===
 
=== Plan ===
 
Continue running on the FULL LHe target at 200nA
 
Continue running on the FULL LHe target at 200nA
  
−
=Wednesday, October 20=
+
=Thursday, October 20=
 
=== Past 24 hours ===
 
=== Past 24 hours ===
 
*Beam was Off most of the day starting from ~7:30am for "ARW Tour and Associated Maintenance" and VCG1L1210(SRF Insulating Vacuum)
 
*Beam was Off most of the day starting from ~7:30am for "ARW Tour and Associated Maintenance" and VCG1L1210(SRF Insulating Vacuum)
Line 43: Line 163:
 
*Collected for about 0.3B events
 
*Collected for about 0.3B events
 
*At 7:45 meeting informed MCC about beam position instability   
 
*At 7:45 meeting informed MCC about beam position instability   
−
 
  
 
=== Shift Summary ===
 
=== Shift Summary ===

Latest revision as of 08:44, 26 October 2022

Meetings are via ZoomGov: connect (Meeting ID: 160 199 0060 / Passcode 8394)

MCC whiteboard

Hall D counting room white board

Hall D Logbook

Back to Run Coordination Meetings:Fall2022

Wednesday, October 26

Past 24 hours

  • ~0.42B out of ~2B triggers collected on EMPTY target at 200nA CW since yesterday morning ~9:30am
  • At morning beam was unstable, then afternoon it become more stabile
  • At around 3:20am VTP5H00SPD vacuum pump tripped, on call expert contacted, ~4:00am entered to Tagger Vault with the MCC/ARM, rebooted the pump
  • The main trigger was not recorded on EPICS, Sergey contacted, to fix it, but Sasha needs to take a look on it
  • Thursday's RF recovery for ~4hrs (MCC will try to keep it strictly for 4hrs)
  • Justin Stevens will take over at 1:30pm

Shift Summary

Issues

  • Numerous issues with RootSpy crashes/freezes
  • Alex killed all the running RootSpy process https://logbooks.jlab.org/entry/4068653
  • Issues with trigger scalers, Sergey needed to reboot DAQ couple of times
  • There is one FCAL channel dropping HV -> https://logbooks.jlab.org/entry/4069105
  • CSS workspace was not starting with 'css_new' command, Sergey and Hovanes contacted
  • VTP5H00SPD vacuum pump tripped, fixed ~4:00am

Plan

  • Continue taking production data at 200nA on EMPTY target till "RF recovery day"

Tuesday, October 25

Past 24 hours

  • ~2B triggers collected on He target at 200nA CW since yesterday update, overall after filling target collected ~13.7B events
  • Mostly beam was very stabile
  • Tim found some correlation between atmospheric pressure and NMR signal https://logbooks.jlab.org/entry/4067493
  • Sergey did the BCAL baseline calibration
  • ~2hrs beam was down this morning due to blown fuse in injector site
  • Thursday During Day shift planned RF recovery for ~4hrs

Shift Summary

Issues

  • ROCBCAL10 was unresponsive, Needed to restart DAQ (during Swing)
  • RootSpy crashed couple times

Plan

  • Empty the Target and take production data at 200nA

Monday, October 24

Past 24 hours

  • ~1.7B triggers collected on He target at 200nA CW since yesterday update, overall after filling target collected ~11B events
  • Mostly beam was very stabile over the weekend
  • Monday early morning ~2:30am beam goes OFF, the reason was injector slit water flow interlock issue, at around 7am it was recovered and MCC started beam delivery
  • There was one FCAL base which is locked-up, shift crew reset it -> https://logbooks.jlab.org/entry/4067100
  • BCAL FADC occupancy plots -> https://logbooks.jlab.org/entry/4067264
  • No planned "Maintenance day" yet

Shift Summary

Issues

Plan

  • Continue taking production data at 200nA with FULL TGT.


Sunday, October 23

Past 24 hours

  • ~2.8B triggers collected on He target at 200nA CW since yesterday update (~11am)
  • Asked MCC for Rapid Access to change the broken FCal fADC (~1hrs down time)
  • Mostly beam was very stabile(Hall-A is not in business), except starting at ~5:30am-6:00am MCC is having some instabilities with 1L12 HV which is causing the instabilities for Halls D and C, but it is not a major issue for now, and MCC will continue beam delivery to halls

Shift Summary

Issues

Plan

  • Continue taking production data at 200nA with FULL TGT.

Saturday, October 22

Past 24 hours

  • past 24 hours we collected ~3.43B total triggers on the He4 target with 200nA beam.
  • Starting from run_num 111140 one of FCal fADC is broken
  • Beam was OFF from 16:37 till 19:58 issues with RF 1L10 HV
  • After Beam is back continued taking data at 200nA FULL target
  • ~00:15 DAQ rate dropped to 0kHz, Contacted to Sergey, 1st power cycled rocTOF2, which brings trigger rate to 12.5kHz, but still way far from nominal 50kHz
  • It turns out one of FCAL outer ring fADC is broken (trying to reach expert to replace it), will continue running with this condition until it's been replaced

Shift Summarys

Issues

Plan

  • Continue taking production data at 200nA with FULL LHe target over the weekend.


Friday, October 21

Past 24 hours

  • Continued taking data at 200nA with EMPTY Target Cell till ~15:00, collected ~0.65B events, overall collected data for past period 1.533B events
  • 15:00 Beam was OFF for ~2.5 hours for Injector Restoration
  • At 16:10 started filling the target with LHe
  • ~17:40 Beam was back, started collecting data with 200nA and FULL target
  • 17:58 FDC:hv:2:c3:3n channel HV tripped, tried to turn it ON, with different ways, but continuously it was tripping, contacted to Lubomir he changed the HV from 500V to 400V https://logbooks.jlab.org/entry/4064442
  • 19:20 Target alarm comes ON, the Pulse Tube Refrigerator(PTR) was switched off resulting in increase of the target cell temperatures from 4.27 to about 4.6 K
  • Same Target Issue happened at 20:50 again, switched back ON the PTR, and in both cases contacted to Chris Carlin, he will keep an eye on it
  • 22:10 asked for Rapid Access to let Chris Carlin take a look on the heater/cooler control box
  • The CDC Pressure Alarm is coming ON more frequently than it was
  • Collected data for LHe ~1.9B events

Shift Summary

Issues

Plan

Continue running on the FULL LHe target at 200nA

Thursday, October 20

Past 24 hours

  • Beam was Off most of the day starting from ~7:30am for "ARW Tour and Associated Maintenance" and VCG1L1210(SRF Insulating Vacuum)
    • During dawn time Mark and Malte replaced some Fcal bases (2?)
    • Vlad and Arshak did some alignment on the prototype
    • Kondo conducted some work on GEM-TRD
  • Restored Beam to the Hall-D at ~22:30, after 15 hours of downtime
  • After Beam restoration Harp Scan was performed and started run on the Empty LHe4 Cell target with 200nA
  • Collected for about 0.3B events
  • At 7:45 meeting informed MCC about beam position instability

Shift Summary

Issues

Plan

Continue running on the Empty target at 200nA

Retrieved from "https://halldweb1.jlab.org/wiki/index.php?title=Run_Planning_Meeting_Notes,_October_20_-_October_26,_2022&oldid=116769"