Difference between revisions of "Run Planning Meeting Notes, October 20 - October 26, 2022"
From GlueXWiki
(→Past 24 hours) |
(→Past 24 hours) |
||
(14 intermediate revisions by the same user not shown) | |||
Line 9: | Line 9: | ||
Back to [[Run Coordination Meetings:Fall2022]] | Back to [[Run Coordination Meetings:Fall2022]] | ||
− | = | + | =Wednesday, October 26= |
=== Past 24 hours === | === Past 24 hours === | ||
− | *Continued taking data at 200nA with EMPTY Target Cell till ~ | + | *~0.42B out of ~2B triggers collected on EMPTY target at 200nA CW since yesterday morning ~9:30am |
+ | *At morning beam was unstable, then afternoon it become more stabile | ||
+ | *At around 3:20am VTP5H00SPD vacuum pump tripped, on call expert contacted, ~4:00am entered to Tagger Vault with the MCC/ARM, rebooted the pump | ||
+ | *The main trigger was not recorded on EPICS, Sergey contacted, to fix it, but Sasha needs to take a look on it | ||
+ | *Thursday's RF recovery for ~4hrs (MCC will try to keep it strictly for 4hrs) | ||
+ | *Justin Stevens will take over at 1:30pm | ||
+ | |||
+ | === Shift Summary === | ||
+ | *[https://logbooks.jlab.org/entry/4068594 Day Oct-24] | ||
+ | *[https://logbooks.jlab.org/entry/4068905 Swing Oct-24] | ||
+ | *[https://logbooks.jlab.org/entry/4069310 Owl Oct-25/26] | ||
+ | *[https://logbooks.jlab.org/entry/4069558 Early Riser Oct-25] | ||
+ | |||
+ | === Issues === | ||
+ | *Numerous issues with RootSpy crashes/freezes | ||
+ | *Alex killed all the running RootSpy process https://logbooks.jlab.org/entry/4068653 | ||
+ | *Issues with trigger scalers, Sergey needed to reboot DAQ couple of times | ||
+ | *There is one FCAL channel dropping HV -> https://logbooks.jlab.org/entry/4069105 | ||
+ | *CSS workspace was not starting with 'css_new' command, Sergey and Hovanes contacted | ||
+ | *VTP5H00SPD vacuum pump tripped, fixed ~4:00am | ||
+ | |||
+ | === Plan === | ||
+ | *Continue taking production data at 200nA on EMPTY target till "RF recovery day" | ||
+ | |||
+ | =Tuesday, October 25= | ||
+ | === Past 24 hours === | ||
+ | *~2B triggers collected on He target at 200nA CW since yesterday update, overall after filling target collected ~13.7B events | ||
+ | *Mostly beam was very stabile | ||
+ | *Tim found some correlation between atmospheric pressure and NMR signal https://logbooks.jlab.org/entry/4067493 | ||
+ | *Sergey did the BCAL baseline calibration | ||
+ | *~2hrs beam was down this morning due to blown fuse in injector site | ||
+ | *Thursday During Day shift planned RF recovery for ~4hrs | ||
+ | |||
+ | === Shift Summary === | ||
+ | *[https://logbooks.jlab.org/entry/4067515 Day Oct-24] | ||
+ | *[https://logbooks.jlab.org/entry/4067740 Swing Oct-24] | ||
+ | *[https://logbooks.jlab.org/entry/4068147 Owl Oct-24/25] | ||
+ | *[https://logbooks.jlab.org/entry/4068334 Early Riser Oct-25] | ||
+ | |||
+ | === Issues === | ||
+ | *ROCBCAL10 was unresponsive, Needed to restart DAQ (during Swing) | ||
+ | *RootSpy crashed couple times | ||
+ | |||
+ | === Plan === | ||
+ | *Empty the Target and take production data at 200nA | ||
+ | |||
+ | =Monday, October 24= | ||
+ | === Past 24 hours === | ||
+ | *~1.7B triggers collected on He target at 200nA CW since yesterday update, overall after filling target collected ~11B events | ||
+ | *Mostly beam was very stabile over the weekend | ||
+ | *Monday early morning ~2:30am beam goes OFF, the reason was injector slit water flow interlock issue, at around 7am it was recovered and MCC started beam delivery | ||
+ | *There was one FCAL base which is locked-up, shift crew reset it -> https://logbooks.jlab.org/entry/4067100 | ||
+ | *BCAL FADC occupancy plots -> https://logbooks.jlab.org/entry/4067264 | ||
+ | *No planned "Maintenance day" yet | ||
+ | |||
+ | === Shift Summary === | ||
+ | |||
+ | *[https://logbooks.jlab.org/entry/4066750 Day Oct-23] | ||
+ | *[https://logbooks.jlab.org/entry/4066883 Swing Oct-23] | ||
+ | *[https://logbooks.jlab.org/entry/4067176 Owl Oct-23/23] | ||
+ | *[https://logbooks.jlab.org/entry/4067369 Early Riser Oct-24] | ||
+ | |||
+ | === Issues === | ||
+ | *https://cebaf.jlab.org/dtm/open-events?event_id=17195 | ||
+ | |||
+ | === Plan === | ||
+ | *Continue taking production data at 200nA with FULL TGT. | ||
+ | |||
+ | |||
+ | =Sunday, October 23= | ||
+ | === Past 24 hours === | ||
+ | *~2.8B triggers collected on He target at 200nA CW since yesterday update (~11am) | ||
+ | *Asked MCC for Rapid Access to change the broken FCal fADC (~1hrs down time) | ||
+ | *Mostly beam was very stabile(Hall-A is not in business), except starting at ~5:30am-6:00am MCC is having some instabilities with 1L12 HV which is causing the instabilities for Halls D and C, but it is not a major issue for now, and MCC will continue beam delivery to halls | ||
+ | |||
+ | === Shift Summary === | ||
+ | |||
+ | *[https://logbooks.jlab.org/entry/4066120 Day Oct-22] | ||
+ | *[https://logbooks.jlab.org/entry/4066343 Swing Oct-22] | ||
+ | *[https://logbooks.jlab.org/entry/4066506 Owl Oct-22/23] | ||
+ | *[https://logbooks.jlab.org/entry/4066646 Early Riser Oct-23] | ||
+ | *[https://logbooks.jlab.org/entry/4066750 Day Oct-23] | ||
+ | |||
+ | === Issues === | ||
+ | *https://logbooks.jlab.org/entry/4066264 -> FCal fADC replacement | ||
+ | |||
+ | === Plan === | ||
+ | *Continue taking production data at 200nA with FULL TGT. | ||
+ | |||
+ | =Saturday, October 22= | ||
+ | === Past 24 hours === | ||
+ | * past 24 hours we collected ~3.43B total triggers on the He4 target with 200nA beam. | ||
+ | * Starting from run_num 111140 one of FCal fADC is broken | ||
+ | * Beam was OFF from 16:37 till 19:58 issues with RF 1L10 HV | ||
+ | * After Beam is back continued taking data at 200nA FULL target | ||
+ | * ~00:15 DAQ rate dropped to 0kHz, Contacted to Sergey, 1st power cycled rocTOF2, which brings trigger rate to 12.5kHz, but still way far from nominal 50kHz | ||
+ | * It turns out one of FCAL outer ring fADC is broken (trying to reach expert to replace it), will continue running with this condition until it's been replaced | ||
+ | |||
+ | === Shift Summarys === | ||
+ | * [https://logbooks.jlab.org/entry/4064773 Early Riser Oct-21] | ||
+ | * [https://logbooks.jlab.org/entry/4065074 Day Oct-21] | ||
+ | * [https://logbooks.jlab.org/entry/4065456 Swing Oct-21] | ||
+ | * [https://logbooks.jlab.org/entry/4065653 Owl Oct-21/22] | ||
+ | * [https://logbooks.jlab.org/entry/4065892 Early Riser Oct-22] | ||
+ | |||
+ | === Issues === | ||
+ | * https://logbooks.jlab.org/entry/4065801 ->FCAL fADC broken | ||
+ | * https://logbooks.jlab.org/entry/4066109 ->Gloun-03 reboot | ||
+ | |||
+ | === Plan === | ||
+ | *Continue taking production data at 200nA with FULL LHe target over the weekend. | ||
+ | |||
+ | |||
+ | =Friday, October 21= | ||
+ | === Past 24 hours === | ||
+ | *Continued taking data at 200nA with EMPTY Target Cell till ~15:00, collected ~0.65B events, overall collected data for past period 1.533B events | ||
*15:00 Beam was OFF for ~2.5 hours for Injector Restoration | *15:00 Beam was OFF for ~2.5 hours for Injector Restoration | ||
*At 16:10 started filling the target with LHe | *At 16:10 started filling the target with LHe | ||
Line 19: | Line 134: | ||
*Same Target Issue happened at 20:50 again, switched back ON the PTR, and in both cases contacted to Chris Carlin, he will keep an eye on it | *Same Target Issue happened at 20:50 again, switched back ON the PTR, and in both cases contacted to Chris Carlin, he will keep an eye on it | ||
*22:10 asked for Rapid Access to let Chris Carlin take a look on the heater/cooler control box | *22:10 asked for Rapid Access to let Chris Carlin take a look on the heater/cooler control box | ||
− | *Collected data for LHe ~1. | + | *The CDC Pressure Alarm is coming ON more frequently than it was |
+ | *Collected data for LHe ~1.9B events | ||
=== Shift Summary === | === Shift Summary === | ||
Line 31: | Line 147: | ||
*https://logbooks.jlab.org/entry/4064492 | *https://logbooks.jlab.org/entry/4064492 | ||
*https://logbooks.jlab.org/entry/4064536 | *https://logbooks.jlab.org/entry/4064536 | ||
+ | *https://logbooks.jlab.org/entry/4064580 (Chris comment) | ||
+ | *https://logbooks.jlab.org/entry/4064899 (Alexander comment) | ||
+ | |||
=== Plan === | === Plan === | ||
Continue running on the FULL LHe target at 200nA | Continue running on the FULL LHe target at 200nA | ||
− | = | + | =Thursday, October 20= |
=== Past 24 hours === | === Past 24 hours === | ||
*Beam was Off most of the day starting from ~7:30am for "ARW Tour and Associated Maintenance" and VCG1L1210(SRF Insulating Vacuum) | *Beam was Off most of the day starting from ~7:30am for "ARW Tour and Associated Maintenance" and VCG1L1210(SRF Insulating Vacuum) | ||
Line 44: | Line 163: | ||
*Collected for about 0.3B events | *Collected for about 0.3B events | ||
*At 7:45 meeting informed MCC about beam position instability | *At 7:45 meeting informed MCC about beam position instability | ||
− | |||
=== Shift Summary === | === Shift Summary === |
Latest revision as of 08:44, 26 October 2022
Meetings are via ZoomGov: connect (Meeting ID: 160 199 0060 / Passcode 8394)
Hall D counting room white board
Back to Run Coordination Meetings:Fall2022
Contents
Wednesday, October 26
Past 24 hours
- ~0.42B out of ~2B triggers collected on EMPTY target at 200nA CW since yesterday morning ~9:30am
- At morning beam was unstable, then afternoon it become more stabile
- At around 3:20am VTP5H00SPD vacuum pump tripped, on call expert contacted, ~4:00am entered to Tagger Vault with the MCC/ARM, rebooted the pump
- The main trigger was not recorded on EPICS, Sergey contacted, to fix it, but Sasha needs to take a look on it
- Thursday's RF recovery for ~4hrs (MCC will try to keep it strictly for 4hrs)
- Justin Stevens will take over at 1:30pm
Shift Summary
Issues
- Numerous issues with RootSpy crashes/freezes
- Alex killed all the running RootSpy process https://logbooks.jlab.org/entry/4068653
- Issues with trigger scalers, Sergey needed to reboot DAQ couple of times
- There is one FCAL channel dropping HV -> https://logbooks.jlab.org/entry/4069105
- CSS workspace was not starting with 'css_new' command, Sergey and Hovanes contacted
- VTP5H00SPD vacuum pump tripped, fixed ~4:00am
Plan
- Continue taking production data at 200nA on EMPTY target till "RF recovery day"
Tuesday, October 25
Past 24 hours
- ~2B triggers collected on He target at 200nA CW since yesterday update, overall after filling target collected ~13.7B events
- Mostly beam was very stabile
- Tim found some correlation between atmospheric pressure and NMR signal https://logbooks.jlab.org/entry/4067493
- Sergey did the BCAL baseline calibration
- ~2hrs beam was down this morning due to blown fuse in injector site
- Thursday During Day shift planned RF recovery for ~4hrs
Shift Summary
Issues
- ROCBCAL10 was unresponsive, Needed to restart DAQ (during Swing)
- RootSpy crashed couple times
Plan
- Empty the Target and take production data at 200nA
Monday, October 24
Past 24 hours
- ~1.7B triggers collected on He target at 200nA CW since yesterday update, overall after filling target collected ~11B events
- Mostly beam was very stabile over the weekend
- Monday early morning ~2:30am beam goes OFF, the reason was injector slit water flow interlock issue, at around 7am it was recovered and MCC started beam delivery
- There was one FCAL base which is locked-up, shift crew reset it -> https://logbooks.jlab.org/entry/4067100
- BCAL FADC occupancy plots -> https://logbooks.jlab.org/entry/4067264
- No planned "Maintenance day" yet
Shift Summary
Issues
Plan
- Continue taking production data at 200nA with FULL TGT.
Sunday, October 23
Past 24 hours
- ~2.8B triggers collected on He target at 200nA CW since yesterday update (~11am)
- Asked MCC for Rapid Access to change the broken FCal fADC (~1hrs down time)
- Mostly beam was very stabile(Hall-A is not in business), except starting at ~5:30am-6:00am MCC is having some instabilities with 1L12 HV which is causing the instabilities for Halls D and C, but it is not a major issue for now, and MCC will continue beam delivery to halls
Shift Summary
Issues
- https://logbooks.jlab.org/entry/4066264 -> FCal fADC replacement
Plan
- Continue taking production data at 200nA with FULL TGT.
Saturday, October 22
Past 24 hours
- past 24 hours we collected ~3.43B total triggers on the He4 target with 200nA beam.
- Starting from run_num 111140 one of FCal fADC is broken
- Beam was OFF from 16:37 till 19:58 issues with RF 1L10 HV
- After Beam is back continued taking data at 200nA FULL target
- ~00:15 DAQ rate dropped to 0kHz, Contacted to Sergey, 1st power cycled rocTOF2, which brings trigger rate to 12.5kHz, but still way far from nominal 50kHz
- It turns out one of FCAL outer ring fADC is broken (trying to reach expert to replace it), will continue running with this condition until it's been replaced
Shift Summarys
Issues
- https://logbooks.jlab.org/entry/4065801 ->FCAL fADC broken
- https://logbooks.jlab.org/entry/4066109 ->Gloun-03 reboot
Plan
- Continue taking production data at 200nA with FULL LHe target over the weekend.
Friday, October 21
Past 24 hours
- Continued taking data at 200nA with EMPTY Target Cell till ~15:00, collected ~0.65B events, overall collected data for past period 1.533B events
- 15:00 Beam was OFF for ~2.5 hours for Injector Restoration
- At 16:10 started filling the target with LHe
- ~17:40 Beam was back, started collecting data with 200nA and FULL target
- 17:58 FDC:hv:2:c3:3n channel HV tripped, tried to turn it ON, with different ways, but continuously it was tripping, contacted to Lubomir he changed the HV from 500V to 400V https://logbooks.jlab.org/entry/4064442
- 19:20 Target alarm comes ON, the Pulse Tube Refrigerator(PTR) was switched off resulting in increase of the target cell temperatures from 4.27 to about 4.6 K
- Same Target Issue happened at 20:50 again, switched back ON the PTR, and in both cases contacted to Chris Carlin, he will keep an eye on it
- 22:10 asked for Rapid Access to let Chris Carlin take a look on the heater/cooler control box
- The CDC Pressure Alarm is coming ON more frequently than it was
- Collected data for LHe ~1.9B events
Shift Summary
Issues
- https://logbooks.jlab.org/entry/4064442
- https://logbooks.jlab.org/entry/4064492
- https://logbooks.jlab.org/entry/4064536
- https://logbooks.jlab.org/entry/4064580 (Chris comment)
- https://logbooks.jlab.org/entry/4064899 (Alexander comment)
Plan
Continue running on the FULL LHe target at 200nA
Thursday, October 20
Past 24 hours
- Beam was Off most of the day starting from ~7:30am for "ARW Tour and Associated Maintenance" and VCG1L1210(SRF Insulating Vacuum)
- During dawn time Mark and Malte replaced some Fcal bases (2?)
- Vlad and Arshak did some alignment on the prototype
- Kondo conducted some work on GEM-TRD
- Restored Beam to the Hall-D at ~22:30, after 15 hours of downtime
- After Beam restoration Harp Scan was performed and started run on the Empty LHe4 Cell target with 200nA
- Collected for about 0.3B events
- At 7:45 meeting informed MCC about beam position instability
Shift Summary
Issues
Plan
Continue running on the Empty target at 200nA