Locking in JANA

General Comments

• Be careful when creating / filling histograms & trees.
• Be careful when modifying / reading JEventProcessor class member data.
• Be careful when returning early from functions: Make sure you release the lock first.
• Avoid nested locks.

Basics: Factories & Plugin Processors

• Plugin Processors: For each plugin, there is a total of one instance of the processor object, regardless of how many threads you run with.
  • Thus, since the same object is shared amongst all threads, you must be careful about modifying it.
• Factories: Each thread has it's own instance of each factory.
  • Thus, you can easily modify them without worrying about interfering with other threads.
  • However, sharing/modifying objects for a factory between the threads is difficult, and you must be careful about doing so.

(Plugin) Processor Variables

Basics

• Do not place use any locks in JEventProcessor::init() or ::fini(): they are unnecessary, and a waste of time.
  • JANA is guaranteed to be single-threaded in these functions.

• Do not use or modify any global variables, unless you are in a lock.
  • Or even better, just avoid them at all possible. It's terrible programming practice.

Class members

• Do NOT use or modify any class member-variables that you define for your JEventProcessor (i.e. in header file), unless you are in a lock (or in init(), fini()). These variables can be modified by other threads. This locking can be done via:

LockState(); //ACQUIRE PROCESSOR LOCK
UnlockState(); //RELEASE PROCESSOR LOCK

• For data updated in brun() and read in evnt(), it is sometimes unfeasible to work around this, such as when working with calibration constants. As long as the run number doesn't change, this should still be OK (they should just be overwriting the same value).
  • However, running your program over data with two different runs may result in unpredictable behavior.

• Just to be safe, have as few class member variables as possible, and reduce the amount of time spent in locks. In other words, use function-scope (local) variables instead.
  • Where possible of course. E.g. you can't avoid it with histograms, but you need to lock to fill those anyway.

Creating Histograms or TTrees in a plugin processor

• If you are creating histograms or trees (outside of JEventProcessor::init() or ::fini()): Use the global ROOT JANA lock functions below.
  • This is because they depend on gDirectory being constant.

japp->RootWriteLock(); //ACQUIRE ROOT LOCK
japp->RootUnLock(); //RELEASE ROOT LOCK

Filling Histograms

• If you are filling histograms, you do NOT need to grab the global ROOT JANA lock functions. Try to instead grab a lock with a smaller scope, so that other threads can write to other histograms that don't interfere with yours.
  • In you are directly in the plugin processor, there's a special function to do this, that locks for the current plugin only. It is:

japp->RootFillLock(this); //ACQUIRE ROOT FILL LOCK //this: JEventProcessor pointer
japp->RootFillUnLock(this); //RELEASE ROOT FILL LOCK //this: JEventProcessor pointer

Filling TTrees

hd_root.root

• If you are filling TTrees to the global ROOT file (i.e. hd_root.root), write to the TTrees with the global ROOT JANA lock functions below:
  • This is because TTree::Fill() periodically writes to the file, modifying it.

japp->RootWriteLock(); //ACQUIRE ROOT LOCK
japp->RootUnLock(); //RELEASE ROOT LOCK

Other Output Files

• If you are filling TTrees to a local ROOT file (i.e. NOT hd_root.root), you do NOT need to grab the global ROOT JANA lock functions. Try to instead grab a lock with a smaller scope, so that other threads can write to other trees that don't interfere with yours.
  • Grab the same for every tree that is written to the same output file. This is because TTree::Fill() periodically writes to the file, modifying it.
  • This can be done with:

japp->WriteLock("MyFileName"); //ACQUIRE FILE LOCK
japp->Unlock("MyFileName"); //RELEASE FILE LOCK

Analysis Actions

• Just use the pre-generated code and it will work.
  • Unless you're creating TTrees and not saving them into a unique file. In which case, follow the instructions in the appropriate sections.

• The pre-generated code acquires a action-local lock, such that you can fill the histograms in the current action without interfering with threads filling in other actions.

Nested Locks

Overview

• Don't EVER acquire a lock from within another lock. This is extremely dangerous, and can easily result in deadlock: program hangs, because two threads are waiting for each other to unlock.
  • If you think you need to do a nested lock, you are probably doing it wrong.
  • If you absolutely must have a nested lock, be sure to acquire & release the locks in the exact same order everywhere. This is of course, very difficult to guarantee, because once someone starts doing it one plugin, someone else may have a different convention in another plugin ...

Avoiding Nested Locks

• To avoid nested locking, when you're in a lock:
  • Never query JANA for calibration constants
  • Never query JANA for global parameters
  • Never query JANA for objects (JEventLoop::Get()) //This is also potentially incredibly slow
  • Never call the event writers to save your EVIO skims, REST files, ROOT trees, etc.
  • Never call any function that you don't understand exactly what it's doing. It might be acquiring a lock underneath.