HOWTO use AmpTools on the JLab farm with MPI

Load MPI module

On the ifarm you can load the MPI module with

module load mpi/openmpi-4.0.1

providing the binaries below to compile (mpicxx) and run (mpirun) MPI commands

which mpicxx
which mpirun

AmpTools Compilation MPI

This example was done in csh on ifarm1901

1) Download latest AmpTools release

git clone git@github.com:mashephe/AmpTools.git

2) Set AMPTOOLS directory

setenv AMPTOOLS_HOME $PWD/AmpTools/
setenv AMPTOOLS $AMPTOOLS_HOME/AmpTools/

3) Put root-config in your path (assumes ROOTSYS set by some other setup script)

setenv PATH $ROOTSYS/bin:$PATH

4) Build main AmpTools library with MPI support (temporary branch to support openmpi version 4 on ifarm)

cd $AMPTOOLS/AmpTools
git checkout jrs_fixamp
make MPI=1

Fitter Compilation with MPI

The only MPI dependent part of halld_sim is fitMPI.cc is an optional build for MPI fits, analogous to the usual fit.cc without MPI. You can build the fitMPI executable with the following commands (requires git pull of the halld_sim master branch after 1/13/22)

cd $HALLD_SIM_HOME/src/programs/AmplitudeAnalysis/fitMPI/
scons -u install

Performing Fits Interactively

The fitMPI executable is run with mpirun

mpirun N fitMPI -c YOURCONFIG.cfg

where N is the number of parallel processes to use in the fit and YOURCONFIG.cfg is your usual config file. Note: additional command line parameters can be used as well, as needed.