Difference between revisions of "Getting started with GlueX Software"

From GlueXWiki
Jump to: navigation, search
(Replaced svn+ssh method with file method)
 
(46 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 
This is derived from the original web document "Getting started with HDGeant"
 
This is derived from the original web document "Getting started with HDGeant"
  
D. Lawrence  11/11/04
+
D. Lawrence  11/11/04 <br>
 
+
D. Lawrence  4/6/06  Updated <br>
D. Lawrence  4/6/06  Updated
+
D. Lawrence  12/5/06 Converted to Wiki <br>
 
+
D. Lawrence  1/18/07 Finished Converting to Wiki <br>
D. Lawrence  12/5/06 Converted to Wiki
+
D. Lawrence  2/4/08 Tweaked for calibDB among others <br>
 
+
D. Lawrence 2/17/09 Overhauled to reflect current state of code <br>
D. Lawrence  1/18/07 Finished Converting to Wiki
+
M. Ito 1/5/10 Added HDDS to list of 3rd party software <br>
 
+
D. Lawrence 9/30/14 Replace "make" with "scons" <br>
D. Lawrence  2/4/08 Tweaked for calibDB among others
+
  
 
=Overview=
 
=Overview=
Line 21: Line 20:
 
# Getting and compiling the source code
 
# Getting and compiling the source code
 
# Configuring and running the ''hdgeant'' program
 
# Configuring and running the ''hdgeant'' program
# Creating a [http://www.jlab.org/JANA JANA]-based program for reconstruction and histogramming
+
# Running one of the Hall-D ''DANA''-based reconstruction programs (This could involve customizing the C++ program for your needs or using one of the pre-built plugins to create ROOT TTrees) 
# Running said JANA program and analyzing the results
+
 
 +
 
  
 
=Getting and compiling the source code=
 
=Getting and compiling the source code=
----------------------------------------
 
The Hall-D specific source code is currently being kept in a
 
Subversion repository
 
on the JLab CUE. You must have a CUE account and you must
 
belong to the "halld" unix group. (Contact the JLab computer
 
center for help with both of these.)
 
  
 +
== Required 3rd Party Packages ==
 
The source relies on
 
The source relies on
several software packages that were not written for Hall-D
+
several software packages that are not kept in the Hall-D
and are ''not'' kept in its repository. These must be downloaded
+
repository (with the exception of HDDS, which is kept there). These must be downloaded
and installed prior to compiling the Hall-D source. For convenience,
+
and installed prior to compiling the Hall-D source. Here is
some versions known to have compiled/linked are here.  
+
a list of what you will need.  
  
 
'''JANA'''
 
'''JANA'''
 
* [http://www.jlab.org/JANA/downloads.php JANA downloads]
 
* [http://www.jlab.org/JANA/downloads.php JANA downloads]
Make sure you set your JANA_HOME environment variable.
+
Make sure you set your JANA_HOME environment variable
 +
to point to the directory containing the ''bin'', ''lib'', and ''include''
 +
directories after building JANA.
 +
 
 +
<small>(Note that the source for JANA is maintained in an svn repository located here: [https://phys12svn.jlab.org/repos/JANA https://phys12svn.jlab.org/repos/JANA]
 +
with the tagged versions located here: [https://phys12svn.jlab.org/repos/tags https://phys12svn.jlab.org/repos/tags])</small>
  
You will also need to set your JANA_CALIB_URL environment variable as
 
is described below.
 
  
 
'''CERNLIB'''
 
'''CERNLIB'''
Line 51: Line 49:
  
 
Check [[CERNLIB_Installation | here]] for possible issues with CERNLIB installation.
 
Check [[CERNLIB_Installation | here]] for possible issues with CERNLIB installation.
 +
 +
If you don't already have it installed, you will also need Motif. You can download OpenMotif from [http://www.ist-inc.com/motif/index.html here] for many platforms (including Mac OS X).
 +
  
 
'''ROOT'''
 
'''ROOT'''
Line 61: Line 62:
  
  
'''Note''': You only need the XERCES packages if you're going to be modifying the  geometry or the data model.
 
  
 
'''XERCES'''
 
'''XERCES'''
 +
* [http://mirror.metrocast.net/apache//xerces/c/2/sources/xerces-c-src_2_8_0.tar.gz xerces-c-src_2_8_0.tar.gz]
 
* [http://www.jlab.org/Hall-D/software/xerces-c-src_2_7_0.tar.gz xerces-c-src_2_7_0.tar.gz]
 
* [http://www.jlab.org/Hall-D/software/xerces-c-src_2_7_0.tar.gz xerces-c-src_2_7_0.tar.gz]
 
* [http://www.jlab.org/Hall-D/software/xerces-c-src_2_5_0.tar.gz xerces-c-src_2_5_0.tar.gz]
 
* [http://www.jlab.org/Hall-D/software/xerces-c-src_2_5_0.tar.gz xerces-c-src_2_5_0.tar.gz]
 
(note that binaries exist on the JLab CUE for certain platforms:)
 
(note that binaries exist on the JLab CUE for certain platforms:)
  
* <sub>[http://www.jlab.org/Hall-D/software/xerces-c-src_2_7_0.Linux-athlon Linux-athlon]</sub>
 
* <sub>[http://www.jlab.org/Hall-D/software/xerces-c-src_2_7_0.Linux-i686 Linux-i686]</sub>
 
* <sub>[http://www.jlab.org/Hall-D/software/xerces-c-src_2_7_0.Linux-x86_64 Linux-x86_64]</sub>
 
* <sub>[http://www.jlab.org/Hall-D/software/xerces-c-src_2_7_0.SunOS-sparc SunOS-sparc]</sub>
 
  
 +
'''Note''': You only need the XERCES Perl module if you're going to be modifying the data model.
  
 
'''XERCES Perl module'''
 
'''XERCES Perl module'''
Line 79: Line 77:
  
  
'''[http://xml.apache.org/xalan-j XALAN-Java]'''
+
'''Note''': You only need the XALAN package if you're going to be modifying the data model AND you want to convert between hddm template and schema files. You can install either the java version or the C version. See the source for the hddm-schema and schema-hddm perl scripts for how to select which to use.
* [http://www.jlab.org/Hall-D/software/xerces-c-src_2_7_0.Linux-i686/tools/jars/xalan.jar xalan.jar]
+
  
 +
'''[http://xml.apache.org/xalan-j XALAN-Java]''' ([http://www.jlab.org/Hall-D/software/xerces-c-src_2_7_0.Linux-i686/tools/jars/xalan.jar xalan.jar]) <br>
 +
'''[http://xml.apache.org/xalan-c XALAN-C]'''
  
  
Download and install the needed packages from the above locations.
+
'''HDDS'''
  
<p><hr width="50%"></p>
+
The HDDS package contains all of the geometry information for GlueX. See the wiki page "[[HOWTO use the stand-alone HDDS system]]" for instruction on how to build it.
  
 
+
== Getting the Hall-D Source ==
We'll assume you are working on a machine outside of JLab.
+
The Hall-D specific source code is kept in a
(All of these instructions will work onsite as well.)
+
Subversion repository on the JLab CUE. The repository can be accessed via anonymous svn checkout
 +
(even from offsite) but you must have a CUE account and you must
 +
belong to the "halld" unix group in order to commit changes back into it. (Contact the JLab computer
 +
center for help on getting a CUE account.)  
  
 
First, you will need to create a directory where your files will
 
First, you will need to create a directory where your files will
 
be kept. (I use a directory called ''HallD'' in my home directory.)
 
be kept. (I use a directory called ''HallD'' in my home directory.)
Set your HALLD_HOME environment variable to point to this directory.
 
  
You should also set your OSNAME environment variable to indicate the
+
Next, you can check out the code:
OS and architecture on which you are working.
+
  
As a summary, here are the environment variables you should have set. Strictly speaking,
+
===CHECKING OUT THE CODE===
not all of these are required, but this is a set that is known to be sufficient
+
1. Check that subversion is installed on your system. Many Linux
 +
distributions now install subversion with the development tools
 +
so it may already be there (check for an executable named "svn").
  
setenv OS `/bin/uname -s`
+
2. Check out the source:
setenv ARCH `/bin/uname -p`
+
setenv OSNAME ${OS}-${ARCH}
+
setenv JANA_HOME /group/12gev_phys/builds/LATEST
+
setenv HALLD_HOME ${HOME}/HallD
+
setenv JANA_PLUGIN_PATH ${HALLD_HOME}/lib/${OSNAME}
+
setenv JAVAROOT /apps
+
setenv HALLD_EXTERNALPACKAGES /group/halld/Software/ExternalPackages
+
setenv XERCESCROOT ${HALLD_EXTERNALPACKAGES}/xerces-c-src_2_7_0.${OSNAME}
+
setenv XERCES_INCLUDE ${HALLD_EXTERNALPACKAGES}/include
+
setenv XERCES_LIB ${HALLD_EXTERNALPACKAGES}/lib/${OSNAME}
+
setenv PERL5LIB ${HALLD_EXTERNALPACKAGES}/perl_mods/lib/site_perl/5.8.2/i686-linux
+
  
in addition, your PATH and (DY)LD_LIBRARY_PATH environment variables should be set to
+
You will probably want to get the most recent stable release. These
include some of these directories:
+
are tagged in the repository in the top level "tags" directory. To get
 +
a list of available tagged releases, issue the following subversion command:
  
  setenv PATH ${PATH}:${JANA_HOME}/bin/${OSNAME}:${HALLD_HOME}/bin:${HALLD_HOME}/bin/${OSNAME}:${HALLD_EXTERNALPACKAGES}/bin:${HALLD_EXTERNALPACKAGES}/bin/${OSNAME}:${ROOTSYS}/bin:${CERN}/${CERN_LEVEL}/bin
+
  svn ls https://halldsvn.jlab.org/repos/tags
setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:${ROOTSYS}/lib:${XERCES_LIB}
+
  
NOTES:
+
You should see a list of directories, some with names like: sim-recon-YYYY-MM-DD
 +
where YYYY-MM-DD are the year, month and day the release was made.
  
- You MUST set your JAVAROOT environment variable or else the
+
For example, if you wanted to check out the ''sim-recon-2010-03-29'' release
schema-hddm script will fail with a very misleading error message. This
+
you would enter the following:
should point to the directory containing bin/java. For example, if
+
your java executable is /usr/bin/java (i.e. this is what "which java"
+
returns) then you should set JAVAROOT to /usr.
+
  
- The HALLD_EXTERNALPACKAGES environment variable is not required. It is
+
svn co https://halldsvn.jlab.org/repos/tags/sim-recon-2010-03-29
just convienient for defining the other environment variables since things
+
svn co https://halldsvn.jlab.org/repos/trunk/calib
like xerces must be install in a non-standard place on the CUE.
+
  
 +
where the second line obtains a copy of the calibration constants from the
 +
repository where they are being kept temporarily until an actual database
 +
is developed.
  
==CHECKING OUT THE CODE==
+
Note that if you check out a tagged release, you should '''not''' commit any
1. Check that subversion is installed on your system. Many Linux
+
changes to that source back into the repository (in subversion, you can
distributions now install subversion with the development tools
+
commit changes to tagged code just as easily as to the main trunk.)
so it may already be there (check for an executable named "svn").
+
 
 +
If you want to develop code to contribute back into the repository for
 +
others to use, you should checkout from the trunk of the tree:
 +
 
 +
svn co https://halldsvn.jlab.org/repos/trunk/sim-recon
 +
 
 +
===Setting up your Environment===
 +
Several environment variables are needed to compile and run the Hall-D software.
 +
The following is a list of environment variables related to 3rd party packages:
 +
 
 +
* JANA_HOME
 +
* XERCESCROOT
 +
* CERN
 +
* CERN_LEVEL
 +
* ROOTSYS
 +
* HDDS_HOME
 +
 
 +
Environment variables needed that identify locations of Hall-D files
 +
 
 +
* JANA_CALIB_URL
 +
* JANA_GEOMETRY_URL
 +
* HALLD_HOME
 +
* HALLD_MY  ''(optional)''
 +
 
 +
Example:
  
2. Check out the needed files with:
+
Suppose you checked out the sim-recon from the trunk into a directory called /home/Bob/GlueX. Then you would use the following environment variables:  
 +
 +
setenv HALLD_HOME /home/Bob/GlueX/sim-recon
 +
setenv JANA_CALIB_URL file:///home/Bob/GlueX/calib
 +
setenv JANA_GEOMETRY_URL xmlfile://${HALLD_HOME}/src/programs/Simulation/hdds/main_HDDS.xml
  
svn co https://halldsvn.jlab.org/repos/trunk/src
+
If you wish to edit programs and then link against libraries in a common area you can do so through
 +
use of the [[Getting started with Hall-D Software on the JLab CUE|HALLD_MY]] environment variable
 +
whose use is explained [[Getting started with Hall-D Software on the JLab CUE|here]].
  
If your svn binary doesn't support the https access method, you can
 
use the following in a pinch by accessing the repository directly from
 
your CUE account. Note this means if you want to work on the files
 
on a remote system, you'll have to copy them there and them copy
 
them back to the CUE in order to commit any changes.
 
  
First, log into a CUE computer like jlabl1.jlab.org. Then ...
+
in addition, you'll probably want to set your PATH environment variable to include the $HALLD_HOME/bin/$BMS_OSNAME directory to avoid having type in the full path every time you run a program.
  svn co file:///group/halld/Repositories/svnroot/trunk/src
+
  
 
==COMPILING THE CODE==
 
==COMPILING THE CODE==
  
===Compiling the Libraries===
+
If you have checked out a fresh and complete set of source code (e.g. a tagged release) then you should run "scons install" in the src directory to do a top-level build. Building the source directories in the right sequence is important in order for all dependencies to be satisfied. The SConstruct and SConscript files are set up to build in the proper sequence when invoked from the top-level directory ''src''. When doing this you should make sure that you have not defined the HALLD_MY environment variable.  
1. Make sure your HALLD_HOME and JANA_HOME environment variables are set and cd into src/libraries and type "make". This should build all of the Hall-D libraries and place them in $HALLD_HOME/lib/$OSNAME.
+
  
If you do not need to modify the geometry, then you can skip item 2. below and go straight to item 3..
+
scons -j32 install
 +
 
 +
After performing a top-level build one can modify source and invoke "scons -u install" only in the specific sub-directory without having to do a top-level build every time.
 +
 
 +
scons -u install
 +
 
 +
There are a [[build script examples|set of scripts that give examples]] of how to install the various GlueX software packages. In many cases, the scripts can be run to do the installation automatically, as long as some of the choices made for directory location are acceptable to you.
 +
 
 +
 
 +
=== Modifying a Program===
 +
 
 +
Programs and plugins generally are kept in the sim-recon/src/programs directory tree. To modify a program one can either edit the code where it is an re-run "scons -u" or copy the whole directory and modify the copy.  
 +
 
 +
Example: to build a private version of the ''hd_ana'' program called ''hd_ana_bob'' do the following:
 +
 +
cd $HALLD_HOME/src/programs/Analysis
 +
cp -r hd_ana hd_ana_bob
 +
... edit SConscript file to include hd_ana_bob ...
 +
cd hd_ana_bob
 +
mv hd_ana.cc hd_ana_bob.cc
 +
scons -u install -c
 +
.... edit files ...
 +
scons -u install
 +
 
 +
Notice that the file including ''main()'' was renamed from ''hd_ana.cc'' to ''hd_ana_bob.cc''. This is because the generic build system uses the basename of the ''main()''-containing file as the name of the executable. You also *must* edit the SConscript file in the parent directory of the directory containing your source to include your source directory. See [[SCons Build System|more details here]].
 +
 
 +
The ''scons -u install -c'' step is needed to clear the objects and dependency files for the hd_ana.cc file which now does not exist by that name.
 +
 
 +
===Modifying a Library===
 +
 
 +
Code useful to more than one program should be kept in a library. Library code is stored in the sim-recon/src/libraries directory tree. Among other things, this is where all of the reconstruction code is kept. To modify code in a library, just go to the appropriate directory, edit the code and invoke "scons -u install".
 +
 
 +
Example: to change the way the Forward Calorimeter reconstruction works, do the following:
 +
 +
cd $HALLD_HOME/src/libraries/FCAL
 +
.... edit files ....
 +
scons -u install
 +
cd $HALLD_HOME/src/programs/Analysis
  
 
===Modifying the Geometry===
 
===Modifying the Geometry===
2. cd into "src/programs/Simulation/hdds". Here you will find a number
+
If you do not need to modify the geometry, then you can skip this item and go straight to "RUNNING HDGEANT"..
of XML files that define the geometry of the GlueX detector in the simulation.
+
Modify the appropriate XML file(s) and then do a "make hddsGeant3.F". This will build
+
the hdds-geant executable and run it on the main_HDDS.xml to
+
produce hddsGeant3.F. The hddsGeant3.F file is FORTRAN source
+
which contains all of the geometry definitions for the entire
+
GlueX detector. After generating the hddsGeant3.F file, you will
+
need to copy it into ../HDGeant.  
+
  
Note that if you make changes to the geometry that you feel should be propagated
+
The geometry for the simulation is kept in a set of XML files in the copy of HDDS you are using (see [[Getting started with GlueX Software#Required 3rd Party Packages|Required 3rd Party Packages]] above).
back into the repository, you should contact Richard Jones (richard.t.jones@uconn.edu)
+
who is the acting gatekeeper for the geometry and simulation just to make sure he's
+
aware of the changes and that they are implemented in a consistent way.
+
+
===Building HDGeant===
+
3. Now cd into the src/programs/Simulation/HDGeant directory and just
+
type "make". Don't worry if it seems to take a long time to
+
compile hddsGeant3.F. This is a HUGE file so just be patient.
+
When it is all done, you should end up with a few executables
+
including hdgeant. They will, by default, be installed in the
+
$HALLD_HOME/bin/$OSNAME directory.
+
(for example ~/HallD/bin/Linux-i686).
+
  
==Example Script==
+
If you are using your own version of HDDS, you can modify the appropriate XML file(s) and then do a "scons install" in your $HDDS_HOME directory. This will build the hdds-geant executable and run it on the main_HDDS.xml to
 +
produce a file named ''hddsGeant3.F''. The ''hddsGeant3.F'' file is FORTRAN source which contains all of the geometry definitions for the entire GlueX detector. ''hddsGeant3.F'', will
 +
be compiled and put into a library in a subdirectory of $HDDS_HOME/lib.
  
Assuming that you have the supporting software (i. e., non-GlueX-related) built on your machine, there is an [[example build script|example script]] that summarizes the steps necessary to build JANA and the entire Hall D source tree.
+
Note that if you make changes to the geometry that you feel should be propagated back into the repository, you should contact Richard Jones (richard.t.jones@uconn.edu) who is the acting gatekeeper for the geometry and simulation just to make sure he's aware of the changes and that they are implemented in a consistent way.
  
 
=RUNNING HDGEANT=
 
=RUNNING HDGEANT=
I've really only used the batch version of HDGeant called ''hdgeant''
+
The simulation program comes in 2 forms ''hdgeant'' for batch usage and ''hdgeant++'' for interactive usage.
so that is what these instructions focus on.
+
These instructions focus on the batch version ''hdgeant''.
  
 
I'll just give a couple of hints here:
 
I'll just give a couple of hints here:
Line 195: Line 229:
 
# When running hdgeant, make sure the control.in
 
# When running hdgeant, make sure the control.in
 
#: file exists in the current working directory. This can be found in the
 
#: file exists in the current working directory. This can be found in the
#: src/programs/Simulation/HDGeant directory.
+
#: sim-recon/src/programs/Simulation/HDGeant directory.
 +
#:
 +
#: The file is in ASCII format and has lots of comments to help you configure it to suit your needs.
 
#:
 
#:
# Make sure you JANA_CALIB_URL environment variable is set
+
# Make sure your JANA_CALIB_URL environment variable is set
#: and points to the location of the calibration files. At this point,
+
#: and points to the location of the CCDB version you wish to use. Some
#: the calibration files are text files stored in the repository
+
#: instructions on this [[SQLite-form of the CCDB database|can be found here]].
#: and can be checked out via:
+
#: (If offsite, you should consider using an SQLite file)
#: ''svn co https://halldsvn.jlab.org/repos/trunk/calib''
+
 
#:
 
#:
 
#: Example:
 
#: Example:
#: The root of your calibration files tree is ''/home/jdoe/HallD/calib''
+
#: setenv JANA_CALIB_URL sqlite:///home/jdoe/HallD/ccdb.sqlite
#: setenv JANA_CALIB_URL file:///home/jdoe/HallD/calib
+
  
  
Line 218: Line 252:
  
 
An HDDM file can be generated using any of the following programs:
 
An HDDM file can be generated using any of the following programs:
* '''genr8''' (src/programs/Simulation/genr8) generator for specific reaction via isobar model
+
* '''genr8''' (sim-recon/src/programs/Simulation/genr8) generator for specific reaction via isobar model
* '''bggen''' (src/programs/Simulation/bggen) generator for full hadronic photoproduction spectrum
+
* '''bggen''' (sim-recon/src/programs/Simulation/bggen) generator for full hadronic photoproduction spectrum
* '''genphoton''' (src/programs/Simulation/genphoton) external particle gun for single photons
+
* '''genphoton''' (sim-recon/src/programs/Simulation/genphoton) external particle gun for single photons
* '''genpi''' (src/programs/Simulation/genpi) external particle gun for single pions
+
* '''genpi''' (sim-recon/src/programs/Simulation/genpi) external particle gun for single pions
  
The program genr8_2_hddm (src/programs/Simulation/genr8_2_hddm) can be used to
+
The program genr8_2_hddm (sim-recon/src/programs/Simulation/genr8_2_hddm) can be used to
convert the output of genr8 into HDDM format.
+
convert the output of ''genr8'', ''genphoton'', and ''genpi'' into HDDM format.
  
The input files still contain some "pristine" hits. Some smearing of these is still required to  
+
After setting up your control.in file and creating any generated events file (if your not
 +
using a built-in generator) then simply invoke ''hdgeant'' with no arguments. This will
 +
produce an hddm formatted output file with a name specified in the control.in file.
 +
 
 +
The simulation produces several output files. The default name of the one with the
 +
detector responses that you want is called ''hdgeant.hddm'' (unless you specified something
 +
else in the control.in file".
 +
 
 +
The ''hdgeant.hddm'' file still contains some "pristine" hits. Some smearing of these is required to  
 
accurately model detector responses. In general, the calorimeter values are smeared  
 
accurately model detector responses. In general, the calorimeter values are smeared  
 
inside of the reconstruction program while drift chamber hits are are smeared  
 
inside of the reconstruction program while drift chamber hits are are smeared  
 
using an intermediate program called ''mcsmear''.
 
using an intermediate program called ''mcsmear''.
 
The simulation produces several output files. The default name of the one with the
 
detector responses that you want is called ''hdgeant.hddm''.
 
  
 
=Reading in Simulated Data=
 
=Reading in Simulated Data=
  
<!--There are a few ways to look at the output data from hdgeant. The first
+
In addition to the hddm tools (sim-recon/src/programs/Utilities) that can be used to scan
is to  build the hddm package (src/packages/Analysis/hddm) Note that
+
data in an hddm file, there are several programs in the Hall-D arsenal that
yoou will need XERCES installed. There is a tool called ''hddm-xml''
+
can read this in as well. All of them rely on the libHDDM.a library so new
provided by this package that will
+
programs can access the data too with minimal effort.
display the contents of an HDDM file in XML form. This is useful to at least see that
+
you are getting hits.-->
+
 
+
Simulated data can be accessed from a DANA-based program.
+
DANA is the Hall-D implementation
+
of the JANA reconstruction framework. The DANA framework can read in an
+
HDDM file using the libHDDM.a library (src/programs/libraries/HDDM) and create
+
C++ objects from it that can then be used to access the event data.
+
  
Some programs already exist that can be used verify the hits are there and look
+
Several tools exist that can be used to look at the simulated data file, all of
at single events. For any kind of sophisticated analysis however, you will
+
which have the full reconstruction software built-in. Below are descriptions
likely need to write a custom C++ program.
+
of the most useful ones.  
  
 
==hd_dump==
 
==hd_dump==
A simple utility program is available called "hd_dump" which is based on the DANA
+
A simple utility program is available called "hd_dump" which is based on the JANA
framework. The source code for this program already exists in the
+
framework.
src directory tree. To build it:
+
#cd into src/library and do a "make"
+
#cd into src/programs/Analysis/hd_dump and do "make"
+
  
Running hd_dump with no arguments will print a usage message. Essentially through,
+
Running ''hd_dump'' with no arguments will print a usage message. Essentially though,
 
you just give it the name of the hdgeant.hddm file when you run it. It will
 
you just give it the name of the hdgeant.hddm file when you run it. It will
 
list the objects available event by event (pausing for user input after each event).
 
list the objects available event by event (pausing for user input after each event).
Line 266: Line 294:
 
print the contents of all objects of type ''objectname'' as well.
 
print the contents of all objects of type ''objectname'' as well.
  
==Making ROOT histograms in DANA==
+
Example of using hd_dump:
 +
 +
>hd_dump -DDMCThrown hdgeant.hddm
 +
 +
Registered factories: (56 total)
 +
 +
Name:            nrows:  tag:
 +
---------------- ------- --------------
 +
DBCALMCResponse    15                                                       
 +
DBCALGeometry        1                                                       
 +
DBCALShower          2                                                       
 +
DBCALTruthShower    6                                                       
 +
DBCALPhoton          2                                                       
 +
DHDDMBCALHit        11                                                       
 +
DCDCHit            10                                                       
 +
DCDCTrackHit        10                                                       
 +
DFDCHit            51                                                       
 +
DFDCPseudo          5                                                       
 +
DFDCCathodeCluster  12                                                       
 +
DFDCSegment          1                                                       
 +
DFDCIntersection    4                                                       
 +
DFDCPseudo          8    "WIRESONLY"                                         
 +
DFDCPseudo          5    "CORRECTED"                                         
 +
DFCALGeometry        1                                                       
 +
DSCTruthHit          1                                                       
 +
DSCHit              1                                                       
 +
DTOFMCResponse      2                                                       
 +
DTOFHit              2                                                       
 +
DTOFGeometry        1                                                       
 +
DHDDMTOFHit          2                                                       
 +
DTOFHit              2    "MC"                                               
 +
DTrack              1    "ALT2"                                             
 +
DTrack              1    "ALT3"                                             
 +
DTrack              1                                                       
 +
DTrackCandidate      1                                                       
 +
DTrackCandidate      1    "CDC"                                               
 +
DTrackCandidate      1    "FDC"                                               
 +
DTrackCandidate      1    "FDCCathodes"                                       
 +
DTrackCandidate      1    "FDCpseudo"                                         
 +
DTrackCandidate      1    "THROWN"                                           
 +
DMCTrackHit        24                                                       
 +
DMCThrown            1                                                       
 +
DMCTrajectoryPoint  2                                                       
 +
DTrack              1    "THROWN"                                           
 +
DTrackFitter        1                                                       
 +
DTrackFitter        1    "ALT1"                                             
 +
DTrackHitSelector    1                                                       
 +
DTrackHitSelector    1    "ALT1"                                             
 +
DPhoton              2                                                       
 +
DParticle            1                                                       
 +
DParticle            1    "THROWN"                                           
 +
 +
DMCThrown:
 +
  q: x(cm): y(cm): z(cm): E(GeV):    t(ns): p(GeV/c): theta(deg): phi(deg): type: pdgtype: myid: parentid: mech:
 +
----------------------------------------------------------------------------------------------------------------
 +
  +1    0.0    0.0  65.0  1.694  -999.000    1.688      16.329    -7.040    8        0    1        0    0
 +
 
 +
== hdview2==
 +
The ''hdview2'' program is a simple event viewer written in ROOT. It can be used to help visualize the event using 2-D projections. Not all features work on it and it is geared a little toward charged particle tracking at the moment. Start it by just passing it the name of an hddm file on the command line.
 +
 
 +
Example of using hdview2:
 +
 
 +
>hdview2 hdgeant.hddm
 +
 +
[[Image:20090217_hdview2.gif|700px]]
 +
 
 +
== hd_root ==
 +
The ''hd_root'' program can be used to make histograms and trees in a ROOT file. The program itself only creates an empty ROOT file and then cycles through events. The usefulness comes in through plugins that make and fill the histos and trees. The idea is that one can specify any number of plugins on the command line and the histograms and trees created by all of them will go into a single ROOT file. It is important to note that reconstruction algorithms are all compiled into hd_root, but it is left to the plugins to actually activate them.
 +
 
 +
Example: Using hd_root to make ROOT file with TTree containing information on CDC hits.
 +
 +
hd_root --plugin=cdc_hists --auto_activate=DCDCTrackHit hdgeant.hddm
 +
 
 +
== janaroot plugin ==
 +
 
 +
In the most general case, one can use the ''janaroot'' plugin to create a separate ROOT file with trees filled with the objects of interest. Since this plugin creates its own ROOT file, it can be used with any of the Hall-D reconstruction programs including ''hd_root'' and ''hd_ana''.
 +
 
 +
Example: Using the janaroot plugin to create TTrees with generated particles and reconstructed photons
 +
 +
hd_ana --plugin=janaroot --auto_activate=DMCThrown --auto_activate=DPhoton hdgeant.hddm
 +
 
 +
==Making your own custom program==
 +
 
 +
If you are not familiar or comfortable with using plugins then you may just want to
 +
create your own custom program. It is probably easiest to just copy the source for
 +
''hd_root'' and modify it to create and fill the histograms/trees that you want.
  
 
The main idea behind JANA is that it passes around C++ objects that contain the
 
The main idea behind JANA is that it passes around C++ objects that contain the
Line 273: Line 386:
 
[http://www.jlab.org/Hall-D/software/HDSoftware_Documentation/ doxygen generated on-line documentation].
 
[http://www.jlab.org/Hall-D/software/HDSoftware_Documentation/ doxygen generated on-line documentation].
  
Several programs exist that can be used to make ROOT histograms and Trees using
 
HDDM files as inputs. There is no such thing, however, as a generic HDDM to ROOT converter
 
program. The data model is large enough that most jobs will only ever need a
 
fraction of the information so writing out everything would use up resources
 
uneccessarily. The programs that are available can be found in the src/program/analysis
 
directory.
 
  
 
===Using plugins===
 
===Using plugins===
Line 292: Line 399:
 
create a separate directory in the ROOT file to place their histograms/trees. This allows
 
create a separate directory in the ROOT file to place their histograms/trees. This allows
 
mulitple plugins to be attached to the same executable without risk of conflict between  
 
mulitple plugins to be attached to the same executable without risk of conflict between  
histogram names. Source code for the plugins resides in the src/programs/Analysis/plugins
+
histogram names. Source code for the plugins resides in the sim-recon/src/programs/Analysis/plugins
 
directory.
 
directory.
  
 
Although a plugin can be attached to any DANA program, a generic program
 
Although a plugin can be attached to any DANA program, a generic program
''hd_root'' exists that serves as a shell specifically written for this task. For example, to
+
''hd_root'' exists that serves as a shell specifically written for this task (see above). For example, to
 
create a root file with the thrown values histograms and the histograms used to
 
create a root file with the thrown values histograms and the histograms used to
 
study the acceptance of the GlueX detector, type this:
 
study the acceptance of the GlueX detector, type this:
  
  >hd_root --plugin=mcthrown_hists --plugin=acceptance_hists hdgeant.hddm
+
  hd_root --plugin=mcthrown_hists --plugin=acceptance_hists hdgeant.hddm
  
This will create a ROOT file called ''hd_tree.root''. Inside the file would be two
+
This will create a ROOT file called ''hd_root.root''. Inside the file would be two
 
directories called "THROWN" and "ACCEPTANCE" containing the histograms produced
 
directories called "THROWN" and "ACCEPTANCE" containing the histograms produced
 
by the respective plugins.
 
by the respective plugins.
  
 
You can create your own plugin using one of the existing plugins as a template.
 
You can create your own plugin using one of the existing plugins as a template.
 
===Building a custom executable===
 
 
The last option is to just build your own custom
 
executable that can be used to define and fill your histograms.
 
To do this, take a look at the example program:
 
 
src/programs/Analysis/hd_ana
 
 
Try building and using it as-is first to make sure it all works.
 
It will produce a couple of sample histograms. To customize
 
''hd_ana'', the only files you'll
 
need to modify are the MyProcessor.cc and MyProcessor.h (All of the
 
work is done in MyProcessor.cc).
 

Latest revision as of 07:16, 30 September 2014

This is derived from the original web document "Getting started with HDGeant"

D. Lawrence 11/11/04
D. Lawrence 4/6/06 Updated
D. Lawrence 12/5/06 Converted to Wiki
D. Lawrence 1/18/07 Finished Converting to Wiki
D. Lawrence 2/4/08 Tweaked for calibDB among others
D. Lawrence 2/17/09 Overhauled to reflect current state of code
M. Ito 1/5/10 Added HDDS to list of 3rd party software
D. Lawrence 9/30/14 Replace "make" with "scons"

Overview

This document gives the bare minimum to get you up and going with running the GlueX GEANT-based simulation program hdgeant and looking at the output. More details are given in other HOWTOs (to be written).


The basic steps to running and analyzing a GlueX simulation are:

  1. Getting and compiling the source code
  2. Configuring and running the hdgeant program
  3. Running one of the Hall-D DANA-based reconstruction programs (This could involve customizing the C++ program for your needs or using one of the pre-built plugins to create ROOT TTrees)


Getting and compiling the source code

Required 3rd Party Packages

The source relies on several software packages that are not kept in the Hall-D repository (with the exception of HDDS, which is kept there). These must be downloaded and installed prior to compiling the Hall-D source. Here is a list of what you will need.

JANA

Make sure you set your JANA_HOME environment variable to point to the directory containing the bin, lib, and include directories after building JANA.

(Note that the source for JANA is maintained in an svn repository located here: https://phys12svn.jlab.org/repos/JANA with the tagged versions located here: https://phys12svn.jlab.org/repos/tags)


CERNLIB

Make sure you set your CERN and CERN_LEVEL environment variables and that the cernlib script is in your path as it will be used by the makefiles.

Check here for possible issues with CERNLIB installation.

If you don't already have it installed, you will also need Motif. You can download OpenMotif from here for many platforms (including Mac OS X).


ROOT

Make sure your ROOTSYS environment variable is set and that the root-config script is in your path. Also, make sure the ROOT shared libraries directory (usually $ROOTSYS/lib) is in your LD_LIBRARY_PATH (DYLD_LIBRARY_PATH on OS X) environment variable.


XERCES

(note that binaries exist on the JLab CUE for certain platforms:)


Note: You only need the XERCES Perl module if you're going to be modifying the data model.

XERCES Perl module


Note: You only need the XALAN package if you're going to be modifying the data model AND you want to convert between hddm template and schema files. You can install either the java version or the C version. See the source for the hddm-schema and schema-hddm perl scripts for how to select which to use.

XALAN-Java (xalan.jar)
XALAN-C


HDDS

The HDDS package contains all of the geometry information for GlueX. See the wiki page "HOWTO use the stand-alone HDDS system" for instruction on how to build it.

Getting the Hall-D Source

The Hall-D specific source code is kept in a Subversion repository on the JLab CUE. The repository can be accessed via anonymous svn checkout (even from offsite) but you must have a CUE account and you must belong to the "halld" unix group in order to commit changes back into it. (Contact the JLab computer center for help on getting a CUE account.)

First, you will need to create a directory where your files will be kept. (I use a directory called HallD in my home directory.)

Next, you can check out the code:

CHECKING OUT THE CODE

1. Check that subversion is installed on your system. Many Linux distributions now install subversion with the development tools so it may already be there (check for an executable named "svn").

2. Check out the source:

You will probably want to get the most recent stable release. These are tagged in the repository in the top level "tags" directory. To get a list of available tagged releases, issue the following subversion command:

svn ls https://halldsvn.jlab.org/repos/tags

You should see a list of directories, some with names like: sim-recon-YYYY-MM-DD where YYYY-MM-DD are the year, month and day the release was made.

For example, if you wanted to check out the sim-recon-2010-03-29 release you would enter the following:

svn co https://halldsvn.jlab.org/repos/tags/sim-recon-2010-03-29
svn co https://halldsvn.jlab.org/repos/trunk/calib

where the second line obtains a copy of the calibration constants from the repository where they are being kept temporarily until an actual database is developed.

Note that if you check out a tagged release, you should not commit any changes to that source back into the repository (in subversion, you can commit changes to tagged code just as easily as to the main trunk.)

If you want to develop code to contribute back into the repository for others to use, you should checkout from the trunk of the tree:

svn co https://halldsvn.jlab.org/repos/trunk/sim-recon

Setting up your Environment

Several environment variables are needed to compile and run the Hall-D software. The following is a list of environment variables related to 3rd party packages:

  • JANA_HOME
  • XERCESCROOT
  • CERN
  • CERN_LEVEL
  • ROOTSYS
  • HDDS_HOME

Environment variables needed that identify locations of Hall-D files

  • JANA_CALIB_URL
  • JANA_GEOMETRY_URL
  • HALLD_HOME
  • HALLD_MY (optional)

Example:

Suppose you checked out the sim-recon from the trunk into a directory called /home/Bob/GlueX. Then you would use the following environment variables:

setenv HALLD_HOME /home/Bob/GlueX/sim-recon
setenv JANA_CALIB_URL file:///home/Bob/GlueX/calib
setenv JANA_GEOMETRY_URL xmlfile://${HALLD_HOME}/src/programs/Simulation/hdds/main_HDDS.xml

If you wish to edit programs and then link against libraries in a common area you can do so through use of the HALLD_MY environment variable whose use is explained here.


in addition, you'll probably want to set your PATH environment variable to include the $HALLD_HOME/bin/$BMS_OSNAME directory to avoid having type in the full path every time you run a program.

COMPILING THE CODE

If you have checked out a fresh and complete set of source code (e.g. a tagged release) then you should run "scons install" in the src directory to do a top-level build. Building the source directories in the right sequence is important in order for all dependencies to be satisfied. The SConstruct and SConscript files are set up to build in the proper sequence when invoked from the top-level directory src. When doing this you should make sure that you have not defined the HALLD_MY environment variable.

scons -j32 install

After performing a top-level build one can modify source and invoke "scons -u install" only in the specific sub-directory without having to do a top-level build every time.

scons -u install

There are a set of scripts that give examples of how to install the various GlueX software packages. In many cases, the scripts can be run to do the installation automatically, as long as some of the choices made for directory location are acceptable to you.


Modifying a Program

Programs and plugins generally are kept in the sim-recon/src/programs directory tree. To modify a program one can either edit the code where it is an re-run "scons -u" or copy the whole directory and modify the copy.

Example: to build a private version of the hd_ana program called hd_ana_bob do the following:

cd $HALLD_HOME/src/programs/Analysis
cp -r hd_ana hd_ana_bob

... edit SConscript file to include hd_ana_bob ...

cd hd_ana_bob
mv hd_ana.cc hd_ana_bob.cc
scons -u install -c
.... edit files ...
scons -u install

Notice that the file including main() was renamed from hd_ana.cc to hd_ana_bob.cc. This is because the generic build system uses the basename of the main()-containing file as the name of the executable. You also *must* edit the SConscript file in the parent directory of the directory containing your source to include your source directory. See more details here.

The scons -u install -c step is needed to clear the objects and dependency files for the hd_ana.cc file which now does not exist by that name.

Modifying a Library

Code useful to more than one program should be kept in a library. Library code is stored in the sim-recon/src/libraries directory tree. Among other things, this is where all of the reconstruction code is kept. To modify code in a library, just go to the appropriate directory, edit the code and invoke "scons -u install".

Example: to change the way the Forward Calorimeter reconstruction works, do the following:

cd $HALLD_HOME/src/libraries/FCAL
.... edit files ....
scons -u install
cd $HALLD_HOME/src/programs/Analysis

Modifying the Geometry

If you do not need to modify the geometry, then you can skip this item and go straight to "RUNNING HDGEANT"..

The geometry for the simulation is kept in a set of XML files in the copy of HDDS you are using (see Required 3rd Party Packages above).

If you are using your own version of HDDS, you can modify the appropriate XML file(s) and then do a "scons install" in your $HDDS_HOME directory. This will build the hdds-geant executable and run it on the main_HDDS.xml to produce a file named hddsGeant3.F. The hddsGeant3.F file is FORTRAN source which contains all of the geometry definitions for the entire GlueX detector. hddsGeant3.F, will be compiled and put into a library in a subdirectory of $HDDS_HOME/lib.

Note that if you make changes to the geometry that you feel should be propagated back into the repository, you should contact Richard Jones (richard.t.jones@uconn.edu) who is the acting gatekeeper for the geometry and simulation just to make sure he's aware of the changes and that they are implemented in a consistent way.

RUNNING HDGEANT

The simulation program comes in 2 forms hdgeant for batch usage and hdgeant++ for interactive usage. These instructions focus on the batch version hdgeant.

I'll just give a couple of hints here:

  1. When running hdgeant, make sure the control.in
    file exists in the current working directory. This can be found in the
    sim-recon/src/programs/Simulation/HDGeant directory.
    The file is in ASCII format and has lots of comments to help you configure it to suit your needs.
  2. Make sure your JANA_CALIB_URL environment variable is set
    and points to the location of the CCDB version you wish to use. Some
    instructions on this can be found here.
    (If offsite, you should consider using an SQLite file)
    Example:
    setenv JANA_CALIB_URL sqlite:///home/jdoe/HallD/ccdb.sqlite


The control.in file should be edited to suit your specific simulation. Most importantly, you need to define the source of events. There are 3 main options which are documented in the example control.in file. They are:

  • coherent bremstrahlung photon generator(built-in)
  • single particle gun(built-in)
  • read events from an external file in HDDM (Hall-D Data Model) format.

An HDDM file can be generated using any of the following programs:

  • genr8 (sim-recon/src/programs/Simulation/genr8) generator for specific reaction via isobar model
  • bggen (sim-recon/src/programs/Simulation/bggen) generator for full hadronic photoproduction spectrum
  • genphoton (sim-recon/src/programs/Simulation/genphoton) external particle gun for single photons
  • genpi (sim-recon/src/programs/Simulation/genpi) external particle gun for single pions

The program genr8_2_hddm (sim-recon/src/programs/Simulation/genr8_2_hddm) can be used to convert the output of genr8, genphoton, and genpi into HDDM format.

After setting up your control.in file and creating any generated events file (if your not using a built-in generator) then simply invoke hdgeant with no arguments. This will produce an hddm formatted output file with a name specified in the control.in file.

The simulation produces several output files. The default name of the one with the detector responses that you want is called hdgeant.hddm (unless you specified something else in the control.in file".

The hdgeant.hddm file still contains some "pristine" hits. Some smearing of these is required to accurately model detector responses. In general, the calorimeter values are smeared inside of the reconstruction program while drift chamber hits are are smeared using an intermediate program called mcsmear.

Reading in Simulated Data

In addition to the hddm tools (sim-recon/src/programs/Utilities) that can be used to scan data in an hddm file, there are several programs in the Hall-D arsenal that can read this in as well. All of them rely on the libHDDM.a library so new programs can access the data too with minimal effort.

Several tools exist that can be used to look at the simulated data file, all of which have the full reconstruction software built-in. Below are descriptions of the most useful ones.

hd_dump

A simple utility program is available called "hd_dump" which is based on the JANA framework.

Running hd_dump with no arguments will print a usage message. Essentially though, you just give it the name of the hdgeant.hddm file when you run it. It will list the objects available event by event (pausing for user input after each event). If you give it any "'-Dobjectname" options on the command line, it will attempt to print the contents of all objects of type objectname as well.

Example of using hd_dump:

>hd_dump -DDMCThrown hdgeant.hddm

Registered factories: (56 total)

Name:             nrows:  tag:
---------------- ------- --------------
DBCALMCResponse     15                                                         
DBCALGeometry        1                                                         
DBCALShower          2                                                         
DBCALTruthShower     6                                                         
DBCALPhoton          2                                                         
DHDDMBCALHit        11                                                         
DCDCHit             10                                                         
DCDCTrackHit        10                                                         
DFDCHit             51                                                         
DFDCPseudo           5                                                         
DFDCCathodeCluster  12                                                         
DFDCSegment          1                                                         
DFDCIntersection     4                                                         
DFDCPseudo           8    "WIRESONLY"                                          
DFDCPseudo           5    "CORRECTED"                                          
DFCALGeometry        1                                                         
DSCTruthHit          1                                                         
DSCHit               1                                                         
DTOFMCResponse       2                                                         
DTOFHit              2                                                         
DTOFGeometry         1                                                         
DHDDMTOFHit          2                                                         
DTOFHit              2    "MC"                                                 
DTrack               1    "ALT2"                                               
DTrack               1    "ALT3"                                               
DTrack               1                                                         
DTrackCandidate      1                                                         
DTrackCandidate      1    "CDC"                                                
DTrackCandidate      1    "FDC"                                                
DTrackCandidate      1    "FDCCathodes"                                        
DTrackCandidate      1    "FDCpseudo"                                          
DTrackCandidate      1    "THROWN"                                             
DMCTrackHit         24                                                         
DMCThrown            1                                                         
DMCTrajectoryPoint   2                                                         
DTrack               1    "THROWN"                                             
DTrackFitter         1                                                         
DTrackFitter         1    "ALT1"                                               
DTrackHitSelector    1                                                         
DTrackHitSelector    1    "ALT1"                                               
DPhoton              2                                                         
DParticle            1                                                         
DParticle            1    "THROWN"                                             

DMCThrown:
  q: x(cm): y(cm): z(cm): E(GeV):    t(ns): p(GeV/c): theta(deg): phi(deg): type: pdgtype: myid: parentid: mech:
----------------------------------------------------------------------------------------------------------------
 +1    0.0    0.0   65.0   1.694  -999.000     1.688      16.329    -7.040     8        0     1         0     0

hdview2

The hdview2 program is a simple event viewer written in ROOT. It can be used to help visualize the event using 2-D projections. Not all features work on it and it is geared a little toward charged particle tracking at the moment. Start it by just passing it the name of an hddm file on the command line.

Example of using hdview2:

>hdview2 hdgeant.hddm

20090217 hdview2.gif

hd_root

The hd_root program can be used to make histograms and trees in a ROOT file. The program itself only creates an empty ROOT file and then cycles through events. The usefulness comes in through plugins that make and fill the histos and trees. The idea is that one can specify any number of plugins on the command line and the histograms and trees created by all of them will go into a single ROOT file. It is important to note that reconstruction algorithms are all compiled into hd_root, but it is left to the plugins to actually activate them.

Example: Using hd_root to make ROOT file with TTree containing information on CDC hits.

hd_root --plugin=cdc_hists --auto_activate=DCDCTrackHit hdgeant.hddm

janaroot plugin

In the most general case, one can use the janaroot plugin to create a separate ROOT file with trees filled with the objects of interest. Since this plugin creates its own ROOT file, it can be used with any of the Hall-D reconstruction programs including hd_root and hd_ana.

Example: Using the janaroot plugin to create TTrees with generated particles and reconstructed photons

hd_ana --plugin=janaroot --auto_activate=DMCThrown --auto_activate=DPhoton hdgeant.hddm

Making your own custom program

If you are not familiar or comfortable with using plugins then you may just want to create your own custom program. It is probably easiest to just copy the source for hd_root and modify it to create and fill the histograms/trees that you want.

The main idea behind JANA is that it passes around C++ objects that contain the data of interest. In order to access the data you want, you must first figure out what object(s) contain it. You can browse or search for the objects you need in the doxygen generated on-line documentation.


Using plugins

All DANA executables can have plugins attached at run time to extend the functionality of the program. For those who are unfamiliar with the term, a plugin is just a dynamically linked object that contains routines that can be accessed by a running executable.

Several plugins have been written that can add histograms/trees to a ROOT file. For example, the "mcthrown_hists plugin will create histograms and fill them with the "thrown" values from Monte Carlo data. There is also a cdc_hists plugin that will produce histograms related specifically to the CDC. In general, these plugins will create a separate directory in the ROOT file to place their histograms/trees. This allows mulitple plugins to be attached to the same executable without risk of conflict between histogram names. Source code for the plugins resides in the sim-recon/src/programs/Analysis/plugins directory.

Although a plugin can be attached to any DANA program, a generic program hd_root exists that serves as a shell specifically written for this task (see above). For example, to create a root file with the thrown values histograms and the histograms used to study the acceptance of the GlueX detector, type this:

hd_root --plugin=mcthrown_hists --plugin=acceptance_hists hdgeant.hddm

This will create a ROOT file called hd_root.root. Inside the file would be two directories called "THROWN" and "ACCEPTANCE" containing the histograms produced by the respective plugins.

You can create your own plugin using one of the existing plugins as a template.