https://halldweb1.jlab.org/wiki/index.php?title=SCons_Build_System&feed=atom&action=historySCons Build System - Revision history2024-03-29T13:30:00ZRevision history for this page on the wikiMediaWiki 1.24.1https://halldweb1.jlab.org/wiki/index.php?title=SCons_Build_System&diff=80873&oldid=prevMarki: Text replacement - "http://argus.phys.uregina.ca/cgi-bin/private" to "https://halldweb.jlab.org/doc-private"2017-02-24T21:13:43Z<p>Text replacement - "http://argus.phys.uregina.ca/cgi-bin/private" to "https://halldweb.jlab.org/doc-private"</p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 21:13, 24 February 2017</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 2:</td>
<td colspan="2" class="diff-lineno">Line 2:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>''10/30/2013  David Lawrence''</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>''10/30/2013  David Lawrence''</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>The first general build system used by Hall-D was called the [<del class="diffchange diffchange-inline">http</del>://<del class="diffchange diffchange-inline">argus</del>.<del class="diffchange diffchange-inline">phys</del>.<del class="diffchange diffchange-inline">uregina.ca</del>/<del class="diffchange diffchange-inline">cgi</del>-<del class="diffchange diffchange-inline">bin/</del>private/DocDB/ShowDocument?docid=473 Build Management System (BMS)] and was base on the standard "make" utility. This was chosen [<del class="diffchange diffchange-inline">http</del>://<del class="diffchange diffchange-inline">argus</del>.<del class="diffchange diffchange-inline">phys</del>.<del class="diffchange diffchange-inline">uregina.ca</del>/<del class="diffchange diffchange-inline">cgi</del>-<del class="diffchange diffchange-inline">bin/</del>private/DocDB/ShowDocument?docid=71 for a couple of reasons] over a few competitors, (including the then new '''scons''' tool). Despite the lack of creativity in the name, the BMS system was successfully used for a number of years to build the [https://halldsvn.jlab.org/repos/trunk/sim-recon/ sim-recon] software for the GlueX experiment in Hall-D. As the amount of source code in sim-recon grew, the compile time required grew too. By 2013, it would take 15-20 minutes to build '''sim-recon''' from a fresh checkout of the code. This was not a problem for everyone since the most common use-case was to build '''sim-recon''' once and then work on only a small part of the software and rebuild that as needed. However, whenever the '''sim-recon''' directory was updated from the repository after a number of changes had been made, it was good practice to do a "make clean" and rebuild everything. In addition, certain power users would routinely need to check out the source and build from scratch on different platforms or to test different revisions.  </div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>The first general build system used by Hall-D was called the [<ins class="diffchange diffchange-inline">https</ins>://<ins class="diffchange diffchange-inline">halldweb</ins>.<ins class="diffchange diffchange-inline">jlab</ins>.<ins class="diffchange diffchange-inline">org</ins>/<ins class="diffchange diffchange-inline">doc</ins>-private/DocDB/ShowDocument?docid=473 Build Management System (BMS)] and was base on the standard "make" utility. This was chosen [<ins class="diffchange diffchange-inline">https</ins>://<ins class="diffchange diffchange-inline">halldweb</ins>.<ins class="diffchange diffchange-inline">jlab</ins>.<ins class="diffchange diffchange-inline">org</ins>/<ins class="diffchange diffchange-inline">doc</ins>-private/DocDB/ShowDocument?docid=71 for a couple of reasons] over a few competitors, (including the then new '''scons''' tool). Despite the lack of creativity in the name, the BMS system was successfully used for a number of years to build the [https://halldsvn.jlab.org/repos/trunk/sim-recon/ sim-recon] software for the GlueX experiment in Hall-D. As the amount of source code in sim-recon grew, the compile time required grew too. By 2013, it would take 15-20 minutes to build '''sim-recon''' from a fresh checkout of the code. This was not a problem for everyone since the most common use-case was to build '''sim-recon''' once and then work on only a small part of the software and rebuild that as needed. However, whenever the '''sim-recon''' directory was updated from the repository after a number of changes had been made, it was good practice to do a "make clean" and rebuild everything. In addition, certain power users would routinely need to check out the source and build from scratch on different platforms or to test different revisions.  </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>The solution to the slow-build issue is to do parallel builds where one takes advantage of multiple cores on a single computer by compiling, linking, and installing several things at the same time. The difficulty with this is that some files must be built before others. For example, a library must be built before it can be linked into a program. Good old "make" can handle this, but only if the Makefiles are written in such a way that it knows the entire dependency tree and can work out the sequencing. Alas, the BMS system was written to work ''recursively''. This means that the Makefile in the libraries directory would simply run another instance of ''make'' in its sub-directories. Thus, when the parallel feature of ''make'' was turned on, BMS would choke at some point and fail to complete the build. Two options were available (well, OK, probably  a lot more than 2, but these two were the only ones that made sense for our narrow section of the code-niverse). The first was to re-write the Makefiles to work using includes that would allow it to form the entire dependency tree and thus, efficiently use parallel builds. The second was to do something similar with SCons. Both would really need to be written from scratch so the amount of work was assumed to be more or less equivalent. An SCons based system had a few things that were appealing though:</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>The solution to the slow-build issue is to do parallel builds where one takes advantage of multiple cores on a single computer by compiling, linking, and installing several things at the same time. The difficulty with this is that some files must be built before others. For example, a library must be built before it can be linked into a program. Good old "make" can handle this, but only if the Makefiles are written in such a way that it knows the entire dependency tree and can work out the sequencing. Alas, the BMS system was written to work ''recursively''. This means that the Makefile in the libraries directory would simply run another instance of ''make'' in its sub-directories. Thus, when the parallel feature of ''make'' was turned on, BMS would choke at some point and fail to complete the build. Two options were available (well, OK, probably  a lot more than 2, but these two were the only ones that made sense for our narrow section of the code-niverse). The first was to re-write the Makefiles to work using includes that would allow it to form the entire dependency tree and thus, efficiently use parallel builds. The second was to do something similar with SCons. Both would really need to be written from scratch so the amount of work was assumed to be more or less equivalent. An SCons based system had a few things that were appealing though:</div></td></tr>
</table>Markihttps://halldweb1.jlab.org/wiki/index.php?title=SCons_Build_System&diff=59269&oldid=prevDavidl at 01:49, 26 July 20142014-07-26T01:49:12Z<p></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 01:49, 26 July 2014</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 36:</td>
<td colspan="2" class="diff-lineno">Line 36:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>=== Simple build with one thread ===</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>=== Simple build with one thread ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If running from the top-most directory (this should be named ''src'' and should contain the ''SConstruct'' file)</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If running from the top-most directory (this should be named ''src'' and should contain the ''SConstruct'' file)</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>  scons</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>  scons <ins class="diffchange diffchange-inline">install</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If running in a subdirectory, you need to add the "-u" option to tell scons to search parent directories until it finds the ''SConstruct'' file.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If running in a subdirectory, you need to add the "-u" option to tell scons to search parent directories until it finds the ''SConstruct'' file.</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>  scons -u</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>  scons -u <ins class="diffchange diffchange-inline">install</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>=== Simple build with multiple threads ===</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>=== Simple build with multiple threads ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If running from the top-most directory with 4 threads (this should be named ''src'' and should contain the ''SConstruct'' file)</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If running from the top-most directory with 4 threads (this should be named ''src'' and should contain the ''SConstruct'' file)</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>  scons -j4</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>  scons -j4 <ins class="diffchange diffchange-inline">install</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If running in a subdirectory with 4 threads, you need to add the "-u" option to tell scons to search parent directories until it finds the ''SConstruct'' file.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If running in a subdirectory with 4 threads, you need to add the "-u" option to tell scons to search parent directories until it finds the ''SConstruct'' file.</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>  scons -u -j4</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>  scons -u -j4 <ins class="diffchange diffchange-inline">install</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>You can (and probably should) specify the number of threads such that it equals the number of cores on the computer you are using. Note that I did see issues on my MacBook Pro where compiling with 4 threads where it eventually used up all of the RAM and caused it to go into swap mode which slowed the whole machine to a crawl. You should keep an eye on the RAM usage to know how many threads it's safe to use.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>You can (and probably should) specify the number of threads such that it equals the number of cores on the computer you are using. Note that I did see issues on my MacBook Pro where compiling with 4 threads where it eventually used up all of the RAM and caused it to go into swap mode which slowed the whole machine to a crawl. You should keep an eye on the RAM usage to know how many threads it's safe to use.</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 55:</td>
<td colspan="2" class="diff-lineno">Line 55:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>Run this from any directory, but only files associated with that directory or its direct descendants will be cleaned.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>Run this from any directory, but only files associated with that directory or its direct descendants will be cleaned.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>  scons -c</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>  scons <ins class="diffchange diffchange-inline">install </ins>-c</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>You can also use the "-j" option here to do this multi-threaded, but I don't think any speedup will be very noticeable.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>You can also use the "-j" option here to do this multi-threaded, but I don't think any speedup will be very noticeable.</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 63:</td>
<td colspan="2" class="diff-lineno">Line 63:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>By default, SBMS hides the sometimes quite verbose compile and link commands. To show these commands as they are executed, set the SHOWBUILD variable to "1" (note that this was adopted from the ''gemc'' build system used for CLAS12)</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>By default, SBMS hides the sometimes quite verbose compile and link commands. To show these commands as they are executed, set the SHOWBUILD variable to "1" (note that this was adopted from the ''gemc'' build system used for CLAS12)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>  scons -u -j4 SHOWBUILD=1</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>  scons -u -j4 <ins class="diffchange diffchange-inline">install </ins>SHOWBUILD=1</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>The above example would be used for running in a subdirectory and with 4 threads. SCons is smart enough not to intermix commands from multiple threads when printing to the screen so this is safe.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>The above example would be used for running in a subdirectory and with 4 threads. SCons is smart enough not to intermix commands from multiple threads when printing to the screen so this is safe.</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 73:</td>
<td colspan="2" class="diff-lineno">Line 73:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>   </div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>   </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>  install directory:  src/../${BMS_OSNAME}  (aka  sim-recon/${BMS_OSNAME} )</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>  install directory:  src/../${BMS_OSNAME}  (aka  sim-recon/${BMS_OSNAME} )</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">You'll notice in all of the commands given above, the "''install''" target is specified explicitly. Without this, the binaries will still be built, but only in the variant dir. This is a useful feature if you are modifying the source in a distribution that you want to make sure builds cleanly before the binaries are deployed to the install directory. This follows the standard form used when distributing software for unix systems that typically used separate "make" and "make install" phases.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
</table>Davidlhttps://halldweb1.jlab.org/wiki/index.php?title=SCons_Build_System&diff=54668&oldid=prevDavidl: Add instructions for using OptionallyBuild()2014-02-18T12:59:14Z<p>Add instructions for using OptionallyBuild()</p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 12:59, 18 February 2014</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 85:</td>
<td colspan="2" class="diff-lineno">Line 85:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>== Adding to the ''sim-recon'' source tree ==</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>== Adding to the ''sim-recon'' source tree ==</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>If you wish to add a library, program, or plugin to the ''sim-recon'' source tree, the easiest thing to do is find an existing item and duplicate it. Create a directory at the appropriate place for you source and you can probably copy the SConscript file from a sibling directory without modification. You will, though <del class="diffchange diffchange-inline">probably </del>need to modify the SConscript file in the parent directory to include your new directory in the list of places it needs to read SConscript files from.</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>If you wish to add a library, program, or plugin to the ''sim-recon'' source tree, the easiest thing to do is find an existing item and duplicate it. Create a directory at the appropriate place for you source and you can probably copy the SConscript file from a sibling directory without modification. You will, though need to modify the SConscript file in the parent directory to include your new directory in the list of places it needs to read SConscript files from.</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">There are two options when specifying your new directory in its parent's SConscript file. The first will automatically build your package whenever the rest of the source tree is built. The second will only build it if scons is invoked while in your directory or, alternatively, in its parent's, but with your directory explicitly listed on the command line. Note that if you go for the first option (and assuming you committed this to the subversion repository), then you are declaring this directory a required part of the sim-recon build. This will then be required for all sim-recon users so be prepared to maintain this package and respond to issues that may arise on various platforms people build and run sim-recon on.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">To add the package as a required part of sim-recon, add the directory to the ''subdirs'' list:</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"> subdirs = ['acceptance_hists', 'b1pi_hists', 'MyNewPackage']</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"> SConscript(dirs=subdirs, exports='env osname', duplicate=0)</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">To add the package as an optional one that doesn't get built unless the user explicitly requests it:</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"> sbms.OptionallyBuild(env, ['phys_tree', 'mc_tree', 'MyNewPackage'])</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>== Extending SBMS ==</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>== Extending SBMS ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If you need to add support for a new package to SBMS then there are two ways to do it. The first is to modify the ''src/SBMS/sbms.py" file to add support for the desired package (there are multiple examples already in there). The second is to put this support in a new file in the "src/SBMS" directory. This directory is added to the PYTHONPATH automatically in the top-level SConstruct file so anything you put there can be easily brought in to an SConscript file using the standard "import" syntax from Python.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If you need to add support for a new package to SBMS then there are two ways to do it. The first is to modify the ''src/SBMS/sbms.py" file to add support for the desired package (there are multiple examples already in there). The second is to put this support in a new file in the "src/SBMS" directory. This directory is added to the PYTHONPATH automatically in the top-level SConstruct file so anything you put there can be easily brought in to an SConscript file using the standard "import" syntax from Python.</div></td></tr>
</table>Davidlhttps://halldweb1.jlab.org/wiki/index.php?title=SCons_Build_System&diff=52489&oldid=prevDavidl: /* Features */2013-11-05T16:25:28Z<p><span dir="auto"><span class="autocomment">Features</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 16:25, 5 November 2013</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 24:</td>
<td colspan="2" class="diff-lineno">Line 24:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># Final library file names are based on the directory name. For example, a library whose source is contained in a directory named "TOF" will have all of the source in that directory compiled and placed in a library named "libTOF.a". This is to make it easier for someone trying to find the code for a library, or the library that is made from some code.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># Final library file names are based on the directory name. For example, a library whose source is contained in a directory named "TOF" will have all of the source in that directory compiled and placed in a library named "libTOF.a". This is to make it easier for someone trying to find the code for a library, or the library that is made from some code.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># Final program file names are based either on the source file name defining "main()". For example, the ''<del class="diffchange diffchange-inline">hd_ana</del>'' program will have it's "main()" routine defined in a file named ''<del class="diffchange diffchange-inline">hd_ana</del>.cc'' (or <del class="diffchange diffchange-inline">possible </del>''<del class="diffchange diffchange-inline">hd_ana</del>.F'' or ''<del class="diffchange diffchange-inline">hd_ana</del>.c'', ... you get the idea)</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># Final program file names are based either on the source file name defining "main()" <ins class="diffchange diffchange-inline">or the directory name holding the code (sbms.executables vs. sbms.executable)</ins>. For example, the ''<ins class="diffchange diffchange-inline">hdgeant</ins>'' program will have it's "main()" routine defined in a file named ''<ins class="diffchange diffchange-inline">hdgeant</ins>.cc'' (or <ins class="diffchange diffchange-inline">possibly </ins>''<ins class="diffchange diffchange-inline">hdgeant</ins>.F'' or ''<ins class="diffchange diffchange-inline">hdgeant</ins>.c'', ... you get the idea)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:* Multiple programs in a single source directory are supported, but all source files that do not define "main()" will have their object files linked into all programs made from source that directory.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:* Multiple programs in a single source directory are supported, but all source files that do not define "main()" will have their object files linked into all programs made from source that directory.</div></td></tr>
</table>Davidlhttps://halldweb1.jlab.org/wiki/index.php?title=SCons_Build_System&diff=52487&oldid=prevDavidl: /* Features */2013-11-05T15:53:32Z<p><span dir="auto"><span class="autocomment">Features</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 15:53, 5 November 2013</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 18:</td>
<td colspan="2" class="diff-lineno">Line 18:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># Supports parallel builds! Just invoke with the "-j" argument. e.g. to run with 16 threads do "scons -j16" (or "scons -u -j16" if not in ''src'')</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># Supports parallel builds! Just invoke with the "-j" argument. e.g. to run with 16 threads do "scons -j16" (or "scons -u -j16" if not in ''src'')</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># Centralized configuration. Most SConscript files are very simple and often duplicate others. This allows almost all of the details on how to build to be kept in a central location so that they apply to the entire source tree. For example, the list of DANA libraries exists in only one place <del class="diffchange diffchange-inline">to </del>adding a new library will not require one to edit 50 SConscript files!</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># Centralized configuration. Most SConscript files are very simple and often duplicate others. This allows almost all of the details on how to build to be kept in a central location so that they apply to the entire source tree. For example, the list of DANA libraries exists in only one place <ins class="diffchange diffchange-inline">so </ins>adding a new library will not require one to edit 50 SConscript files!</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># In most cases, all source files in a given directory are compiled. For programs and plugins, the compiled objects are all linked into the final executable. For libraries, all objects are added to the library. (This is to try and force better housekeeping.)</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># In most cases, all source files in a given directory are compiled. For programs and plugins, the compiled objects are all linked into the final executable. For libraries, all objects are added to the library. (This is to try and force better housekeeping.)</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 24:</td>
<td colspan="2" class="diff-lineno">Line 24:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># Final library file names are based on the directory name. For example, a library whose source is contained in a directory named "TOF" will have all of the source in that directory compiled and placed in a library named "libTOF.a". This is to make it easier for someone trying to find the code for a library, or the library that is made from some code.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># Final library file names are based on the directory name. For example, a library whose source is contained in a directory named "TOF" will have all of the source in that directory compiled and placed in a library named "libTOF.a". This is to make it easier for someone trying to find the code for a library, or the library that is made from some code.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># Final program file names are based on the source file name defining "main()". For example, the ''hd_ana'' program will have it's "main()" routine defined in a file named ''hd_ana.cc'' (or possible ''hd_ana.F'' or ''hd_ana.c'', ... you get the idea)</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># Final program file names are based <ins class="diffchange diffchange-inline">either </ins>on the source file name defining "main()". For example, the ''hd_ana'' program will have it's "main()" routine defined in a file named ''hd_ana.cc'' (or possible ''hd_ana.F'' or ''hd_ana.c'', ... you get the idea)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:* Multiple programs in a single source directory are supported, but all source files that do not define "main()" will have their object files linked into all programs made from source that directory.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:* Multiple programs in a single source directory are supported, but all source files that do not define "main()" will have their object files linked into all programs made from source that directory.</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 31:</td>
<td colspan="2" class="diff-lineno">Line 31:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>#:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># Cleaning via "scons -c" will only clean the current directory and its direct descendants. Note that this contradicts the default SCons behavior which is to clean all dependencies even ones in sibling and cousin directories (e.g. when doing a clean in a program directory that depends on libraries, the library directories would also be cleaned if the default SCons behavior were used). The entire source tree is cleaned if one invokes ''scons -c'' from the top-level ''src'' directory.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div># Cleaning via "scons -c" will only clean the current directory and its direct descendants. Note that this contradicts the default SCons behavior which is to clean all dependencies even ones in sibling and cousin directories (e.g. when doing a clean in a program directory that depends on libraries, the library directories would also be cleaned if the default SCons behavior were used). The entire source tree is cleaned if one invokes ''scons -c'' from the top-level ''src'' directory.</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>== Examples ==</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>== Examples ==</div></td></tr>
</table>Davidlhttps://halldweb1.jlab.org/wiki/index.php?title=SCons_Build_System&diff=52387&oldid=prevDavidl at 14:18, 30 October 20132013-10-30T14:18:09Z<p></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 14:18, 30 October 2013</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 88:</td>
<td colspan="2" class="diff-lineno">Line 88:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If you wish to add a library, program, or plugin to the ''sim-recon'' source tree, the easiest thing to do is find an existing item and duplicate it. Create a directory at the appropriate place for you source and you can probably copy the SConscript file from a sibling directory without modification. You will, though probably need to modify the SConscript file in the parent directory to include your new directory in the list of places it needs to read SConscript files from.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If you wish to add a library, program, or plugin to the ''sim-recon'' source tree, the easiest thing to do is find an existing item and duplicate it. Create a directory at the appropriate place for you source and you can probably copy the SConscript file from a sibling directory without modification. You will, though probably need to modify the SConscript file in the parent directory to include your new directory in the list of places it needs to read SConscript files from.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>== <del class="diffchange diffchange-inline">Adding to </del>SBMS ==</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>== <ins class="diffchange diffchange-inline">Extending </ins>SBMS ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If you need to add support for a new package to SBMS then there are two ways to do it. The first is to modify the ''src/SBMS/sbms.py" file to add support for the desired package (there are multiple examples already in there). The second is to put this support in a new file in the "src/SBMS" directory. This directory is added to the PYTHONPATH automatically in the top-level SConstruct file so anything you put there can be easily brought in to an SConscript file using the standard "import" syntax from Python.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If you need to add support for a new package to SBMS then there are two ways to do it. The first is to modify the ''src/SBMS/sbms.py" file to add support for the desired package (there are multiple examples already in there). The second is to put this support in a new file in the "src/SBMS" directory. This directory is added to the PYTHONPATH automatically in the top-level SConstruct file so anything you put there can be easily brought in to an SConscript file using the standard "import" syntax from Python.</div></td></tr>
</table>Davidlhttps://halldweb1.jlab.org/wiki/index.php?title=SCons_Build_System&diff=52386&oldid=prevDavidl: Created page with "==History== ''10/30/2013 David Lawrence'' The first general build system used by Hall-D was called the [http://argus.phys.uregina.ca/cgi-bin/private/DocDB/ShowDocument?docid=47..."2013-10-30T14:17:43Z<p>Created page with "==History== ''10/30/2013 David Lawrence'' The first general build system used by Hall-D was called the [http://argus.phys.uregina.ca/cgi-bin/private/DocDB/ShowDocument?docid=47..."</p>
<p><b>New page</b></p><div>==History==<br />
''10/30/2013 David Lawrence''<br />
<br />
The first general build system used by Hall-D was called the [http://argus.phys.uregina.ca/cgi-bin/private/DocDB/ShowDocument?docid=473 Build Management System (BMS)] and was base on the standard "make" utility. This was chosen [http://argus.phys.uregina.ca/cgi-bin/private/DocDB/ShowDocument?docid=71 for a couple of reasons] over a few competitors, (including the then new '''scons''' tool). Despite the lack of creativity in the name, the BMS system was successfully used for a number of years to build the [https://halldsvn.jlab.org/repos/trunk/sim-recon/ sim-recon] software for the GlueX experiment in Hall-D. As the amount of source code in sim-recon grew, the compile time required grew too. By 2013, it would take 15-20 minutes to build '''sim-recon''' from a fresh checkout of the code. This was not a problem for everyone since the most common use-case was to build '''sim-recon''' once and then work on only a small part of the software and rebuild that as needed. However, whenever the '''sim-recon''' directory was updated from the repository after a number of changes had been made, it was good practice to do a "make clean" and rebuild everything. In addition, certain power users would routinely need to check out the source and build from scratch on different platforms or to test different revisions. <br />
<br />
The solution to the slow-build issue is to do parallel builds where one takes advantage of multiple cores on a single computer by compiling, linking, and installing several things at the same time. The difficulty with this is that some files must be built before others. For example, a library must be built before it can be linked into a program. Good old "make" can handle this, but only if the Makefiles are written in such a way that it knows the entire dependency tree and can work out the sequencing. Alas, the BMS system was written to work ''recursively''. This means that the Makefile in the libraries directory would simply run another instance of ''make'' in its sub-directories. Thus, when the parallel feature of ''make'' was turned on, BMS would choke at some point and fail to complete the build. Two options were available (well, OK, probably a lot more than 2, but these two were the only ones that made sense for our narrow section of the code-niverse). The first was to re-write the Makefiles to work using includes that would allow it to form the entire dependency tree and thus, efficiently use parallel builds. The second was to do something similar with SCons. Both would really need to be written from scratch so the amount of work was assumed to be more or less equivalent. An SCons based system had a few things that were appealing though:<br />
<br />
# The CLAS12 group in Hall-B had been using scons successfully for a few years<br />
# SCons is based on Python which continues to become more popular<br />
#* (the online group in Hall-D had plans to use it as the default scripting language for online systems)<br />
# I would have had to spend the next 10 years trying to defend "''make'' to those in the "Nuevo esta Grande" Gang<br />
<br />
So, an effort to create a build system based on SCons was started. The name SBMS just stands for "SCons Build Management System" which pays homage to the old BMS color-challenged name.<br />
<br />
==Features==<br />
SBMS implements the following features/conventions:<br />
<br />
# Supports parallel builds! Just invoke with the "-j" argument. e.g. to run with 16 threads do "scons -j16" (or "scons -u -j16" if not in ''src'')<br />
#:<br />
# Centralized configuration. Most SConscript files are very simple and often duplicate others. This allows almost all of the details on how to build to be kept in a central location so that they apply to the entire source tree. For example, the list of DANA libraries exists in only one place to adding a new library will not require one to edit 50 SConscript files!<br />
#:<br />
# In most cases, all source files in a given directory are compiled. For programs and plugins, the compiled objects are all linked into the final executable. For libraries, all objects are added to the library. (This is to try and force better housekeeping.)<br />
#:<br />
# Final library file names are based on the directory name. For example, a library whose source is contained in a directory named "TOF" will have all of the source in that directory compiled and placed in a library named "libTOF.a". This is to make it easier for someone trying to find the code for a library, or the library that is made from some code.<br />
#:<br />
# Final program file names are based on the source file name defining "main()". For example, the ''hd_ana'' program will have it's "main()" routine defined in a file named ''hd_ana.cc'' (or possible ''hd_ana.F'' or ''hd_ana.c'', ... you get the idea)<br />
#:<br />
#:* Multiple programs in a single source directory are supported, but all source files that do not define "main()" will have their object files linked into all programs made from source that directory.<br />
#:<br />
# Final plugin names are based on the directory name just as with libraries.<br />
#:<br />
# Cleaning via "scons -c" will only clean the current directory and its direct descendants. Note that this contradicts the default SCons behavior which is to clean all dependencies even ones in sibling and cousin directories (e.g. when doing a clean in a program directory that depends on libraries, the library directories would also be cleaned if the default SCons behavior were used). The entire source tree is cleaned if one invokes ''scons -c'' from the top-level ''src'' directory.<br />
<br />
<br />
== Examples ==<br />
<br />
=== Simple build with one thread ===<br />
If running from the top-most directory (this should be named ''src'' and should contain the ''SConstruct'' file)<br />
scons<br />
<br />
If running in a subdirectory, you need to add the "-u" option to tell scons to search parent directories until it finds the ''SConstruct'' file.<br />
scons -u<br />
<br />
<br />
=== Simple build with multiple threads ===<br />
If running from the top-most directory with 4 threads (this should be named ''src'' and should contain the ''SConstruct'' file)<br />
scons -j4<br />
<br />
If running in a subdirectory with 4 threads, you need to add the "-u" option to tell scons to search parent directories until it finds the ''SConstruct'' file.<br />
scons -u -j4<br />
<br />
You can (and probably should) specify the number of threads such that it equals the number of cores on the computer you are using. Note that I did see issues on my MacBook Pro where compiling with 4 threads where it eventually used up all of the RAM and caused it to go into swap mode which slowed the whole machine to a crawl. You should keep an eye on the RAM usage to know how many threads it's safe to use.<br />
<br />
<br />
=== Cleaning ===<br />
Run this from any directory, but only files associated with that directory or its direct descendants will be cleaned.<br />
<br />
scons -c<br />
<br />
You can also use the "-j" option here to do this multi-threaded, but I don't think any speedup will be very noticeable.<br />
<br />
<br />
=== Show the build commands ===<br />
By default, SBMS hides the sometimes quite verbose compile and link commands. To show these commands as they are executed, set the SHOWBUILD variable to "1" (note that this was adopted from the ''gemc'' build system used for CLAS12)<br />
<br />
scons -u -j4 SHOWBUILD=1<br />
<br />
The above example would be used for running in a subdirectory and with 4 threads. SCons is smart enough not to intermix commands from multiple threads when printing to the screen so this is safe.<br />
<br />
== Install Location==<br />
SBMS uses the [http://www.scons.org/doc/HTML/scons-user.html#chap-separate Variant Directory] feature to place built binary files outside of the source tree. This helps keep the source tree less cluttered. The variant directory is really a mirror directory structure of the source directory tree and should be considered a temporary staging area for binaries, including the object files. The installation directory is different and contains only the final libraries+headers, programs, and plugins. The topmost directory of the variant directory is a hidden directory in ''src'' whose name is simply "src/.$BMS_OSNAME" where the BMS_OSNAME environment variable is used to provide a platform dependent directory name. The installation directory is parallel to the src directory and does not contain the "." in the name so that it is not hidden.<br />
<br />
variant directory: src/.${BMS_OSNAME} (aka sim-recon/src/.${BMS_OSNAME} )<br />
<br />
install directory: src/../${BMS_OSNAME} (aka sim-recon/${BMS_OSNAME} )<br />
<br />
<br />
Inside the install directory a standard directory structure exists:<br />
<br />
├── Linux_CentOS6-x86_64-clang3.2<br />
│ ├── bin<br />
│ ├── include<br />
│ ├── plugins<br />
│ └── lib<br />
<br />
<br />
== Adding to the ''sim-recon'' source tree ==<br />
If you wish to add a library, program, or plugin to the ''sim-recon'' source tree, the easiest thing to do is find an existing item and duplicate it. Create a directory at the appropriate place for you source and you can probably copy the SConscript file from a sibling directory without modification. You will, though probably need to modify the SConscript file in the parent directory to include your new directory in the list of places it needs to read SConscript files from.<br />
<br />
== Adding to SBMS ==<br />
If you need to add support for a new package to SBMS then there are two ways to do it. The first is to modify the ''src/SBMS/sbms.py" file to add support for the desired package (there are multiple examples already in there). The second is to put this support in a new file in the "src/SBMS" directory. This directory is added to the PYTHONPATH automatically in the top-level SConstruct file so anything you put there can be easily brought in to an SConscript file using the standard "import" syntax from Python.</div>Davidl