Difference between revisions of "HOWTO use AmpTools on the JLab farm GPUs"
From GlueXWiki
(→AmpTools Compilation with CUDA) |
(→AmpTools Compilation with CUDA) |
||
Line 28: | Line 28: | ||
=== AmpTools Compilation with CUDA === | === AmpTools Compilation with CUDA === | ||
− | This example was done in csh on sciml1902. | + | This example was done in csh for the Titan RTX cards on sciml1902. |
'''1)''' Download latest AmpTools release | '''1)''' Download latest AmpTools release | ||
Line 45: | Line 45: | ||
'''5)''' Put root-config in your path | '''5)''' Put root-config in your path | ||
setenv PATH $ROOTSYS/bin:$PATH | setenv PATH $ROOTSYS/bin:$PATH | ||
+ | |||
+ | '''6)''' Edit the Makefile to pass the appropriate GPU architecture for that machine to the cuda complier | ||
+ | CUDA_FLAGS := -m64 -arch=sm_75 | ||
+ | |||
+ | '''7)''' Build main AmpTools library with GPU support | ||
+ | make GPU=1 | ||
=== Performing Fits Interactively === | === Performing Fits Interactively === | ||
=== Submitting Batch Jobs === | === Submitting Batch Jobs === |
Revision as of 11:48, 26 April 2021
Contents
Access through SLURM
JLab currently provides 4 NVidia Titan RTX cards. They can be accessed through SLURM, where N is the number of requested cards (1-4):
>salloc --gres gpu:TitanRTX:N --partition gpu --nodes 1
An interactive shell (e.g. bash) on the node with requested allocation can be opened with srun:
>srun --pty bash
Information about the cards, cuda version and usage is displayed with this command:
>nvidia-smi +-----------------------------------------------------------------------------+ | NVIDIA-SMI 418.87.01 Driver Version: 418.87.01 CUDA Version: 10.1 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 TITAN RTX Off | 00000000:3E:00.0 Off | N/A | | 41% 27C P8 2W / 280W | 0MiB / 24190MiB | 0% Default | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+
AmpTools Compilation with CUDA
This example was done in csh for the Titan RTX cards on sciml1902.
1) Download latest AmpTools release
wget https://github.com/mashephe/AmpTools/archive/refs/tags/v0.11.0.tar.gz
2) Extract files
tar -xvf v0.11.0.tar.gz
3) Load cuda environment module
module add cuda setenv CUDA_INSTALL_PATH /usr/local/cuda
4) Set AMPTOOLS directory
setenv AMPTOOLS $PWD/AmpTools
5) Put root-config in your path
setenv PATH $ROOTSYS/bin:$PATH
6) Edit the Makefile to pass the appropriate GPU architecture for that machine to the cuda complier
CUDA_FLAGS := -m64 -arch=sm_75
7) Build main AmpTools library with GPU support
make GPU=1