Frequently Asked Question
This article explores some of the most commonly used modules in HPC environments, categorizing them based on their functionality. From mathematical libraries like LAPACK and CUDA to data management tools such as HDF5 and NetCDF, data handling, and software development.
Math Libraries
- OpenBLAS: Optimized Basic Linear Algebra Subprograms.
- LAPACK: Linear Algebra Package for numerical linear algebra.
- FFTW: Fast Fourier Transform library.
- MKL: Intel Math Kernel Library for optimized mathematical computations.
- CUDA and cuDNN: NVIDIA's library and GPU-accelerated libraries for deep neural networks.
OpenBLAS is an open-source implementation of the Basic Linear Algebra Subprograms (BLAS) library, designed for high-performance linear algebra operations.
$ module use /app/utils/modules $ module load OpenBLAS-0.3.21
LAPACK (Linear Algebra Package) is a widely used library designed to perform a range of linear algebra computations, including solving systems of linear equations, eigenvalue problems, and singular value decompositions.
$ module use /app/utils/modules $ module load lapack-3.7.1
FFTW (Fastest Fourier Transform in the West) is an open-source library designed for computing the discrete Fourier transform (DFT) and its various generalizations efficiently.
$ module load fftw/3.3.4/intel-15.0.2
Note: Other releases and builds for FFTW are available.
Intel Math Kernel Library (MKL) is a highly optimized, widely used library for mathematical computations, particularly in the fields of scientific computing, engineering, and data analysis. It provides a comprehensive set of routines for various mathematical operations, including linear algebra, fast Fourier transforms (FFT), vector mathematics, and statistical functions.
$ module load composer_xe/2017.2.174
CUDA and cuDDN are highly optimized programming model and GPU-accelerated library developed by NVIDIA for deep learning applications.
$ module load cuda/12.1 # For cuda 12.1 $ module load cuda/11.8 # For cuda 11.8
Note: The above two modules are only accessible only from the A100 GPU nodes.
Data Formats and I/O Libraries
- HDF5: Hierarchical Data Format for storing and managing large datasets.
- NetCDF: Network Common Data Form for scientific data.
HDF5 (Hierarchical Data Format version 5) is an open-source file format and set of tools designed to manage, store, and organize large amounts of complex data.
$ module use /app/utils/modules $ module load hdf5-1.14.3
NetCDF (Network Common Data Form) is an open-source software library and data format designed for the creation, access, and sharing of scientific data.
$ module load netcdf/4.3.3.1/intel-15.0.1_parallel
Note: Other releases and builds for NetCDF are available.
Compression and General Utilities
- zlib: Compression library for data compression.
- Boost: Collection of C++ libraries for general programming.
zlib is a widely-used open-source library that provides data compression and decompression functionalities.
$ module use /apps/common/modules $ module load zlib-1.2.13
Boost is a collection of open-source, peer-reviewed libraries that extend the functionality of C++ by providing solutions for common programming challenges. These libraries cover a wide range of functionality, including data structures, algorithms, memory management, multithreading, and input/output (I/O) operations.
$ module use /apps/utils/modules $ module load boost/gcc/1_69_0
Build and Configuration Tool
CMake is an open-source, cross-platform build system generator that automates the process of compiling and building software projects. It is widely used to manage the build process in complex, multi-platform environments.
$ module use apps/utils/modules $ module load cmake-3.30.5
Note: There are other releases available on Aziz.
Programming and Scripting
- JDK: Java Development Kit for building and running Java applications.
- Go: Open-source programming language designed for efficient and scalable software development.
- MATLAB: High-level computing environment for numerical computation, visualization, and programming.
- R: Statistical computing and graphics language widely used in data analysis and research.
- Python: Versatile programming language with extensive libraries for scientific computing and machine learning.
The Java Development Kit is an open-source toolkit for developing and running Java applications. It provides essential tools such as a compiler, debugger, and libraries, enabling efficient Java software development across platforms.
$ module use apps/utils/modules $ module load jdk-9.0.4
Go is an open-source programming language designed for simplicity, efficiency, and scalability. It is widely used for building modern, high-performance applications, particularly in cloud computing and server-side development.
$ module use apps/utils/modules $ module load go-1.21.3
MATLAB is a high-level programming and interactive environment for numerical computation, data analysis, and visualization. It is extensively used in engineering, science, and academia for developing algorithms and modeling systems.
$ module use apps/common/modules $ module load matlab-R2022b
R is a powerful programming language and environment specifically tailored for statistical computing and graphics. It is popular in data science and research for tasks such as data manipulation, analysis, and visualization.
$ module use apps/utils/modules $ module load R-3.5.0
Python is a versatile, high-level programming language known for its readability and extensive libraries. It is widely used across domains, including web development, scientific computing, artificial intelligence, and data analysis.
$ module use apps/utils/modules $ module load anaconda3-2024.02