Versions Compared

Old Version 20

changes.mady.by.user David William Botsch

Saved on

compared with

New Version Current

changes.mady.by.user David William Botsch

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Cluster charges are based on accessing the cluster and running jobs on the cluster – a single monthly "all you can eat" fee is charged for each month during which the cluster is used.

All user processes must be run as cluster jobs in the cluster job queuing system.

Head Node:

The head node is a Dell PowerEdge R740xd with approximately 9 TeraBytes of shared disk space available to users. Via NFS, user home directories are shared out to all of the compute nodes.

...

This class comprises two Dell PowerEdge R640 systems. Each node has 256GB of memory and 2 Intel Xeon Gold 6136 processors. Each individual processor has 12 cores and 2 threads per core. This equals 48 virtual CPUs per node.

rb2u Compute Nodes:

At present, three Eleven of the twelve nodes in this class are online . We are working to provision the rest. (one hardware failure). Each node has 32GB of memory and 2 Intel Xeon E5-2620 processors. Each individual processor has 6 cores and 2 threads per core. This equals 24 virtual CPUs per node.

rb1u Compute Nodes:

None Six of these nodes are yet available. We are working to provision them the rest once a new network switch is installed. Each node has 24GB of memory and 2 Intel Xeon DP E56020 processors. Each individual processor has 4 cores and 2 threads per core. This equals 16 virtual CPUs per node.

Network Switches:

Two Dell S3148 network switches are installed to connect all the nodes. Each node is connected to the switch via Gigabit Ethernet.

Scientific Codes on the Cluster:

If you don't see a code listed below, please let us know. We will do our best to install new codes upon your request. We will also install commercial codes where you Bring Your Own License.

MPI Families

OpenMPIhttp://www.open-mpi.orgThe Open MPI Project is an open source  Message Passing Interface  implementation that is developed and maintained by a consortium of academic, research, and industry partners.
MPICHhttp://www.mpich.orgMPICH is a high-performance and widely portable implementation of the Message Passing Interface (MPI) standard (MPI-1, MPI-2 and MPI-3). 

...

SciPyhttp://www.scipy.orgSciPy is a Python-based ecosystem of open-source software for mathematics, science, and engineering
NumPyhttp://www.numpy.orgBase N-dimensional array package
pyspreadhttps://pyspread.gitlab.io/pyspread is a non-traditional spreadsheet application that is based on and written in the programming language Python


Code Block
module load py3.6-numpy
module load pyspread
pyspread


IO Libraries

Adioshttp://www.olcf.ornl.gov/center-projects/adios

The Adaptable IO System (ADIOS).

HDF5/PHDF5http://www.hdfgroup.org/HDF5

A general purpose library and file format for storing scientific data.

NetCDFhttp://www.unidata.ucar.edu/software/netcdfFortran, C, and C++ libraries for the Unidata network Common Data
PnetCDFhttp://cucis.ece.northwestern.edu/projects/PnetCDF

A Parallel NetCDF library

ScionLibhttp://www.fz-juelich.de/ias/jsc/EN/Expertise/Support/Software/SIONlib/_node.html

Scalable I/O Library for Parallel Access to Task-Local Files.

...

Boosthttp://www.boost.org

Free peer-reviewed portable C++ source libraries.

FFTWhttp://www.fftw.org

Fast Fourier Transform library.

Hyprehttp://www.llnl.gov/casc/hypre

Scalable algorithms for solving linear systems of equations.

MFEMhttp://mfem.org

Lightweight, general, scalable C++ library for finite element methods.

MUMPShttp://mumps.enseeiht.fr

A MUltifrontal Massively Parallel Sparse direct Solver.

PETSChttp://www.mcs.anl.gov/petsc

Portable Extensible Toolkit for Scientific Computation.

PT-SCOTCHhttp://www.labri.fr/perso/pelegrin/scotch

Graph, mesh and hypergraph partitioning library using MPI.

SLEPC http://slepc.upv.esA library for solving large scale sparse eigenvalue problems.
SuperLUhttp://crd.lbl.gov/~xiaoye/SuperLUA general purpose library for the direct solution of linear equations.
Trilinos https://trilinos.org/An object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems. 

Capacitance Extraction/Field Solving

FasterCaphttps://www.fastfieldsolvers.com/fastercap.htmA powerful three- and two-dimensional capactiance extraction program


Code Block
languagebash
/usr/local/fastercap/FasterCap_6.0.7/FasterCap [-b]


Density Functional Approaches

ABINITGNU7/OpenMPIhttps://www.abinit.org/

ABINIT is a software suite to calculate the optical, mechanical, vibrational, and other observable properties of materials. Starting from the quantum equations of density functional theory, you can build up to advanced applications with perturbation theories based on DFT, and many-body Green's functions (GW and DMFT) .
ABINIT can calculate molecules, nanostructures and solids with any chemical composition, and comes with several complete and robust tables of atomic potentials.


Code Block
languagebash
module load lapack
module load blas
module load abinit


Device

...

, Process, Particle, and Finite Element Method Simulation

Archimedes/Rappture 
https://www.gnu.org/software/archimedes
 https://nanohub.org/infrastructure/rappture
Archimedes belongs to the well-known family of TCAD software, i.e. tools utilized to assist the development of technologically relevant products. In particular, this package assists engineers in designing and simulating submicron and mesoscopic semiconductor devices.
Rappture is the GUI for Archimedes.


Code Block
languagebash
module load rappture

...


DevSim
https://devsim.org/
DEVSIM is semiconductor device simulation software which uses the finite volume method. It solves partial differential equations on a mesh. The Python interface allows users to specify their own equations.

 Code Block
languagebash
bash
module load devsim
source /usr/local/intel/mkl/bin/mklvars.sh intel64

ElmerGNU7/OpenMPIhttps://www.csc.fi/web/elmer/elmer
Elmer is an open source multiphysical simulation software mainly developed by CSC - IT Center for Science (CSC). Elmer development was started 1995 in collaboration with Finnish Universities, research institutes and industry. After it's open source publication in 2005, the use and development of Elmer has become international.
Elmer includes physical models of fluid dynamics, structural mechanics, electromagnetics, heat transfer and acoustics, for example. These are described by partial differential equations which Elmer solves by the Finite Element Method (FEM).

 Code Block
languagebash
module load elmer

HOOME-blueSingularity
http://glotzerlab.engin.umich.edu/hoomd-blue/index.html
https://glotzerlab-software.readthedocs.io/en/latest/software.html
https://glotzerlab-software.readthedocs.io/en/latest/
HOOMD-blue is a  general-purpose  particle simulation toolkit. It scales from a single CPU to thousands.

 Code Block
languagebash
module load singularity
# Use this package for non-mpi jobs
singularity exec /usr/local/glotzerlab/glotzerlab-software-master-latest.simg [command-in-glotzerlab-package]
# Use this package for MPI jobs
singularity exec /usr/local/glotzerlab/glotzerlab-software-mpi-flux.simg [command-in-glotzerlab-package]

OpenFOAM v6 and v8
https://openfoam.comA Computational Fluid Dynamics (CFD) package. OpenFOAM has an extensive range of features to solve anything from complex fluid flows involving chemical reactions, turbulence and heat transfer, to solid dynamics and electromagnetics.

 Code Block
languagebash
bash
source /usr/local/openFOAM/OpenFOAM-6/etc/bashrc # for version 6
source /usr/local/openFOAM/OpenFOAM-8/etc/bashrc # for version 8
mkdir -p $FOAM_RUN
run


Linear Algebra Operations
 
      
LAPACK
http://www.netlib.org/lapack/LAPACK is written in Fortran 90 and provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. 

 Code Block
languagebash
module load lapack

SCALAPACKGNU7/OpenMPIhttp://www.netlib.org/scalapack/
ScaLAPACK is a library of high-performance linear algebra routines for parallel distributed memory machines. ScaLAPACK solves dense and banded linear systems, least squares problems, eigenvalue problems, and singular value problems.

 Code Block
languagebash
module load scalapack
...


Math Libraries
      
Intel Math Kernel Library (MKL) 
https://software.intel.com/en-us/mkl
Intel Math Kernel Library (Intel MKL) is a library of optimized math routines for science, engineering, and financial applications. Core math functions include BLAS, LAPACK, ScaLAPACK, sparse solvers, fast Fourier transforms, and vector math. The routines in MKL are hand-optimized specifically for Intel processors.

 Code Block
languagebash
bash
source /usr/local/intel/mkl/bin/mklvars.sh intel64
 


Matrix and Vector Operations
     
BLAShttp://netlib.org/blas/The BLAS (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations.
(OpenBLAS is also available... see above)

 Code Block
languagebash
module load blas
...


Tomographic Data Processing and Image Reconstruction
     
TomoPy
https://tomopy.readthedocs.io/
https://tomopycli.readthedocs.io/
Tomographic reconstruction creates three-dimensional views of an object by combining two-dimensional images taken from multiple directions, for example, this is how a CAT (computer-aided tomography) scanner generates 3D views of the heart or brain.
Included is the tomopy-cli commandline interface.

 Code Block
languagebash
bash
module load miniconda
conda activate tomopy
source /usr/local/miniconda3/envs/tomopy/bin/complete_tomopy.sh


Cluster HowTos: