Lots of news this time. Here are the items that follow:
- KNL Jobs Now “Cost” Less
- Login Node Etiquette
- Shared Queue on Bebop
- Perfect Your Image in Print
- Jenkins Continuous Integration
- Requesting More Time
- Upcoming Changes to Blues
- Tons of New and Updated Software
KNL Jobs Now “Cost” Less
Since Bebop arrived last summer, we have been benchmarking the Broadwell and KNL nodes, and collecting user experiences with their performance. The consensus is that a KNL core is not as productive as a Broadwell core for most LCRC codes. To compensate, starting April 1, 2018, a KNL core hour will cost 0.585 bank core-hours, while a Broadwell core hour stays at 1.0 bank core-hours. It now becomes advantageous to preferentially run optimized community codes such as VASP, Quantum Espresso, NAMD and LAMMPS on the KNLs. Notes on how to use these codes are available on the LCRC web site. We may make further adjustments in the future, based on more experience with KNL.
Login Node Etiquette
Login nodes are only meant for operations such as editing, copying, compilation and submitting jobs to the compute nodes. The login nodes are used by our whole community. Please do not run long jobs (> 10 minutes) on the login nodes as they slow down the response time of the node to other users. You can submit short jobs (< 1 hr) for testing/debugging on the shared queue (see below). MPI has been disabled on login nodes (due to misuse), and long jobs running on a login node will be killed by the system administrators.
Shared Queue on Bebop
LCRC is now offering a shared queue on Bebop. The new job queue is named “bdws.”.
The main purpose of this queue is for users to have a place for short term testing of their workflow. This queue is comprised of 8 Broadwell nodes with a maximum walltime of 1 hour per job.
Each node in this queue allows up to 4 processes per CPU. This means you can have up to 4 full node jobs on one single node at a time, or any variation in between. There is no user limit, just this CPU limit, so nodes may have several users running at once until the CPU limit is reached. Because of this, you should not expect to see the performance you would expect on the normal queues. Jobs will be scheduled on the least used nodes first in order to help with performance.
LCRC users can submit to as many of these nodes as they require per job. Unlike the other Bebop queues, this queue only charges for the cores used, as opposed to a whole node per job. If you only use 18 cores for 1 hour, this is how much will be charged to your project.
Perfect Your Image in Print
Please remember to acknowledge LCRC in your publications, e.g., “We gratefully acknowledge use of the Bebop [or Blues] cluster in the Laboratory Computing Resource Center at Argonne National Laboratory.” Yes, we sometimes take a look at publications from list reported by the PI in their annual project report.
Jenkins Continuous Integration
LCRC is now running a Jenkins continuous integration server. LCRC users can login with their LCRC account to: https://jenkins-ci.lcrc.anl.gov
Builds that run on this service do not get charged any time to user projects.
What is Jenkins?
Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software. We currently have 1 build slave where builds/tasks get scheduled. This is a Broadwell node with the same specifications as our other Broadwell nodes. If you wish to use the LCRC software stack (via modules) in your job, please add this as the first step to your build process: source /etc/profile
Requesting More Time
PIs can request more time for their project at any point during the year via the project management web pages: https://accounts.lcrc.anl.gov/
When you request more time,
please provide a detailed account of how you came up with the requested amount. This should include the typical problem size (nodes/cores), the wall-time for a run, and total number of runs planned. These numbers should be based on runs that have already been conducted or projections based on previous experience on Bebop. Additional time will be for the current quarter on Bebop, unless you say otherwise. These requests are reviewed by the Allocations Committee in the context of the overall usage of the cluster. Smaller requests are handled by the LCRC staff.
Upcoming Changes to Blues
This Summer, we plan to upgrade the software on Blues to match Bebop, which will streamline operations and enable some new capabilities planned for Blues. This will involve changing Blues to CentOS 7.4, updating libraries and tools, and installing Modules and Slurm. We will provide details and guidance at a later date.
Tons of New and Updated Software
|Intel-17.0.4||Intel c/c++/Fortran programming languages|
|GCC-7.1.10||Gnu c/c++ programming languages|
|singularity-2.4.2||HPC container technology|
|arpack-ng-3.5.0||Collection of Fortran77 subroutines designed to solve large scale eigenvalue problems</td|
|boost-1.66.0||Boost provides free peer-reviewed portable C++ source libraries|
|chemzone-2.2||Simplified but extremely fast solver for analysis of plasma discharges and neutral reactive flows such as in combustion or chemical vapor deposition|
|cmake-3.10.1||Open-source, cross-platform family of tools designed to build, test and package software|
|darshan-3.1.5||Scalable HPC I/O characterization tool|
|eigen-3.3.3||High-level C++ library of template headers for linear algebra, matrix and vector operations, geometrical transformations, numerical solvers and related algorithms|
|fftw-3.3.7||Software library for computing discrete Fourier transforms developed by Matteo Frigo and Steven G. Johnson at the Massachusetts Institute of Technology|
|gdbm-1.13||GNU dbm (or GDBM, for short) is a library of database functions that use extensible hashing and work similar to the standard UNIX dbm|
|gettext-0.19.8.1||Internationalization and localization system commonly used for writing multilingual programs on Unix-like computer operating systems|
|hdf5-1.10.1||Data model, library, and file format for storing and managing data|
|intel-mkl-2018.1.163||Intel Math Kernel Library|
|libint-2.4.2||Software stack for computing integrals used in molecular quantum mechanics|
|libpsm2-10.3-17||Intel opa-psm2 library|
|llvm-5.0.1||Collection of modular and reusable compiler and toolchain technologies|
|lz4-1.7.5||LZ4 – Extremely fast compression|
|mathematica-10.2.0||High-powered computation with thousands of Wolfram Language functions, natural language input, real-world data, mobile support|
|metis-5.1.0||Set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices|
|mumps-5.1.1||Multifrontal Massively Parallel sparse direct Solver|
|noaa/wct-4.0.6||NOAA’s Weather and Climate Toolkit (WCT)|
|OpenFOAM-5.4.0||Open source software for CFD from the OpenFOAM Foundation|
|openmpi-1.10.7||A High Performance Message Passing Library|
|parallel-20170322||GNU parallel is a shell tool for executing jobs in parallel using one or more computers|
|parmetis-4.0.3||ParMETIS is an MPI-based parallel library that implements a variety of algorithms for partitioning unstructured graphs, meshes, and for computing fill-reducing orderings of sparse matrices|
|petsc-3.8.1||Suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations|
|pkgconf-1.4.0||Helps configure compiler and linker flags for development frameworks|
|py-appdirs-1.4.3||Module for determining appropriate platform-specific dirs|
|py-cython-0.25.2||C-Extensions for Python|
|py-h5py-2.7.1||A Pythonic interface to the HDF5 binary data format|
|py-mpi4py-3.0.0||Python Message Passing Library|
|py-numpy-1.13.3||NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices|
|py-packaging-16.8||Core utilities for Python packages|
|py-pyparsing-2.2.0||Alternative approach to creating and executing simple grammars, vs. the traditional lex/yacc approach, or the use of regular expressions|
|py-scipy-0.19.1||Python-based ecosystem of open-source software for mathematics, science, and engineering|
|py-setuptools-35.0.2||Fully-featured, actively-maintained, and stable library designed to facilitate packaging Python projects|
|py-six-1.10.0||Six is a Python 2 and 3 compatibility library|
|readline-7.0||Provides a set of functions for use by applications that allow users to edit command lines as they are typed in|
|rosetta-2017.52.59948||Library based object-oriented software suite which provides a robust system for predicting and designing protein structures, protein folding mechanisms, and protein-protein interactions|
|scotch-6.0.4||Software package and libraries for sequential and parallel graph partitioning, static mapping and clustering, sequential mesh and hypergraph partitioning, and sequential and parallel sparse matrix block ordering|
|sqlite-3.20.0||SQLite is a relational database management system contained in a C programming library|
|suite-sparse-5.1.0||Suite of sparse matrix software|
|SuiteSparse-4.5.6||Suite of sparse matrix software|
|superlu-5.2.1||SuperLU is a general purpose library for the direct solution of large, sparse, nonsymmetric systems of linear equations|
|superlu-dist-5.2.2||Distributed memory, MPI based SuperLU|
|sxdefectalign-2.2||The charge model|
|taskfarmer-0.2-1||Utility to farm tasks onto a compute node|
|tcl-8.6.6||Tool Command Language|
|tcpdx-1.4||TCP proxy utility|
|texlive-live||TeX Live is an easy way to get up and running with the TeX document production system|
|tkdiff-4.2||A graphical diff viewer based on the Tk framework|
|visit-2.13.0||Interactive, scalable, visualization, animation and analysis tool|