Laboratory Computing Resource Center

Lots of news this time. Here are the items that follow:

• KNL Jobs Now “Cost” Less
• Login Node Etiquette
• Shared Queue on Bebop
• Perfect Your Image in Print
• Jenkins Continuous Integration
• Requesting More Time
• Upcoming Changes to Blues
• Tons of New and Updated Software

KNL Jobs Now “Cost” Less

Since Bebop arrived last summer, we have been benchmarking the Broadwell and KNL nodes, and collecting user experiences with their performance. The consensus is that a KNL core is not as productive as a Broadwell core for most LCRC codes. To compensate, starting April 1, 2018, a KNL core hour will cost 0.585 bank core-hours, while a Broadwell core hour stays at 1.0 bank core-hours. It now becomes advantageous to preferentially run optimized community codes such as VASP, Quantum Espresso, NAMD and LAMMPS on the KNLs. Notes on how to use these codes are available on the LCRC web site. We may make further adjustments in the future, based on more experience with KNL.

Login Node Etiquette

Login nodes are only meant for operations such as editing, copying, compilation and submitting jobs to the compute nodes. The login nodes are used by our whole community. Please do not run long jobs (> 10 minutes) on the login nodes as they slow down the response time of the node to other users. You can submit short jobs (< 1 hr) for testing/debugging on the shared queue (see below). MPI has been disabled on login nodes (due to misuse), and long jobs running on a login node will be killed by the system administrators.

Shared Queue on Bebop

LCRC is now offering a shared queue on Bebop. The new job queue is named “bdws.”.
The main purpose of this queue is for users to have a place for short term testing of their workflow. This queue is comprised of 8 Broadwell nodes with a maximum walltime of 1 hour per job.

Each node in this queue allows up to 4 processes per CPU. This means you can have up to 4 full node jobs on one single node at a time, or any variation in between. There is no user limit, just this CPU limit, so nodes may have several users running at once until the CPU limit is reached. Because of this, you should not expect to see the performance you would expect on the normal queues. Jobs will be scheduled on the least used nodes first in order to help with performance.

LCRC users can submit to as many of these nodes as they require per job. Unlike the other Bebop queues, this queue only charges for the cores used, as opposed to a whole node per job. If you only use 18 cores for 1 hour, this is how much will be charged to your project.

Perfect Your Image in Print

Please remember to acknowledge LCRC in your publications, e.g., “We gratefully acknowledge use of the Bebop [or Blues] cluster in the Laboratory Computing Resource Center at Argonne National Laboratory.” Yes, we sometimes take a look at publications from list reported by the PI in their annual project report.

Jenkins Continuous Integration

LCRC is now running a Jenkins continuous integration server. LCRC users can login with their LCRC account to: https://jenkins-ci.lcrc.anl.gov
Builds that run on this service do not get charged any time to user projects.

What is Jenkins?
Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software. We currently have 1 build slave where builds/tasks get scheduled. This is a Broadwell node with the same specifications as our other Broadwell nodes. If you wish to use the LCRC software stack (via modules) in your job, please add this as the first step to your build process: source /etc/profile

Requesting More Time

PIs can request more time for their project at any point during the year via the project management web pages: https://accounts.lcrc.anl.gov/

When you request more time,
please provide a detailed account of how you came up with the requested amount. This should include the typical problem size (nodes/cores), the wall-time for a run, and total number of runs planned. These numbers should be based on runs that have already been conducted or projections based on previous experience on Bebop. Additional time will be for the current quarter on Bebop, unless you say otherwise. These requests are reviewed by the Allocations Committee in the context of the overall usage of the cluster. Smaller requests are handled by the LCRC staff.

Upcoming Changes to Blues

This Summer, we plan to upgrade the software on Blues to match Bebop, which will streamline operations and enable some new capabilities planned for Blues. This will involve changing Blues to CentOS 7.4, updating libraries and tools, and installing Modules and Slurm. We will provide details and guidance at a later date.

Tons of New and Updated Software