Using Jazz  |  Getting Started  |  About the LCRC  |  Presentations  |  Status  |  FAQ  |  Search   |   Main Page  

Welcome to the LCRC Linux Cluster: Jazz

Welcome to Argonne's Laboratory Computing Resource Center (LCRC). This is the Getting Started page.

The topics covered here should cover everything that a new user who is moderately experienced with clusters should need to know to get started on Jazz. Links to other documents with more information on specific topics are also provided.

Complete detailed documentation may be found on the Using Jazz pages.

How to... Information
Get Help The LCRC Support Team and Application Engineers can assist you with your questions and requests. For system related issues you will receive the fastest service with email sent to However, phone and in person assistance is available during business hours.

We hope you can find much of the information you'll need through the LCRC web pages. The home page is From that page, you will find links to software and other documentation, online tutorials, mailing lists and current status information.

Get an Account Any ANL employees may request an account on Jazz. To apply for an account, please fill out a Jazz Account Request. An initial starter allocation of computer hours will be provided. Information is available on the LCRC web pages for allocations, projects and related policies.

Non-ANL employees must be associated with an active Jazz project. More information on projects can be found on the Projects web page.

Connect and Log In Once you have an account and its associated password, you may log onto the Jazz cluster. Software that uses the ssh protocol must be used to connect to the cluster for interactive sessions. See Accessing Jazz for detailed information on using ssh and the associated scp command.

To log onto the Jazz cluster from a unix machine, you would ssh to the cluster alias with the command:

      ssh -l <jazz_username>
Once you have connected, you will be prompted for your Jazz password. "Accessed Denied" error messages at login usually mean there is a password or username problem; contact Jazz support for help.

The name refers to all currently available user login hosts. Users are placed onto the real hosts in round-robin fashion, as a result they may end up on a different host from one login to the next. Because the real hosts may change without notice, we suggest that you use the alias and not the real hostnames to log onto the cluster.

Set Up Your Environment Once you are logged onto the Jazz cluster, you will be sitting at a Unix shell prompt in your home ($HOME) directory. If you have never used UNIX before, you will need to learn some basic commands before you will be able to submit jobs. Some helpful links may be found on the training resources page.

Your account will probably be setup with the default shell tcsh. Two other shells, bash and zsh are also provided. To change your shell, go to your personal accounts web page, which you can get to from the LCRC Accounts page.

Jazz uses the MCS SoftEnv package to manage access to applications on the cluster. The first time a users log onto Jazz, a '.soft' configuration file will automatically be created in the users's home directory. This file will be set up with the default applications environment - typically there will be a single line with '@default' in the file. Depending on the user's applications, further modifications to the .soft file may be necessary.

SoftEnv man pages are available in the default environment (use 'man softenv'). The command 'softenv' will list all available applications. Users wishing to gain a more complete understanding of how softenv works may read the complete softenv documentation.

Store Files There are two file systems on Jazz that are available to users: the home directory space, and the PVFS file system.

Your home directory, /home/<username>, is on a global filesystem (GFS). This filesystem is mounted across all nodes in the cluster. Users should use the /home/<username> path in all scripts and jobs. In general, there should be no need to manually copy files out to the nodes.

In addition to the home filesystem, there is a /pvfs/scratch/<username> directory located on a PVFS filesystem. This filesystem is also mounted across the entire cluster. Please do not use /pvfs/scratch/<username> as a permanent storage location. It is a scratch area and as such, all data in it may be destroyed without prior notice at any time.

For information about how to achieve the best performance and avoid the various problems please read the Jazz File Systems page.

Run on the Compute Nodes Access to Jazz compute nodes (named j[1-350]) is provided via a scheduler and an associated batch queuing application (currently PBS is used for both). To run an application, the user submits a 'job' to the scheduler. The scheduler will then determine, based on LCRC policies, available and requested resources and other jobs waiting in the queue, when the job should run. At that time, the scheduler hands the job to the queuing application which initiates the job.

When a user's job starts, it is given a list of compute nodes. This list is stored in the $PBS_NODEFILE file. Access to compute nodes is restricted to the user who job is assigned to run on those nodes.

There are man pages available for all PBS commands. Users should read the 'qsub' and 'qstat' man pages first. Complete documentation to PBS, including a users guide, may be found on the Jazz Applications Docmentation pages.

The Running Jobs document provides details about how to submit jobs, writing job scripts, etc.

Please read the Jazz Policies page which includes information about the job scheduling policy.

Manage Email When you get a Jazz account, a .forward file is automatically created for you. This file tells the Jazz mail server to send any mail addressed to you to the address stored in that file. Initially, this will be the address you provided on your account request. See the User Mail page for information about how to modify your forward file and how to read and send mail within the LCRC system.

There are a number of mailing lists associated with LCRC and Jazz. A complete list of the mailing lists and access to their archives may be found on LCRC Mailing Lists page. Two important mailing lists are the and the lists.

List Description Used by LCRC administrators for long term information such as cluster policies. All users on Jazz are members of this mailing list. Used by the administrators to communicate operational status that would be of interest to the active user community (for example, short emergency outages). Membership in the notify list is optional. Users are not automatically subscribed to notify, instead membership on that maillist is managed by the users with the 'notifyme' command:
'notifyme -y' start notifications
'notifyme -n' stop notifications

Request a New Project Shortly after getting your Jazz account you should plan on joining an existing Jazz project or requesting a new project. Jazz projects are managed by the Primary Investigator (PI) for the project.

If you wish to join an existing project, please go to the Project Membership page. You will need to use your Jazz username and password to log onto the page. Once you have logged on, click on the name of the project you wish to join. This will bring up a page with details about the project. Click on the 'Request Membership' button at the bottom of the page. An email will be sent to the Primary Investigator for the project, informing them that you wish to become a member of their project. The PI is responsible for making you a member of the project. Once that has happened, you will receive email with instructions and the name of the project to use for job submissions.

If you will be the PI for a new project, you should use the 'New Project Request' form to apply for approval and an allocation. The data from the form will be sent to the LCRC Allocation Board. Once approval has been given and the allocation set, you will receive mail indicating the project name and the starting alloction. Only current Jazz users may request a new project, please use your Jazz username and password to log on.

Once you have a project name, you will want to set your default project to that name. Use the lcrc-qbank command to query or set your default project. To set the default:

         lcrc-qbank -s default <projectname> 
For detailed information about managing your allocations, please see the Managing Your Allocations page.
Get More Help If, after reading the documentation, you have questions or require assistance, please contact the LCRC Support Team.

Help Security/Privacy Notice Disclaimer