Getting Started

Welcome to Argonne's Laboratory Computing Resource Center (LCRC). This is the Getting Started page.
The topics covered here should cover everything that a new user who is moderately experienced with clusters should need to know to get started on Fusion. Links to other documents with more information on specific topics are also provided.

How to... Information
Get Help We hope you can find much of the information you'll need through the LCRC web pages. The home page is From that page, you will find links to software and other documentation, online tutorials, mailing lists and current status information.

The LCRC Support Team and Application Engineers can assist you with your questions and requests that aren't answered by the web pages. Please contact them via email at

Get an Account

Any ANL employees may request an account on Fusion. To apply for an account, please fill out a Fusion Account Request. An initial starter allocation of computer hours will be provided. Information is available on the LCRC web pages for allocations, projects and related policies.

Non-ANL employees must be associated with an active Fusion project. More information on projects can be found on the Projects web page.

Connect and Log In

Once you have an account and its associated password, you may log onto the Fusion cluster. Software that uses the ssh protocol must be used to connect to the cluster for interactive sessions. See Accessing Fusion for detailed information on using ssh and the associated scp command.

To log onto the Fusion cluster from a unix machine, you would ssh to the cluster alias with the command:

      ssh -l <fusion_username> 

Once you have connected, you will be prompted for your Fusion password. "Accessed Denied" error messages at login usually mean there is a password or username problem; contact Fusion support for help.

The name refers to all currently available user login hosts. Users are placed onto the real hosts in round-robin fashion, as a result they may end up on a different host from one login to the next. Because the real hosts may change without notice, we suggest that you use the alias and not the real hostnames to log onto the cluster.

Set Up Your Environment Once you are logged onto the Fusion cluster, you will be sitting
at a Unix shell prompt in your home ($HOME) directory. If you
have never used UNIX before, you will need to learn some basic
commands before you will be able to submit jobs.

Your account will probably be setup with the default shell tcsh. Two other shells, bash and zsh are also provided. To change
your shell, go to your personal accounts web page, which you
can get to from the
LCRC Accounts

Fusion uses the MCS SoftEnv package to manage access to applications
on the cluster. The first time a users log onto Fusion, a '.soft'
configuration file will automatically be created in the users's
home directory. This file will be set up with the default
applications environment - typically there will be a single line
with '@default' in the file. Depending on the user's
applications, further modifications to the .soft file may be

SoftEnv man pages are available in the default environment (use
'man softenv'). The command 'softenv' will list all available
applications. Users wishing to gain a more complete understanding
of how softenv works may read the href="/info/Software/Softenv"> complete softenv

Store Files

There are two file systems on Fusion that are available to users:
the home directory space, and the PVFS file system.

Your home directory, /home/<username>, is on a global
filesystem (GFS). This filesystem is mounted across all nodes in
the cluster. Users should use the /home/<username> path in all
scripts and jobs. In general, there should be no need to
manually copy files out to the nodes.

In addition to the home
filesystem, there is a /pvfs/scratch/<username> directory
located on a PVFS filesystem. This filesystem is also mounted
across the entire cluster. Please do not use
/pvfs/scratch/<username> as a permanent storage location. It is
a scratch area and as such, all data in it may be destroyed
without prior notice at any time.

For information
about how to achieve the best performance and avoid the various
problems please read the

Fusion File Systems

Transfering Files

Globus Online is a free online service that simplifies data movement -- with Globus Online, you hand-off data movement tasks to a hosted service that manages the entire operation: monitoring performance and errors, retrying failed transfers, correcting problems automatically whenever possible, and reporting status. Command line and web-based interfaces are available. The command line interface, which requires only ssh to be installed on the client, is the method of choice for grid-based workflows.
Click here for more information or go here to sign up for an account.

Run on the Compute Nodes

Access to Fusion compute nodes (named f[1-320]) is provided via a
scheduler and an associated batch queuing application
(currently PBS is used for both). To run an application, the
user submits a 'job' to the scheduler. The scheduler will then
determine, based on LCRC
, available and requested resources and other jobs
waiting in the queue, when the job should run. At that time,
the scheduler hands the job to the queuing application which
initiates the job.

When a user's job starts, it is given a list of compute nodes.
This list is stored in the $PBS_NODEFILE file. Access to
compute nodes is restricted to the user who job is assigned
to run on those nodes.

There are man pages available for all PBS commands. Users
should read the 'qsub' and 'qstat' man pages first. Complete
documentation to PBS, including a users guide, may be found on
the Fusion
Applications Docmentation pages

The Running Jobs document provides
details about how to submit jobs, writing job scripts, etc.

Please read the
Fusion Policies
page which includes information about the job scheduling

Manage Email There are a number of mailing lists associated with LCRC and
Fusion. A complete list of the mailing lists and access to their
archives may be found on href="/info/MailingLists">LCRC Mailing Lists page.
Two important mailing lists are the and
the lists.

List Description

Used by LCRC administrators for long term information such
as cluster policies. All users on Fusion are members of this
mailing list.

Used by the administrators to communicate operational
status that would be of interest to the active user
community (for example, short emergency outages).
Membership in the notify list is optional. Users are not
automatically subscribed to notify, instead membership on
that maillist is managed by the users with the
'notifyme' command:

'notifyme -y' start notifications
'notifyme -n' stop notifications
Request a New Project

Shortly after getting your Fusion account you should plan on
joining an existing Fusion project or requesting a new project.
Fusion projects are managed by the Primary Investigator (PI)
for the project.

If you wish to join an existing project,
please go to the

Project Membership page
. You will need to use your
Fusion username and password to log onto the page. Once
you have logged on, click on the name of the project you wish to join.
This will bring up a page with details about the project.
Click on the 'Request Membership' button at the bottom of the page.
An email will be sent to the Primary Investigator for the
project, informing them that you wish to become a member
of their project. The PI is responsible for making you
a member of the project. Once that has happened, you will
receive email with instructions and the name of the project
to use for job submissions.

If you will be the PI for a new project, you should use the href="">'New Project
Request' form to apply for approval and an allocation. The
data from the form will be sent to the LCRC Allocation Board.
Once approval has been given and the allocation set, you will
receive mail indicating the project name and the starting
alloction. Only current Fusion users may request a new project,
please use your Fusion username and password to log on.

Once you have a project name, you will want to set your
default project to that name. Use the lcrc-qbank command to
query or set your default project. To set the default:

         lcrc-qbank -s default <projectname> 

For detailed information about managing your allocations, please
see the href="/info/ManagingAllocations">
Managing Your Allocations page.

Get More Help

If, after reading the documentation, you have questions or require
assistance, please contact the
LCRC Support Team.