Getting Started

Welcome to Argonne's Laboratory Computing Resource Center (LCRC). This is the Getting Started page.
The topics covered here should cover everything that a new user who is moderately experienced with clusters should need to know to get started on Fusion. Links to other documents with more information on specific topics are also provided.

How to... Information
Get Help We hope you can find much of the information you'll need through the LCRC web pages. The home page is http://www.lcrc.anl.gov. From that page, you will find links to software and other documentation, online tutorials, mailing lists and current status information.

The LCRC Support Team and Application Engineers can assist you with your questions and requests that aren't answered by the web pages. Please contact them via email at support@lcrc.anl.gov.
Get an Account Any ANL employees may request an account on Fusion. To apply for an account, please fill out a Fusion Account Request. An initial starter allocation of computer hours will be provided. Information is available on the LCRC web pages for allocations, projects and related policies.

Non-ANL employees must be associated with an active Fusion project. More information on projects can be found on the Projects web page.

Connect and Log In Once you have an account and its associated password, you may log onto the Fusion cluster. Software that uses the ssh protocol must be used to connect to the cluster for interactive sessions. See Accessing Fusion for detailed information on using ssh and the associated scp command.

To log onto the Fusion cluster from a unix machine, you would ssh to the cluster alias fusion.lcrc.anl.gov with the command: 

 
      ssh -l <fusion_username> fusion.lcrc.anl.gov
Once you have connected, you will be prompted for your Fusion password. "Accessed Denied" error messages at login usually mean there is a password or username problem; contact Fusion support for help.

The fusion.lcrc.anl.gov name refers to all currently available user login hosts. Users are placed onto the real hosts in round-robin fashion, as a result they may end up on a different host from one login to the next. Because the real hosts may change without notice, we suggest that you use the alias and not the real hostnames to log onto the cluster.

Set Up Your Environment Once you are logged onto the Fusion cluster, you will be sitting at a Unix shell prompt in your home ($HOME) directory. If you have never used UNIX before, you will need to learn some basic commands before you will be able to submit jobs.

Your account will probably be setup with the default shell tcsh. Two other shells, bash and zsh are also provided. To change your shell, go to your personal accounts web page, which you can get to from the LCRC Accounts page.

Fusion uses the MCS SoftEnv package to manage access to applications on the cluster. The first time a users log onto Fusion, a '.soft' configuration file will automatically be created in the users's home directory. This file will be set up with the default applications environment - typically there will be a single line with '@default' in the file. Depending on the user's applications, further modifications to the .soft file may be necessary.

SoftEnv man pages are available in the default environment (use 'man softenv'). The command 'softenv' will list all available applications. Users wishing to gain a more complete understanding of how softenv works may read the complete softenv documentation.

Store Files There are two file systems on Fusion that are available to users: the home directory space, and the PVFS file system.

Your home directory, /home/<username>, is on a global filesystem (GFS). This filesystem is mounted across all nodes in the cluster. Users should use the /home/<username> path in all scripts and jobs. In general, there should be no need to manually copy files out to the nodes.

In addition to the home filesystem, there is a /pvfs/scratch/<username> directory located on a PVFS filesystem. This filesystem is also mounted across the entire cluster. Please do not use /pvfs/scratch/<username> as a permanent storage location. It is a scratch area and as such, all data in it may be destroyed without prior notice at any time.

For information about how to achieve the best performance and avoid the various problems please read the Fusion File Systems page.

Transfering Files Globus Online is a free online service that simplifies data movement -- with Globus Online, you hand-off data movement tasks to a hosted service that manages the entire operation: monitoring performance and errors, retrying failed transfers, correcting problems automatically whenever possible, and reporting status. Command line and web-based interfaces are available. The command line interface, which requires only ssh to be installed on the client, is the method of choice for grid-based workflows.
Click here for more information or go here to sign up for an account.
Run on the Compute Nodes Access to Fusion compute nodes (named f[1-320]) is provided via a scheduler and an associated batch queuing application (currently PBS is used for both). To run an application, the user submits a 'job' to the scheduler. The scheduler will then determine, based on LCRC policies, available and requested resources and other jobs waiting in the queue, when the job should run. At that time, the scheduler hands the job to the queuing application which initiates the job.

When a user's job starts, it is given a list of compute nodes. This list is stored in the $PBS_NODEFILE file. Access to compute nodes is restricted to the user who job is assigned to run on those nodes.

There are man pages available for all PBS commands. Users should read the 'qsub' and 'qstat' man pages first. Complete documentation to PBS, including a users guide, may be found on the Fusion Applications Docmentation pages.

The Running Jobs document provides details about how to submit jobs, writing job scripts, etc.

Please read the Fusion Policies page which includes information about the job scheduling policy.

Manage Email There are a number of mailing lists associated with LCRC and Fusion. A complete list of the mailing lists and access to their archives may be found on LCRC Mailing Lists page. Two important mailing lists are the fusion-users@lcrc.anl.gov and the notify@lcrc.anl.gov lists.

List Description
fusion-users@lcrc.anl.gov Used by LCRC administrators for long term information such as cluster policies. All users on Fusion are members of this mailing list.
notify@lcrc.anl.gov Used by the administrators to communicate operational status that would be of interest to the active user community (for example, short emergency outages). Membership in the notify list is optional. Users are not automatically subscribed to notify, instead membership on that maillist is managed by the users with the 'notifyme' command:
'notifyme -y' start notifications
'notifyme -n' stop notifications
Request a New Project Shortly after getting your Fusion account you should plan on joining an existing Fusion project or requesting a new project. Fusion projects are managed by the Primary Investigator (PI) for the project.

If you wish to join an existing project, please go to the Project Membership page. You will need to use your Fusion username and password to log onto the page. Once you have logged on, click on the name of the project you wish to join. This will bring up a page with details about the project. Click on the 'Request Membership' button at the bottom of the page. An email will be sent to the Primary Investigator for the project, informing them that you wish to become a member of their project. The PI is responsible for making you a member of the project. Once that has happened, you will receive email with instructions and the name of the project to use for job submissions.

If you will be the PI for a new project, you should use the 'New Project Request' form to apply for approval and an allocation. The data from the form will be sent to the LCRC Allocation Board. Once approval has been given and the allocation set, you will receive mail indicating the project name and the starting alloction. Only current Fusion users may request a new project, please use your Fusion username and password to log on.

Once you have a project name, you will want to set your default project to that name. Use the lcrc-qbank command to query or set your default project. To set the default: 

            
         lcrc-qbank -s default <projectname>
For detailed information about managing your allocations, please see the Managing Your Allocations page.
Get More Help If, after reading the documentation, you have questions or require assistance, please contact the LCRC Support Team.