Using Jazz  |  Getting Started  |  About the LCRC  |  Presentations  |  Status  |  FAQ  |  Search   |   Main Page  

Using PBS on Jazz: commands, options and examples

This document provides examples of how to submit jobs on the Jazz cluster. It also provides examples of PBS commands that can be used to query the status of jobs. For an introduction to using PBS and running jobs on Jazz, see Running Jobs on the Jazz Cluster.
How To Examples and Results
Submit a job request Use qsub to submit a job, either interactive or a script.

To run a script, specify the script at the end of the qsub command, after all of the qsub options.

Run the job1 script with any 10 nodes for a maximum of 1 hour and 30 minutes: 

qsub -l nodes=10,walltime=01:30:00 job1

For an interactive job, use the -I option. Upon activation of an interactive job, the user will be placed onto the master node in an interactive shell.

Request any 10 nodes for 1 hour and 30 minutes in interactive mode: 

qsub -I -l nodes=10,walltime=01:30:00
Request 10 large memory nodes (ones with 2GB of memory), for the default time of 15 minutes: 
qsub -I -l nodes=10:bigmem
Delete a job from the queue To delete a job from the queue, use the qdel command. This command is very simple and has only two options, one to force deletion of the job and one to delay the time between when the server sends the SIGTERM and when the server sends the SIGKILL.

Cancel job 34586: 

qdel 34586
If the job failed to cancel (indicating that PBS is confused about its state in some fashion), you might try again with the force option: 
qdel -W force 34586
Query queue and job information To find out information about the state of the queue, the state of particular jobs, etc., use the qstat command. There are many options and ways to use qstat. Use man qstat if you wish to understand all the options.

To see a summary of all jobs in all queues: 

qstat -a
To see the details such as which nodes, when it was submitted, reason for current state - if available, etc. for job 34465: 
qstat -f 34465
Query node information To find out the node state, properties, number of free nodes, etc., use the nodes command. This is a Jazz only command that uses the output from the official PBS pbsnodes command, man pbsnodes is available on all Jazz systems.

To find out how many truly free nodes are currently available (does not include reserved nodes in the count): 

nodes -f -c
To get the count of free nodes with the bigmem property: 
nodes -f -p bigmem
To get the count of free nodes with 2GB of memory and working ip over myrinet: 
nodes -f -c -p bigmem -p myrinet-ip
To list all nodes administratively offline (note the lack of the -c option): 
nodes -s offline
To list all nodes reserved for reservation R34460: 
nodes -r R34460
To get the count of nodes from reservation R34460 actively running a job: 
nodes -c -r R34460 -s job
To get information about all options for nodes: 
nodes -h

Display various data from the current state of the system

Usage: nodes 

Options:
  -h             prints this message
  -c             print just a count
  -a             all nodes, overrides -s and -f
  -f             only free nodes. DEFAULT.
  -s      only nodes in  state
  -p   only nodes with  property. 
                 multiple allowed - and'd together, 
                 only nodes having all properties.

Notes:
    If -a or -s are not used, only free nodes will be listed.
Query reservations To find out information about any reservations, either active or pending, use the rstat command.

To see a summary of all reservations: 

rstat -a
To see the details such as which nodes, authorized users, etc. for the reservation R34465: 
rstat -f R34465

Help Security/Privacy Notice Disclaimer