Using Gaussian on the LCRC servers is fairly straightforward. First you must define your environment correctly by adding a softenv key for Gaussian. The following softenv keywords are available:
+g03 Gaussian 03, latest version (C.02)
+g03-c.02 Gaussian 03, version C.02
+g09 Gaussian 09, latest version (A.02)
+g09-a.02 Gaussian 09, version A.02
+g09-d.01 Gaussian 09, version D.01
We recommend that you use the latest version of Gaussian. To setup your environment for this version of Gaussian add the softenv key +g09-d.01 as the first line in your ~/.soft file.
Gaussian uses the Linda parallel system rather than MPI to communicate between nodes. You need to pass directives to Gaussian to specify which nodes to use and how many processes to use on each node. The simplest way to do this is to start a Linda worker on each node and spawn a number of threads equal to the number of cores on the node. Unfortunately, the node names will not be defined until your job starts. You PBS script will need to determine and pass this information to Gaussian at run time.
The following PBS script will define the correct Linda directives for Gaussian and writes them to a header file. It then concatenates the header file and your input through a pipe to Gaussian. Note that on the second line is where you define the PBS resource ppn equal to 8 on fusion and 16 on blues.
#PBS -N water
#PBS -l nodes=2:ppn=16
#PBS -l walltime=0:10:00
#PBS -j oe
#PBS -A lcrc-app-engr
np=`wc -l < $PBS_NODEFILE`
nn=`sort -u $PBS_NODEFILE | wc -l`
node_list=`sort -u $PBS_NODEFILE| paste -s -d','`
echo Number of nodes is $nn
echo Number of processors is $np
echo Node list is $node_list
#Create a header file that contains g09 keywords
#to run efficiently on a distributed memory server.
echo '%NProcShared='$shared >header
echo '%LindaWorkers='$node_list >>header
### Set up Gaussian environment
# The following command directs the scratch
# files from g09 to the local scratch drives on
# lcrc servers.
### Run the g09 calculation by prepending the
# header generated above to the user's Gaussian input
# before g09 reads it.
cat header water.com | g09 > water.log
Sample PBS scripts can also be found at these links for blues and fusion.
Note that it is extremely important that you define the environmental variable GAUSS_SCRDIR to be the local disk on the compute node ($TMPDIR). Otherwise Gaussian will use the working directory on a shared file system as scratch space. This can slow down the LCRC servers and may aggravate the sysadmins, as well as other users.
Efficient use of Gaussian is greatly aided by correctly defining the amount of memory and disk space available on each node with directives to Gaussian. Note that the compute nodes on blues and fusion do not have the same amount of memory or disk space. On a regular compute node of fusion we recommend putting the following Gaussian commands at the top of your Gaussian input.
Add the link0 command "%Mem=24GB" to the beginning of your Gaussian input and the keyword "MaxDisk=100GB" to the route section of your Gaussian input for jobs running on Fusion.
For blues you should add the link0 command "%Mem=60GB" to the beginning of your Gaussian input and the keyword "MaxDisk=500MB" to the route section of your Gaussian input.
You can also use a Default.Route file in your working directory to define the amount of disk space and memory used by Gaussian. Sample Default.Route files can be found at these links for blues and fusion..
Note that Gaussian 09, version D.01, was built and optimized on Nehalem processor. It will not use the AVX instructions on blue's sandy-bridge processor. Therefore Gaussian will tend to run faster on fusion than on blues.