Centre for Atmospheric Science
How to use p-TOMCAT on HECToR
HECToR is a Cray XT4 massively parallel system. The processors are about the same performance as the Dobson cluster machines. There is a very fast network - faster than dobson or HPCx. Tests show that p-TOMCAT does not suffer as much slowdown at high resolutions due to the improved network on HECToR.
You can log in to HECToR (using ssh only) at login.hector.ac.uk, which is the front-end machine of the cluster: but you cannot log into the other compute nodes. The /home directory is only mounted on the login node, it cannot be seen on the compute/batch nodes. Any data that needs to be accessed by the batch jobs must be copied onto your /work directory.
An important point to note is that the cluster machines are based on Intel processors. Therefore you MUST recompile your programs before you can run them on Hector. Also, binary format on these machines is little-endian rather than big-endian on the other servers. If you write binary files from your Fortran programs on rome (say), you need to add the appropriate compiler option to read them correctly on HECToR.
Setting up your environment
In order to run p-TOMCAT and compile it correctly you need to set up your environment. In your .profile file in your home directory, add the following lines at the bottom:
# make sure we load the right modules module unload netcdf module unload PrgEnv-pgi module load PrgEnv-pathscale module load xtpe-quadcore# adjust paths to get my pathscale version of netcdf LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/n02/n02/emgdc/netcdf export LD_LIBRARY_PATH PATH=$PATH:/home/n02/n02/emgdc/bin; export PATH
p-TOMCAT doesn't work correctly with the default PGI compiller so the module commands above switch to the PathScale compiler. Also, the netcdf module supplied on the system does not work with p-TOMCAT as it uses the PGI compiler. A version compiled with the PathScale compiler is available in my home directory and the above sets access to it.
All the files required to run p-TOMCAT have been transferred to HECToR and are in the directories under /work/n02/n02/emgdc/tomcat/data. The sample HECToR tom.run script has the correct directory names in the namelist for the model.
The forcing files are in /work/n02/n02/emgdc/data/ECMWF. We have all
the forcing files from 1995- present day. I will keep the files up to
date. If anyone notices any missing or
wants any earlier years please let me know.
Jobs are submitted on HECToR using the qsub command (as on Dobson). qstat is used to check the status of jobs.