How to use the IBM POWER5 Shared Memory System

POWER5 System Configuration

The POWER5 system includes login and compute nodes. Initially the system has two p575 compute nodes and one p550 login node. Compute nodes have eight 1.9GHz, single-core, POWER5 processors and 32GB of memory. The login node has two 1.65GHz, dual-core POWER5 processors and 8GB of memory. Login and compute nodes are running AIX as their operating system.

Login and compute nodes share a 2TB scratch file system (/scratch). IBM's General Parallel File System (gpfs) is used to implement this shared file system. User home directories are served from the same external file server used for the university Linux cluster (henry2). The same home directories are mounted on the Linux cluster nodes and the shared memory system nodes. Mass storage file systems (/ncsu/volume1 and /ncsu/volume2) are available from the login node (but are not available from the compute nodes).

All access to the compute nodes is through the LSF resource manager. Interactive use of the system is limited to the login node.

Logging on to the POWER5 System

Use ssh to p5login.hpc.ncsu.edu to access the POWER5 system login node. Unity usernames and passwords are used for authentication on the POWER5 system.

Default shell for users is tcsh. System customization for ease of use is implemented only for the default shell. Users should take care in creating files - keeping in mind that files may be shared by both the POWER5 system and the university Linux cluster.

File Storage Systems

• /home

  The Power5 system and the henry2 cluster SHARE the SAME /home file system. To make system dependent customization simplier, the Power5 system home directory is a subdirectory "p5" under /home/userid (where userid is the account user name. Thus the henry2 home directory for user "desmith" would be /home/desmith and the Power5 home directory would be /home/desmith/p5.

  /home is available from each of the Power5 nodes.

  A daily backup is made of files in /home. A single backup copy is kept of each file. These backups are kept for 30 days. That is if a file is altered, the altered version will replace the previous version during the daily backup. If a file is deleted the backup copy of that file will be kept for 30 days. Daily backup occurs overnight.

• /scratch

  This type of storage system is intended to provide storage space required for running jobs. /scratch is world writable. Users should create a subdirectory for their use and set appropriate permissions on their subdirectory. /scratch uses fibre channel connected disks directly attached to each Power5 node and IBM's General Parallel File System (GPFS) to provide good I/O performance.

  /scratch is available from each of the Power5 nodes.

  Files stored in /scratch are not backed up and are subject to a periodic purge; that is, files that have not been accessed in several days may be automatically removed.

• /ncsu/volume[12]

  The HPC mass storage system is available for storage in excess of what can be accomodated in /home. These are the same file systems available from other HPC login nodes.

  /ncsu/volume[12] is only available from the Power5 lgoin mode.

  A daiily backup is made of files in /ncsu/volume[12]. A single backup copy is kept of each file. These backups are kept for 30 days. That is if a file is altered, the altered version will replace the previous version during the daily backup. If a file is deleted the backup copy of that file will be kept for 30 days. Daily backup occurs overnight.

Compiling and Running Jobs

See How to Compile and Run jobs on the Power5 Shared Memory System.