Lab/Seminar Lecture 1
10/25/1994
11/01/1994
PVM
Victor Batista
PARALLEL PROGRAMING ON COUPLED CLUSTERS OF WORKSTATIONS OR CM5 SUPERCOMPUTERS
-----------------------------------------------------------------------------
USING PVM OR CMMD MESSAGE-PASSING SUBROUTINES:
---------------------------------------------
Introduction:
------------
In this file you will find all the information you need in order to write and
run spmd programs with calls to pvm or cmmd message-passing subroutines.
It includes all the examples presented in the lectures as well as references to
all available documentation about pvm and CMMD message passing subroutines.
For convenience it was written as a sequence of "baby-steps" that should
be followed sequentially.
It also contains information about "MD_SPMD" version 1.0 [April, 1994], a
parallel and scalable algorithm for doing short range molecular dynamics on
distributed memory parallel architectures such as clusters of workstations
or CM-5 supercomputers. The program is loaded in the directory /usr/var/tmp/pvm
as a tar-file named md_spmd.tar
Steps to make pvm work from your account in the HPC cluster:
-----------------------------------------------------------
>From your home directory prompt in basie type:
cp .cshrc cshrc_bak
cp .rhosts rhosts_bak
cp /usr/var/tmp/pvm/.cshrc .
cp /usr/var/tmp/pvm/cshrc.stub .
cp /usr/var/tmp/pvm/cshrc.stub .
cp /usr/var/tmp/pvm/.rhosts .
Edit the file named cshrc.stub and replace the name of the home directory from
/usr3/graduate/batista to your HPC home directory name. Then edit the file
named .rhosts and replace the login names from batista to your login names.
Then type,
source .cshrc
Now you can try pvm by typing some basic commands. Type:
pvm
you will get:
basie% pvm
pvm>
then type,
add billie
you will get:
pvm> add billie
1 successful
HOST DTID
billie 80000
pvm>
then type,
halt
you will get:
pvm> halt
libpvm [t40001]: mxfer() EOF on pvmd sock
basie%
If you got these output lines, pvm3.3.3 is working for you on the HPC
cluster. CONGRATULATIONS!!.
Steps for working with examples:
--------------------------------
>From your home directory prompt in basie type:
mkdir pvm3
cd pvm3
mkdir bin
cd bin
mkdir SGI5
cd ..
mkdir examples
cd examples
cp /usr/var/tmp/pvm/* .
Example 1: (main.c)
----------
This example, brings you with all the information you need in order to
write and compile your own spmd programs with pvm calls.
It provides you with three general functions presented in the file
named tools.c for building dynamically a cluster of hosts where processors
are being spawn, for sending and receiving arrays of data
(with similar characteristics as those of the CMMD library), and for
finishing with the participation of hosts in the pool. For learning how
to compile your own programs take a look at the Makefile where all the
flags and include files are defined.
Edit the file named Makefile and replace the name of MY_HOME DIR from
/usr3/graduate/batista to your home directory name. Then type
make
you will get something like:
basie% make
cc -g -I/usr/local/pvm3/include -o main main.o tools.o /usr/local/pvm3/lib/SGI5/libpvm3.a -lsun -lm
mv main /home_directory_name/pvm3/bin/SGI5
basie%
Then start pvm by typing,
pvm
you will get:
basie% pvm
pvm>
Then type ,
quit
you will get:
pvm> quit
pvmd still running.
basie%
Then type,
main
You will get the output of the program main.c This is an array that has
been initialized piecewise by each of the computers of the HPC cluster.
Example 2: (spmd.c)
----------
Edit the file called Makefile and replace all strings "main" by "spmd",
except for those "main.h".
Then type,
make
you will get something like:
basie% make
cc -g -I/usr/local/pvm3/include -c spmd.c
cc -g -I/usr/local/pvm3/include -o spmd spmd.o tools.o /usr/local/pvm3/lib/SGI5/libpvm3.a -lsun -lm
mv spmd /home_directory_name/pvm3/bin/SGI5
basie%
Then start pvm by typing,
pvm
you will get:
basie% pvm
pvm>
Then quit typing,
quit
you will get:
pvm> quit
pvmd still running.
basie%
Then run the program typing,
spmd
you will get:
basie% spmd
me = 0 mytid = 262146
token ring done
basie%
This is the output of the program that simulates a token going around a ring
of processors. This is one of the examples presented in the reference manual.
More examples are loaded in the directory /usr/local/pvm3.3.3/examples
and are discussed in the reference manual "PVM 3 USER's GUIDE AND REFERENCE
MANUAL".
How to make pvm run from your account on conx:
----------------------------------------------
Go to your home directory prompt on conx and get from basie the files named
.rhosts .cshrc and cshrc.stub typing,
cp .cshrc cshrc_bak
cp .rhosts rhosts_bak
ftp basie
you'll somthing like:
Connected to basie.bu.edu.
220 basie.bu.edu FTP server ready.
Name (basie:your login name):
331 Password required for batista.
Password:
230 User your name logged in.
ftp>
then type,
prompt
ftp> prompt
Interactive mode off.
ftp>
mget .cshrc cshrc.stub .rhosts
you'll get:
mget .cshrc?
200 PORT command successful.
150 Opening ASCII mode data connection for '.cshrc' (5208 bytes).
226 Transfer complete.
local: .cshrc remote: .cshrc
5402 bytes received in 0.039 seconds (1.4e+02 Kbytes/s)
mget cshrc.stub?
200 PORT command successful.
150 Opening ASCII mode data connection for 'cshrc.stub' (1709 bytes).
226 Transfer complete.
local: cshrc.stub remote: cshrc.stub
1808 bytes received in 0.042 seconds (42 Kbytes/s)
mget .rhosts?
ftp> get .rhosts
200 PORT command successful.
150 Opening ASCII mode data connection for '.rhosts' (943 bytes).
226 Transfer complete.
local: .rhosts remote: .rhosts
986 bytes received in 0.0087 seconds (1.1e+02 Kbytes/s)
ftp>
Then type,
quit
And you'll get the prompt back.
Edit the file named cshrc.stub and replace the name of the home directory from
/usr3/graduate/batista to your conx home directory name. Then edit the file
named .rhosts and replace the login names from batista to your login names.
Now type,
source .cshrc
Now you can try pvm by typing some basic commands. Type:
pvm
you will get:
conx% pvm
pvm>
then type,
add edsac
you will get:
pvm> add edsac
1 successful
HOST DTID
edsac 80000
pvm>
then type,
add billie
and you'll get:
pvm> add billie
1 successful
HOST DTID
billie c0000
pvm>
then type,
add illiac
and you'll get:
pvm> add illiac
1 successful
HOST DTID
illiac 100000
pvm>
the type,
conf
and you'll get:
pvm> conf
4 hosts, 1 data format
HOST DTID ARCH SPEED
conx 40000 SUN4 1000
edsac 80000 CM5 1000
billie c0000 SGI5 1000
illiac 100000 CM5 1000
pvm>
Then type,
halt
you will get:
pvm> halt
libpvm [t40001]: mxfer() EOF on pvmd sock
conx%
Now you are able to make a cluster of SGI, SUN4 and CM5 computers that
can communicate and run MIMD programs!!!. Wooow!.
How to test example 1 (main.c), on conx using PVM calls:
-------------------------------------------------------
Type,
mkdir pvm3
cd pvm3
mkdir bin
cd bin
mkdir SUN4
cd ..
mkdir examples
cd examples
using the ftp command mget all of the files you have in your directory
home_directory/pvm3/examples in basie.
Edit the file named Makefile and replace the lines that read
PVM_ARCH = SGI
ARCHLIB = -lsun
by the lines,
PVM_ARCH = SUN4
ARCHLIB =
Comment out lines 18 and 19 so they will read
#CC = cc
#PREPROCESSOR = cc -E
and uncomment lines 21 and 22 to,
CC = gcc
PREPOCESSOR = gcc -E
Replace the name of MY_HOME_DIR from /usr3/graduate/batista to your home
directory name in conx.
Edit file main.h and comment out the line that defines SGI,
/*
#define SGI
*/
and uncomment lines that read
#define SUN4
#define RAND_MAX 2147483646
Now type,
make
you'll get something like,
conx% make
gcc -g -I/usr/local/pvm3/include -c main.c
gcc -g -I/usr/local/pvm3/include -c tools.c
gcc -g -I/usr/local/pvm3/include -o main main.o tools.o /usr/local/pvm3/lib/SUN4/libpvm3.a -lm
mv main /you_home_dir/pvm3/bin/SUN4
conx%
Start up pvm console typing,
pvm
you will get,
conx% pvm
pvm>
Then quit typing,
quit
you will get,
pvm> quit
pvmd still running.
conx%
Edit the file Makefile and replace all strings "spmd" by strings "main".
In order to run it, just type
main
Before the program is over, open another window in basie type
pvm, and see that hosts are being added to the pool. you can type
"conf" in order to see all the hosts already added. At the end
the deamon is deleted.
The output is once again the array that has been initialized by each of
the hosts of the pool.
It also interesting to use prism and see how the array evolves after
each communication step. In order to do this, start by typing
setenv DISPLAY name_of_the_terminal:0.0
prism /your_home_dir/pvm3/bin/SUN4/main
Then put a break in lines 43 and 49. Click "Run".
You can go to basie and type
pvm
basie% pvm
pvmd already running.
pvm> conf
10 hosts, 1 data format
HOST DTID ARCH SPEED
conx 40000 SUN4 1000
basie 2c0000 SGI5 1000
miles 300000 SGI5 1000
bird 340000 SGI5 1000
billie 380000 SGI5 1000
ellington 3c0000 SGI5 1000
monk 400000 SGI5 1000
marsalis 440000 SGI5 1000
quincy 480000 SGI5 1000
fitzgerald 4c0000 SGI5 1000
pvm>
Now you can highlight the variable x, and you'll see that it has been
initialized only for its first 10 elements. Now click "Continue", you can
see that after one communication step, host 0 has communicated with host #9
and has received elements 90-99, subsequent steps show how the array is being
updated with data coming from all the other hosts of the pool.
How to run example 1 (main.c), on conx, using CMMD synchronous message
----------------------------------------------------------------------
passing functions instead of using PVM calls:
---------------------------------------------
All you have to do is to recompile the program with the appropriate flags.
Edit the file Makefile, comment out "PVM flags" and uncomment
"CMMD flags". Comment out the line that reads
mv main $(XDIR)
Edit file named "main.h" and uncomment lines for CMMD library
#define CMMDRUN
#include
#if CMMD_VERSION == 20
#include
#endif
Comment out lines for PVM library
/*
#define PVMRUN
#include
*/
set lenx to 16 instead of 10.
Now you are ready to compile it by typing,
make
you'll get,
you'll get,
conx% make
gcc -g -c main.c
gcc -g -c tools.c
cmmd-ld -comp gcc -g -o main main.o tools.o -lm
conx%
You can run it with prism, by opening a window in illiac
(make sure you are in the directory pvm3/examples and that you have set the
environment to your terminal)
then type,
prism -node main
and proceed as before: place a break stop in lines 43 and 49 and click "Run".
You can highlight variable "x" and see how it evolves in subsequent steps.
It is also interesting to see arrays aux and bux.
Scalable and Parallel Short Range Molecular Dynamics (SPaSM):
------------------------------------------------------------
Molecular Dynamics (MD) is perhaps the broadest interdisciplinary field with
the participation of Computer Scientists, Engineers, Material Scientists,
Physicists, Chemists, Metheorologists, Biologists, etc.
Effective use of parallel architectures seems to hold the greatest potential
for making MD able to deal with systems that are beyond the limits of speed and
memory of modern computers.
The file md_spmd.tar contains "MD_SPMD" version 1.0 [April, 1994], a
parallel and scalable algorithm for doing short range molecular dynamics on
distributed memory parallel architectures such as clusters of workstations
or CM-5 supercomputers. See "Parallel Scalable Message-Passing Multi-Cell
Molecular Dynamics on Coupled Cluster of Workstations" by Victor S. Batista
and David F. Coker, Submitted to Parallel Computing (1994).
It is traditionally designed, especially with regard to keeping track of
consistency of parameters, restart files, output trajectories and so forth.
The algorithm is numerically equivalent to the Verlet Neighbor List method.
It is parallelized minimizing interprocessor communication at the expense of
some computational redundancy. Equations of motion are integrated using Gear
scheme method to fourth order. All calculations are performed using double
precision. It calculates energy, pressure and virial as well as the pair
distribution and velocity autocorrelation functions for nobel gases that
interact with Lennard-Jones potentials.
The code has been written in ANSI C for greater portability and to have
the flexibility of dynamic memory allocation. Prototypes have been provided
for all functions.
Once you have untar the file md_spmd.tar you can compile it by running make
on a UNIX system. (Make sure the Makefile has your own home_directory)
On other systems, you are on your own; take a look at the Makefile for hints.
Start pvm and run the program by typing "main". Running the program with the
default input definitions (in main.h), and parameters (argon_10_c) will start
a short calculation, 100 integration steps, of a system of ten cubic cells
with 500 argon atoms each on an fcc lattice that is being melted at 1000 K.
NO WARRANTY is offered on the software supplied here -- use it at your own
risk. NO RESTRICTIONS are imposed on use and distribution of the code -- you
are free to use this software for any application, modify it in any way, use
pieces in other programs, and give it away to anyone else. You are encouraged
to extend these rights to others who receive this software from you, following
the philosophy and intentions of the Free Software Foundation.
You are asked to include a brief acknowledgement in any scientific papers
you write using this code; I would appreciate receiving preprints or reprints
of such papers. I would also appreciate receiving reports of bugs and/or
problems encountered in porting this code to other parallel multicomputers.
Documentation about PVM:
-----------------------
All the documentation available to users of pvm can be found in the
"PVM 3 USER's GUIDE AND REFERENCE MANUAL" as a postcript file named ug.ps
in the directory /usr/var/tmp/pvm of basie.
File refcard.ps is postcript file with the Quick Reference Guide, Release 3.3
and writeup.ps contains a brief description of the PVM project for Release 3.0
Documentation about CMMD:
------------------------
All the documentation available to users of the CMMD library as well as
the description of hostless spmd programs for the CM5 can be
found in the reference manuals that can be accessed from conx typing
cmview
Documentation about Short Range Molecular Dynamimcs:
---------------------------------------------------
- "A parallel scalable approach to short range molecular dynamics" by
R. Giles and P. Tamayo. Proceedings of Conference on Scalable High
Performance Computing (1992).
- "50 Gflops Molecular Dynamics on the Connection Machine 5" by
P.S. Lomdhal, P. Tamayo, N. Gronbech-Jensen and D.M Beazley. Proceedings
of the conference on scalable high performance computing, 1993.
- "Message-passing multi-cell molecular dynamics on the connection machine 5"
by D. M. Beazley and P.S. Lomdhal. Parallel Computing (1994), 173-195.
- "Parallel and Scalable Short Range Molecular Dynamics on Coupled Clusters
of Workstations", by V.S. Batista and D.F. Coker. Submitted to Parallel
Computing (1994).
Date of last update: November 1, 1994.