Lab/Seminar Lecture 1
                              10/25/1994
                              11/01/1994

				 PVM

			    Victor Batista


PARALLEL PROGRAMING ON COUPLED CLUSTERS OF WORKSTATIONS OR CM5 SUPERCOMPUTERS
-----------------------------------------------------------------------------
USING PVM OR CMMD MESSAGE-PASSING SUBROUTINES:
---------------------------------------------

Introduction: 
------------

In this file you will find all the information you need in order to write and 
run spmd programs with calls to pvm or cmmd message-passing subroutines.
It includes all the examples presented in the lectures as well as references to
all available documentation about pvm and CMMD message passing subroutines.
For convenience it was written as a sequence of "baby-steps" that should
be followed sequentially. 
It also contains information about "MD_SPMD" version 1.0 [April, 1994], a 
parallel and scalable algorithm for doing short range molecular dynamics on 
distributed memory parallel architectures such as clusters of workstations 
or CM-5 supercomputers. The program is loaded in the directory /usr/var/tmp/pvm
as a tar-file named md_spmd.tar

Steps to make pvm work from your account in the HPC cluster:
-----------------------------------------------------------
>From your home directory prompt in basie type:

cp .cshrc cshrc_bak
cp .rhosts rhosts_bak
cp /usr/var/tmp/pvm/.cshrc .
cp /usr/var/tmp/pvm/cshrc.stub .
cp /usr/var/tmp/pvm/cshrc.stub .
cp /usr/var/tmp/pvm/.rhosts .

Edit the file named cshrc.stub and replace the name of the home directory from
/usr3/graduate/batista to your HPC home directory name. Then edit the file 
named .rhosts and replace the login names from batista to your login names.

Then type,

source .cshrc

Now you can try pvm by typing some basic commands. Type:

pvm     
                  you will get:
 
                  basie% pvm
                  pvm>

then type,
add billie 
                  you will get:

                  pvm> add billie
                  1 successful
                                        HOST     DTID 
                                      billie    80000
                  pvm>

then type,

halt
                  you will get:

                  pvm> halt
                  libpvm [t40001]: mxfer() EOF on pvmd sock
                  basie%

If you got these output lines, pvm3.3.3 is working for you on the HPC
cluster. CONGRATULATIONS!!.

Steps for working with examples:
--------------------------------

>From your home directory prompt in basie type:

mkdir pvm3
cd pvm3
mkdir bin
cd bin
mkdir SGI5
cd ..
mkdir examples
cd examples
cp /usr/var/tmp/pvm/* .

Example 1: (main.c)
----------
This example, brings you with all the information you need in order to
write and compile your own spmd programs with pvm calls.
It provides you with three general functions presented in the file
named tools.c for building dynamically a cluster of hosts where processors 
are being spawn, for sending and receiving arrays of data 
(with similar characteristics as those of the CMMD library), and for 
finishing with the participation of hosts in the pool. For learning how
to compile your own programs take a look at the Makefile where all the
flags and include files are defined.

Edit the file named Makefile and replace the name of MY_HOME DIR from
/usr3/graduate/batista to your home directory name. Then type 

make
                          you will get something like:

                          basie% make
                          cc -g -I/usr/local/pvm3/include -o main main.o tools.o /usr/local/pvm3/lib/SGI5/libpvm3.a -lsun -lm
                          mv main /home_directory_name/pvm3/bin/SGI5
                          basie%

Then start pvm by typing,

pvm
                          you will get:

                          basie% pvm
                          pvm>

Then type ,

quit
                          you will get:

                          pvm> quit

                          pvmd still running.
                          basie%

Then type,

main

You will get the output of the program main.c This is an array that has
been initialized piecewise by each of the computers of the HPC cluster.


Example 2: (spmd.c)
----------

Edit the file called Makefile and replace all strings "main" by "spmd", 
except for those "main.h".

Then type,

make
                            you will get something like:
 
                            basie% make 
                                cc -g -I/usr/local/pvm3/include -c spmd.c
                                cc -g -I/usr/local/pvm3/include -o spmd spmd.o tools.o /usr/local/pvm3/lib/SGI5/libpvm3.a -lsun -lm
                                mv spmd /home_directory_name/pvm3/bin/SGI5
                            basie%

Then start pvm by typing,

pvm

                            you will get:

                            basie% pvm
                            pvm>

Then quit typing,

quit 
                            you will get:

                            pvm> quit

                            pvmd still running.
                            basie%

Then run the program typing,

spmd
                            you will get:

                            basie% spmd
                            me = 0 mytid = 262146
                            token ring done
                            basie%

This is the output of the program that simulates a token going around a ring
of processors. This is one of the examples presented in the reference manual.
More examples are loaded in the directory /usr/local/pvm3.3.3/examples
and are discussed in the reference manual "PVM 3 USER's GUIDE AND REFERENCE 
MANUAL". 


How to make pvm run from your account on conx:
----------------------------------------------

Go to your home directory prompt on conx and get from basie the files named
.rhosts .cshrc and cshrc.stub typing,

cp .cshrc cshrc_bak
cp .rhosts rhosts_bak

ftp basie
                               you'll somthing like:

                               Connected to basie.bu.edu.
                               220 basie.bu.edu FTP server ready.
                               Name (basie:your login name):
                               331 Password required for batista.
                               Password:
                               230 User your name logged in.
                               ftp>

then type,

prompt
                               ftp> prompt
                               Interactive mode off.
                               ftp>

mget .cshrc cshrc.stub .rhosts
 
                               you'll get:

                               mget .cshrc? 
                               200 PORT command successful.
                               150 Opening ASCII mode data connection for '.cshrc' (5208 bytes).
                               226 Transfer complete.
                               local: .cshrc remote: .cshrc
                               5402 bytes received in 0.039 seconds (1.4e+02 Kbytes/s)
                               mget cshrc.stub? 
                               200 PORT command successful.
                               150 Opening ASCII mode data connection for 'cshrc.stub' (1709 bytes).
                               226 Transfer complete.
                               local: cshrc.stub remote: cshrc.stub
                               1808 bytes received in 0.042 seconds (42 Kbytes/s)
                               mget .rhosts? 
                               ftp> get .rhosts
                               200 PORT command successful.
                               150 Opening ASCII mode data connection for '.rhosts' (943 bytes).
                               226 Transfer complete.
                               local: .rhosts remote: .rhosts
                               986 bytes received in 0.0087 seconds (1.1e+02 Kbytes/s)
                               ftp>

Then type,

quit

And you'll get the prompt back.

Edit the file named cshrc.stub and replace the name of the home directory from
/usr3/graduate/batista to your conx home directory name. Then edit the file 
named .rhosts and replace the login names from batista to your login names.

Now type, 

source .cshrc

Now you can try pvm by typing some basic commands. Type:

pvm     
                  you will get:
 
                  conx% pvm
                  pvm>

then type,
 
add edsac 
                  you will get:

                  pvm> add edsac
                  1 successful
                                        HOST     DTID 
                                       edsac    80000
                  pvm>

then type,

add billie
                  and you'll get:

                  pvm> add billie
                  1 successful
                                        HOST     DTID
                                      billie    c0000
                  pvm>
then type,

add illiac
                  and you'll get:

                  pvm> add illiac
                  1 successful
                                        HOST     DTID
                                      illiac   100000
                  pvm>
the type,

conf
                  and you'll get:

                  pvm> conf
                  4 hosts, 1 data format
                                       HOST     DTID     ARCH   SPEED
                                       conx    40000     SUN4    1000
                                      edsac    80000      CM5    1000
                                     billie    c0000     SGI5    1000
                                     illiac   100000      CM5    1000
                  pvm>
Then type,

halt
                  you will get:

                  pvm> halt
                  libpvm [t40001]: mxfer() EOF on pvmd sock
                  conx%

Now you are able to make a cluster of SGI, SUN4 and CM5 computers that
can communicate and run MIMD programs!!!. Wooow!.


How to test example 1 (main.c), on conx using PVM calls:
-------------------------------------------------------

Type,

mkdir pvm3
cd pvm3
mkdir bin
cd bin
mkdir SUN4
cd ..
mkdir examples
cd examples

using the ftp command mget all of the files you have in your directory
home_directory/pvm3/examples in basie.

Edit the file named Makefile and replace the lines that read

PVM_ARCH =   SGI
ARCHLIB  =  -lsun

by the lines,

PVM_ARCH  =  SUN4
ARCHLIB   =

Comment out lines 18 and 19 so they will read

#CC             =             cc
#PREPROCESSOR   =             cc -E

and uncomment lines 21 and 22 to,

CC              =            gcc
PREPOCESSOR     =            gcc -E

Replace the name of MY_HOME_DIR from /usr3/graduate/batista to your home
directory name in conx.

Edit file main.h and comment out the line that defines SGI,
/*
#define SGI
*/
and uncomment lines that read

#define SUN4
#define RAND_MAX 2147483646

Now type,

make
                                you'll get something like,

                                conx% make
                                gcc -g -I/usr/local/pvm3/include -c main.c
                                gcc -g -I/usr/local/pvm3/include -c tools.c
                                gcc -g -I/usr/local/pvm3/include -o main main.o tools.o /usr/local/pvm3/lib/SUN4/libpvm3.a  -lm
                                mv main /you_home_dir/pvm3/bin/SUN4
                                conx%

Start up pvm console typing,

pvm

                                 you will get,

                                 conx% pvm
                                 pvm>
Then quit typing,

quit
 
                                 you will get,
                                 pvm> quit

                                 pvmd still running.
                                 conx%


Edit the file Makefile and replace all strings "spmd" by strings "main".
In order to run it, just type

main

Before the program is over, open another window in basie type
pvm, and see that hosts are being added to the pool. you can type 
"conf" in order to see all the hosts already added. At the end
the deamon is deleted.

The output is once again the array that has been initialized by each of
the hosts of the pool.

It also interesting to use prism and see how the array evolves after
each communication step. In order to do this, start by typing

setenv DISPLAY name_of_the_terminal:0.0
prism /your_home_dir/pvm3/bin/SUN4/main

Then put a break in lines 43 and 49. Click "Run".
You can go to basie and type 

pvm


                                   basie% pvm
                                   pvmd already running.
                                   pvm> conf
                                   10 hosts, 1 data format
                                                HOST     DTID     ARCH   SPEED
                                                conx    40000     SUN4    1000
                                               basie   2c0000     SGI5    1000
                                               miles   300000     SGI5    1000
                                                bird   340000     SGI5    1000
                                              billie   380000     SGI5    1000
                                           ellington   3c0000     SGI5    1000
                                                monk   400000     SGI5    1000
                                            marsalis   440000     SGI5    1000
                                              quincy   480000     SGI5    1000
                                          fitzgerald   4c0000     SGI5    1000
                                   pvm>

Now you can highlight the variable x, and you'll see that it has been 
initialized only for its first 10 elements. Now click "Continue", you can 
see that after one communication step, host 0 has communicated with host #9 
and has received  elements 90-99, subsequent steps show how the array is being
updated with data coming from all the other hosts of the pool.



How to run example 1 (main.c), on conx, using CMMD synchronous message 
----------------------------------------------------------------------
passing functions instead of using PVM calls:
---------------------------------------------

All you have to do is to recompile the program with the appropriate flags.
Edit the file Makefile, comment out "PVM flags" and uncomment
"CMMD flags". Comment out the line that reads

mv main $(XDIR)

Edit file named "main.h" and uncomment lines for CMMD library

#define CMMDRUN
#include 
#if CMMD_VERSION == 20
#include 
#endif

Comment out lines for PVM library
/*
#define PVMRUN
#include
*/

set lenx to 16 instead of 10.

Now you are ready to compile it by typing,

make
                                        you'll get,

                                        you'll get,

                                        conx% make
                                        gcc -g -c main.c
                                        gcc -g -c tools.c
                                        cmmd-ld -comp gcc -g -o main main.o tools.o  -lm
                                        conx%


You can run it with prism, by opening a window in illiac 

(make sure you are in the directory pvm3/examples and that you have set the
environment to your terminal)

then type,

prism -node main

and proceed as before: place a break stop in lines 43 and 49 and click "Run".
You can highlight variable "x" and see how it evolves in subsequent steps.
It is also interesting to see arrays aux and bux.


Scalable and Parallel Short Range Molecular Dynamics (SPaSM):
------------------------------------------------------------
Molecular Dynamics (MD) is perhaps the broadest interdisciplinary field with 
the participation of Computer Scientists, Engineers, Material Scientists, 
Physicists, Chemists, Metheorologists, Biologists, etc. 
Effective use of parallel architectures seems to hold the greatest potential
for making MD able to deal with systems that are beyond the limits of speed and
memory of modern computers.

The file md_spmd.tar contains "MD_SPMD" version 1.0 [April, 1994], a 
parallel and scalable algorithm for doing short range molecular dynamics on 
distributed memory parallel architectures such as clusters of workstations 
or CM-5 supercomputers. See "Parallel Scalable Message-Passing Multi-Cell 
Molecular Dynamics on Coupled Cluster of Workstations" by Victor S. Batista 
and David F. Coker, Submitted to Parallel Computing (1994).
It is traditionally designed, especially with regard to keeping track of 
consistency of parameters, restart files, output trajectories and so forth.
The algorithm is numerically equivalent to the Verlet Neighbor List method. 
It is parallelized minimizing interprocessor communication at the expense of
some computational redundancy. Equations of motion are integrated using Gear
scheme method to fourth order. All calculations are performed using double 
precision. It calculates energy, pressure and virial as well as the pair 
distribution and velocity autocorrelation functions for nobel gases that
interact with Lennard-Jones potentials.
The code has been written in ANSI C for greater portability and to have
the flexibility of dynamic memory allocation.  Prototypes have been provided 
for all functions.  
Once you have untar the file md_spmd.tar you can compile it by running make 
on a UNIX system. (Make sure the Makefile has your own home_directory)
On other systems, you are on your own; take a look at the Makefile for hints.
Start pvm and run the program by typing "main". Running the program with the 
default input definitions (in main.h), and parameters (argon_10_c) will start 
a short calculation, 100 integration steps, of a system of ten cubic cells 
with 500 argon atoms each on an fcc lattice that is being melted at 1000 K.

NO WARRANTY is offered on the software supplied here -- use it at your own
risk.  NO RESTRICTIONS are imposed on use and distribution of the code -- you
are free to use this software for any application, modify it in any way, use
pieces in other programs, and give it away to anyone else.  You are encouraged
to extend these rights to others who receive this software from you, following
the philosophy and intentions of the Free Software Foundation.

You are asked to include a brief acknowledgement in any scientific papers
you write using this code; I would appreciate receiving preprints or reprints
of such papers.  I would also appreciate receiving reports of bugs and/or
problems encountered in porting this code to other parallel multicomputers.


Documentation about PVM:
-----------------------
All the documentation available to users of pvm can be found in the 
"PVM 3 USER's GUIDE AND REFERENCE MANUAL" as a postcript file named ug.ps 
in the directory /usr/var/tmp/pvm of basie.
File refcard.ps is postcript file with the Quick Reference Guide, Release 3.3
and writeup.ps contains a brief description of the PVM project for Release 3.0


Documentation about CMMD:
------------------------
All the documentation available to users of the CMMD library as well as
the description of hostless spmd programs  for the CM5 can be 
found in the reference manuals that can be accessed from conx typing
cmview

Documentation about Short Range Molecular Dynamimcs:
---------------------------------------------------
- "A parallel scalable approach to short range molecular dynamics" by
   R. Giles and P. Tamayo. Proceedings of Conference on Scalable High
   Performance Computing (1992).

- "50 Gflops Molecular Dynamics on the Connection Machine 5" by 
   P.S. Lomdhal, P. Tamayo, N. Gronbech-Jensen and D.M Beazley. Proceedings
   of the conference on scalable high performance computing, 1993.

- "Message-passing multi-cell molecular dynamics on the connection machine 5"
   by D. M. Beazley and P.S. Lomdhal. Parallel Computing (1994), 173-195.

- "Parallel and Scalable Short Range Molecular Dynamics on Coupled Clusters
   of Workstations", by V.S. Batista and D.F. Coker. Submitted to Parallel 
   Computing (1994).



This document has been prepared by Professor Azer Bestavros <best@cs.bu.edu> as the WWW Home Page for CS-551, which is part of the NSF-funded undergraduate curriculum on parallel computing at BU.

Date of last update: November 1, 1994.