Lab/Seminar Lecture 1 10/25/1994 11/01/1994 PVM Victor Batista PARALLEL PROGRAMING ON COUPLED CLUSTERS OF WORKSTATIONS OR CM5 SUPERCOMPUTERS ----------------------------------------------------------------------------- USING PVM OR CMMD MESSAGE-PASSING SUBROUTINES: --------------------------------------------- Introduction: ------------ In this file you will find all the information you need in order to write and run spmd programs with calls to pvm or cmmd message-passing subroutines. It includes all the examples presented in the lectures as well as references to all available documentation about pvm and CMMD message passing subroutines. For convenience it was written as a sequence of "baby-steps" that should be followed sequentially. It also contains information about "MD_SPMD" version 1.0 [April, 1994], a parallel and scalable algorithm for doing short range molecular dynamics on distributed memory parallel architectures such as clusters of workstations or CM-5 supercomputers. The program is loaded in the directory /usr/var/tmp/pvm as a tar-file named md_spmd.tar Steps to make pvm work from your account in the HPC cluster: ----------------------------------------------------------- >From your home directory prompt in basie type: cp .cshrc cshrc_bak cp .rhosts rhosts_bak cp /usr/var/tmp/pvm/.cshrc . cp /usr/var/tmp/pvm/cshrc.stub . cp /usr/var/tmp/pvm/cshrc.stub . cp /usr/var/tmp/pvm/.rhosts . Edit the file named cshrc.stub and replace the name of the home directory from /usr3/graduate/batista to your HPC home directory name. Then edit the file named .rhosts and replace the login names from batista to your login names. Then type, source .cshrc Now you can try pvm by typing some basic commands. Type: pvm you will get: basie% pvm pvm> then type, add billie you will get: pvm> add billie 1 successful HOST DTID billie 80000 pvm> then type, halt you will get: pvm> halt libpvm [t40001]: mxfer() EOF on pvmd sock basie% If you got these output lines, pvm3.3.3 is working for you on the HPC cluster. CONGRATULATIONS!!. Steps for working with examples: -------------------------------- >From your home directory prompt in basie type: mkdir pvm3 cd pvm3 mkdir bin cd bin mkdir SGI5 cd .. mkdir examples cd examples cp /usr/var/tmp/pvm/* . Example 1: (main.c) ---------- This example, brings you with all the information you need in order to write and compile your own spmd programs with pvm calls. It provides you with three general functions presented in the file named tools.c for building dynamically a cluster of hosts where processors are being spawn, for sending and receiving arrays of data (with similar characteristics as those of the CMMD library), and for finishing with the participation of hosts in the pool. For learning how to compile your own programs take a look at the Makefile where all the flags and include files are defined. Edit the file named Makefile and replace the name of MY_HOME DIR from /usr3/graduate/batista to your home directory name. Then type make you will get something like: basie% make cc -g -I/usr/local/pvm3/include -o main main.o tools.o /usr/local/pvm3/lib/SGI5/libpvm3.a -lsun -lm mv main /home_directory_name/pvm3/bin/SGI5 basie% Then start pvm by typing, pvm you will get: basie% pvm pvm> Then type , quit you will get: pvm> quit pvmd still running. basie% Then type, main You will get the output of the program main.c This is an array that has been initialized piecewise by each of the computers of the HPC cluster. Example 2: (spmd.c) ---------- Edit the file called Makefile and replace all strings "main" by "spmd", except for those "main.h". Then type, make you will get something like: basie% make cc -g -I/usr/local/pvm3/include -c spmd.c cc -g -I/usr/local/pvm3/include -o spmd spmd.o tools.o /usr/local/pvm3/lib/SGI5/libpvm3.a -lsun -lm mv spmd /home_directory_name/pvm3/bin/SGI5 basie% Then start pvm by typing, pvm you will get: basie% pvm pvm> Then quit typing, quit you will get: pvm> quit pvmd still running. basie% Then run the program typing, spmd you will get: basie% spmd me = 0 mytid = 262146 token ring done basie% This is the output of the program that simulates a token going around a ring of processors. This is one of the examples presented in the reference manual. More examples are loaded in the directory /usr/local/pvm3.3.3/examples and are discussed in the reference manual "PVM 3 USER's GUIDE AND REFERENCE MANUAL". How to make pvm run from your account on conx: ---------------------------------------------- Go to your home directory prompt on conx and get from basie the files named .rhosts .cshrc and cshrc.stub typing, cp .cshrc cshrc_bak cp .rhosts rhosts_bak ftp basie you'll somthing like: Connected to basie.bu.edu. 220 basie.bu.edu FTP server ready. Name (basie:your login name): 331 Password required for batista. Password: 230 User your name logged in. ftp> then type, prompt ftp> prompt Interactive mode off. ftp> mget .cshrc cshrc.stub .rhosts you'll get: mget .cshrc? 200 PORT command successful. 150 Opening ASCII mode data connection for '.cshrc' (5208 bytes). 226 Transfer complete. local: .cshrc remote: .cshrc 5402 bytes received in 0.039 seconds (1.4e+02 Kbytes/s) mget cshrc.stub? 200 PORT command successful. 150 Opening ASCII mode data connection for 'cshrc.stub' (1709 bytes). 226 Transfer complete. local: cshrc.stub remote: cshrc.stub 1808 bytes received in 0.042 seconds (42 Kbytes/s) mget .rhosts? ftp> get .rhosts 200 PORT command successful. 150 Opening ASCII mode data connection for '.rhosts' (943 bytes). 226 Transfer complete. local: .rhosts remote: .rhosts 986 bytes received in 0.0087 seconds (1.1e+02 Kbytes/s) ftp> Then type, quit And you'll get the prompt back. Edit the file named cshrc.stub and replace the name of the home directory from /usr3/graduate/batista to your conx home directory name. Then edit the file named .rhosts and replace the login names from batista to your login names. Now type, source .cshrc Now you can try pvm by typing some basic commands. Type: pvm you will get: conx% pvm pvm> then type, add edsac you will get: pvm> add edsac 1 successful HOST DTID edsac 80000 pvm> then type, add billie and you'll get: pvm> add billie 1 successful HOST DTID billie c0000 pvm> then type, add illiac and you'll get: pvm> add illiac 1 successful HOST DTID illiac 100000 pvm> the type, conf and you'll get: pvm> conf 4 hosts, 1 data format HOST DTID ARCH SPEED conx 40000 SUN4 1000 edsac 80000 CM5 1000 billie c0000 SGI5 1000 illiac 100000 CM5 1000 pvm> Then type, halt you will get: pvm> halt libpvm [t40001]: mxfer() EOF on pvmd sock conx% Now you are able to make a cluster of SGI, SUN4 and CM5 computers that can communicate and run MIMD programs!!!. Wooow!. How to test example 1 (main.c), on conx using PVM calls: ------------------------------------------------------- Type, mkdir pvm3 cd pvm3 mkdir bin cd bin mkdir SUN4 cd .. mkdir examples cd examples using the ftp command mget all of the files you have in your directory home_directory/pvm3/examples in basie. Edit the file named Makefile and replace the lines that read PVM_ARCH = SGI ARCHLIB = -lsun by the lines, PVM_ARCH = SUN4 ARCHLIB = Comment out lines 18 and 19 so they will read #CC = cc #PREPROCESSOR = cc -E and uncomment lines 21 and 22 to, CC = gcc PREPOCESSOR = gcc -E Replace the name of MY_HOME_DIR from /usr3/graduate/batista to your home directory name in conx. Edit file main.h and comment out the line that defines SGI, /* #define SGI */ and uncomment lines that read #define SUN4 #define RAND_MAX 2147483646 Now type, make you'll get something like, conx% make gcc -g -I/usr/local/pvm3/include -c main.c gcc -g -I/usr/local/pvm3/include -c tools.c gcc -g -I/usr/local/pvm3/include -o main main.o tools.o /usr/local/pvm3/lib/SUN4/libpvm3.a -lm mv main /you_home_dir/pvm3/bin/SUN4 conx% Start up pvm console typing, pvm you will get, conx% pvm pvm> Then quit typing, quit you will get, pvm> quit pvmd still running. conx% Edit the file Makefile and replace all strings "spmd" by strings "main". In order to run it, just type main Before the program is over, open another window in basie type pvm, and see that hosts are being added to the pool. you can type "conf" in order to see all the hosts already added. At the end the deamon is deleted. The output is once again the array that has been initialized by each of the hosts of the pool. It also interesting to use prism and see how the array evolves after each communication step. In order to do this, start by typing setenv DISPLAY name_of_the_terminal:0.0 prism /your_home_dir/pvm3/bin/SUN4/main Then put a break in lines 43 and 49. Click "Run". You can go to basie and type pvm basie% pvm pvmd already running. pvm> conf 10 hosts, 1 data format HOST DTID ARCH SPEED conx 40000 SUN4 1000 basie 2c0000 SGI5 1000 miles 300000 SGI5 1000 bird 340000 SGI5 1000 billie 380000 SGI5 1000 ellington 3c0000 SGI5 1000 monk 400000 SGI5 1000 marsalis 440000 SGI5 1000 quincy 480000 SGI5 1000 fitzgerald 4c0000 SGI5 1000 pvm> Now you can highlight the variable x, and you'll see that it has been initialized only for its first 10 elements. Now click "Continue", you can see that after one communication step, host 0 has communicated with host #9 and has received elements 90-99, subsequent steps show how the array is being updated with data coming from all the other hosts of the pool. How to run example 1 (main.c), on conx, using CMMD synchronous message ---------------------------------------------------------------------- passing functions instead of using PVM calls: --------------------------------------------- All you have to do is to recompile the program with the appropriate flags. Edit the file Makefile, comment out "PVM flags" and uncomment "CMMD flags". Comment out the line that reads mv main $(XDIR) Edit file named "main.h" and uncomment lines for CMMD library #define CMMDRUN #include#if CMMD_VERSION == 20 #include #endif Comment out lines for PVM library /* #define PVMRUN #include */ set lenx to 16 instead of 10. Now you are ready to compile it by typing, make you'll get, you'll get, conx% make gcc -g -c main.c gcc -g -c tools.c cmmd-ld -comp gcc -g -o main main.o tools.o -lm conx% You can run it with prism, by opening a window in illiac (make sure you are in the directory pvm3/examples and that you have set the environment to your terminal) then type, prism -node main and proceed as before: place a break stop in lines 43 and 49 and click "Run". You can highlight variable "x" and see how it evolves in subsequent steps. It is also interesting to see arrays aux and bux. Scalable and Parallel Short Range Molecular Dynamics (SPaSM): ------------------------------------------------------------ Molecular Dynamics (MD) is perhaps the broadest interdisciplinary field with the participation of Computer Scientists, Engineers, Material Scientists, Physicists, Chemists, Metheorologists, Biologists, etc. Effective use of parallel architectures seems to hold the greatest potential for making MD able to deal with systems that are beyond the limits of speed and memory of modern computers. The file md_spmd.tar contains "MD_SPMD" version 1.0 [April, 1994], a parallel and scalable algorithm for doing short range molecular dynamics on distributed memory parallel architectures such as clusters of workstations or CM-5 supercomputers. See "Parallel Scalable Message-Passing Multi-Cell Molecular Dynamics on Coupled Cluster of Workstations" by Victor S. Batista and David F. Coker, Submitted to Parallel Computing (1994). It is traditionally designed, especially with regard to keeping track of consistency of parameters, restart files, output trajectories and so forth. The algorithm is numerically equivalent to the Verlet Neighbor List method. It is parallelized minimizing interprocessor communication at the expense of some computational redundancy. Equations of motion are integrated using Gear scheme method to fourth order. All calculations are performed using double precision. It calculates energy, pressure and virial as well as the pair distribution and velocity autocorrelation functions for nobel gases that interact with Lennard-Jones potentials. The code has been written in ANSI C for greater portability and to have the flexibility of dynamic memory allocation. Prototypes have been provided for all functions. Once you have untar the file md_spmd.tar you can compile it by running make on a UNIX system. (Make sure the Makefile has your own home_directory) On other systems, you are on your own; take a look at the Makefile for hints. Start pvm and run the program by typing "main". Running the program with the default input definitions (in main.h), and parameters (argon_10_c) will start a short calculation, 100 integration steps, of a system of ten cubic cells with 500 argon atoms each on an fcc lattice that is being melted at 1000 K. NO WARRANTY is offered on the software supplied here -- use it at your own risk. NO RESTRICTIONS are imposed on use and distribution of the code -- you are free to use this software for any application, modify it in any way, use pieces in other programs, and give it away to anyone else. You are encouraged to extend these rights to others who receive this software from you, following the philosophy and intentions of the Free Software Foundation. You are asked to include a brief acknowledgement in any scientific papers you write using this code; I would appreciate receiving preprints or reprints of such papers. I would also appreciate receiving reports of bugs and/or problems encountered in porting this code to other parallel multicomputers. Documentation about PVM: ----------------------- All the documentation available to users of pvm can be found in the "PVM 3 USER's GUIDE AND REFERENCE MANUAL" as a postcript file named ug.ps in the directory /usr/var/tmp/pvm of basie. File refcard.ps is postcript file with the Quick Reference Guide, Release 3.3 and writeup.ps contains a brief description of the PVM project for Release 3.0 Documentation about CMMD: ------------------------ All the documentation available to users of the CMMD library as well as the description of hostless spmd programs for the CM5 can be found in the reference manuals that can be accessed from conx typing cmview Documentation about Short Range Molecular Dynamimcs: --------------------------------------------------- - "A parallel scalable approach to short range molecular dynamics" by R. Giles and P. Tamayo. Proceedings of Conference on Scalable High Performance Computing (1992). - "50 Gflops Molecular Dynamics on the Connection Machine 5" by P.S. Lomdhal, P. Tamayo, N. Gronbech-Jensen and D.M Beazley. Proceedings of the conference on scalable high performance computing, 1993. - "Message-passing multi-cell molecular dynamics on the connection machine 5" by D. M. Beazley and P.S. Lomdhal. Parallel Computing (1994), 173-195. - "Parallel and Scalable Short Range Molecular Dynamics on Coupled Clusters of Workstations", by V.S. Batista and D.F. Coker. Submitted to Parallel Computing (1994).
Date of last update: November 1, 1994.