Lecture 08
                              10/06/1994

Prallel Programming Models
--------------------------
So far, we have been introduced to a particular program model (the
SIMD) model, which is the model underlying the C* programming
language. The SIMD model (like many other programming models) is tied
up closely to an architectural model. This makes "programs" written in
that model easier to map on the targetted architectures (e.g. CM-2, or
CM-5), but not very useful for "any" architecture. This is why such
programming models are "architecture dependent". It would be very
interesting to be able to develop programs using an "architecture
independent" model. This has been the case with serial machines,
mainly because the architectural variations amongst them didn't
warrant the development of "specialized" models. With parallel
computers this is not the case anymore. One may argue that the
traditional "sequential" programming model should be used as the
architecture independent programming model for parallel computers, and
thet the task of mapping the program onto the architecture should be
delegated to optimizing "parallelizing" compilers. This argument holds
only if compilers could efficiently extract parallelism. Generally
speaking, this has not proven to be the case. 


The UNITY programming model
---------------------------
UNITY is an example of an architecture-independent programming model
that has been proposed by Shandi and Misra. UNITY is not (although it
is usually perceived as) a programming language; it is merely a way of
expressing computation so as not to "hide" potential parallelism. The
philosophy of UNITY is that an algorithm should be specified at a very
high level of abstraction (almost close to a specification) and then
refined so as to bring closer to the architectural model. Also, UNITY
provides for a mechanism to verify the correctness of algorithms (by
proving invariants, for examples) --- something that is seriously
lacking in almost all other approaches.

UNITY program structure
-----------------------

    program --> Program     "program-name"
                  declare   "declare-section"
                  always    "always-section"
                  initially "initially-section"
                  assign    "assign-section"
                end

The "program-name" is a string of text. The "declare-section" consists 
of variable declarations (we will use C-like notation, although
original UNITY uses a Pascal-like notation -- as we said earlier
syntax is not an issue here, since we are not defining a programming
language per se, bu only a programming model, a frame of mind). The
"always-section" is used to define variables that are functions of
others (think about them as macro definitions). The
"initially-section" defines the initial values of variables. The
"assign-section" contains a set of "assignment" statements that define
how the variables are allowed to change their states.


UNITY program execution
-----------------------
The program "execution" starts in any state that is defined by the
"initially-section". A step of a computation involves "chosing" an
assignment from the "assign-section" and "applying" it. As a result of
a computation step, the state of the program changes (i.e. each change
of state is considered a step). The program is said to have
terminated, if it reaches a "fixed point"; i.e. no more application of
an assign statement will ever change its state.


The Assignment statement
------------------------
The assignment statement is defined for both a single variable 
(e.g. x = 5)  or for multiple variables (e.g. x, y, z = 0, 1, 2),
where x gets 0, y gets 1, z gets 2. It is assumed that all assignment
are performed atomically and concurrently. So, for example the
statement 

	x, y = y, x

will result in swapping atomically the values of x and y.

Assignment statements may be composed. The first composition operation
is the parallel composition || which means that the two assignments
are to be executed concurrently and atomically. Thus, 

        x = y  ||  y = x

will also result in swapping atomically the values of x and y.

An assignment statement (whether with a single variable or with
multiple variables) may also be enumerated based on a condition (you
may want to think about this as a case-by-case assignment). For
example:

        x = +y   if y>=0 ~
            -y   if y<0 

will result in x been assigned the absolute value of y.

An assignment statement (whether with a single variable or with
multiple variables) may also be quantified (you
may want to think about this as a forall assignment). For
example:

        < || i : 0 <= i < N :: A[i] = max(A[i], B[i]) >

will assign to all the elements of A[i], for 0<=i<N, the maximum of
A[i] and B[i]. Notice that the quantification is specified with the
||operator, which means that the assignments are to be done
concurrently and atomically.

A variable may appear more than once on the left-hand-side of an
assignment statement (or on the left-hand-side of multiple assignments
composed using the || operator). It is the responsibility of the
"programmer" to ensure that the value assigned to that variable is
identical in all assignments. 


The Assign-section
------------------
To make an "assign-section", individual assignment statements may be
composed together using the [] operator, which can be thought of as
the "choice" operator. For example the execution of 

       x = y [] y = z

means that "randomly but fairly" one of the two assign statements on
either side of the [] operator will be "chosen" and executed. The
fairness condition says that if we execute the above composed
statements for an infinite amount of time, both components ( x = y )
and ( y = z ) will have a chance to execute infinitely often.

An assignment statement (whether with a single variable or with
multiple variables) may also be quantified using the [] operator. For
example: 

        < [] i : 0 <= i < N :: A[i] = 0 >

means that a value for i is chosen randomly (but fairly) from all
possible values between 0 and N-1, and the assignment A[i] = 0 is
performed. 

To run a program, an assign statement from the assign section is
chosen at random and executed and the process is repeated forever
(until a fixed point is reached). If the whole program consists of one
assignment statement then that statement is executed over and over.
For example the following assign-section

    < [] i,j : (0 <= i < N) && (0 <= j < N) :: A[i][j] = 0 if i != j ~
                                                         1 if i == j >

will "eventually" result in the program reaching a fixed point where
only the diagonal elements of the matrix A[][] are 1s and all other
elements are 0. 

A very important thing to notice about UNITY programs is that there is
no "sequential" composition operator between statements! Such a
composition, however, could be synthesized. For simplicity we will
assume that it is possible to identify a sequence of statements to be
executed in a strict sequential order. We should understand, however,
that such "serialization" has to be (and could be) implemented using
UNITY's simple || and [] compositions. 


The Initially-section
---------------------
The syntax of this section is identical to the "assign-section",
except for two conditions: that the assignments should not be circular
(i.e. no variable defined in terms of itself) and that there exists a
ordering of the assignments that will result in only "defined"
variables being used to define other variables. The non-circular
requirement guarantees that the execution of the "initially-section"
could be terminated after a finite number of iterations. The existence
of an ordering guarantees that there is a way for variables to be
initialized before being used to initialize other variables. An
<assignment-section> that guarantees these two conditions is called
"proper". The "initially-section" is a proper <assignment-section>!


The Always-section
------------------
The syntax of this section is identical to the "initially-section".
Variables appearing on the left-hand-side of an assignment in an
"always-section" is called a transparent variable because it could be
replaced in the "assign-section" with its right-hand-side.  In the
"always-section" transparent variables are usually defined in terms of
other transparent or non-transparent variables. Transparent variables
cannot appear on the left-hand-side of any assignments in the
"initially-section" or in the "assign-section". 


Some Examples of UNITY programs:
-------------------------------

Program BubbleSort
  ...
  assign
    < [] i : 0<=i<N :: A[i],A[i+1] = A[i+1],A[i]  if  A[i] > A[i+1] >

end


Program ParallelBubbleSort1
  ...
  assign
    < || i : 0<=i<N && even(i) :: A[i],A[i+1]=A[i+1],A[i] if A[i] > A[i+1] >
  []< || i : 0<=i<N &&  odd(i) :: A[i],A[i+1]=A[i+1],A[i] if A[i] > A[i+1] >
end


Program ParallelBubbleSort2
  ...
  initially
    j = 0 
  assign
     j = 1 - j
  [] < || i : 0<=i<N && j == i mod 2 :: 
       A[i],A[i+1]=A[i+1],A[i] if A[i] > A[i+1] >
end



/*
   Compute an array of bionomial coefficients using the identity

          n      n-1        n-1
           C  =     C    +     C
            k        k-1        k

*/
Program BionomialCoefficients
  ...
  intially
    < || n : 0<=n<N :: c[n][0],c[n][n] = 1,1 >
  assign
    < || n,k : 0<k<n && 0<n<N :: c[n][k] = c[n-1][k-1] + c[n-1][k] >
end


Some Comments
-------------
Notice that the order of execution for a UNITY program statements is
arbitrary.  For example, in the BionomialCoefficient program, some
c[n][k] may be assigned a value even when c[n-1][k-1] or c[n-1][k] has
not been computed. At fixed point, the correct values will prevail.

Generally speaking, replacing a || with a [] composition (or vice
versa) is not correct. For some programs, however, this may be
possible. For example, replacing || with [] will work for the above
programs!


Time complexity of a UNITY program
----------------------------------
The operational model of a program (how it is executed on a computer)
is straightforward for programs written in conventional sequential
languages executing on conventional von Newman architecture. It is
simply the number of "operations" (or steps) executed by the hardware.
For UNITY the notion of a "step" (where an operation is performed) is
not clear because there is no underlying architecture. One way to
"measure" work is to compute the number of state changes needed for a
program to reach a fixed point. For example, the BubbleSort program
described above will go through at most 0.5*N*(N-1) states before
reaching a fixed point. The ParallelBubbleSort programs on the other
hand will go through a maximum of N-1 states before reaching a fixed
point. We define the time complexity of a UNITY program as the maximum
number of state changes that must be traversed before reaching a fixed
point.