
\chapter{The Design of Cube Calculus Machine Co-processor}

% \setcounter{page}{71}
\label{CHEN-Chapter5}

In our design, the cube calculus machine acts as a co-processor to the
host computer, and it will be realized on the DEC-PERLE-1 board. Therefore,
the architecture of DEC-PERLE-1 is our only design constraint.

The {\bf input FIFO} and the {\bf output FIFO} on DEC-PERLE-1 allowed us to significantly 
simplify the
communication between the host and the DEC-PELRE-1 board. This feature lets
the host and the DEC-PERLE-1 board {\bf to work asynchronously}. Therefore, we use
these two FIFOs as the way of communicating between the host and
the CCM.

By using the input and output FIFOs, the communication between the host 
and the CCM is as follows. The host just puts instructions into the 
input FIFO, and receives the results from the output FIFO. On the other 
side, the CCM takes an instruction from the input FIFO, executes this
instruction and puts the results back into the output FIFO. 
This is shown in Figure \ref{fig:des7}.

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/des7.eps}}}
\caption{Communication between the host and the CCM}
\label{fig:des7}
\end{figure}

As described in Chapter~\ref{CHEN-Chapter4}, the width of FIFOs is 32 bits, which means
the data transferred between the host and the CCM is 32-bit-wide.
At this time, we want to keep the CCM as simple as possible,
so we just use {\bf fixed-length instructions}, and the width of all
instructions will be 32 bits. This means that the opcode and the actual 
data of a CCM instruction are both included in one {\bf 32-bit-wide word.}

\section{Executing Patterns}
\label{CHEN-section5.1}

Before we design the {\bf CCM instructions}, we need to know what kind of 
execution patterns happen often in practical applications of the CCM, 
and how can our design be able to execute cube operations efficiently
for these execution patterns. 

\begin{figure}[H]
\hbox{
\begin{minipage}{1.45in}
\centerline{\hbox{
\psfig{figure=fig/patn1.eps,width=1.1in}
}}
\centerline{(a)}
\end{minipage}
\begin{minipage}{1.45in}
\centerline{\hbox{
\psfig{figure=fig/patn2.eps,width=1.1in}
}}
\centerline{(b)}
\end{minipage}
\begin{minipage}{1.45in}
\centerline{\hbox{
\psfig{figure=fig/patn3.eps,width=1.1in}
}}
\centerline{(c)}
\end{minipage}
\begin{minipage}{1.45in}
\centerline{\hbox{
\psfig{figure=fig/patn4.eps,width=1.1in}
}}
\centerline{(d)}
\end{minipage}
}
\caption{Cube operation patterns}
\label{fig:patns}
\end{figure}

We found {\bf four patterns} based on the analysis of many algorithms in logic
synthesis, such as satisfiability, tautology, complementation, 
solving of equations and others that must be speeded-up.
These four basic patterns are shown in Figure \ref{fig:patns}. 
Many practical complex patterns can be created by repeating or combining
these basic patterns:
\begin{enumerate}
\item[Pattern (a)] (Figure \ref{fig:patns} (a)) is the general form of 
combinational cube operations. A combinational cube operation produces one resultant cube.
\item[Pattern (b)] (Figure \ref{fig:patns} (b)) is the general form of 
sequential cube operations. A sequential
cube operation produces as many as $n$ resultant cubes, where $n$ is the 
number of variables in the operand cubes. 

\item[Pattern (c)] (Figure \ref{fig:patns} (c)) is used in some combinational
cube operations on an array of
cubes, for example, the result of intersection operation on an array of cubes 
($A_1 \cdot A_2 \cdots A_n$) is a single cube or an empty cube. 

\item[Pattern (d)] (Figure \ref{fig:patns} (d)) can be used both in combinational
and sequential cube operations.
\end{enumerate}

A combinational operation example is a cofactor operation on an array of cubes: \\
\centerline{$\vec{C}|_{A} = (C_1|_{A}\; C_2|_{A} \ldots C_n|_{A})$}
A sequential operation example is a sharp operation on two arrays of cubes,
and this is the most complicated case:

\begin{align*}
\vec{C} = \vec{A} \# \vec{B} 
=& (A_1\;A_2\ldots A_m) \# (B_1\;B_2\ldots B_n) \\
=&\Big(\big((A_1\# B_1\; A_2\# B_1\ldots A_m\# B_1)\# B_2\big)\cdots\#B_n\Big)\\
=&\big((C_1^1 \# B_2 \; C_2^1 \# B_2 \ldots C_k^1 \# B_2) \cdots \# B_n\big)\\
 & \;\vdots
\end{align*}

\noindent
where ($C_1^1\, C_2^1 \ldots C_k^1$) is the result of 
operation ($A_1 \# B_1 \, A_2 \# B_1 \ldots A_m \# B_1$). As we can see
from the equation, the basic step for sharp operation on two arrays of cubes
is the sharp operation on one array of cubes and one cube. This is what
Pattern (d) describes. Therefore, the pattern of sharp operation on two 
arrays of cubes repeats pattern (d) as many times as the number of
cubes in the array of cubes $\vec{B}$.

It can be seen from these execution patterns that the same cube operation 
is executed very many times before another kind of cube operation is 
executed in a practical application. Also, sometimes one operand 
cube does not change in subsequent operations or comes from the result
of the previous cube operation. Thus, we have the following design 
considerations:
\begin{itemize}
\item We need an {\it accumulator} register for pattern (c), this 
      {\it accumulator} can be set by the user or it receives the data being 
      the result of a previous cube operation. As discussed in Chapter~\ref{CHEN-Chapter2}, 
      most cube operations have two operand cubes. Thus, we need
      another {\it general} data register to store another operand cube.
\item The CCM can execute cube operations by just accepting operand
      cube(s) without re-setting the {\it instruction} register.
\end{itemize}

\section{The Design of the CCM}
\label{CHEN-section5.2}

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/ndes1.eps,height=5.7in}}}
\caption{The Block Diagram of Our Design}
\label{fig:ndes1}
\end{figure}

The block diagram of our design is shown in Figure \ref{fig:ndes1}. 
In this design, there are 5 data buses, 2 banks of memories, Global 
Control Unit (GCU), ILU and it's controller Operation Control Unit (OCU),
two address units, registers, tri-state buffers and three multiplexers.
The following section will discuss them in detail. The control signals
are not shown in the figure, and they are all generated by GCU, which
means that all components of the CCM work together under the control 
of the GCU.


\subsection{Data Bus}
\label{CHEN-section5.2.1}

Five data buses are used in the CCM. They are described as follows:
\begin{description}
\item $IBus$ is the short name of  {\bf input FIFO data bus}.
      The CCM receives the instruction from the input FIFO through
      this data bus. Only input FIFO can write this data bus.
\item $OBus$ is the short form of {\bf output FIFO data bus}.
      The CCM puts the results into output FIFO through this data bus.
      Only the ILU can write this data Bus.
\item $ABus$ is the short name of {\bf Address data bus}. The CCM sets
      the contents of two address units AddrA and AddrB and one address 
      register AddrR through this data bus. The input FIFO and two address 
      units can write this data bus, and they are controlled by three
      control signals: {\it EnIFifoA}, {\it EnAddrA} and {\it EnAddrB},
      which control the corresponding tri-state buffers\footnote
     {There are programmable tri-state buffer resources in Xilinx XC3090 FPGA}.
\item $DBusA$ is the short name of {\bf Data Bus A}. This data bus
      connects to the input FIFO, memory bank A (MEM\_A) and the 
      input and output of the ILU. The input FIFO, MEM\_A and the ILU
      can write this data bus, and they are controlled by three control 
      signals: {\it EnIFifoD}, {\it MemARW}, and {\it EnIluA}, which 
      control the corresponding tri-state buffers.
\item $DBusB$ is the short name of {\bf Data Bus B}. This data bus
      connects to the memory bank B (MEM\_B), the input and output of 
      the ILU. The MEM\_B and the ILU can write this data bus,
      and they are controlled by two control signals: {\it MemBRW} and 
      {\it EnIluB}, which control the corresponding tri-state buffers.
\end{description}
The examples that show how to use these buses will be given in section~\ref{CHEN-section6.2}.

\subsection{Memory and Address Units}
\label{CHEN-section5.2.2}

In this design, we use two banks of memory, MEM\_A and MEM\_B, to store 
intermediate results. Each bank of memory connects to one data bus: MEM\_A
connects to the {\it DBusA} and {\it MEM\_B} connects to the {\it DBusB}.

The address signals of MEM\_A come from {\it Address Unit A} ({\it AddrA} 
for short). The address signals of MEM\_B come from {\it Address Unit B} 
({\it AddrB} for short). The contents of these two address units can be
set or incremented under the control of GCU. These two address units are
realized by 18-bit-wide loadable-up-counters. A {\it Address Register} 
({\it AddrR} for short) is used to store an address data shown on 
{\it ABus}. As we mentioned before, {\it ABus} can be written by {\it IBus},
{\it AddrA} or {\it AddrB}, so the address data could be one of these 
three sources. The example of using {\it AddrR} is shown in section~\ref{CHEN-section6.2}.

The control signal {\it MemARW} controls the MEM\_A in read mode or write 
mode, which means the MemARW can read from or write to the data bus
{\it DBusA}. When the {\it MemARW} is set, the MEM\_A is in read mode, 
otherwise, the MEM\_A is in write mode. The control signal {\it MemBRW}
controls the MEM\_B in the same manner.

\subsection{Registers}
\label{CHEN-section5.2.3}

\begin{figure}[H]
\hbox{\hspace{0.4in}\psfig{figure=fig/instreg.eps,width=5in}}
\caption{The content of {\it instruction} register}
\label{fig:instreg}
\end{figure}

There are six registers used by ILU in our design ({\it AddrR} register is
mentioned in the above section). They are described as follows:

\begin{description}
\item {\tt Accu} is an {\it accumulator register} used to store one operand
      cube for the cube operation. It is 30 bits wide.
\item {\tt Data} is a general {\it data register} used to store one operand
      cube for the cube operation. It is 30 bits wide.
\item {\tt Water} is a 15 bits wide register used to store {\it water} signals.
\item {\tt Rightedge} is a 15 bits wide register used to store {\it right\_edge}
      signals.
\item {\tt Inst} is a 21 bit wide register used to store cube operation 
      instruction. The content of the {\it inst} register is shown
      in Figure \ref{fig:instreg}. The meaning of these nine fields is the following:

\begin{enumerate}
\item[{\it p1}] field represents whether the first pre-relation/pre-operation is valid or not.
\item[{\it p2}] field represents whether the second pre-relation/pre-operation
      is valid or not when {\it p1} = 1.
\item[{\it sc}] is sequential/combinational bit. When it is 1, the operation
      is a sequential operation, otherwise, the operation is combinational operation.
\item[{\it pm}] is prime bit. When it is 1, the operation is a complex 
      combinational operation, otherwise, the operation is a simple combinational operation.
\item[{\it ao}] is {\it and\_or} bit. When it is 1, the relation type of
      the operation is ``AND", otherwise, the relation type is ``OR".
\item[operations] {\it rel}, {\it bef}, {\it act} and {\it aft} are the four bitwise
      functions used to describe the operation (see Chapter~\ref{CHEN-Chapter2} and~\ref{CHEN-Chapter3}).
\item[{\tt PRPO}] is a 24 bits wide register used to store two pairs of
      pre-relation/pre-operation. The content of the {\it prpo} register
      is shown in Figure \ref{fig:prporeg}.  The meaning of these eight 
      fields are as follows:

\begin{figure}[H]
\hbox{\hspace{0.4in}\psfig{figure=fig/prporeg.eps,width=5.4in}}
\caption{The content of {\it prpo} register}
\label{fig:prporeg}
\end{figure}


%      \hspace{0.01in}\psfig{figure=fig/prporeg.eps,width=5.4in} 
%
%      \vspace{-0.1in}
%      All these eight signals (represented by these eight field) are
%      discussed in section~ 3.5.

     {\it pand\_or1}, {\it prel1}, {\it pcmp1}, {\it pval1} and {\it poper1}
     are the partial pre-relation type, partial pre-relation,
     pre-relation compare type,  pre-relation compare value and 
     pre-operation for the first pair of pre-relation/pre-operation,
     respectively. 

     {\it pand\_or2}, {\it prel2}, {\it pcmp2}, {\it pval2} and {\it poper2}
     are the partial pre-relation type, partial pre-relation,
     pre-relation compare type,  pre-relation compare value and 
     pre-operation for the second pair of pre-relation/pre-operation,
     respectively.

     For the more information about pre-relation/pre-operation, please
     see section~\ref{CHEN-section3.7}.

\end{enumerate}
\end{description}

As shown in Figure \ref{fig:ndes1}, the input of these six
registers is connected to either {\it DBusA} or {\it DBusB}. The signal
{\it ASrc} controls to which bus the {\it Accu} is connected. The
signal {\it OSrc} controls to which bus the {\it data} register is connected.

Every register has a load signal used to load data from its inputs, and 
all load signals are generated by GCU (they are not shown in Figure \ref{fig:ndes1}). 

There is one more register called {\it config} register (not shown in 
Figure \ref{fig:ndes1}), and it will be discussed in section~\ref{CHEN-section5.3}.

\begin{figure}[H]
\begin{center}
\begin{minipage}{2.8in}
\centerline{\hbox{
\psfig{figure=fig/buses0.eps,width=2.5in}
}}
% \centerline{(a)}
\centerline{(a) Simplified block diagram}
\end{minipage}
% 
\begin{minipage}{2.8in}
\centerline{\hbox{
\psfig{figure=fig/buses1.eps,width=2.5in}
}}
% \centerline{(b)}
\centerline{(b) InputFifo $\rightarrow$ data path $\rightarrow$ OutputFifo}
\end{minipage}
\vspace{0.2in} \\
%
% second row
%
\begin{minipage}{2.8in}
\centerline{\hbox{
\psfig{figure=fig/buses2.eps,width=2.5in}
}}
% \centerline{(c)}
\centerline{(c) InputFifo $\rightarrow$ datapath $\rightarrow$ {\it DataA}}
\end{minipage}
%
\begin{minipage}{2.8in}
\centerline{\hbox{
\psfig{figure=fig/buses3.eps,width=2.5in}
}}
% \centerline{(d)}
\centerline{(d) InputFifo $\rightarrow$ data path $\rightarrow$ MEM\_B}
\end{minipage}
\end{center}
\label{fig:dataflow}
\caption{The dataflow modes of the CCM. I}
\end{figure}

\begin{figure}[H]
\begin{center}
\begin{minipage}{2.8in}
\centerline{\hbox{
\psfig{figure=fig/buses4.eps,width=2.5in}
}}
% \centerline{(e)}
\centerline{(e) MEM\_A $\rightarrow$ data path $\rightarrow$ MEM\_B}
\end{minipage}
%
\begin{minipage}{2.8in}
\centerline{\hbox{
\psfig{figure=fig/buses5.eps,width=2.5in}
}}
% \centerline{(f)}
\centerline{(f) MEM\_B $\rightarrow$ data path $\rightarrow$ MEM\_A} 
\end{minipage}
\end{center}
\label{fig:dataflow-1}
\caption{The dataflow modes of the CCM. II}
\end{figure}

\subsection{Dataflow mode}
\label{CHEN-section5.1.4}

A simplified block diagram of the CCM is shown in Figure \ref{fig:dataflow}
(a). It can be seen that the CCM has two data buses that connect the
input/output FIFOs, memory and data path (ILU in the CCM) together. 

This two-bus structure has better performance than the single bus 
structure. Suppose we use single bus structure, which means that the 
input/output FIFOs, memory and ILU are connected together by one data
bus. For a sequence of cube operations that read data from memory and
write the results back to the memory, the algorithm would be: \\
\nvs\ccmasm{1. for (i=0; i++; i<n)} \\
\nvs\ccmasm{2. \{ \quad set MEM be the writer and ILU be the reader of the
data bus}\\
\nvs\ccmasm{3. \qquad read data from MEM to ILU} \\
\nvs\ccmasm{4. \qquad execute cube operation} \\
\nvs\ccmasm{5. \qquad set MEM be the reader and ILU be the writer of the
data bus}\\
\nvs\ccmasm{6. \qquad write data from ILU to MEM} \\
\ccmasm{7. \}}\\
It is easy to observe that the lines 2 through 6 are executed $n$ times. 
With our two data buses structure, the algorithm is now changed to:\\
\nvs\ccmasm{1. set MEM be the writer and ILU be the reader of the data bus A}\\
\nvs\ccmasm{2. set MEM be the reader and ILU be the writer of the data bus B}\\
\nvs\ccmasm{3. for (i=0; i++; i<n)} \\
\nvs\ccmasm{4. \{ \quad read data from MEM to ILU through data bus A} \\
\nvs\ccmasm{5. \qquad execute cube operation} \\
\nvs\ccmasm{6. \qquad write data from ILU to MEM through data bus B} \\
\ccmasm{7. \}}\\
This time, the lines 1 and 2 are outside of loop and are only executed
one time in our architecture. Therefore, we improve the performance 
by using two data buses.
% The full discussion of the advantage of multiple data buses can be found
% in the literatures like [?,?,?].
Some of useful dataflow modes are shown in Figure \ref{fig:dataflow} (b) to 
(f). The examples of using these modes are given in Chapter~\ref{CHEN-chapter6}.

\section{Instructions and Their Encoding}
\label{CHEN-section5.3}

The CCM has two categories of instructions called ``CCM instructions", 
{\it config} instructions and {\it execute} instructions. The {\it config}
instructions set the CCM to be ready to execute a specified cube operation.
The {\it execute} instructions let the CCM executes cube operation(s) 
currently set in the {\it instruction} register.

There are three {\it config} instructions: {\it Set Accumulator},
{\it Set Tri-state Buffers} and {\it Set Registers}. And there are 
two {\it execute} instructions: {\it Execute} and {\it Loop}. 
This section will discuss these instructions and their encoding 
in detail (the examples are given in section~\ref{CHEN-section6.2}).

\subsection{Set Accumulator}
\label{CHEN-section5.3.1}

The ``set accumulator" instruction loads the data into the accumulator
({\it Accu}) from its input. The encoding of the instruction is as follows:

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/ninst1.eps}}}
\end{figure}

\vspace{-10pt}
\noindent The first two bits ``01" is the opcode of this CCM instruction.
The {\it 30-bit data} in the instruction will be shown on the 
bus {\it IBus}. For loading the correct data into {\it Accu}, the 
control bits {\it EnIFifoD} and {\it Asrc} (see Figure \ref{fig:ndes1}) 
must be set properly (by issuing {\bf set tri-state buffers} and
{\it set register} instructions) before issuing this instruction.
For example, when {\it EnIFifoD} is 1 and {\it Asrc} is 0, this 
instruction will load the data shown on bits 29 to 0 of the instruction 
word into the {\bf Accumulator}.

\subsection{Set Tri-state Buffers}
\label{CHEN-section5.3.2}

The {\bf ``set tri-state buffers''} instruction is used to set the control bits of the
{\bf tri-state buffers} that control
the data flow. Some useful dataflow modes are discussed in section~\ref{CHEN-section5.2.4}. 
There are 8 bits of this kind in our design. 

These 8 bits are registered by an 8 bits register in the CCM, and this
register can be set by one CCM instruction. As we discussed in section~\ref{CHEN-section5.2.1},
the three data buses (ABus, DBusA and DBusB) have more than one possible 
driver, but at any given time, there is only one driver for each
data bus. If there were more than one driver for a given bus at a 
given time, then the FPGA chip would be destroyed permanently.

For protecting the hardware from destroying by a ``bad program", in our
design, the {\it set tri-state buffers} instruction can only set one
control bit a time, and a special circuit is used to check potential
contention (multiple drivers). The idea of this special circuit 
is shown in Figure \ref{fig:ntribufs}. 

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/ntribufs.eps,width=3.5in}}}
\caption{Avoiding contention which would result from multiple drivers}
\label{fig:ntribufs}
\end{figure}

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/tbuftim.eps,width=5in}}}
\caption{Timing diagram of special circuit for avoiding bus contention}
\label{fig:tbuftim}
\end{figure}

As shown in Figure \ref{fig:ntribufs}, two control bits ($cntrbit1$
and $cntrbit2$) control two tri-state buffers that drive one data bus
(the tri-state buffer and data bus are not shown in the figure).
At any given time, at most one control bit can be set to 1. 

For understanding how this circuit works, let's see a timing diagram
shown in Figure \ref{fig:tbuftim}. At time point 0ns, all signals
are 0.  For setting $cntrbit1$ to 1, the signal $databit$ is set to 1 
first at 25ns, then there is a raising edge on the signal $ld\_bit1$ 
at 50ns. As shown in the figure, after a little delay, the signal
$cntrbit1$ is set to 1. 

Now let us try to set $cntrbit2$ to 1 to create a bus contention. The 
signal $databit$ is set to 1 at 125ns, then there is a raising edge on 
the signal $ld\_bit2$ at 150ns. Since one of two control bits ($cntrbit1$)
is 1, then both inputs of gate 2 (NAND gate) are 1's, thus, the 
output of gate 2 is 0. Then the raising edge on the signal $ld\_bit2$
can not go through the gate 4 (AND gate), which means that the raising
edge can not reach the clock input of DFF2 (D flip-flop), thus Q output
of the DFF2 does not change. Therefore, this circuit ensures that at most
one control bit can be set to 1. 

For setting $cntrbit2$ to 1 at this time, two steps are needed.
First step is to set $cntrbit1$ to 0 at 250ns (please note that the signal
$databit$ is set to 0 before the raising edge on the signal
$ld\_bit1$), then second step is to set $cntrbit2$ to 1 at 350ns. 
By using this kind of circuit, nothing happens when the CCM encounters 
a ``bad instruction" that tries to create multiple drivers.
This circuit can be described by the following VHDL code:

\vspace{3pt}
{\footnotesize
 \renewcommand{\baselinestretch}{1}
 \begin{alltt}
   ...
   signal databit, ld_bit1, ld_bit2, cntrbit1, cntrbit2 : std_logic;
   signal dff1clk, dff2clk, gate2output: std_logic;
   ...
   begin
     ...
     DFF1: dff port map ( d=>databit, clk=>dff1clk, q=>cntrbit1);
     DFF2: dff port map ( d=>databit, clk=>dff2clk, q=>cntrbit2);
  
     gate2output <= not ((cntrbit1 or cntrbit2) and databit);
     dff1clk <= ld_bit1 and gate2output;
     dff2clk <= ld_bit2 and gate2output;
     ...
   end;
 \end{alltt}
 \renewcommand{\baselinestretch}{1.37}
}

\vspace*{- 10pt}
The encoding of the ``set tri-state buffers" instruction is as follows:

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/ninst2.eps}}} 
\end{figure}

\noindent The first three bits ``000" is the opcode of this CCM instruction.
The bit 25 is the $databit$ signal in Figure \ref{fig:ntribufs}.
The bits 28 to 26 ({\tt mmm} in the encoding format) is the ``address"
of these eight control bits of tri-state buffers. The address of these 
control bits is as follows: 000 is EnAddrA, 001 is EnAddrB, 010 is 
EnIFifoA, 011 is MemARW, 100 is EnIluA, 101 is EnIFifoD, 110 is MemBRW,
and 111 is EnIluB (all these eight control bits are shown in Figure \ref{fig:ndes1}).

\subsection{Set Registers}
\label{CHEN-section5.3.3}

The {\bf ``set registers''} instruction loads the data into registers (except 
{\it Accu} and {\it Data}) from their inputs. For loading the correct 
data into registers, the tri-state buffers must be set properly before 
issuing this instruction. The encoding of the instruction is as follows:

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/ninst3.eps}}} 
\end{figure}

\noindent The first three bits ``001" is the opcode of this CCM instruction.
The bits 28 to 26 ({\tt mmm} in the encoding format) is the
``address" of the target register. The addresses of these registers are as
follows: 000 is AddrA, 001 is AddrB, 010 is AddrR, 011 is WATER, 100 is
RightEdge, 101 is INST, 110 is CONF, and 111 is PRPO.

AddrA and AddrB are the {\bf Address Units} (see section~\ref{CHEN-section5.2.2}). Since they
can be set in the same way we set registers, the same instruction is used
to set {\it Address Units} and registers. AddrA, AddrB and AddrR are 
18-bit wide, so bits 17 to 0 are used when the target register is one
of them. WATER and RightEdge registers are 15-bit wide, so bits 14 to 0
are used when the target register is one of them.

When the target register is the {\bf instruction register}, 21-bit data is 
needed. The bits 25 to 23 represent the highest 3 bits of the 
{\it instruction} register. The bits 17 to 0 represents the lowest 18 
bits of the {\it instruction} register (see the format of 
{\it instruction} register discussed in section~\ref{CHEN-section5.2.3}). 
When the target register is the PRPO register, the bits 23 to 0 are used
(see the format of {\it prpo} register discussed in section~\ref{CHEN-section5.2.3}).

% vspace{0.1in}
% \centerline{\hbox{\psfig{figure=fig/confreg.eps,width=2.25in}}} 

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/confreg.eps,width=2.5in}}}
\caption{the format of {\it config} register}
\label{fig:confreg}
\end{figure}

When the target register is {\it config} register (CONF), the bits 8 to
0 of the instruction are used. The {\it config} register is the collection
of eight configuration bits of the CCM. The content of {\it config}
register is shown in Figure \ref{fig:confreg}. The meaning of these 
eight bits is as follows:
\begin{itemize}
\item {\it enFinish} determine whether the instructions {\it Execute} and
      {\it Loop} will generate ``finish word" or not. The ``finish word"
      will be discussed in section~\ref{CHEN-section5.3.4}.
\item {\it enMemA} and {\it enMemB} determine whether the memory banks
      are used in cube operation or not. Do not confuse with tri-state
      buffer control signals {\it memARW} and {\it memBRW};
      the signals {\it memARW} and {\it memBRW} determine the operation
      mode ({\it read} or {\it write}) of memory banks (see section~\ref{CHEN-section5.2.2}).
      If {\it enMemA} is set to 1, then the MEM\_A is used in the 
      following cube operation, otherwise MEM\_A is not used. The bit
      {\it enMemB} controls MEM\_B in the same manner. Only {\it Loop}
      instruction will use memory banks, and it will be discussed later
      in this section.
\item {\it CmpSrc}, {\it ASrc} and {\it Osrc} are three ``select" signals
      of the multiplexers (see Figure \ref{fig:ndes1}).
\item {\it toOFifo},{\it toAccu} and {\it toMem} are three output control 
      signals. These three signals tell the GCU just whether or not to 
      generate the corresponding control signals to load data into 
      the output FIFO, and/or {\it Accu}, and/or the memory from their 
      inputs after a resultant cube is generated. The GCU doesn't care about
      where these inputs come from. For using proper dataflow mode, 
      the tri-state buffers must be set properly before executing the
      operation. It is possible to write the resultant cube to all these 
      three targets at the same time. 
\end{itemize}

A cube operation is completely described by functions $relation$, $before$,
$active$, $after$ and {\it pre-relation/pre-operation}. As it can be seen from
this instruction, all these functions can be programmed by users through 
setting registers {\it inst} and {\it prpo} instead of {\bf ``hard-code" values}.
This is similar to microprogramming and makes it easy to execute a {\bf ``new''}
cube operation that is not discussed in this book and can be classified
into one of three classes of cube operations without re-designing of the
entire CCM. For example, the cofactor operation is a ``new" operation to the
CCM. A student was asked by me to implement the cofactor operation on 
the CCM, which did not exist there yet. Therefore, 
he described the cofactor operation using Equation \ref{eq:ccm6}, and 
we derived $before$, $active$ and $relation$ functions. The final equation 
used to describe cofactor operation is shown in Equation \ref{eq:cofactor}.
Now we can perform cofactor operation without changing the design of the CCM.
This is a powerful feature of the presented CCM design and it is hoped that it will find
many applications in CCM assembly programs.
You will be also asked to design ``new'' CCM operations.

\subsection{Execute}
\label{CHEN-section5.3.4}

The {\bf ``execute" instruction} is used to execute only one cube operation. It is
the realization of the executing patterns (a) and (b) (see section~\ref{CHEN-section5.1}).
When the CCM receives this instruction, the CCM loads data into
{\bf Data} register from its input, then executes cube operation on two
operand cubes currently stored in {\bf Accu} and {\bf Data} registers.
The resultant cubes being written to {\it Accu}, and/or memory,
and/or the output FIFO depending on three output control bits
(The address of memory will be automatically increased by one after 
every memory write operation). The encoding of the instruction is as 
follows:

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/ninst4.eps}}}
\end{figure}

\vspace{-10pt}
\noindent The first two bits ``10" is the opcode of this CCM instruction.
The {\bf 30-bit data} in the instruction will be shown on the bus {\bf IBus}
after the instruction is read from the input FIFO. For loading the correct
data into {\bf Data} register, the control bits {\it EnIFifoD} and 
{\it Osrc} must be set properly before issuing this instruction. 

The last step of executing {\bf execute} instruction is that a
{\bf ``finish word''} is pushed into the output FIFO if the bit {\it enFinish} is 1. 
The finish word is a special 32-bit word whose highest two bits 
are set to 10. On the other hand, the highest two bits of the general 
data word (represents cubes) are always 00. The finish word is used to 
separate two arrays of (resultant) cubes of two set of cube operations.

The finish word is necessary for our CCM co-processor. For example, we 
calculate a sharp operations on two arrays of cubes. The sharp operation
is carried out by a set of CCM instructions, and produces an array of 
cubes. Without the finish word, the host computer would never know
where is the end of the resultant cubes even if the host computer fetches
all words in the output FIFO. The question is how the host computer
could determine whether a operation (or a set of operations) is completed
or not. Therefore, we introduced here the concept of finish word to solve
this problem. We can let the CCM generate a finish word after the 
operation is completed. With the finish word, the host computer can tell 
whether a operation is already completed or not. Anther example of use of the
finish word is when we calculate two sharp operations, each of them
produces an array of cubes as the result. Without the finish word, the 
two resultant cubes would be concatenated together, so the host computer
would be not not able to separate these two arrays of cubes. With the finish
word, the host computer has a way to separate these two arrays of cubes.

% There are two kinds of output words pushed into the output FIFO: finish
% word and result word. The result words represent resultant cubes.
% The output words are 32 bits wide, and the resultant cubes are 30 bit wide
% (because there is 15 ITs in the CCM). In our design, the resultant cube
% occupies bits 29 to 0 of the result word, and the bit 31 is always ``0". 
% For the finish word, the bit 31 is always ``1", and the bits 29 to 0 are
% meaningless. The bit 30 is not used, and it is always ``0". 

\subsection{Loop}
\label{CHEN-section5.3.5}

The {\bf ``loop'' instruction} is used to execute multiple cube operations
continuously without fetching the input FIFO. It is the realization
of the executing patterns (c) and (d) (see section~\ref{CHEN-section5.1}). When the CCM
receives {\bf LOOP} instruction, the CCM loads the data from memory into 
{\it Data} register (The control signals {\it MemARW}, {\it MemBRW}, 
{\it EnIluA}, {\it EnIluB}, {\it OSrc} and {\it ASrc} must be set properly
before issuing {\it LOOP} instruction). 

Then the CCM executes cube operation on two operand cubes currently 
stored in {\it Accu} and {\it Data} registers. The resultant cubes are 
written to {\it Accu}, and/or memory, and/or the output FIFO determined
by three output control signals. After that, the CCM loads the next
data from memory into {\it Data} register and executes the same cube 
operation again (The address of memory will be automatically increased
by one after every memory read/write operation). 

This procedure is repeated until the signal {\it AddrEQ} (see Figure
\ref{fig:ndes1}) becomes 1, which means that the memory address (the signal 
{\it Cmpsrc} determines which memory address is used, see Figure 
\ref{fig:ndes1}) is equal to the content of the {\it AddrR} register. 
The encoding of the instruction is as follows:

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/ninst5.eps}}}
\end{figure}

\vspace{-10pt}
\noindent The first two bits ``11" is the opcode of this CCM instruction.
The {\it 30-bit data} in the instruction will be shown on the 
bus {\it IBus}, but typically it is not used.

Similarly as in the {\it execute} instruction, the last step of executing 
the {\it loop} instruction is also that a finish word is pushed into the 
output FIFO if the bit {\it enFinish} is 1.

\section{Global Control Unit}
\label{CHEN-section5.4}

The Global Control Unit (GCU) handles the communication between the host
computer and the CCM, and it is also the controller of the whole CCM.
As mentioned in section~\ref{CHEN-section3.5}, another controller OCU is used to control
the datapath of the CCM. Certainly, we can design a single controller
to control all of them. The reason why we design two controllers in our
CCM is that it is easier to design and test two simple controllers than
one complex controller.

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/state_biu.eps,width=5.8in}}}
\caption{The state diagram of the GCU}
\label{fig:state_biu}
\end{figure}

The algorithm of the CCM is very simple: under the control of GCU, the 
CCM fetches an instruction from the input FIFO; then the CCM executes the
instruction: set the contents of registers, or tri-state buffers, or
executes cube operation(s). After that, the CCM is ready to process
next instruction from the input FIFO. The GCU will remove an empty cube
by using signal {\it empty} (see section~\ref{CHEN-section3.4.2}). The state diagram of 
the GCU is shown in Figure \ref{fig:state_biu}. The signals $opc$ in
the state diagram is the opcode the CCM instruction, and it occupies 
the highest 3 bits of the CCM instruction. The other signals will be 
discussed when we will discuss the related states. Now let us take a look at 
these states.

\begin{itemize}
\item State S0: 
This is the initial state of the GCU. In this state, the
GCU checks if there are CCM instruction(s) in the input FIFO by 
asserting signal {\it FifoInEmpty} (see section~\ref{CHEN-section4.2.6}). If there
are CCM instruction(s) in the input FIFO, then the GCU goes to state
S1, otherwise, the GCU stays in state S0.

\item State S1: 
In this state, the GCU generates signal {\it FifoInRead} 
(see section~\ref{CHEN-section4.2.6}) to fetch a CCM instruction from the input FIFO. This state
has three exits, states S2, S3 and S4. If the instruction is one of 
instructions ``set accumulator'', ``set tri-state buffers'' or ``set
registers'' (the highest bit of opcodes of these three
instructions all are 0's), then the GCU goes to state S2; if the 
instruction is the instruction ``exec'' (the corresponding opcode is 10x),
then the GCU goes to state S4; if the instruction is the ``loop'' instruction
(the corresponding opcode is 11x), the GCU goes to state S3.

\item State S2:
In this state, the GCU generates the load signals to load data into
the corresponding register, or the accumulator, or the 1-bit register that 
store the control bit of the corresponding tri-state buffer. For example,
if the instruction
is ``set instruction register'', then the GCU generates signal {\it ld\_reg},
and the signal {\it ld\_inst} that is used to load data into the 
instruction register is generated as follows: \\
\centerline{$ ld\_inst = ld\_reg \cdot b_{28} 
    \cdot \overline{b_{27}} \cdot b_{26}$ } \\
(please note that the instruction register is encoded as 101,
 see section~\ref{CHEN-section5.3.3}), where $b_{28}$, $b_{27}$ and $b_{26}$ are the 28th to 26th bit of
the CCM instruction. Load signals for other registers and 8 1-bits 
registers for 8 tri-state buffers' control bits are generated similarly.
When the instruction is ``set accumulator", the GCU generates signal
{\it ld\_accu}.

After that, the GCU always goes back to state S0 and becomes ready to 
process the next CCM instruction.

\item State S3:
In this state, the GCU checks if the loop operation is completed by asserting
signal {\it AddrEQ} (see Figure \ref{fig:ndes1} and section~\ref{CHEN-section5.3.5}).
If the loop operation is completed, then the GCU goes back to state S0 and 
becomes ready to process the next CCM instruction; otherwise, the GCU goes to
state S4.

\item State S4:
In this state, the GCU generates signal {\it ld\_data} to loads the data
into the {\it data} register from its input. If the CCM is executing 
``exec'' instruction, the input of the {\it data} register should come from
the {\it IBus}, otherwise (the CCM is executing ``loop'' instruction), the 
input of the {\it data} register should come from one of the two memory
banks. This state has two exits, states P2 and P1. If the first 
pre-relation/pre-operation is used in the operation (represented by 
{\it p1} field of the instruction register, see section~\ref{CHEN-section5.2.3}), then the 
GCU goes to state P2, otherwise, the GCU goes to state P1.

\item State P2:
In this state, the GCU sets the signal {\it prel\_sel} to 00. After that,
the pre-relation/pre-operation circuitry begins to evaluate the first
pre-relation (see section~\ref{CHEN-section3.7}). The GCU 
always goes to state P3 from state P2.

\item State P3:
In this state, the GCU still keeps the signal {\it prel\_sel} to be 00,
and checks if the first pre-relation is satisfied by asserting signal
{\it prel\_res} (see section~\ref{CHEN-section3.7}). If the first pre-relation is 
satisfied (the signal {\it prel\_res} is 1), then the GCU goes to state
P4; otherwise, the GCU will check if the second pre-relation/pre-operation
is used in the operation (represented by {\it p2} field of the 
instruction register, see section~\ref{CHEN-section5.2.3}). If the second 
pre-relation/pre-operation is used, then the
GCU goes to state P5; otherwise, the GCU goes to state P1. 

\item State P4:
Achieving this state means that the first pre-relation is satisfied. The GCU 
calculates the cube operation by using the first pre-operation function
(the signal {\it pre\_sel} keeps 00), and the GCU will generate signal
{\it write\_output}. By using signal {\it write\_output}, the proper 
write signals can be generated to write the resultant cube to {\it Accu},
and/or memory, and/or the output FIFO depending on three output control 
bits if the {\it empty} signal is 0 in this state. For example, the
signal {\it write\_fifo} that writes the result to the output FIFO is
generated as follows:\\
\centerline{$ write\_fifo = write\_output \cdot toOFifo \cdot 
              \overline{empty} $}\\
where the signal {\it toOfifo} is discussed in section~\ref{CHEN-section5.3.3}, and
the signal {\it empty} is discussed in section~\ref{CHEN-section3.4}.
The GCU always goes to state S7 from state P4.

\item State P5:
This state is similar to state P2, and the difference is that the GCU will
check the second pre-relation rather than check the first pre-relation.
The signal {\it prel\_sel} is set to 01 in this state. The GCU always goes
to state P6 from this state.

\item State P6:
In this state, the GCU still keeps the signal {\it prel\_sel} to be 01,
and checks if the second pre-relation is satisfied by asserting signal
{\it prel\_res} (see section~\ref{CHEN-section3.5}). If the second pre-relation is 
satisfied, then the GCU goes to state P7; otherwise, the GCU will 
go to state P1.

\item State P7:
Achieving this state means that the second pre-relation has been satisfied.
The GCU calculates the cube operation by using the second pre-operation
function (the signal {\it pre\_sel} keeps 01), and the GCU will generate
proper write signals (in the same way with state P4) to write the resultant
cube to {\it Accu}, and/or memory, and/or the output FIFO depending on 
three output control bits if the {\it empty} signal is 0 in this state. 
The GCU always goes to state S7 from state P7.

\item State P1:
This state means that the cube operation that is executed on the CCM does
not have the pre-relation/pre-operation or these pre-relations are not 
satisfied. In the state P1, state S5, and state S6, the signal {\it prel\_sel}
is set to 10, which means the cube operation will be carried out by using
relation/operation specified by the {\it instruction} register.
State P1 has two exits, states S5 and S6. If the cube operation is a
sequential operation (The {\it sc} field of the {\it instruction} register
represents whether a operation is sequential or combinational, 
see section~\ref{CHEN-section5.2.3}), then the GCU goes to state S5; otherwise (combinational operation),
the GCU goes to state S6.

\item State S5:
This state means that the operation is a sequential cube operation.
In this state, the GCU will generate signal {\it ilu\_enable} which
enables the ILU to execute sequential operation under the control of
OCU (the control unit of the ILU). After the operation is done, the OCU
will generate signal {\it ilu\_done} to tell the GCU that the cube
operation is done. The GCU will keep checking the signal {\it ilu\_done}
to see if the cube operation is done. If not, the GCU will remain in state
S5; otherwise (the operation is done), the GCU will check if this is a
loop operation by asserting the opcode of the current CCM instruction on
the {\it IBus}. If this is a loop operation ($opc=11x$), then the GCU will
go to state S3, otherwise, the GCU goes back to state S0 and becomes 
ready to process the next CCM instruction.

\item State S6:
This state means that the operation is a combinational cube operation.
In this state, the signal {\it ilu\_enable} keeps 0, which means that all
ITs in the ILU will remain in {\it before} states, or goes to {\it active}
states if the current cube operation is a complex combinational 
cube operation (the {\it pm} field of the {\it instruction} register 
represents whether the operation is complex or not, see section~\ref{CHEN-section5.2.3}) 
and the given IT is a special variable\footnote{Do not confuse states 
{\it before} and {\it active} with the states of the GCU, these two 
states are the states of the state machine inside IT.} (or part of 
special variable). The ILU executes this (complex) 
combinational cube operation by using {\it before} and {\it active} 
functions (see section~\ref{CHEN-section2.3.1} and ~\ref{CHEN-section2.3.2}).

The GCU will generate proper write signals (in the same way with state P4)
to write the resultant cube to {\it Accu}, and/or memory, and/or the output
FIFO depending on three output control bits if the {\it empty} signal 
is 0 in this state. The GCU always goes to state S7 from state S6.

\item State S7:
In this state, the GCU will check if this is a loop operation by asserting
the opcode of the current CCM instruction on the {\it IBus}. If this is 
a loop operation ($opc=11x$), then the GCU will go to state S3, otherwise,
the GCU goes to state S0 and becomes ready to process the next CCM 
instruction.

In this state, the GCU also adjusts the address of the memory if the CCM 
read the data from and/or write the data to the memory bank(s). 
% For the sequential cube operations, this task is carried out by the OCU.

\end{itemize}


\section{Mapping CCM onto PeRLe-1 board}
\label{CHEN-section5.5}

This design of the CCM is mapped onto DEC-PERLE-1 board, and the mapping is 
described by VHDL codes. Since the mapping detail is too tedious to list 
here, we only give the outline of idea in this section. 

\begin{figure}[H]
\centerline{\hbox{\psfig{figure=fig/nmap1.eps,width=5.85in}}}
\caption{The outline of mapping}
\label{fig:nmap1}
\end{figure}

The outline of mapping is shown in Figure \ref{fig:nmap1}.
As shown in the figure, for using Matrix as regular as possible, only ITs
are mapped in it; The GCU is mapped in {\it South-West controller} 
({\it CSW}) because it is easy to control input and output FIFOs from 
the controller, OCU is mapped in {\it South Switch} ({\it SWS}) because it 
is easy to control all matrix chips from the switches. All other parts
are shown in the figure. The components of DEC-PERLE-1 that are represented
by dotted lines are not used in this design. The following section will
discuss how we derived this mapping.

One important mapping principle is that the mapping should be as simple
as possible, which means the designer should try to use direct connections
as more as possible. 

In our design, the input FIFO connects the bus {\it IBus}, and the output
FIFO connects to the bus {\it OBus}. Both these two buses are 32-bit wide.
Since the DEC-PERLE-1 has two 32-bit-wide I/O buses: {\it DBusSW} and {\it DBusNE},
the bus {\it IBus} is mapped to {\it DBusSW}, and the bus {\it OBus} is
mapped to {\it DBusNE}. It is very easy to connect the input FIFO to
{\it DBusSW} and the output FIFO to {\it DBusNE} through {\it Fifo
Switch} (FSW).

Since the GCU is connected to the bus {\it IBus}, and it should be able
to control the input and output FIFOs directly, it should be mapped to
a FPGA chip that connects to {\it IBus} and the control signals of the 
input and output FIFOs directly.
\footnote{ 
We can map GCU to a FPGA chip that does not connect to {\it IBus} and the
control signals of the input and output FIFOs directly, but it will take 
more time to transfer signal between them, see section~\ref{CHEN-section4.2.7}.  }
In the DEC-PERLE-1, the only FPGA chip that meets this requirement is 
{\it South-West Controller} (CSW). Therefore, the GCU is mapped into CSW.

Our design needs two memory banks. It can be seen from Chapter~\ref{CHEN-Chapter4} that
the address signals of the memory banks are connected to two controller
FPGAs CSW and CNE. In our design, the two {\it Address Units} and 
{\it Address Register} should be able to copy their 18-bit-wide contents
to each other, which means they should reside in one FPGA chip; 
and they are also connected to the bus {\it IBus}. So we mapped these two 
{\it Address Units} in CSW. This means also that we use the west and south
memory banks to map two memory banks: MEM\_A and MEM\_B.

The ILU, the data path of our CCM, is mapped into the matrix of DEC-PERLE-1
since the matrix is the only place that is large enough to hold the ILU.
In the previous experiment \cite{foote94}, David W. Foote mapped four ITs
in one Xilinx 3090 FPGA chip, which took about 50\% CLB resources of the FPGA.
In our design, more functions are added to one IT, like COUNT and EMPTY 
blocks; and we still want to reserve some resource for the future 
modifications. Thus, four ITs are mapped into one matrix FPGA chip in our
mapping, which means we need 4 matrix chips to map 15 ITs. We can use
any 4 FPGAs inside matrix to map the ILU, but we want to keep the structure
as regular as possible. So we decided to use one column or one row of the
matrix to map the ILU. The bus {\it DBusA} is connected to the MEM\_A
and the bus {\it IBus}. Suppose MEM\_A is mapped to west memory 
bank\footnote{There exists another choice of mapping MEM\_A to the south 
memory bank. These two mapping choices are symmetrical.},
so the bus {\it DBusA} is routed into the matrix from the west side.
There are only 16 connections from {\it West Switch} to any row of the matrix.
The width of {\it DBusA} is 30-bit, so we use the first column 
FPGAs of the matrix to map ILU that has 15 ITs.

The registers ({\bf Accu, Data, Water, Right, Inst} and {\bf PRPO} are
also mapped in the first column of the matrix because that there is not
enough direct connections between the matrix and the west/south switches
if these registers are mapped into the {\bf west/south switches}.

The last unmapped component is OCU. The OCU is connected to every ITs in
the ILU and the GCU. The only FPGA chip that has direct connection to the
first column FPGAs of matrix and the GCU (mapped in CSW) is {\bf South
Switch} (SWS). Therefore, the OCU is mapped into SWS.

