1. Matrices

A matrix is a rectangular array of numbers, symbols, or expressions, arranged in rows and columns and surrounded by brackets.

For example

$\begin{pmatrix}2 & 6 & 3 & -5\\15 & 8 & 1 & 14\\2 & 2 & 2 & -8\end{pmatrix},$

$\begin{pmatrix}\cos\alpha & -\sin\alpha\\ \sin\alpha & \cos\alpha\end{pmatrix},$

$\begin{pmatrix}e & 3e\\0 & e\\ \frac{1}{2}e^5-\frac{1}{2}e & e^5 \end{pmatrix},$

$\begin{pmatrix}x^2\\y^2\\z^2\\w^2\end{pmatrix}.$

The size or dimension of a matrix is defined as $m \times n$ where $m$ is the number of rows and $n$ is the number of columns.

What is the size of the following matrices:

$\begin{pmatrix}1 & 3 & 7\end{pmatrix}$ $\begin{pmatrix}a & b \\c & d \\e & f\end{pmatrix}$ $\begin{pmatrix}\overline{1} & \overline{7} & \overline{8} \\ \overline{2} & \overline{9} & \overline{0}\end{pmatrix}$ $\begin{pmatrix}7+i \\8-3i \\10+\sqrt{5}i\end{pmatrix}$ $\begin{pmatrix}7\end{pmatrix}$ $\begin{pmatrix}x^2 & 1 & 3 \\6 & x^2 & 5 \\9 & 7 & x^2\end{pmatrix}$
Answer Answer Answer Answer Answer Answer
Matrices with dimensions $1\times n$ and $n\times 1$ are also called vectors. A $1\times n$ matrix is called a column vector and an $n\times 1$ matrix is called a row vector.

Which of the following matrices are vectors? If a matrix below is not a vector, then name all vectors the matrix consists of.

$\begin{pmatrix}1\\2\\0\\1\end{pmatrix}$ $\begin{pmatrix}x & y \\x^2 & y^2 \\v^3 & y^3\end{pmatrix}$ $\begin{pmatrix}-\cos\alpha&\sin\alpha&\tan\alpha\end{pmatrix}$ $\begin{pmatrix}f(2) & g(1) & 3 \\g(6) & f(5) & 2\end{pmatrix},$
$f(x)=x-4$ and $g(x)=\pi x$
$\begin{pmatrix}7+i \\8-3i \\10+\sqrt{5}i\end{pmatrix}$
Answer Answer Answer Answer Answer

A general notation of an $m \times n$ matrix $A$ is

$A=\begin{pmatrix}a_{11} & a_{12} & \dots & a_{1n}\\a_{21} & a_{22} & \dots & a_{2n}\\a_{31} & a_{32} & \dots & a_{3n}\\ \vdots & \vdots & & \vdots\\a_{m1} & a_{m2} & \dots & a_{mn}\end{pmatrix}\hspace{2cm} \text{or}\hspace{2cm} A=(a_{ij}),$

where $i=1,2,\dots , m$ is the row index and $j=1,2,\dots , n$ is the column index. Usually, a capital letter is used for denoting a matrix. Row and column vectors are often denoted by a (noncapital) bold letter, e.g.,

$\mathbf{x}=\begin{pmatrix}x_1 & x_2 & \dots & x_n\end{pmatrix}\hspace{2cm} \text{or}\hspace{2cm} \mathbf{b}=\begin{pmatrix}b_1\\b_2\\\vdots\\b_m\end{pmatrix}.$

What is the $ij^{th}$ element of the matrix \begin{pmatrix}0 & 1 & 2 & 3 & 4 \\5 & 6 & 7 & 8 & 9\\10 & 11 & 12 & 13 & 14\end{pmatrix} when

  • $i=2$ and $j=5$; Answer
  • $i=1$ and $j=2$; Answer
  • $i=5$ and $j=3$. Answer

:?: Let $a_{14}=8$, $a_{27}=0$, $a_{13}=2$, $a_{21}=-4$, $a_{41}=-5$, $a_{12}=1$, $a_{31}=5$, $a_{23}=12$, $a_{25}=-1$, $a_{03}=10$, $a_{26}=7$, $a_{34}=4.$ Complete the matrix by entering the missing values (drag and trop the blue number below)

Matrices can be added, multiplied by a scalar, and multiplied by other matrices. Formulas for the operations are as follows

Operation Formula Example
Sum $(a_{ij})+(b_{ij})=(a_{ij}+b_{ij})$ $\begin{pmatrix}\color{blue}{1} & \color{blue}2\\\color{blue}0 &\color{blue}{-2}\end{pmatrix}+\begin{pmatrix}\color{red}{0} &\color{red}{-1}\\\color{red}1 & \color{red}2\end{pmatrix}=\begin{pmatrix}\color{blue}1+\color{red}0 & \color{blue}2+\color{red}{(-1)}\\\color{blue}0+\color{red}1 & \color{blue}{-2}+\color{red}2\end{pmatrix}=\begin{pmatrix}1 & 1\\1 & 0 \end{pmatrix}$
Scalar multiplication $c\cdot (a_{ij})=(c\cdot a_{ij})$ $\color{blue}{-1}\cdot\begin{pmatrix}1 & 8 \\5 &-2 \end{pmatrix}=$ $\begin{pmatrix}\color{blue}{-1}\cdot 1 & \color{blue}{-1}\cdot 8 \\\color{blue}{-1}\cdot 5 &\color{blue}{-1}\cdot (-2)\end{pmatrix}=\begin{pmatrix}-1 & -8 \\-5 & 2 \end{pmatrix}$
Matrix multiplication Let $AB=(c_{ij})$, then $c_{ij}=\sum^m_{r=1}a_{ir}b_{rj}$,
where $A=(a_{ij})$ is $n\times m$ matrix and $B=(b_{ij})$ is $m\times p$ matrix.
$\begin{pmatrix}\color{blue}2 & \color{blue}3 & \color{blue}4\\ \color{red}1 & \color{red}0 & \color{red}0\end{pmatrix}\begin{pmatrix}\color{LimeGreen}6& \color{Cyan}{10}\\\color{LimeGreen}7 & \color{Cyan}5\\\color{LimeGreen}8 & \color{Cyan}1\end{pmatrix}=$ $\begin{pmatrix}\color{blue}2\cdot \color{LimeGreen}6+\color{blue}3\cdot \color{LimeGreen}7+\color{blue}4\cdot \color{LimeGreen}8 & \color{blue}2\cdot \color{Cyan}{10}+\color{blue}3\cdot \color{Cyan}5+\color{blue}4\cdot \color{Cyan}1\\\color{red}1\cdot \color{LimeGreen}6+\color{red}0\cdot \color{LimeGreen}7+\color{red}0\cdot \color{LimeGreen}8 & \color{red}1\cdot \color{Cyan}{10}+\color{red}0\cdot \color{Cyan}5+\color{red}0\cdot \color{Cyan}1\end{pmatrix}=$ $\begin{pmatrix}65& 39\\6 & 10\end{pmatrix}$

Note that the dimension of matrices is important in the operations of adding and multiplying matrices:

  • two matrices can be added only when they have same dimensions;
  • two matrices can be multiplied if and only if the number of row in first matrix equals to the number of columns in the second matrix, i.e.

$\underset{n\times \color{blue}m}{A}\cdot \underset{\color{blue}m\times p}B=\underset{n\times p}{AB}.$

:?: Click on corresponding elements of the matrices for calculating the elements of the sum (over $\mathbb Z_6$).

Which of the following operations can be performed and what is the result:

$\begin{pmatrix}1 & -1 & 2\\-2 & 1 & 0\end{pmatrix}\!+\!\begin{pmatrix}-2 & 1\\2 & -1\end{pmatrix}$ $\begin{pmatrix}2 \\-1 \\4\end{pmatrix}\begin{pmatrix}2 & -2 & 3\end{pmatrix}$ $3\!\cdot\!\begin{pmatrix}1&0&4\\3&1&-2\\2&1&-1\end{pmatrix}$
Answer Answer Answer
$(-2)\!\cdot\!\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}$ $\begin{pmatrix}1 & 2\\-1 & 3\\5 & -1\end{pmatrix}\begin{pmatrix}-1 & 2\end{pmatrix}$ $\begin{pmatrix}8&-6\\1&-1\\3&2\end{pmatrix}\begin{pmatrix}0&3\\10&6\end{pmatrix}$
Answer Answer Answer

The diagram to the right describes also matrix multiplication.

Here are multiplied matrices $A$ with dimension $4\times 2$ and $B$ with dimension $2\times 3$. The matrix $AB$ is with dimension $4\times 3$.

Let $AB=(c_{ij})$. Then

$c_{12}=a_{11}b_{12}+a_{12}b_{22}$

and

$c_{33}=a_{31}b_{13}+a_{32}b_{23}$.

What are the values of indexes $i$ and $j$ if $c_{ij}=a_{41}b_{11}+a_{42}b_{21}$? Answer

Use IPython for calculating

  • $\begin{pmatrix}0 & 3 & 1\\3 & -1 & 7\\-5 & 14 & -8\\7&2&9\\0&-6&10\end{pmatrix}\begin{pmatrix}6 \\ 1\\-1\end{pmatrix}$
  • $\begin{pmatrix}2 & 10 & 7\\1 & 4 & 3\end{pmatrix}\begin{pmatrix}1 & 6 & 4\\2 & 3 & 1\\2 & 1 &10\end{pmatrix}$ over $\mathbb{Z_3}$.


Solution

Solution

from numpy import array
 
M1 = array([[0,3,1],[3,-1,7],[-5,14,-8],[7,2,9],[0,-6,10]]) # the first matrix
M2 = array([[6],[1],[-1]])                                  # the second matrix
R = numpy.zeros(shape=(len(M1),len(M2.T)))                  # the result matrix filled with zeros
 
for i in range(len(M1)):                                    # loop over rows of M1; row i in matrix M1 is 
    R[i]=sum(M1[i,:]*M2[:,0])                               # multiplied elementwise with column 0 from M2,
                                                            # elements in the product are added
print R

from numpy import array
 
M1 = array([[2,10,7],[1,4,3]])               # the first matrix
M2 = array([[1,6,4],[2,3,1],[2,1,10]])       # the second matrix
R = numpy.zeros(shape=(len(M1),len(M2.T)))   # the product filled with zeros
 
for i in range(len(M1)):
    for  j in range(len(M2.T)):
        R[i,j]=sum(M1[i,:]*M2[:,j])          # M1 row i multiplied elementwise with M2 column j and summed
 
print R%3                                    #for every element in R the residue modulo 3 is calculated


Another important operation on matrices is that of taking the transpose. It is denoted by placing a $T$ as an exponent on the initial matrix.

Operation Formula Example
Transpose $(a_{ij})^T=(a_{ji})$ $\begin{pmatrix}\color{blue}1 & \color{blue}2 & \color{blue}{-6}\\ \color{red}3 & \color{red}5 & \color{red}4\end{pmatrix}^T=\begin{pmatrix}\color{blue}1 & \color{red}3\\ \color{blue}2 & \color{red}5\\ \color{blue}{-6} & \color{red}4\end{pmatrix}$

Thus taking the transpose means reorganize matrix entries so that row of the initial matrix will be columns in the transpose. First row will be first column, second row will be second column and so on.

Which of the matrices

$(a)\quad\begin{pmatrix}2 & 6 & 3\\15 & 8 & 1\\2 & 2 & 2\\ 0 & 1 & 0\\-5 & 14 & -8\end{pmatrix}$

$(b)\quad\begin{pmatrix}3&6&2\\1&8&15\\2&2&2\\0&1&0\\-8&14&-5\end{pmatrix}$

$(c)\quad\begin{pmatrix}-8&14&-5\\0&1&1\\2&2&2\\1&8&15\\3&6&2\end{pmatrix}$

is the transpose of the matrix $\begin{pmatrix}3 & 1 & 2 & 0 & -8\\6 & 8 & 2 & 1 & 14\\2 & 15 & 2 & 0 & -5\end{pmatrix}$? Answer

Let

$A=\begin{pmatrix}1 & 5\\4 & 3\\5 & 4\end{pmatrix}$,

$\mathbf{b}=\begin{pmatrix}1\\2\\3\end{pmatrix}$,

$\mathbf{c}=\begin{pmatrix}1\\2\\3\end{pmatrix}^T$.

Calculate $5\cdot AA^T+\mathbf{b}\mathbf{c}$ by using IPython .

Solution

Solution

from numpy import array
 
M1 = array([[1,5],[4,3],[5,4]])          # the first matrix
M2 = array([[1],[2],[3]])                # the second matrix
M3 = array([[1],[2],[3]])                # the third matrix
R = numpy.zeros(shape=(len(M1),len(M1))) # the product matrix filled with zeros
 
for i in range(len(M1)):
    for  j in range(len(M1)):
        R[i,j]=sum(M1[i,:]*M1.T[:,j])    # M1 row i multiplied elementwise with M1 transp. column j, summed
 
print 5*R+M2*M3.T


Sum of matrices, scalar multiplication, matrix multiplication and the transpose are basic matrix operations. These operations have several useful properties. For example, knowing that $$c\cdot A+c\cdot B=c\cdot (A+B),$$ where $A$ and $B$ are matrices and $c$ is a constant, it is possible to reduce number of multiplications. Thus, instead of

$$-2\cdot\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!(-2)\cdot\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}=\begin{pmatrix}-2\color{red}\cdot10 & -2\color{red}\cdot(-3)\\-2\color{red}\cdot(-8) & -2\color{red}\cdot5\\-2\color{red}\cdot20 & -2\color{red}\cdot3\end{pmatrix}\!+\!\begin{pmatrix}-2\color{red}\cdot(-8) & -2\color{red}\cdot4\\-2\color{red}\cdot6 & -2\color{red}\cdot3\\-2\color{red}\cdot(-15) & -2\color{red}\cdot17\end{pmatrix}\!=\!\begin{pmatrix}-4 & -2\\4 & -16\\-10 & -40\end{pmatrix},$$ which is $12$ multiplications, we can manage only with $6$:

$$-2\cdot\!\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!(-2)\cdot\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}=-2\cdot\!\Bigg[\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}\Bigg]=-2\cdot\!\begin{pmatrix}2 & 1\\-2 & 8\\5 & 20\end{pmatrix}\!=\!\begin{pmatrix}-4 & -2\\4 & -16\\-10 & -40\end{pmatrix}.$$

Let $A$, $B$ and $C$ be matrices such that all the operations below are defined, and let $k$ and $l$ denote some constants (for example real numbers) . Here are some additional useful properties concerning the matrix operations:

Property Example
$[A^T]^T=A$$\Bigg[\begin{pmatrix}-1&2\\4&-7\\3&5\end{pmatrix}^T\Bigg]^T=\Bigg[\begin{pmatrix}-1&4&3\\2&-7&5\end{pmatrix}\Bigg]^T=\begin{pmatrix}-1&2\\4&-7\\3&5\end{pmatrix}$
$A+B=B+A$$\begin{pmatrix}1\\4\\3\end{pmatrix}\!+\!\begin{pmatrix}0\\5\\6\end{pmatrix}=\begin{pmatrix}1+0\\4+5\\3+6\end{pmatrix}=\begin{pmatrix}0+1\\5+4\\6+3\end{pmatrix}=\begin{pmatrix}0\\5\\6\end{pmatrix}\!+\!\begin{pmatrix}1\\4\\3\end{pmatrix}$
$A+(-1)\cdot A=\mathbf{0}$,
where $\mathbf{0}$ denotes matrix consisting of only zeros
$\begin{pmatrix}1&2&4\\3&4&6\end{pmatrix}\!+\!(-1)\!\cdot\!\begin{pmatrix}1&2&4\\3&4&6\end{pmatrix}=\begin{pmatrix}1-1&2-2&4-4\\3-3&4-4&6-6\end{pmatrix}=\begin{pmatrix}0&0&0\\0&0&0\end{pmatrix}$
$k\cdot A+l\cdot A=(k+l)\cdot A$$-4\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}\!+\!5\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}=\begin{pmatrix}-4\\-20\\-16\end{pmatrix}\!+\!\begin{pmatrix}5\\25\\20\end{pmatrix}=\begin{pmatrix}1\\5\\4\end{pmatrix}\quad\text{and}\quad (-4+5)\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}=\begin{pmatrix}1\\5\\4\end{pmatrix}$
In general $AB\neq BA$$\begin{pmatrix}1&2\\3&4\end{pmatrix}\begin{pmatrix}0&1\\1&0\end{pmatrix}=\begin{pmatrix}2&1\\4&3\end{pmatrix}\quad\text{but}\quad \begin{pmatrix}0&1\\1&0\end{pmatrix}\begin{pmatrix}1&2\\3&4\end{pmatrix}=\begin{pmatrix}3&4\\1&2\end{pmatrix}$
$[AB]C=A[BC]$$\Bigg[\begin{pmatrix}1 & 2 & 3\end{pmatrix}\begin{pmatrix}1 & 3\\-1 & -2\\2 & 5\end{pmatrix}\Bigg]\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}=[\begin{pmatrix}5&14\end{pmatrix}]\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}=\begin{pmatrix}19&33\end{pmatrix}$
$\begin{pmatrix}1 & 2 & 3\end{pmatrix}\Bigg[\begin{pmatrix}1 & 3\\-1 & -2\\2 & 5\end{pmatrix}\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}\Bigg]=\begin{pmatrix}1 & 2 & 3\end{pmatrix}\Bigg[\begin{pmatrix}4&7\\-3&-5\\7&12\end{pmatrix}\Bigg]=\begin{pmatrix}19&33\end{pmatrix}$
$[A+B]C=AC+BC$$\Bigg[\begin{pmatrix}1&2\\3&1\end{pmatrix}\!+\!\begin{pmatrix}-1&2\\2&-3\end{pmatrix}\Bigg]\begin{pmatrix}1\\3\end{pmatrix}=\Bigg[\begin{pmatrix}0&4\\5&-2\end{pmatrix}\Bigg]\begin{pmatrix}1\\3\end{pmatrix}=\begin{pmatrix}12\\-1\end{pmatrix}$
$\begin{pmatrix}1&2\\3&1\end{pmatrix}\begin{pmatrix}1\\3\end{pmatrix}\!+\!\begin{pmatrix}-1&2\\2&-3\end{pmatrix}\begin{pmatrix}1\\3\end{pmatrix}=\begin{pmatrix}7\\6\end{pmatrix}\!+\!\begin{pmatrix}5\\-7\end{pmatrix}=\begin{pmatrix}12\\-1\end{pmatrix}$
$(AB)^T=B^TA^T$$\Bigg[\begin{pmatrix}1&-2\\1&-1\end{pmatrix}\begin{pmatrix}2\\1\end{pmatrix}\Bigg]^T=\Bigg[\begin{pmatrix}0\\1\end{pmatrix}\Bigg]^T=\begin{pmatrix}0&1\end{pmatrix}$
$\begin{pmatrix}2\\1\end{pmatrix}^T\begin{pmatrix}1&-2\\1&-1\end{pmatrix}^T=\begin{pmatrix}2&1\end{pmatrix}\begin{pmatrix}1&1\\-2&-1\end{pmatrix}=\begin{pmatrix}0&1\end{pmatrix}$

Note that matrix multiplication is not commutative, i.e. even if both the matrices $AB$ and $BA$ do exist, in general

$$AB\neq BA.$$