# 1. Matrices

A matrix is a rectangular array of numbers, symbols, or expressions, arranged in rows and columns and surrounded by brackets.

For example

 $\begin{pmatrix}2 & 6 & 3 & -5\\15 & 8 & 1 & 14\\2 & 2 & 2 & -8\end{pmatrix},$ $\begin{pmatrix}\cos\alpha & -\sin\alpha\\ \sin\alpha & \cos\alpha\end{pmatrix},$ $\begin{pmatrix}e & 3e\\0 & e\\ \frac{1}{2}e^5-\frac{1}{2}e & e^5 \end{pmatrix},$ $\begin{pmatrix}x^2\\y^2\\z^2\\w^2\end{pmatrix}.$
The size or dimension of a matrix is defined as $m \times n$ where $m$ is the number of rows and $n$ is the number of columns.

What is the size of the following matrices:

 $\begin{pmatrix}1 & 3 & 7\end{pmatrix}$ $\begin{pmatrix}a & b \\c & d \\e & f\end{pmatrix}$ $\begin{pmatrix}\overline{1} & \overline{7} & \overline{8} \\ \overline{2} & \overline{9} & \overline{0}\end{pmatrix}$ $\begin{pmatrix}7+i \\8-3i \\10+\sqrt{5}i\end{pmatrix}$ $\begin{pmatrix}7\end{pmatrix}$ $\begin{pmatrix}x^2 & 1 & 3 \\6 & x^2 & 5 \\9 & 7 & x^2\end{pmatrix}$ Answer $1 \times 3$ Answer $3 \times 2$ Answer $2 \times 3$ Answer $3 \times 1$ Answer $1 \times 1$ Answer $3 \times 3$
Matrices with dimensions $1\times n$ and $n\times 1$ are also called vectors. A $1\times n$ matrix is called a column vector and an $n\times 1$ matrix is called a row vector.

Which of the following matrices are vectors? If a matrix below is not a vector, then name all vectors the matrix consists of.

 $\begin{pmatrix}1\\2\\0\\1\end{pmatrix}$ $\begin{pmatrix}x & y \\x^2 & y^2 \\v^3 & y^3\end{pmatrix}$ $\begin{pmatrix}-\cos\alpha&\sin\alpha&\tan\alpha\end{pmatrix}$ $\begin{pmatrix}f(2) & g(1) & 3 \\g(6) & f(5) & 2\end{pmatrix},$ $f(x)=x-4$ and $g(x)=\pi x$ $\begin{pmatrix}7+i \\8-3i \\10+\sqrt{5}i\end{pmatrix}$ Answer A column vector Answer It is not a vector. The $3\times 2$ matrix has $3$ row vectors: $\begin{pmatrix}x&y\end{pmatrix}$, $\begin{pmatrix}x^2&y^2\end{pmatrix}$, $\begin{pmatrix}x^3&y^3\end{pmatrix}$, and $2$ column vectors: $\begin{pmatrix}x\\x^2\\x^3\end{pmatrix}$, $\begin{pmatrix}y\\y^2\\y^3\end{pmatrix}$. Answer A row vector Answer The $2\times 3$ matrix has $2$ row vectors: $\begin{pmatrix}f(2)&g(1)&3\end{pmatrix}$, $\begin{pmatrix}g(6)&f(5)&2\end{pmatrix}$, and $3$ column vectors: $\begin{pmatrix}f(2)\\g(6)\end{pmatrix}$, $\begin{pmatrix}g(1)\\f(5)\end{pmatrix}$, $\begin{pmatrix}3\\2\end{pmatrix}$. Answer A column vector

A general notation of an $m \times n$ matrix $A$ is

$A=\begin{pmatrix}a_{11} & a_{12} & \dots & a_{1n}\\a_{21} & a_{22} & \dots & a_{2n}\\a_{31} & a_{32} & \dots & a_{3n}\\ \vdots & \vdots & & \vdots\\a_{m1} & a_{m2} & \dots & a_{mn}\end{pmatrix}\hspace{2cm} \text{or}\hspace{2cm} A=(a_{ij}),$

where $i=1,2,\dots , m$ is the row index and $j=1,2,\dots , n$ is the column index. Usually, a capital letter is used for denoting a matrix. Row and column vectors are often denoted by a (noncapital) bold letter, e.g.,

$\mathbf{x}=\begin{pmatrix}x_1 & x_2 & \dots & x_n\end{pmatrix}\hspace{2cm} \text{or}\hspace{2cm} \mathbf{b}=\begin{pmatrix}b_1\\b_2\\\vdots\\b_m\end{pmatrix}.$

What is the $ij^{th}$ element of the matrix \begin{pmatrix}0 & 1 & 2 & 3 & 4 \\5 & 6 & 7 & 8 & 9\\10 & 11 & 12 & 13 & 14\end{pmatrix} when

• $i=2$ and $j=5$; Answer
• $i=1$ and $j=2$; Answer
• $i=5$ and $j=3$. Answer

Let $a_{14}=8$, $a_{27}=0$, $a_{13}=2$, $a_{21}=-4$, $a_{41}=-5$, $a_{12}=1$, $a_{31}=5$, $a_{23}=12$, $a_{25}=-1$, $a_{03}=10$, $a_{26}=7$, $a_{34}=4.$ Complete the matrix by entering the missing values (drag and trop the blue number below)

Matrices can be added, multiplied by a scalar, and multiplied by other matrices. Formulas for the operations are as follows

Operation Formula Example
Sum $(a_{ij})+(b_{ij})=(a_{ij}+b_{ij})$ $\begin{pmatrix}\color{blue}{1} & \color{blue}2\\\color{blue}0 &\color{blue}{-2}\end{pmatrix}+\begin{pmatrix}\color{red}{0} &\color{red}{-1}\\\color{red}1 & \color{red}2\end{pmatrix}=\begin{pmatrix}\color{blue}1+\color{red}0 & \color{blue}2+\color{red}{(-1)}\\\color{blue}0+\color{red}1 & \color{blue}{-2}+\color{red}2\end{pmatrix}=\begin{pmatrix}1 & 1\\1 & 0 \end{pmatrix}$
Scalar multiplication $c\cdot (a_{ij})=(c\cdot a_{ij})$ $\color{blue}{-1}\cdot\begin{pmatrix}1 & 8 \\5 &-2 \end{pmatrix}=$ $\begin{pmatrix}\color{blue}{-1}\cdot 1 & \color{blue}{-1}\cdot 8 \\\color{blue}{-1}\cdot 5 &\color{blue}{-1}\cdot (-2)\end{pmatrix}=\begin{pmatrix}-1 & -8 \\-5 & 2 \end{pmatrix}$
Matrix multiplication Let $AB=(c_{ij})$, then $c_{ij}=\sum^m_{r=1}a_{ir}b_{rj}$,
where $A=(a_{ij})$ is $n\times m$ matrix and $B=(b_{ij})$ is $m\times p$ matrix.
$\begin{pmatrix}\color{blue}2 & \color{blue}3 & \color{blue}4\\ \color{red}1 & \color{red}0 & \color{red}0\end{pmatrix}\begin{pmatrix}\color{LimeGreen}6& \color{Cyan}{10}\\\color{LimeGreen}7 & \color{Cyan}5\\\color{LimeGreen}8 & \color{Cyan}1\end{pmatrix}=$ $\begin{pmatrix}\color{blue}2\cdot \color{LimeGreen}6+\color{blue}3\cdot \color{LimeGreen}7+\color{blue}4\cdot \color{LimeGreen}8 & \color{blue}2\cdot \color{Cyan}{10}+\color{blue}3\cdot \color{Cyan}5+\color{blue}4\cdot \color{Cyan}1\\\color{red}1\cdot \color{LimeGreen}6+\color{red}0\cdot \color{LimeGreen}7+\color{red}0\cdot \color{LimeGreen}8 & \color{red}1\cdot \color{Cyan}{10}+\color{red}0\cdot \color{Cyan}5+\color{red}0\cdot \color{Cyan}1\end{pmatrix}=$ $\begin{pmatrix}65& 39\\6 & 10\end{pmatrix}$

Note that the dimension of matrices is important in the operations of adding and multiplying matrices:

• two matrices can be added only when they have same dimensions;
• two matrices can be multiplied if and only if the number of row in first matrix equals to the number of columns in the second matrix, i.e.

$\underset{n\times \color{blue}m}{A}\cdot \underset{\color{blue}m\times p}B=\underset{n\times p}{AB}.$

Click on corresponding elements of the matrices for calculating the elements of the sum (over $\mathbb Z_6$).

Which of the following operations can be performed and what is the result:

 $\begin{pmatrix}1 & -1 & 2\\-2 & 1 & 0\end{pmatrix}\!+\!\begin{pmatrix}-2 & 1\\2 & -1\end{pmatrix}$ $\begin{pmatrix}2 \\-1 \\4\end{pmatrix}\begin{pmatrix}2 & -2 & 3\end{pmatrix}$ $3\!\cdot\!\begin{pmatrix}1&0&4\\3&1&-2\\2&1&-1\end{pmatrix}$ Answer The operation can not be performed. Answer $\begin{pmatrix}4&-4&6\\-2&2&-3\\8&-8&12\end{pmatrix}$ Answer $\begin{pmatrix}3&0&12\\9&3&-6\\6&3&-3\end{pmatrix}$ $(-2)\!\cdot\!\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}$ $\begin{pmatrix}1 & 2\\-1 & 3\\5 & -1\end{pmatrix}\begin{pmatrix}-1 & 2\end{pmatrix}$ $\begin{pmatrix}8&-6\\1&-1\\3&2\end{pmatrix}\begin{pmatrix}0&3\\10&6\end{pmatrix}$ Answer $\begin{pmatrix} -28&10\\22&-7\\-55&11\end{pmatrix}$ Answer The operation can not be performed. Answer $\begin{pmatrix} -60&-12\\-10&-3\\20&21\end{pmatrix}$

The diagram to the right describes also matrix multiplication.

Here are multiplied matrices $A$ with dimension $4\times 2$ and $B$ with dimension $2\times 3$. The matrix $AB$ is with dimension $4\times 3$.

Let $AB=(c_{ij})$. Then

$c_{12}=a_{11}b_{12}+a_{12}b_{22}$

and

$c_{33}=a_{31}b_{13}+a_{32}b_{23}$.

What are the values of indexes $i$ and $j$ if $c_{ij}=a_{41}b_{11}+a_{42}b_{21}$? Answer

Use IPython for calculating

 $\begin{pmatrix}0 & 3 & 1\\3 & -1 & 7\\-5 & 14 & -8\\7&2&9\\0&-6&10\end{pmatrix}\begin{pmatrix}6 \\ 1\\-1\end{pmatrix}$ $\begin{pmatrix}2 & 10 & 7\\1 & 4 & 3\end{pmatrix}\begin{pmatrix}1 & 6 & 4\\2 & 3 & 1\\2 & 1 &10\end{pmatrix}$ over $\mathbb{Z_3}$.

Solution

Solution

Solution

from numpy import array

M1 = array([[0,3,1],[3,-1,7],[-5,14,-8],[7,2,9],[0,-6,10]]) # the first matrix
M2 = array([[6],[1],[-1]])                                  # the second matrix
R = numpy.zeros(shape=(len(M1),len(M2.T)))                  # the result matrix filled with zeros

for i in range(len(M1)):                                    # loop over rows of M1; row i in matrix M1 is
R[i]=sum(M1[i,:]*M2[:,0])                               # multiplied elementwise with column 0 from M2,
# elements in the product are added
print R

from numpy import array

M1 = array([[2,10,7],[1,4,3]])               # the first matrix
M2 = array([[1,6,4],[2,3,1],[2,1,10]])       # the second matrix
R = numpy.zeros(shape=(len(M1),len(M2.T)))   # the product filled with zeros

for i in range(len(M1)):
for  j in range(len(M2.T)):
R[i,j]=sum(M1[i,:]*M2[:,j])          # M1 row i multiplied elementwise with M2 column j and summed

print R%3                                    #for every element in R the residue modulo 3 is calculated

Another important operation on matrices is that of taking the transpose. It is denoted by placing a $T$ as an exponent on the initial matrix.

Operation Formula Example
Transpose $(a_{ij})^T=(a_{ji})$ $\begin{pmatrix}\color{blue}1 & \color{blue}2 & \color{blue}{-6}\\ \color{red}3 & \color{red}5 & \color{red}4\end{pmatrix}^T=\begin{pmatrix}\color{blue}1 & \color{red}3\\ \color{blue}2 & \color{red}5\\ \color{blue}{-6} & \color{red}4\end{pmatrix}$

Thus taking the transpose means reorganize matrix entries so that row of the initial matrix will be columns in the transpose. First row will be first column, second row will be second column and so on.

Which of the matrices

 $(a)\quad\begin{pmatrix}2 & 6 & 3\\15 & 8 & 1\\2 & 2 & 2\\ 0 & 1 & 0\\-5 & 14 & -8\end{pmatrix}$ $(b)\quad\begin{pmatrix}3&6&2\\1&8&15\\2&2&2\\0&1&0\\-8&14&-5\end{pmatrix}$ $(c)\quad\begin{pmatrix}-8&14&-5\\0&1&1\\2&2&2\\1&8&15\\3&6&2\end{pmatrix}$

is the transpose of the matrix $\begin{pmatrix}3 & 1 & 2 & 0 & -8\\6 & 8 & 2 & 1 & 14\\2 & 15 & 2 & 0 & -5\end{pmatrix}$? Answer

Let

 $A=\begin{pmatrix}1 & 5\\4 & 3\\5 & 4\end{pmatrix}$, $\mathbf{b}=\begin{pmatrix}1\\2\\3\end{pmatrix}$, $\mathbf{c}=\begin{pmatrix}1\\2\\3\end{pmatrix}^T$.

Calculate $5\cdot AA^T+\mathbf{b}\mathbf{c}$ by using IPython .

Solution

Solution

Solution

from numpy import array

M1 = array([[1,5],[4,3],[5,4]])          # the first matrix
M2 = array([[1],[2],[3]])                # the second matrix
M3 = array([[1],[2],[3]])                # the third matrix
R = numpy.zeros(shape=(len(M1),len(M1))) # the product matrix filled with zeros

for i in range(len(M1)):
for  j in range(len(M1)):
R[i,j]=sum(M1[i,:]*M1.T[:,j])    # M1 row i multiplied elementwise with M1 transp. column j, summed

print 5*R+M2*M3.T

Sum of matrices, scalar multiplication, matrix multiplication and the transpose are basic matrix operations. These operations have several useful properties. For example, knowing that $$c\cdot A+c\cdot B=c\cdot (A+B),$$ where $A$ and $B$ are matrices and $c$ is a constant, it is possible to reduce number of multiplications. Thus, instead of

$$-2\cdot\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!(-2)\cdot\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}=\begin{pmatrix}-2\color{red}\cdot10 & -2\color{red}\cdot(-3)\\-2\color{red}\cdot(-8) & -2\color{red}\cdot5\\-2\color{red}\cdot20 & -2\color{red}\cdot3\end{pmatrix}\!+\!\begin{pmatrix}-2\color{red}\cdot(-8) & -2\color{red}\cdot4\\-2\color{red}\cdot6 & -2\color{red}\cdot3\\-2\color{red}\cdot(-15) & -2\color{red}\cdot17\end{pmatrix}\!=\!\begin{pmatrix}-4 & -2\\4 & -16\\-10 & -40\end{pmatrix},$$ which is $12$ multiplications, we can manage only with $6$:

$$-2\cdot\!\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!(-2)\cdot\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}=-2\cdot\!\Bigg[\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}\Bigg]=-2\cdot\!\begin{pmatrix}2 & 1\\-2 & 8\\5 & 20\end{pmatrix}\!=\!\begin{pmatrix}-4 & -2\\4 & -16\\-10 & -40\end{pmatrix}.$$

Let $A$, $B$ and $C$ be matrices such that all the operations below are defined, and let $k$ and $l$ denote some constants (for example real numbers) . Here are some additional useful properties concerning the matrix operations:

Property Example
$[A^T]^T=A$$\Bigg[\begin{pmatrix}-1&2\\4&-7\\3&5\end{pmatrix}^T\Bigg]^T=\Bigg[\begin{pmatrix}-1&4&3\\2&-7&5\end{pmatrix}\Bigg]^T=\begin{pmatrix}-1&2\\4&-7\\3&5\end{pmatrix} A+B=B+A$$\begin{pmatrix}1\\4\\3\end{pmatrix}\!+\!\begin{pmatrix}0\\5\\6\end{pmatrix}=\begin{pmatrix}1+0\\4+5\\3+6\end{pmatrix}=\begin{pmatrix}0+1\\5+4\\6+3\end{pmatrix}=\begin{pmatrix}0\\5\\6\end{pmatrix}\!+\!\begin{pmatrix}1\\4\\3\end{pmatrix}$
$A+(-1)\cdot A=\mathbf{0}$,
where $\mathbf{0}$ denotes matrix consisting of only zeros
$\begin{pmatrix}1&2&4\\3&4&6\end{pmatrix}\!+\!(-1)\!\cdot\!\begin{pmatrix}1&2&4\\3&4&6\end{pmatrix}=\begin{pmatrix}1-1&2-2&4-4\\3-3&4-4&6-6\end{pmatrix}=\begin{pmatrix}0&0&0\\0&0&0\end{pmatrix}$
$k\cdot A+l\cdot A=(k+l)\cdot A$$-4\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}\!+\!5\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}=\begin{pmatrix}-4\\-20\\-16\end{pmatrix}\!+\!\begin{pmatrix}5\\25\\20\end{pmatrix}=\begin{pmatrix}1\\5\\4\end{pmatrix}\quad\text{and}\quad (-4+5)\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}=\begin{pmatrix}1\\5\\4\end{pmatrix} In general AB\neq BA$$\begin{pmatrix}1&2\\3&4\end{pmatrix}\begin{pmatrix}0&1\\1&0\end{pmatrix}=\begin{pmatrix}2&1\\4&3\end{pmatrix}\quad\text{but}\quad \begin{pmatrix}0&1\\1&0\end{pmatrix}\begin{pmatrix}1&2\\3&4\end{pmatrix}=\begin{pmatrix}3&4\\1&2\end{pmatrix}$
$[AB]C=A[BC]$$\Bigg[\begin{pmatrix}1 & 2 & 3\end{pmatrix}\begin{pmatrix}1 & 3\\-1 & -2\\2 & 5\end{pmatrix}\Bigg]\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}=[\begin{pmatrix}5&14\end{pmatrix}]\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}=\begin{pmatrix}19&33\end{pmatrix} \begin{pmatrix}1 & 2 & 3\end{pmatrix}\Bigg[\begin{pmatrix}1 & 3\\-1 & -2\\2 & 5\end{pmatrix}\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}\Bigg]=\begin{pmatrix}1 & 2 & 3\end{pmatrix}\Bigg[\begin{pmatrix}4&7\\-3&-5\\7&12\end{pmatrix}\Bigg]=\begin{pmatrix}19&33\end{pmatrix} [A+B]C=AC+BC$$\Bigg[\begin{pmatrix}1&2\\3&1\end{pmatrix}\!+\!\begin{pmatrix}-1&2\\2&-3\end{pmatrix}\Bigg]\begin{pmatrix}1\\3\end{pmatrix}=\Bigg[\begin{pmatrix}0&4\\5&-2\end{pmatrix}\Bigg]\begin{pmatrix}1\\3\end{pmatrix}=\begin{pmatrix}12\\-1\end{pmatrix}$
$\begin{pmatrix}1&2\\3&1\end{pmatrix}\begin{pmatrix}1\\3\end{pmatrix}\!+\!\begin{pmatrix}-1&2\\2&-3\end{pmatrix}\begin{pmatrix}1\\3\end{pmatrix}=\begin{pmatrix}7\\6\end{pmatrix}\!+\!\begin{pmatrix}5\\-7\end{pmatrix}=\begin{pmatrix}12\\-1\end{pmatrix}$
$(AB)^T=B^TA^T$$\Bigg[\begin{pmatrix}1&-2\\1&-1\end{pmatrix}\begin{pmatrix}2\\1\end{pmatrix}\Bigg]^T=\Bigg[\begin{pmatrix}0\\1\end{pmatrix}\Bigg]^T=\begin{pmatrix}0&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix}^T\begin{pmatrix}1&-2\\1&-1\end{pmatrix}^T=\begin{pmatrix}2&1\end{pmatrix}\begin{pmatrix}1&1\\-2&-1\end{pmatrix}=\begin{pmatrix}0&1\end{pmatrix} Note that matrix multiplication is not commutative, i.e. even if both the matrices AB and BA do exist, in general$$AB\neq BA.$\$

linear_algebra/01_matrices.txt ยท Last modified: 2014/12/14 00:03 by jaan