====== 1. Matrices ======
A **matrix** is a rectangular array of numbers, symbols, or expressions, arranged in rows and columns and surrounded by brackets.
For example
$\begin{pmatrix}2 & 6 & 3 & -5\\15 & 8 & 1 & 14\\2 & 2 & 2 & -8\end{pmatrix},$
$\begin{pmatrix}\cos\alpha & -\sin\alpha\\ \sin\alpha & \cos\alpha\end{pmatrix},$
$\begin{pmatrix}e & 3e\\0 & e\\ \frac{1}{2}e^5-\frac{1}{2}e & e^5 \end{pmatrix},$
$\begin{pmatrix}x^2\\y^2\\z^2\\w^2\end{pmatrix}.$
The **size** or **dimension of a matrix** is defined as $m \times n$ where $m$ is the number of rows and $n$ is the number of columns.
What is the size of the following matrices:
|$\begin{pmatrix}1 & 3 & 7\end{pmatrix}$ | $\begin{pmatrix}a & b \\c & d \\e & f\end{pmatrix}$ | $\begin{pmatrix}\overline{1} & \overline{7} & \overline{8} \\ \overline{2} & \overline{9} & \overline{0}\end{pmatrix}$ | $\begin{pmatrix}7+i \\8-3i \\10+\sqrt{5}i\end{pmatrix}$ | $\begin{pmatrix}7\end{pmatrix}$ | $\begin{pmatrix}x^2 & 1 & 3 \\6 & x^2 & 5 \\9 & 7 & x^2\end{pmatrix}$ |
|++Answer | $1 \times 3$ ++ | ++Answer | $3 \times 2$ ++ | ++Answer | $2 \times 3$ ++ | ++Answer | $3 \times 1$ ++ | ++Answer | $1 \times 1$ ++ | ++Answer | $3 \times 3$ ++|
Matrices with dimensions $1\times n$ and $n\times 1$ are also called **vectors**.
A $1\times n$ matrix is called **a column vector** and an $n\times 1$ matrix is called **a row vector**.
Which of the following matrices are vectors? If a matrix below is not a vector, then name all vectors the matrix consists of.
|$\begin{pmatrix}1\\2\\0\\1\end{pmatrix}$ | $\begin{pmatrix}x & y \\x^2 & y^2 \\v^3 & y^3\end{pmatrix}$ | $\begin{pmatrix}-\cos\alpha&\sin\alpha&\tan\alpha\end{pmatrix}$ | $\begin{pmatrix}f(2) & g(1) & 3 \\g(6) & f(5) & 2\end{pmatrix},$\\ $f(x)=x-4$ and $g(x)=\pi x$ | $\begin{pmatrix}7+i \\8-3i \\10+\sqrt{5}i\end{pmatrix}$ |
|++Answer | A column vector ++ | ++Answer | It is not a vector.\\ The $3\times 2$ matrix has\\ $3$ row vectors: $\begin{pmatrix}x&y\end{pmatrix}$, $\begin{pmatrix}x^2&y^2\end{pmatrix}$, $\begin{pmatrix}x^3&y^3\end{pmatrix}$, and\\ $2$ column vectors: $\begin{pmatrix}x\\x^2\\x^3\end{pmatrix}$, $\begin{pmatrix}y\\y^2\\y^3\end{pmatrix}$. ++ | ++Answer | A row vector ++ | ++Answer | The $2\times 3$ matrix has\\ $2$ row vectors: $\begin{pmatrix}f(2)&g(1)&3\end{pmatrix}$, $\begin{pmatrix}g(6)&f(5)&2\end{pmatrix}$, and\\ $3$ column vectors: $\begin{pmatrix}f(2)\\g(6)\end{pmatrix}$, $\begin{pmatrix}g(1)\\f(5)\end{pmatrix}$, $\begin{pmatrix}3\\2\end{pmatrix}$. ++ | ++Answer | A column vector ++|
A general notation of an $m \times n$ matrix $A$ is
;#;
$A=\begin{pmatrix}a_{11} & a_{12} & \dots & a_{1n}\\a_{21} & a_{22} & \dots & a_{2n}\\a_{31} & a_{32} & \dots & a_{3n}\\ \vdots & \vdots & & \vdots\\a_{m1} & a_{m2} & \dots & a_{mn}\end{pmatrix}\hspace{2cm} \text{or}\hspace{2cm} A=(a_{ij}),$
;#;
where $i=1,2,\dots , m$ is the row index and $j=1,2,\dots , n$ is the column index. Usually, a capital letter is used for denoting a matrix. Row and column vectors are often denoted
by a (noncapital) bold letter, e.g.,
;#;
$\mathbf{x}=\begin{pmatrix}x_1 & x_2 & \dots & x_n\end{pmatrix}\hspace{2cm} \text{or}\hspace{2cm} \mathbf{b}=\begin{pmatrix}b_1\\b_2\\\vdots\\b_m\end{pmatrix}.$
;#;
What is the $ij^{th}$ element of the matrix \begin{pmatrix}0 & 1 & 2 & 3 & 4 \\5 & 6 & 7 & 8 & 9\\10 & 11 & 12 & 13 & 14\end{pmatrix} when
* $i=2$ and $j=5$; ++Answer | 9 ++
* $i=1$ and $j=2$; ++Answer | 1 ++
* $i=5$ and $j=3$. ++Answer | There is no such element in this matrix. The matrix has only 3 rows. ++
:?:
Let $a_{14}=8$, $a_{27}=0$, $a_{13}=2$, $a_{21}=-4$, $a_{41}=-5$, $a_{12}=1$, $a_{31}=5$, $a_{23}=12$, $a_{25}=-1$, $a_{03}=10$, $a_{26}=7$, $a_{34}=4.$ Complete the matrix by entering the missing values (drag and trop the blue number below)
{{url>http://mathwiki.cs.ut.ee/illustrations/matrix.html 431px,197px}}
Matrices can be added, multiplied by a scalar, and multiplied by other matrices. Formulas for the operations are as follows
^Operation ^Formula ^Example^
| Sum | $(a_{ij})+(b_{ij})=(a_{ij}+b_{ij})$ | $\begin{pmatrix}\color{blue}{1} & \color{blue}2\\\color{blue}0 &\color{blue}{-2}\end{pmatrix}+\begin{pmatrix}\color{red}{0} &\color{red}{-1}\\\color{red}1 & \color{red}2\end{pmatrix}=\begin{pmatrix}\color{blue}1+\color{red}0 & \color{blue}2+\color{red}{(-1)}\\\color{blue}0+\color{red}1 & \color{blue}{-2}+\color{red}2\end{pmatrix}=\begin{pmatrix}1 & 1\\1 & 0 \end{pmatrix}$ |
| Scalar multiplication | $c\cdot (a_{ij})=(c\cdot a_{ij})$ | $\color{blue}{-1}\cdot\begin{pmatrix}1 & 8 \\5 &-2 \end{pmatrix}=$ $\begin{pmatrix}\color{blue}{-1}\cdot 1 & \color{blue}{-1}\cdot 8 \\\color{blue}{-1}\cdot 5 &\color{blue}{-1}\cdot (-2)\end{pmatrix}=\begin{pmatrix}-1 & -8 \\-5 & 2 \end{pmatrix}$|
|Matrix multiplication | Let $AB=(c_{ij})$, then $c_{ij}=\sum^m_{r=1}a_{ir}b_{rj}$,\\ where $A=(a_{ij})$ is $n\times m$ matrix and $B=(b_{ij})$ is $m\times p$ matrix. | $\begin{pmatrix}\color{blue}2 & \color{blue}3 & \color{blue}4\\ \color{red}1 & \color{red}0 & \color{red}0\end{pmatrix}\begin{pmatrix}\color{LimeGreen}6& \color{Cyan}{10}\\\color{LimeGreen}7 & \color{Cyan}5\\\color{LimeGreen}8 & \color{Cyan}1\end{pmatrix}=$ $\begin{pmatrix}\color{blue}2\cdot \color{LimeGreen}6+\color{blue}3\cdot \color{LimeGreen}7+\color{blue}4\cdot \color{LimeGreen}8 & \color{blue}2\cdot \color{Cyan}{10}+\color{blue}3\cdot \color{Cyan}5+\color{blue}4\cdot \color{Cyan}1\\\color{red}1\cdot \color{LimeGreen}6+\color{red}0\cdot \color{LimeGreen}7+\color{red}0\cdot \color{LimeGreen}8 & \color{red}1\cdot \color{Cyan}{10}+\color{red}0\cdot \color{Cyan}5+\color{red}0\cdot \color{Cyan}1\end{pmatrix}=$ $\begin{pmatrix}65& 39\\6 & 10\end{pmatrix}$|
Note that **the dimension of matrices is important** in the operations of adding and multiplying matrices:
* two matrices can be added only when they have same dimensions;
* two matrices can be multiplied if and only if the number of row in first matrix equals to the number of columns in the second matrix, i.e.\\
;#;
$\underset{n\times \color{blue}m}{A}\cdot \underset{\color{blue}m\times p}B=\underset{n\times p}{AB}.$
;#;
:?:
Click on corresponding elements of the matrices for calculating the elements of the sum (over $\mathbb Z_6$).
{{url>http://mathwiki.cs.ut.ee/illustrations/matrixsum.html 250px}}
Which of the following operations can be performed and what is the result:
|$\begin{pmatrix}1 & -1 & 2\\-2 & 1 & 0\end{pmatrix}\!+\!\begin{pmatrix}-2 & 1\\2 & -1\end{pmatrix}$ | $\begin{pmatrix}2 \\-1 \\4\end{pmatrix}\begin{pmatrix}2 & -2 & 3\end{pmatrix}$ | $3\!\cdot\!\begin{pmatrix}1&0&4\\3&1&-2\\2&1&-1\end{pmatrix}$ |
|++Answer | The operation can not be performed. ++ | ++Answer | $\begin{pmatrix}4&-4&6\\-2&2&-3\\8&-8&12\end{pmatrix}$ ++ | ++Answer | $\begin{pmatrix}3&0&12\\9&3&-6\\6&3&-3\end{pmatrix}$ ++|
|$(-2)\!\cdot\!\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}$ | $\begin{pmatrix}1 & 2\\-1 & 3\\5 & -1\end{pmatrix}\begin{pmatrix}-1 & 2\end{pmatrix}$| $\begin{pmatrix}8&-6\\1&-1\\3&2\end{pmatrix}\begin{pmatrix}0&3\\10&6\end{pmatrix}$ |
| ++Answer | $\begin{pmatrix} -28&10\\22&-7\\-55&11\end{pmatrix}$ ++ | ++Answer | The operation can not be performed. ++ | ++Answer | $\begin{pmatrix} -60&-12\\-10&-3\\20&21\end{pmatrix}$ ++|
The diagram to the right describes also matrix multiplication.
{{ :linear_algebra:matrix_multiplication_diagram.png|}}
Here are multiplied matrices $A$ with dimension $4\times 2$ and $B$ with dimension $2\times 3$. The matrix $AB$ is with dimension $4\times 3$.
Let $AB=(c_{ij})$. Then
$c_{12}=a_{11}b_{12}+a_{12}b_{22}$
and
$c_{33}=a_{31}b_{13}+a_{32}b_{23}$.
What are the values of indexes $i$ and $j$ if $c_{ij}=a_{41}b_{11}+a_{42}b_{21}$? ++Answer | $i=4$ and $j=1$ ++
Use IPython for calculating
* $\begin{pmatrix}0 & 3 & 1\\3 & -1 & 7\\-5 & 14 & -8\\7&2&9\\0&-6&10\end{pmatrix}\begin{pmatrix}6 \\ 1\\-1\end{pmatrix}$
* $\begin{pmatrix}2 & 10 & 7\\1 & 4 & 3\end{pmatrix}\begin{pmatrix}1 & 6 & 4\\2 & 3 & 1\\2 & 1 &10\end{pmatrix}$ over $\mathbb{Z_3}$.
\\
from numpy import array
M1 = array([[0,3,1],[3,-1,7],[-5,14,-8],[7,2,9],[0,-6,10]]) # the first matrix
M2 = array([[6],[1],[-1]]) # the second matrix
R = numpy.zeros(shape=(len(M1),len(M2.T))) # the result matrix filled with zeros
for i in range(len(M1)): # loop over rows of M1; row i in matrix M1 is
R[i]=sum(M1[i,:]*M2[:,0]) # multiplied elementwise with column 0 from M2,
# elements in the product are added
print R
from numpy import array
M1 = array([[2,10,7],[1,4,3]]) # the first matrix
M2 = array([[1,6,4],[2,3,1],[2,1,10]]) # the second matrix
R = numpy.zeros(shape=(len(M1),len(M2.T))) # the product filled with zeros
for i in range(len(M1)):
for j in range(len(M2.T)):
R[i,j]=sum(M1[i,:]*M2[:,j]) # M1 row i multiplied elementwise with M2 column j and summed
print R%3 #for every element in R the residue modulo 3 is calculated
\\
Another important operation on matrices is that of taking **the transpose**. It is denoted by placing a $T$ as an exponent on the initial matrix.
^Operation ^Formula ^Example^
| Transpose |$(a_{ij})^T=(a_{ji})$ | $\begin{pmatrix}\color{blue}1 & \color{blue}2 & \color{blue}{-6}\\ \color{red}3 & \color{red}5 & \color{red}4\end{pmatrix}^T=\begin{pmatrix}\color{blue}1 & \color{red}3\\ \color{blue}2 & \color{red}5\\ \color{blue}{-6} & \color{red}4\end{pmatrix}$ |
Thus taking the transpose means reorganize matrix entries so that row of the initial matrix will be columns in the transpose. First row will be first column, second row will be second column and so on.
Which of the matrices
$(a)\quad\begin{pmatrix}2 & 6 & 3\\15 & 8 & 1\\2 & 2 & 2\\ 0 & 1 & 0\\-5 & 14 & -8\end{pmatrix}$
$(b)\quad\begin{pmatrix}3&6&2\\1&8&15\\2&2&2\\0&1&0\\-8&14&-5\end{pmatrix}$
$(c)\quad\begin{pmatrix}-8&14&-5\\0&1&1\\2&2&2\\1&8&15\\3&6&2\end{pmatrix}$
is the transpose of the matrix $\begin{pmatrix}3 & 1 & 2 & 0 & -8\\6 & 8 & 2 & 1 & 14\\2 & 15 & 2 & 0 & -5\end{pmatrix}$? ++Answer | $(b)$ ++
Let
$A=\begin{pmatrix}1 & 5\\4 & 3\\5 & 4\end{pmatrix}$,
$\mathbf{b}=\begin{pmatrix}1\\2\\3\end{pmatrix}$,
$\mathbf{c}=\begin{pmatrix}1\\2\\3\end{pmatrix}^T$.
Calculate $5\cdot AA^T+\mathbf{b}\mathbf{c}$ by using IPython .
from numpy import array
M1 = array([[1,5],[4,3],[5,4]]) # the first matrix
M2 = array([[1],[2],[3]]) # the second matrix
M3 = array([[1],[2],[3]]) # the third matrix
R = numpy.zeros(shape=(len(M1),len(M1))) # the product matrix filled with zeros
for i in range(len(M1)):
for j in range(len(M1)):
R[i,j]=sum(M1[i,:]*M1.T[:,j]) # M1 row i multiplied elementwise with M1 transp. column j, summed
print 5*R+M2*M3.T
\\
Sum of matrices, scalar multiplication, matrix multiplication and the transpose are basic matrix operations. These operations have several useful properties. For example, knowing that
$$c\cdot A+c\cdot B=c\cdot (A+B),$$
where $A$ and $B$ are matrices and $c$ is a constant, it is possible to reduce number of multiplications. Thus, instead of
$$-2\cdot\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!(-2)\cdot\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}=\begin{pmatrix}-2\color{red}\cdot10 & -2\color{red}\cdot(-3)\\-2\color{red}\cdot(-8) & -2\color{red}\cdot5\\-2\color{red}\cdot20 & -2\color{red}\cdot3\end{pmatrix}\!+\!\begin{pmatrix}-2\color{red}\cdot(-8) & -2\color{red}\cdot4\\-2\color{red}\cdot6 & -2\color{red}\cdot3\\-2\color{red}\cdot(-15) & -2\color{red}\cdot17\end{pmatrix}\!=\!\begin{pmatrix}-4 & -2\\4 & -16\\-10 & -40\end{pmatrix},$$
which is $12$ multiplications, we can manage only with $6$:
$$-2\cdot\!\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!(-2)\cdot\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}=-2\cdot\!\Bigg[\begin{pmatrix}10 & -3\\-8 & 5\\20 & 3\end{pmatrix}\!+\!\begin{pmatrix}-8 & 4\\6 & 3\\-15 & 17\end{pmatrix}\Bigg]=-2\cdot\!\begin{pmatrix}2 & 1\\-2 & 8\\5 & 20\end{pmatrix}\!=\!\begin{pmatrix}-4 & -2\\4 & -16\\-10 & -40\end{pmatrix}.$$
Let $A$, $B$ and $C$ be matrices such that all the operations below are defined, and let $k$ and $l$ denote some constants (for example real numbers) . Here are some additional useful properties concerning the matrix operations:
^Property ^Example^
|$[A^T]^T=A$|$\Bigg[\begin{pmatrix}-1&2\\4&-7\\3&5\end{pmatrix}^T\Bigg]^T=\Bigg[\begin{pmatrix}-1&4&3\\2&-7&5\end{pmatrix}\Bigg]^T=\begin{pmatrix}-1&2\\4&-7\\3&5\end{pmatrix}$|
|$A+B=B+A$|$\begin{pmatrix}1\\4\\3\end{pmatrix}\!+\!\begin{pmatrix}0\\5\\6\end{pmatrix}=\begin{pmatrix}1+0\\4+5\\3+6\end{pmatrix}=\begin{pmatrix}0+1\\5+4\\6+3\end{pmatrix}=\begin{pmatrix}0\\5\\6\end{pmatrix}\!+\!\begin{pmatrix}1\\4\\3\end{pmatrix}$|
|$A+(-1)\cdot A=\mathbf{0}$,\\ where $\mathbf{0}$ denotes matrix consisting of only zeros|$\begin{pmatrix}1&2&4\\3&4&6\end{pmatrix}\!+\!(-1)\!\cdot\!\begin{pmatrix}1&2&4\\3&4&6\end{pmatrix}=\begin{pmatrix}1-1&2-2&4-4\\3-3&4-4&6-6\end{pmatrix}=\begin{pmatrix}0&0&0\\0&0&0\end{pmatrix}$|
|$k\cdot A+l\cdot A=(k+l)\cdot A$|$-4\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}\!+\!5\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}=\begin{pmatrix}-4\\-20\\-16\end{pmatrix}\!+\!\begin{pmatrix}5\\25\\20\end{pmatrix}=\begin{pmatrix}1\\5\\4\end{pmatrix}\quad\text{and}\quad (-4+5)\!\cdot\!\begin{pmatrix}1\\5\\4\end{pmatrix}=\begin{pmatrix}1\\5\\4\end{pmatrix}$|
|In general $AB\neq BA$|$\begin{pmatrix}1&2\\3&4\end{pmatrix}\begin{pmatrix}0&1\\1&0\end{pmatrix}=\begin{pmatrix}2&1\\4&3\end{pmatrix}\quad\text{but}\quad \begin{pmatrix}0&1\\1&0\end{pmatrix}\begin{pmatrix}1&2\\3&4\end{pmatrix}=\begin{pmatrix}3&4\\1&2\end{pmatrix}$|
|$[AB]C=A[BC]$|$\Bigg[\begin{pmatrix}1 & 2 & 3\end{pmatrix}\begin{pmatrix}1 & 3\\-1 & -2\\2 & 5\end{pmatrix}\Bigg]\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}=[\begin{pmatrix}5&14\end{pmatrix}]\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}=\begin{pmatrix}19&33\end{pmatrix}$|
|:::|$\begin{pmatrix}1 & 2 & 3\end{pmatrix}\Bigg[\begin{pmatrix}1 & 3\\-1 & -2\\2 & 5\end{pmatrix}\begin{pmatrix}1 & 1\\1 & 2\end{pmatrix}\Bigg]=\begin{pmatrix}1 & 2 & 3\end{pmatrix}\Bigg[\begin{pmatrix}4&7\\-3&-5\\7&12\end{pmatrix}\Bigg]=\begin{pmatrix}19&33\end{pmatrix}$|
|$[A+B]C=AC+BC$|$\Bigg[\begin{pmatrix}1&2\\3&1\end{pmatrix}\!+\!\begin{pmatrix}-1&2\\2&-3\end{pmatrix}\Bigg]\begin{pmatrix}1\\3\end{pmatrix}=\Bigg[\begin{pmatrix}0&4\\5&-2\end{pmatrix}\Bigg]\begin{pmatrix}1\\3\end{pmatrix}=\begin{pmatrix}12\\-1\end{pmatrix}$|
|:::|$\begin{pmatrix}1&2\\3&1\end{pmatrix}\begin{pmatrix}1\\3\end{pmatrix}\!+\!\begin{pmatrix}-1&2\\2&-3\end{pmatrix}\begin{pmatrix}1\\3\end{pmatrix}=\begin{pmatrix}7\\6\end{pmatrix}\!+\!\begin{pmatrix}5\\-7\end{pmatrix}=\begin{pmatrix}12\\-1\end{pmatrix}$|
|$(AB)^T=B^TA^T$|$\Bigg[\begin{pmatrix}1&-2\\1&-1\end{pmatrix}\begin{pmatrix}2\\1\end{pmatrix}\Bigg]^T=\Bigg[\begin{pmatrix}0\\1\end{pmatrix}\Bigg]^T=\begin{pmatrix}0&1\end{pmatrix}$ |
|:::|$\begin{pmatrix}2\\1\end{pmatrix}^T\begin{pmatrix}1&-2\\1&-1\end{pmatrix}^T=\begin{pmatrix}2&1\end{pmatrix}\begin{pmatrix}1&1\\-2&-1\end{pmatrix}=\begin{pmatrix}0&1\end{pmatrix}$|
Note that **matrix multiplication is not commutative**, i.e. even if both the matrices $AB$ and $BA$ do exist, in general
$$AB\neq BA.$$
[[linear_algebra:02_equations_and_matrices]]