# Find the inverse of a matrix [[2, 1], [3, 4]]
import numpy as np
A = np.array([[2, 1], [3, 4]])
A_inv = np.linalg.inv(A)
print(A_inv)[[ 0.8 -0.2]
[-0.6 0.4]]Scalar
Scalar is a single number.
Example: x = 6
Vector
Vector is a one-dimensional array of numbers.
Example: x = [1, 2] or usually written as
\[ \begin{bmatrix} 1 \\ 2 \end{bmatrix} \]
Multiplication of Scalar and Vector
Vector can be multiplied by a scalar. For example, if we multiply the vector above by 2, written as
\[ 2 \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \]
then the result is
\[ \begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix} \]
Addition of Scalar and Vector
This operation is not defined. Scalar can’t be added to vector.
Addition of Vectors
Vectors can be added together
\[ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} + \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix} = \begin{bmatrix} 5 \\ 7 \\ 9 \end{bmatrix} \]
When the dimension of the vectors are different, the addition is not defined. The following is NOT allowed
\[ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} + \begin{bmatrix} 4 \\ 5 \\ \end{bmatrix} \]
Vector dot product
Vector dot product is a way to multiply vectors together. The result is a scalar.
\[ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \cdot \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix} = 1 \times 4 + 2 \times 5 + 3 \times 6 = 32 \]
Not:
\[ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \cdot \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix} = \begin{bmatrix} 1 * 4 \\ 2 * 5 \\ 3 * 6 \end{bmatrix} \]
Matrix
Matrix is a two-dimensional array of numbers. Matrix is usually written as
\[ \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \]
Matrix dimension
The dimension of a matrix is the number of rows x columns. The above matrix has dimension of 2 x 3.
Relation to Vector
A vector is a matrix with only one column. The following vector is a matrix with dimension of 3 x 1
\[ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \]
Matrix addition
Matrix can be added together if they have the same dimension. For example, the following is allowed
\[ \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} + \begin{bmatrix} 7 & 8 & 9 \\ 10 & 11 & 12 \end{bmatrix} = \begin{bmatrix} 8 & 10 & 12 \\ 14 & 16 & 18 \end{bmatrix} \]
Matrix dot product
Matrix dot product is defined as follows
\[ \begin{bmatrix} x_{00} & x_{01} \\ x_{10} & x_{11} \end{bmatrix} \cdot \begin{bmatrix} y_{00} \\ y_{10} \end{bmatrix} = \begin{bmatrix} x_{00} y_{00} + x_{01} y_{10} \\ x_{10} y_{00} + x_{11} y_{10} \end{bmatrix} \]
For example:
\[ \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \cdot \begin{bmatrix} 5 \\ 6 \end{bmatrix} = \begin{bmatrix} 1 \times 5 + 2 \times 6 \\ 3 \times 5 + 4 \times 6 \end{bmatrix} = \begin{bmatrix} 17 \\ 39 \end{bmatrix} \]
If X dimension is m x n and Y dimension is n x p, then the result of X dot Y is m x p.
So, m x n dot o x p is not defined if n is not equal to o. Example
\[ \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \cdot \begin{bmatrix} 5 & 6 \\ 7 & 8 \\ 9 & 10 \end{bmatrix} = undefined \]
The order of the dot product is important.
\[ A \cdot B \neq B \cdot A \]
For example, if we flip the order of the above dot product, we get
\[ \begin{bmatrix} 5 \\ 6 \end{bmatrix} \cdot \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} = undefined \]
Matrix dot product is associative:
\[ A \cdot (B \cdot C) = (A \cdot B) \cdot C \]
Matrix Transpose
Matrix transpose is a matrix that is flipped over its diagonal. For example, the transpose of the following matrix is
\[ \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}^T = \begin{bmatrix} 1 & 3 \\ 2 & 4 \end{bmatrix} \]
\[ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}^T = \begin{bmatrix} 1 & 2 & 3 \end{bmatrix} \]
Identity Matrix
Identity matrix is a matrix that doesn’t change any vector when multiplied by that vector
\[ I \cdot \begin{bmatrix} x_0 \\ x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} x_0 \\ x_1 \\ x_2 \end{bmatrix} \]
Example of 2 x 2 identity matrix
\[ \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \]
Example of 3 x 3 identity matrix
\[ \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \]
Inverse Matrix
Inverse matrix is a matrix that when multiplied by the original matrix, results in an identity matrix:
\[ A \cdot A^{-1} = I \]
Linear Equation
- The price of 2 apples and 1 orange is 5 dollar.
- The price of 3 apples and 4 oranges is 10 dollar.
- What is the price of 1 apple and 1 orange?
\[ 2x + 1y = 5 \]
\[ 3x + 4y = 10 \]
It can modeled as a matrix equation
\[ \begin{bmatrix} 2 & 1 \\ 3 & 4 \end{bmatrix} \cdot \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 5 \\ 10 \end{bmatrix} \]
Let’s multiple both side with the inverse of the matrix
\[ \begin{bmatrix} 2 & 1 \\ 3 & 4 \end{bmatrix}^{-1} \cdot \begin{bmatrix} 2 & 1 \\ 3 & 4 \end{bmatrix} \cdot \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 3 & 4 \end{bmatrix}^{-1} \cdot \begin{bmatrix} 5 \\ 10 \end{bmatrix} \]
\[ I \cdot \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 3 & 4 \end{bmatrix}^{-1} \cdot \begin{bmatrix} 5 \\ 10 \end{bmatrix} \]
\[ I \cdot \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0.8 & -0.2 \\ -0.6 & 0.4 \end{bmatrix} \cdot \begin{bmatrix} 5 \\ 10 \end{bmatrix} \]
\[ \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} \]
Features can be represented as vectors and matrices
In machine learning, we can represent features as vectors. For example, if we have a dataset of 100 people, and each person has 3 features: age, height, and weight, then we can represent each person as a vector of 3 elements.
\[ person = \begin{bmatrix} age \\ height \\ weight \end{bmatrix} \]
and we can represent the whole dataset as a matrix of 100 rows and 3 columns.
\[ people = \begin{bmatrix} person_{1}^{T} \\ person_{2}^{T} \\ \vdots \\ person_{100}^{T} \end{bmatrix} = \begin{bmatrix} age_{1} & height_{1} & weight_{1} \\ age_{2} & height_{2} & weight_{2} \\ \vdots & \vdots & \vdots \\ age_{100} & height_{100} & weight_{100} \end{bmatrix} \]
Sigma Notation
Sigma notation is a way to write a sum of many terms. For example, the following is a sum of 5 terms
\[ \sum_{i=1}^{5} x_{i} = x_{1} + x_{2} + x_{3} + x_{4} + x_{5} \]
In a code it’s just a for loop
sum = 0
for i in range(1, 6):
sum += x[i]Product Notation
Product notation is a way to write a product of many terms. For example, the following is a product of 5 terms
\[ \prod_{i=1}^{5} x_{i} = x_{1} \times x_{2} \times x_{3} \times x_{4} \times x_{5} \]
In a code it’s just a for loop
product = 1
for i in range(1, 6):
product *= x[i]