### Introduction

Many papers in statistics and quantitative finance make heavy use of linear algebra, so you need to have a working knowledge of it in order to read and apply them to your trading.

### Vectors

A vector can be thought of as an arrow pointing from the origin to a specific point. Any vector or point can be represented by its coordinates i.e. an array of numbers, such as for a 2-dimensional vector, or for a 3-dimensional one. We usually write a vector as a column:

The **scalar product** of two vectors and in 2-dimensional space is defined as:

This definition can be easily generalized to `n`

dimensional space. Clearly, we cannot take the scalar product of two vectors with different dimensions.

### Matrices

If we have a few vectors with the same dimension, then we can put them side-by-side to form a **matrix**. For example, the vectors

can be combined to produce a matrix:

m is a 3 × 3 matrix. We typically describe the dimensions of a matrix as where m = number of rows and n = number of columns.

A **square** matrix is one with as many rows as columns.

Notation: refers to a specific value in row and column of a matrix . For example, is the number in the second row and third column of .

#### Python Implementation

In Python, the NumPy package deals with linear algebra. The array we learned in the NumPy chapter can be deemed as a vector:

```
import numpy as np
a = np.array([1,2,3])
b = np.array([2,2,2])
c = np.array([3,1,1])
matrix = np.column_stack((a,b,c))
print(matrix)
print(type(matrix))
[out]:
[[1 2 3]
[2 2 1]
[3 2 1]]
[out]: <class 'numpy.ndarray'>
```

It is worth noticing that we used `column_stack()`

here to ensure that the vectors are vertical and placed side-by-side to form a matrix. Without the `column_stack()`

function, the vectors will be made horizontal and stacked on top of one another:

```
matrix2 = np.array([a,b,c])
print(matrix2)
[out]:
[[1 2 3]
[2 2 2]
[3 1 1]]
```

### Matrix Multiplication

How are two matrices multiplied? Suppose . Each entry of matrix is the scalar product of row from matrix with column from matrix . This is best illustrated with an example:

In NumPy, we can multiply matrices with the `dot()`

function:

```
A = np.array([[2,3],[4,2],[2,2]])
B = np.array([[4,2],[4,6]])
x = np.dot(A,B)
print x
[out]:
[[20 22]
[24 20]
[16 16]]
```

Since matrix multiplication is defined in terms of scalar products, the matrix product exists only if has as many columns as has rows. It's useful to remember this shorthand: (m × n) × (n × p) = (m × p) which means that an (m × n) matrix multiplied by an (n × p) matrix yields an (m × p) matrix.

Reversing the order of multiplication results in an error since B does not have as many columns as A has rows:

```
x = np.dot(B,A)
```

A natrual consequence of this fact is that matrix multiplication is **not commutative**. In other words, in general.

### Inverse

An **identity** matrix is a square matrix with ones on the main diagonal and zeros elsewhere. Here is an identity matrix:

Multiplying any matrix by an identity matrix (of the correct shape) is like multiplying a number by 1. Concretely, if is an matrix, then:

is the **inverse** matrix of a square matrix if:

Some caveats:

- A rectangular matrix will not have an inverse, but it may have a pseudoinverse (not covered in this tutorial).
- A square matrix may not have an inverse i.e. it may be "singular".
- If a square matrix has an inverse, then its inverse is unique.

Inverse matrices are computed using the Gauss-Jordan method. In NumPy, we use the `linalg.inv()`

function to do it:

```
print(matrix)
print('\n-------------------------\n')
print(np.linalg.inv(matrix))
[out]:
[[1 2 3]
[2 2 1]
[3 2 1]]
-------------------------
[[ 0. -1. 1. ]
[-0.25 2. -1.25]
[ 0.5 -1. 0.5 ]]
```

Now let's check if the multiplication is :

```
inverse = np.linalg.inv(matrix)
print(np.dot(matrix, inverse))
print('\n-------------------------\n')
print(np.dot(inverse,matrix))
[out]:
[[ 1.00000000e+00 -6.66133815e-16 6.66133815e-16]
[ 0.00000000e+00 1.00000000e+00 1.11022302e-16]
[ 0.00000000e+00 -2.22044605e-16 1.00000000e+00]]
-------------------------
[[ 1.00000000e+00 -4.44089210e-16 -2.22044605e-16]
[ 6.66133815e-16 1.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00]]
```

Not surprisingly, we ended up with an identity matrix. We can form a non-invertible matrix by making one of its rows a multiple of another:

```
singular = np.array([[1,2,3],[1,2,3],[3,3,3]])
inv = np.linalg.inv(singular)
[out]: numpy.linalg.linalg.LinAlgError: Singular matrix
```

### Linear Equations

A common problem in linear algebra is solving linear equations. Consider the following linear equations:

If we let:

Then the linear equations above can be written as

If A is invertible, then we can multiply on both sides of the equation to obtain the solution:

As long as exists, we can compute it to solve the linear equations:

```
A = np.array([[2,1,-1],[-3,-1,2],[-2,1,2]])
b = np.array([[8],[-11],[-3]])
inv_A = np.linalg.inv(A)
print np.dot(inv_A, b)
[out]:
[[ 2.]
[ 3.]
[-1.]]
```

The solution is `x`

= 2, `y`

= 3, `z`

= −1. However, computing the inverse matrix is not recommended, since it is numerically unstable i.e. small rounding errors can dramatically affect the result.

Instead, NumPy solves linear equations by LU decomposition:

```
print np.linalg.solve(A, b)
[out]:
[[ 2.]
[ 3.]
[-1.]]
```

Of course, we get the same solution. We can check the correctness of the solution by substituting `x`

, `y`

and `z`

into the linear equations.

### Summary

In this chapter we have introduced vectors, matrices, inverse matrices and linear equations. Some applications in finance include: finding arbitrage opportunities by solving linear equations, computing portfolio variance, etc. In the next chapter, we will introduce modern portfolio theory and CAPM.