LINEAR ALGEBRA (Part-5) - Linear Dependence

LINEAR ALGEBRA (Part-5)

Linear Dependence

Coefficient

It is a scalar number used to multiply a variable.
Like 6z means 6 times z, and "z" is a variable, so 6 is a coefficient.

Linear Combination

If we take a set of matrices then multiply each of them by a scalar/coefficient and add all of them together then the resultant matrix is a linear combination of the set of matrices.
Note that all the matrices involved in a linear combination need to have the same dimension (otherwise matrix addition would not be possible).

Let's consider n number of matrices A₁,A₂,......,A_n. all has same dimensions.
Lets also consider n scalars/coefficients α₁,α₂,......,α_n
So, linear combination B will be B = α₁A₁ + α₂A₂ + ...... + α_nA_n
Let's see few ways of calculation linear Combination / coefficients.

Way 1

Holding the equation, B = α₁A₁ + α₂A₂ + ...... + α_nA_n.
We consider n=2, α₁=2, α₂=-1 and

A1 =	[
		0	2
		1	1
	]
A2 =	[
		3	0
		1	1
	]

Here, α₁A₁ and α₂A₂ will be

α1A1 =			[						[							[
	2	❌		0	2		=			2❌0		2❌2		=			0		4
				1	1					2❌1		2❌1					2		2
			]						]							]
α2A2 =			[													[
	-1	❌		3	0		=			-1❌3		-1❌0		=			-3		0
				1	1					-1❌1		-1❌1					-1		-1
			]													]

So, B = α₁A₁ + α₂A₂ will be

B =	[							[							[							[
		0		4		✚			-3		0		=			0-3		4-0		=			-3		4
		2		2					-1		-1					2-1		2-1					1		1
	]							]							]							]

Way 2

Holding the equation, B = α₁A₁ + α₂A₂ + ...... + α_nA_n.
We consider n=3, A₁ = [0 1 2], A₂ = [1 0 -1], A₃ = [1 0 0] and B = [1 1 1]

Here, α₁A₁, α₂A₂ and α₃A₃ will be
α₁A₁ = α₁ ❌ [0 1 2] = [0 α₁ 2α₁]
α₂A₂ = α₂ ❌ [1 0 -1] = [α₂ 0 -α₂]
α₃A₃ = α₃ ❌ [1 0 0] = [α₃ 0 0]
So, B = α₁A₁ + α₂A₂ + α₃A₃ will be,
B = [0 α₁ 2α₁] + [α₂ 0 -α₂] + [α₃ 0 0]
B = [0+α₂+α₃ α₁+0+0 2α₁-α₂+0] = [α₂+α₃ α₁ 2α₁-α₂]
as B=[1 1 1] so we get, B=[1 1 1]=[α₂+α₃ α₁ 2α₁-α₂]
α₂+α₃=1
α₁=1
2α₁-α₂=1
Putting α₁ = 1 in 2α₁-α₂ = 1 we get,
2❌1-α₂ = 1
α₂ = 2-1 = 1
Putting α₂ = 1 in α₂+α₃=1 we get,
1+α₃ = 1
α₃ = 1-1 = 0
So the coefficients α₁ = 1,α₂ = 1 and α₃ = 0.

Way 3

Holding the equation, B = α₁A₁ + α₂A₂ + ...... + α_nA_n.
We consider n=3, A₁ = [1 1 1], A₂ = [1 1 1], A₃ = [1 1 1] and B = [3 3 3]

Here, α₁A₁, α₂A₂ and α₃A₃ will be
α₁A₁ = α₁ ❌ [1 1 1] = [α₁ α₁ α₁]
α₂A₂ = α₂ ❌ [1 1 1] = [α₂ α₂ α₂]
α₃A₃ = α₃ ❌ [1 1 1] = [α₃ α₃ α₃]
So, B = α₁A₁ + α₂A₂ + α₃A₃ will be,
B = [α₁ α₁ α₁] + [α₂ α₂ α₂] + [α₃ α₃ α₃]
B = [α₁+α₂+α₃ α₁+α₂+α₃ α₁+α₂+α₃]
as B=[3 3 3] so we get, B=[3 3 3] = [α₁+α₂+α₃ α₁+α₂+α₃ α₁+α₂+α₃]
α₁+α₂+α₃ = 3
α₁+α₂+α₃ = 3
α₁+α₂+α₃ = 3
Here the coefficients can be many different values Like,
α₁ = 1, α₂ = 1 and α₃ = 1
α₁ = 1, α₂ = 0 and α₃ = 2
α₁ = 0, α₂ = 0 and α₃ = 3
α₁ = 2, α₂ = 1 and α₃ = 0
etc. etc.

Linear Dependence

Let V = { A₁, A₂, ... ,A_n} be a collection of vectors. If n > 2 and at least one of the vectors in V can be written as a linear combination of the others, then V is said to be linearly dependent otherwise linearly independent.

Span

It’s the Set of all the linear combinations of a number vectors.

SOURCE

1) Deep Learning (Ian Goodfellow, Yoshua Bengio, Aaron Courville)

2) https://www.mathsisfun.com/definitions/coefficient.html

3) https://www.statlect.com/matrix-algebra/linear-combinations

4) https://www.cliffsnotes.com/study-guides/algebra/linear-algebra/real-euclidean-vector-spaces/linear-independence

5) https://www.cliffsnotes.com/study-guides/algebra/linear-algebra/real-euclidean-vector-spaces/linear-combinations-and-span