### Linear Independence Vectors are linearly dependent (think redundant) if at least one of them is a linear combination of the others. In other words, the vectors $v_{1}, v_{2}, \dots v_{k}$ are linearly independent when $a_{1}v_{1}+a_{2}v_{2}+\dots+a_{k}v_{k}=0$ only holds if $a_{1}= a_{2} =\dots= a_{k}=0$ . ### Vector Operations ### Dot product: $a\cdot b =a^{\mathrm{T}}b =a_{1}b_{1}+a_{2}b_{2}+\dots+a_{n}b_{n} = \lvert a \rvert~\lvert b \rvert \sin\theta$ where $\theta$ is the angle between vectors. Also known as inner product. The outer product is $ab^{\mathrm{T}}$. ### Length $|v| = \sqrt{ v_{1}^{2}+v_{2}^{2}+\dots+v_{d}^{d} }$ ### Angle between two vectors $\cos\theta= \frac{ v\cdot w}{ \lvert v \rvert ~\lvert w \rvert}$ #### Cross Product The area/volume enclosed by the vectors—it tracks perpendicularity while the dot product measures parallelism. $\vec{a} \times \vec{b} = \lvert \lvert a \rvert \rvert ~ \lvert \lvert b \rvert \rvert \sin(\theta)n = \det\begin{bmatrix} \hat{i} & \hat{j} & \hat{k} \\ a_{1} & a_{2} & a_{3} \\ b_{1} & b_{2} & b_{3}\end{bmatrix} =\begin{bmatrix}0 & -a_{3} & a_{2} \\ a_{3} & 0 & -a_{1} \\ -a_{2} & a_{1} & 0 \end{bmatrix}\vec{b}$ - _θ_ is the angle between **a** and **b** in the plane containing them - ‖**a**‖ and ‖**b**‖ are the magnitudes of vectors **a** and **b** - **n** is a unit vector perpendicular to the plane containing **a** and **b** according to the right hand rule ### Inner Product Vectors $u$ and $v$ are orthogonal if their inner product is zero: $u \cdot v = \sum_{i=1}^{n}u_{i}v_{i}$ The inner product is symmetric and linear. Any set of non-zero vectors $\{ v_{1}, v_{2}, \dots , v_{k}\}$ that are pairwise orthogonal are linearly independent. *Proof:* $v_{1}\cdot \sum_{i=1}^{k}\alpha_{i}v_{i}=\alpha_{1} \lvert v_{1} \rvert^{2}=0$ for each $v_{1}$ so $\alpha_{i}$ are all zero ### Generalized pythagorean theorem Let $\{ v_{1}, v_{2}, \dots , v_{k}\}$ be pairwise orthogonal. Then $\lvert v_{1} + \dots + v_{k} \rvert^{2} = \lvert v_{1} \rvert^{2} + \dots + \lvert v_{k} \rvert^{2}$ *Proof:* expand left side as an inner product ### Cauchy-Schwartz Inequality $\lvert a^{\mathrm{T}}b \rvert \leq \lvert \lvert a \rvert \rvert ~ \lvert \lvert b \rvert \rvert$ for example, $(a_{1} + \dots + a_{n})^{2} \leq n(a_{1}^{2}+\dots+a_{n}^{2})$. This gives us the triangle inequality: $ \lvert \lvert u + v \rvert \rvert \leq \lvert u \rvert + \lvert v \rvert $