ORTHOGONALITY AND LEAST SQUARES

Orthogonal Vectors Norm Unit Vector Orthonormal Vectors Orthogonal Complement Orthogonal Projection Cauchy-Schwarz Inequality Angle Between Two Vectors Gram-Schmidt Process QR Factorialization Transpose Symmetric and Skew-symmetric Matrices Orthogonal Matrices Least Squares Solutions Matrix of an Orthogonal Projection

Least Squares Solutions How to Examples Exercise How to: When we have an equation such as A*x = b, where A is a matrix, b is a vector, and x is our unknown, we can put A and b into an augmented matrix and solve it by putting it into reduced row-echelon form. However, you know that sometimes when solving an augmented matrix we find an inconsistency, that is a row that looks like this: '0 0 ... 0 : 1'. The least-squares solutions of an equation are the best approximations of x that can be found. This approximation is sometimes denoted x*. To find the least-squares solutions we use what is called the normal equation:      A'*A*x = A'*b Here, A' denotes the transpose of A. This normal equation is guaranteed to always be consistent (see book for proof). In the case where the kernal of A is the zero vector, then the product A'*A will be invertible and we can find a unique least-squares solution:      x = inv(A'*A)*A'*b      Examples: Exercise: