Math21b, Spring 2009, Linear Algebra and Differential Equations,

Row and column picture

One of the most important insights of linear algebra is obtained from the role of columns and rows. The columns are helpful to describe the image, the rows helpful to describe the kernel.

The column picture tells that A x = x₁ v₁ + ... + x_n v_n is a linear combination of columns. The row picture tells that A x = 0 means that x is perpendicular to the rows of A. One can state this as

 ker(A)^T = im (A^T)

which relates the transposed matrix A^T of the matrix A with the orthogonal complement V^T of the column space V.

Data

Duality reflects also in statistics If you have two data sets like x = (1,3,1,-2,3) and y = (2,1,-5,5,1). Then there are two ways to describe these data. The first is to see the data as two vectors in 5 dimensional space. The cosine of the angle between X = x-E[x] and Y = y-E[y] is called the correlation of the data. The second dual picture is to visualize the data points (1,2),(3,1),(1,-5),(-2,5),(3,1) in the plane. The regression line passes through the point E[x],E[y] and has the slope m = cos(alpha) ||Y||/||X|| = (X.Y)/||X||² because Y-m X is minimized if m X is the projection of Y onto X Again, if the data are written as a matrix, then the two pictures correspond to taking the transpose of the matrix.

A = |  1  2 | 
    |  3  1 |
    |  1 -5 |
    | -2  5 |
    |  3  1 |

A^T = 
    | 1 3  1 -2 3 |
    | 2 1 -5  5 1 |