21 B
Mathematics Math21b Spring 2007
Linear Algebra and Differential Equations
Course Head: Oliver Knill
Office: SciCtr 434
Send questions of potential general interest to

Question:What was the solution of the DVD challenge in the final review? Answer: the differential equation was
 f" +f = 2sin(t)
This is a case, where the right hand side is a homogenous solution of
 f'' + f = 0
The cookbook method tells to try A t cos(t) + B t sin(t). A solution is
 - t cos(t)
The solver of the problem used complex methods to solve it. Something like
 t(-e^it - e^-it)/2
or this plus a solution of the homogeneous solution. Note that any solution plus a homogeneous solution is also a special solution.
Question:What is the difference between the matrices
 A= | .2  .4 |
    | .4, .8 |
 B= | 1   0 |
    | 2   0 |   ?
Aren't they both projection matrices onto the line spanned by the vector [1,2]T?
Answer: Yes, both are projections. We have A2 = A and B2=B. But only A is an orthogonal projection, the second one is not. The second has the vector [0,1]T in the kernel and not a vector perpendicular to [1,2]T. We usually look at orthogonal projections.
Question:Does the song in the mathematica project has to be original? Answer:
Absolutely not. Here is an example, with a different sound hull:
You can hear the song with the sound player to the right. You might get error messages with the code provided in the notebook. If that is the case, please change the definition of songlengh to:
Question:What is the difference between a projection and a orthogonal projection like in TF 9 of practice exam 3? Answer: We consider only orthogonal projections. If you take an orthogonal projection A and conjugate it to a similar matrix B = S^-1 A S, then B has a kernel which is no more perpendicular to A. One can still call it a projection. For example
  | 1 1 |
  | 0 0 |
projects on the x-axes, but the kernel is [-1,1]. The orthogonal projection on the x axes is
  | 1 0 |
  | 0 0 |
Question:For a problem in the review as well as problem 3a in practice test 3, in part b, it was bluntly stated that the eigenvalue of 10 matches [1 1 1 1 1 1]. Also, in yesterdays review, you said the the eigenvalue was 31 for the matrix with 6's across the diagonal and 5 everywhere else. How did you so easily get these eigenvalues? Answer: If the sum of the entries in each row is constant then [1,1,1....1] is an eigenvector and the eigenvalue is the sum in each row. For example, one sees immediately that 7 is an eigenvalue of
 |  2 5 |
 |  6 1 |
If you have a 6x6 matrix with 5 everywhere and 6 in the diagonal, then the sum of the entries over each row is 31. So, the eigenvalue is 31.
Question:Can you explain again how we get the eigenvalues and eigenvectors of a permutation matrix like in practice exam 1 question 3? Answer: The case when the characteristic polynomial is
   f(L) = L^n-1=
is rather special. It means that the eigenvalue L is a n'th root of 1. All these roots are located on a regular hexagon on a circle of radius 1. They are given by
   L = e 2 pi i k/n 
Lets say n=6 in the following: if we write
  v = | v1 |
      | v2 |
      | v3 |
      | v4 |
      | v5 |
      | v6 |
we have
                 | v2 |        | v1 |
                 | v3 |        | v2 |
           A v = | v4 |   = L  | v3 |
                 | v5 |        | v4 |
                 | v6 |        | v5 |
                 | v1 |        | v6 |
you know that for the eigenvalue L, we have for the coefficients of the eigenvectors:
    v2=L v1, v3=L v2, v4=L v3, v5= L v4, v6=L v1
We can chose v1=1 because eigenvectors can be scaled and still are eigenvectors. This means that
                | L0 |
                | L1 |
                | L2 |
           v =  | L3 |
                | L4 |
                | L5 |
is an eigenvector to the eigenvalue L. We have 6 eigenvalues and for each eigenvalue such an eigenvector. See also problems 25,38 in the book. For any permutation matrix we can get an eigenbasis like this.
Question:To the announcement that in chapter 7.4, we do not cover pages 335-338. Does this mean that the section about eigenvalues of linear transformations will not be covered in the second midterm? Answer: It says what it says: we do not cover the material of pages 335-338. That section deals with egienvalues of general linear transformations which we did not cover yet (section 4.2). Of course you have to know about eigenvalues and eigenvectors of linear transformations on R^n which are given by matrices. That will be an integral part of the exam. But the pages under consideration deal with eigenvalues and eigenfectors of transformations like T(f) = f', the derivative which is a linear transformation, but a linear transformation on an infinite dimensional space.
Question:What happens if ATA is not invertible in the least square solution x* = (ATA )-1AT b of A x = b? Answer: The square matrix ATA is not invertible if and only if it has a nontrivial kernel and this is the case if and only if A has a nontrivial kernel. In that case, the least square solution is not unique. If x is a least square solution and Ay=0, then also (x+y) is a least square solution because the distances |Ax-b| = |A(x+y)-b| are the same.
Remark: In applications, A AT is always invertible. A data fitting situation, where A AT is not invertible would be, if we wanted to fit a set of data with functions 1,x,x^2,x+x^2 for example. Since the forth function is a linear combination of the second and third function, we have ambuiguty with fitting.
Question:Is the trivial space X={0} a linear space? Answer: yes, all three properties of a linear space are satisfied: it contains 0, if x is in X, then r X is in X for every r and if x,y are in X, then x+y are in X. (there is just not a lot of choice for the last two conditions).
Question:To question practice exam TF 20: could ker(T) equal im(T) if the transformation were from R4 to R4? Answer: The dimension formula does not prevent this and indeed:
       |  0 1 0 0 |
       |  0 0 0 0 |
       |  0 0 0 1 |
       |  0 0 0 0 |
has the property that ker(T)=im(T). Now you can maybe see that ker(T)=im(T) is possible if the dimension of the space is even and not possible if the dimension of the space is odd.
Question:Is there any way to tell what the plane equation is, given two vectors? Like, for instance, the equation for the plane spanned by say <5,0,-1,2> and <8,1,4,4>? Answer: In four dimensions, a 2 dimensional plane is given by two equations. These equations mean that the vector [x,y,z,w]T is in the kernel of the matrix
 A = |5,0,-1,2|
     |8,1,4,4 |
 rref(A) = | 1,0,-1/5,2/5|
           | 0,1,28/5,4/5|
the kernel satisfies
 y+z 28/5 + w 4/5 = 0
 y=-z 28/5 -w 4/5
so that
 |x|     | 1/5 |      |-2/5|
 |y|     |-28/5|      |-4/5|
 |z| = z | 1   |  + w |  0 |
 |w|     | 0   |      |  1 |
and the kernel is spanned by [1,-28,5,0] and [-2,-4,0,5]. The equations for your plane are
 x-28y+5z =0
Geometrically, this is the intersection of 2 three dimensional hyperplanes in R4. Cool, isn't it? In three dimensions, we operated with the awkward cross product to get the equation. We know now how to deal with such questions in 2007 dimensions too, where the cross product is an object in a 2007x2006/2 = 2013021 dimensional space and no more a vector in the same space.
Question:I was reviewing the homework solutions from lesson 2.2, and I was wondering, is there a difference between parallel projections and orthogonal projections? Answer: We use the notion "orthogonal projection". It is sometimes called "parallel projection" emaphases the kernel of the projection. The name "orthogonal projection" emphases that we project orthogonally onto the image of the transformation.
Question:Is a 1x1 matrix a scalar or is it considered as a Matrix? Answer: It is both. It is a scalar but also a 1x1 matrix. Matrices form a larger class of objects than scalars. It is still common to call it a scalar, like one calls a 1xn matrix a row vector or a nx1 matrix a column vector.
Question:Which page should I bookmark, the iSites page or the FAS page? Answer: It is safe to bookmark the FAS page. We will make links to all of the content from there. It is also possible to use the hybrid version, where the actual course website is contained within a iFrame. While content managment systems (CMS) like iSites have many advantages (like that websites can be maintained by everybody), there are also serious drawbacks:
  • In a standard unix setup, one can sync entire directory trees in a fraction of a second using the Unix utilty "rsync". This allows to work on the page offline or tar it up. It does not matter whether one page or 1000 files have changed, unix tools works fast and are extremely reliable.
  • Writing with online editors to modify content is not the same than working with a real and familar text editor, which one can have syntax highlighting, which can be programmed to do some tasks, where one can search or match strings etc.
  • Links in CMS are often unintuitive and difficult to remember and communicate.
  • A standard Unix directory structure is search engine friendly and users can easly grab the entire page for offline reading.
  • Currently it is not possible to change the style and look of course pages in iSites.
Please send questions and comments to
Math21b | Oliver Knill | Spring 2007 | Department of Mathematics | Faculty of Art and Sciences | Harvard University