harvard logo

Mathematics 21b, Spring 2006

Linear Algebra and Differential Equations

course head: Clifford Taubes
office: science center 504
email: chtaubes
office hours: Tue 10-11:30, Fri 1:30-2:30


Syllabus

The subject: This is a course on linear algebra and differential equations with a special section that also introduces statistical techniques as used in the sciences. As everyone in the course will be learning linear algebra and learning about differential equations, an introductory digression is in order to explain what these subjects are about and why they should be understood by practicing scientists. These discussions constitute Parts 1 and 2 of the digression. The digression has a third part that says some things about the role of statistics in the sciences.
Part 1: The digression starts with the following definition of science:

The purpose of science is to predict future behavior from present circumstance.


Here is a somewhat simplistic elaboration:

The goal of a scientific program is to predict the outcome of experiments done in the future from some prescribed amount of presently known data.


For example, Watson and Cricks original proposal for genetic inheritance asserts that knowledge of the sequence of bases (adenine, guanine, cytosine, or thymine) along the DNA in a living cell is sufficient to predict the sorts of proteins that the cell can produce. Watson and Crick made this proposal based on their understanding from experiments of the workings of a cell, and then further experimental work subsequently verified that their proposal is fundamental for understanding how cells encode their intrinsic operating instructions.
By the way, note how theory and experiment work hand in hand: Experiment tells us the present state of the world, theory provides the prediction for the future, and future experiments either falsify the theory or are consistent with its predictions. In this regard, a scientific theory must be falsifiable. To a first approximation, a scientific theory can be viewed in the following abstract manner: The known data constitutes a labeled collection of numbers, x = (x1, ..., xn); here and below, integer subscripts play the role of labels. Meanwhile, data that arises from future experiments can always be labeled so as to constitutes a second ordered set of numbers, (y1, . . . , yN) from the initial data (x1, ..., xn). Of course, the latter are not known until the experiments are carried out. Granted this notation, a theory in science must give a well defined and reproducible method for predicting the future data, (y1, . . ., yN), from the initial data, (x1, . . ., xn). Thus, a theory can be viewed as a function that assigns an ordered collection of N numbers (the y's) to the collection of n numbers (the x's). Of course, these experiments, once performed, provide a labeled set of N real data values, (y1real,... , yNreal); and if each ykreal is close to its predicted value, yk, then the theory can be said to be an accurate description of reality. Of course, if some ykreal is far from its prediction, yk, the theory needs some revising. The simplest non-constant functions are the linear functions; these have the schematic form 

 y1=a11 x1+a12 x2+... +a1nxn 
 ...
 ...
 yN=aN1 x1+aN2 x2+...+aNn xn


where the collection aij are numbers. Thus, a scientific theory that is based on such a linear function would have to specify the collection aij and then, the knowledge of the input data (x1, . . ., xn) predicts the output (y1, . . . , yN), of the future experiments using the preceding equation.
As it turns out, the linear functions are among the most relevant to the sciences. This is because an appropriate linear function usually provides the mathematical equivalent of a `first approximation' for a predictive description of any given phenomena. In any event, a good grasp of the mathematics of linear functions is a prerequisite for further explorations because the techniques that are used to study more complicated functions employ most of the mathematics for linear functions.
The subject of Linear algebra concerns the mathematics of linear functions.


Part 2: It is often the case that the quantity of interest in an experiment can be viewed as a function of some auxiliary variable. Indeed, consider the case when the quantity of interest changes with time, and so can be viewed as a function of time. Think of time as a variable, t, that can take values on the real line, and then the quantity of interest at time t can be written as a function of t. This is to say that there is a function u(t) of one variable t, and the values of this function at time t are defined to be those of the quantity of interest. Here is a hypothetical example: Radioactive iodine is inadvertently dumped in a reservoir at time t=today. The concentration of iodine in the reservoir at any given future time can be called u(t), and so the assignment t to u(t) defines a function of time.
In any case, a full theoretical understanding of the time varying behavior of what is represented abstractly by one or several functions of time entails predicting their values at future times from present data. In the reservoir example, the present data might consist of the concentration of iodine measured today and, in addition, the average rates of inflow, outflow and evaporation of water from the reservoir.
As it turns out, theoretical predictions for the future behavior of real world quantities are often expressed via a system of equations that relate the rate of change of the quantities of interest at any given time to the values of the quantities at the same time. For example, such a theory for predicting the time dependence of some quantity that is modeled by the function u(t) would have the form

du/dt = f(u),


where the function f is specified by the theory. Such an equation is a simple example of a differential equation. In general, a differential equation can be said to be any equation for a function or set of functions that constrains the functions and their derivatives in some specified manner. The subject of differential equations concerns techniques for finding solutions to differential equations. The subject also concerns techniques for estimating properties of interest of hypothetical solutions without knowledge of their explicit form.
Linear algebra enters the differential equation story in the following way: Just as linear functions are good first approximations for modeling many phenomenon, so differential equations that involve linear functions also offer reasonable first approximations in many situations. As is explained in this course, differential equations that involve only the linear functions can almost always be solved by closed form expressions using linear algebra. In this regard, note that most of the known techniques for dealing with non-linear differential equations involve generalizations of those that work for the class of linear differential equations.
Because of this relation between the subject of linear algebra and the subject of differential equations, this course studies these subjects together.


Part 3: As I remarked at the outset, there is a special section in this course that focuses not just on linear algebra and differential equations, but on statistics as well. Below, I lay out a guide as to who should be in the special section. But first comes a discussion of the role of statistics in the sciences. To start, statistics and its partner, probability, constitute the mathematics of uncertainty. To elaborate, note that uncertainty arises in the sciences in three ways: First, it is the rare measurement that can be done with absolute precision. This is to say that repeated measurements of a particular item will typically differ. Granted this uncertainty, one role of statistics is to supply an answer to the following question: Are the theoretical predictions consistent with the output data given that both output and input measurements are knowingly imprecise? Uncertainty also appears because most theoretical models do not take into account every possible factor. Indeed, theories must discard factors so as to insure tractable calculations. The strategy then is to discard factors with presumably small influence while estimating the error in having done so. This done, statistics provides the mathematics for estimating such errors.
Finally, certain natural phenomena have an inherent probabilistic nature. A prime example comes from physics where quantum mechanics dictates that the submicroscopic world is ruled not be deterministic laws but by probabilistic ones. To summarize:

Statistics provides the tools for incorporating uncertainties in the evaluation of a given scientific theory.


Meanwhile, probability provides various theoretical models for how the values of repeated measurements should distribute themselves. Linear algebra and statistics are related for the very simple reason that many results in statistics can be formulated using linear equations. For example, the spread in the suite of values from repeated measurements of a particular system is an important clue as to how the system works. As it turns out, the relative frequencies of appearances of the various possible values from such measurements is often a linear function of the relative frequencies of the various values that can appear by repeatedly measuring the input data. By input data, I mean things such as the temperature at which the experiment is carried out, the ambient pressure, the precise concentration of various chemicals used, etc. Note that this is not to say that the output values are linear functions of the input values. Rather, the assertion is that the frequency of the occurrence of any particular output value is often a linear function of the frequencies of the occurrence of the input values.
Differential equations enter the story here by virtue of the fact that it is often the case that the frequencies for the appearances of various real world quantities are time dependent. This said, the theoretical predictions for such time dependence are most often obtained using a differential equation for the functions whose values at a given time represent these same frequencies.


The sections: This course is taught in relatively small, separately meeting sections rather than in a big and rather impersonal room all together. This means that each of you will be assigned a section based on your request (instructions for how to make this request are given below). You then attend all meetings of your assigned section, hand in your homework to the assigned section, and collect your graded homework and graded exams from the assigned section. Most sections will be of size on the order of 25 students.
All sections are deemed regular sections except for one specially named section, the bio/statistics section. The various regular sections teach the same material on the same week to week schedule, give the same weekly homework assignments, and take the same version of the midterm exam and final exam. If you opt for a regular section, choose the one whose scheduled meeting time best fits with your schedule.
As just noted, there is one very special section of this course, the bio/statistics section. The curriculum as planned for the bio/statistics section differs substantially from that of the regular sections starting from the very beginning. Thus, transfer to and from the bio/statistics section after the first week of lectures is very much discouraged. I will be teaching the bio/statistics section from 11:30-1 on Tuesdays and Thursdays.
What follows tersely summarizes the curricula of the two sorts of sections:
  • The regular section covers a great deal of linear algebra. Any one completing a regular section of this course will see all of the basics of linear algebra that might be required for higher level courses in the physical sciences, social sciences, and in mathematics, both pure and applied. In particular, the regular sections teach
    • methods to solve systems of linear equations,
    • methods to analyze and solve systems of linear differential equations,
    • methods to solve discrete linear dynamical systems such as Markov processes,
    • least square fitting of data with arbitrary function sets,
    • the basics of Fourier series and its use to solve partial differential equations.
  • The bio/statistics sections will cover less linear algebra than the regular sections, providing instead basics of statistics and probability. Moreover, the linear algebra and also the differential equations will be taught using examples from statistics and probability, with many coming from applications in the life sciences and in bioinformatics. In this regard, no apriori background in the life sciences is required here; this section makes sense for those with interests in other physical sciences and in the social sciences as well.
    Those in the bio/statistics section of this course will see the core of the linear algebra and differential equations from the regular sections, and also see enough statistics and probability to take various higher level statistics courses (such as Stat 111 and Stat 139).
By the way, successful completion of any section of this course will provide you with a level of sophistication in mathematics that you will not get by learning linear algebra elsewhere, either at Harvard or at any other institution. This sophistication alone will serve you well in your future science courses, and in your future scientific career. Syllabi for the two kinds of sections are provided below.


Math 21b Regular section syllabus. The course will cover topics from the following chapters of Linear Algebra with Applications by Bretscher and from the indicated supplementary material. 

 Chapter 1.1:                 Introduction to linear systems.
 Chapter 1.2:                 Matrices and Gauss-Jordan elimination
 Chapter 1.3:                 On solutions to linear systems
 Chapter 2.1:                 Linear transformations and their derivatives
 Chapter 2.2:                 Linear transformations in geometry
 Chapter 2.3:                 Inverse of a linear transformation
 Chapter 2.4:                 Matrix products
 Chapter 3.1:                 Image and kernel
 Chapter 3.2:                 Subspaces, bases and linear independence
 Chapter 3.3:                 Dimension
 Chapter 3.4:                 Coordinates
 Chapter 5.1:                 Orthonormal bases and orthogonal projections
 Chapter 5.2:                 Gram-Schmidt and QR factorization
 Chapter 5.3:                 Orthogonal transformations
 Chapter 5.4:                 Least squares and data fitting
 Chapter 6.1:                 Introduction to determinants
 Chapter 6.2:                 Determinants
 Chapter 7.1:                 Introduction to eigenvalues
 Chapter 7.2:                 Eigenvalues
 Chapter 7.3:                 Eigenvectors
 Chapter 7.4:                 Diagonalization
 Chapter 7.5:                 Complex eigenvalues
 Chapter 7.6:                 Stability
 Chapter 8.1:                 Symmetric matrices
 Chapter 9.1:                 Introduction to differential equations
 Chapter 9.2:                 Differential equations
 Chapter 9.4:                 Nonlinear systems
 Handout:                     Function spaces
 Chapter 9.3:                 Differential operators
 Handout:                     Fourier series
 Handout:                     Introduction to partial differential equations
 Handout:                     Partial differential equations


Math 21b Bio/statistics section syllabus. The bio/statistics section syllabus will cover the linear algebra topics, and the probability and statistic topics that are listed below. The linear algebra material will come from the following chapters of Linear Algebra with Applications by Bretscher. 

 Chapter 1.1:                 Introduction to linear systems.
 Chapter 1.2:                 Matrices and Gauss-Jordan elimination
 Chapter 1.3:                 On solutions of linear systems
 Chapter 2.1:                 Linear transformations and their derivatives
 Chapter 2.2:                 Linear transformations in geometry
 Chapter 2.3:                 Inverse of a linear transformation
 Chapter 2.4:                 Matrix products
 Chapter 3.1:                 Image and kernel
 Chapter 3.2:                 Subspaces, bases and linear independence
 Chapter 3.3:                 Dimension
 Chapter 3.4:                 Coordinates
 Chapter 5.1:                 Orthonormal bases and orthogonal projections
 Chapter 5.2:                 Gram-Schmidt and QR factorization
 Chapter 5.3:                 Orthogonal transformations
 Chapter 5.4:                 Least squares and data fitting
 Chapter 6.1:                 Introduction to determinants
 Chapter 6.2:                 Determinants
 Chapter 7.1:                 Introduction to eigenvalues
 Chapter 7.2:                 Eigenvalues
 Chapter 7.3:                 Eigenvectors
 Chapter 7.4:                 Diagonalization 
 Chapter 7.5:                 Complex eigenvalues
 Chapter 7.6:                 Stability
 Chapter 8.1:                 Symmetric matrices
 Chapter 9.1:                 Introduction to differential equations
 Chapter 9.2:                 Differential equations
 Chapter 9.4:                 Stability in nonlinear systems


The probability and statistics topics will include material of the following sort:

  • Introduction to probability: Set theoretic definitions, conditional probability, independence, Bayes' theorem.
  • Common discrete and continuous probability distributions, their properties, and their use
  • Random variables and associated constructions.
  • Independence and dependence, correlation matrices.
  • The Chebychev Theorem and its uses: Why we care about the mean and standard deviation.
  • Central limit theorem and its uses.
  • Transition matrices and Markov chains.
  • Use of data to estimate model parameters (point estimation)
  • Use of data to derive probability distributions for model parameters.
  • Statistical tests for `random' fluctuations.
  • Testing for dependent data.
  • Fitting lines and curves to data.
  • Identifying functional dependencies in a large dimensional data set.


As noted in the introduction, most of these topics will be introduced in a life science context, but no specific life science background is required.




Math21b, Linear Algebra and Applications, Spring 2006, Clifford Taubes, Harvard University, Faculty of Arts and Sciences, Harvard University