This study outline for the last part of the course provides a framework that you can fill in with your lecture notes, problem sets, and if needed the text on reserve in Cabot. To make this last part easier, we've indicated which section of the text each topic appears in.

Exam Format: You should expect the exam to be about 12 problems long and you should expect about 3 of those problems to be on probability.

Practice Problems:We have also posted some probability practice problems and answers for you here.

Review Session: The probability review will be on Thursday, Jan 15th at 4-5:30pm in Science Center Hall E (in the basement).

Office Hours:
Jonathan Kaplan SC428b - Tue (20th) through Fri (23rd) 2-4pm

Pitman, Section 1.3

• The language of events in probability
• Experiment
• Outcome
• Outcome space
• Event
• The language of sets
• Universe
• Subset
• Empty set
• Venn diagrams as visual tools
• See also pset 1, #3
  
• Complement
• Union
• Intersection
• Disjoint
  
• Properties of set operations
• Understanding in pictures with Venn diagrams
• Understanding in words with logic
• Dictionary between the language of events and sets
• See figure in Pitman
• Outcome space
• Event
• Impossible event
• Not A
• Either A or B
• Both A and B
• A and B are mutually exclusive
• The probability set function
• Allows us to say precisely what probability is.
• Its a function that assigns to an event, a real number that reflects the likelihood of that event.  
• Properties we require of it
• Always non-negative
• The outcome space has probability one
  
• Additivity of probabilities for unions of disjoint events
• See also pset 1, #4
  
• Examples
• Can translate our examples with cards and coins into this language
• See also pset 1, #1, #2
  
• Additional properties following from the definition
• Complement
• Subset
• There are various interpretations of probability, but the mathematics of probability is independent of interpretation

Pitman, Section 1.4

• Conditional probability
• Definition
• P(A|B) is the probability that event A occurs given that the event B has occurred
  
• Note the new use of the old symbol P. A|B is not a set!
  
• Properties
• Probability conditioned on an event B is again a set function
• See also pset 2, #2, #3
  
• Multiplication rule
• Just a restatement of the definition of conditional probability in a form that's sometimes useful
• See also pset 2, #1
  
• Independent events
• Two events are independent if the probability of both of them occurring (that is, the probability of their intersection) is the product of the probabilities each of them occurring.
• See also pset 2, #5
  
• Rule of Averaged Conditional Probability
• Partition
• Proof follows from
  
• Definition of partition
  
• Additivity of probability set function
• Use of multiplication rule  for each term above

Pitman, Section 1.5

• Bayes' Rule
• Motivation through disease testing example
• Follows from definition of conditional probability
  
• and Rule of Averaged Conditional Probability
• Bayes' Rule is simply a generalization of this result for a partition of the outcome space into more than two parts
• Interview question with coins
• Monty Hall problem
• See also pset 3, #1, #2, #3
  

Pitman, Section 2.1

• Repeated trials and binomial distribution
• Repeated coin-flips as prototypical example
• Also applied to repeated dice rolls and repeated missile tests
  
• Binomial theorem
• See also pset 1, #5 and pset 4, #1b
  

Pitman, Section 3.1

• Discrete random variable (RV)
  
• Definition: A random variable is an outcome of a numerical measurement in an experiment.
• Random: The outcome of the measurement cannot be determined ahead of time.
  
• Variable: The outcome of the measurement is one of many possible values.
  
• Examples
  
• Dice
• Birthweight
• Time to failure of an electronic device
• The space of an RV is the set of values it can take.
• An RV whose space is a discrete subset of the real numbers (like the whole numbers from 1 to n) is called a discrete RV.
• An RV whose space is a continuous subset of the real numbers (like the interval from 1 to n) is a called a continuous RV.
• Probability density function or probability distribution function (PDF)
• Definition: The function f(k) gives the probability that the random variable X takes on a value k
  
• Properties of PDF that follow from properties of probability set function
• Always nonnegative
• Sum of the value of the function over all elements of outcome space is 1
• Note: The domain of a PDF is often extended to all real numbers by setting it equal to zero for values the random variable doesn't take.
• Uniform distribution
• See also pset 3, #4
  
• Binomial distribution
• See also pset 4, #1b
• See also earlier section on repeated trials
• Function of a random variable
• Is again a random variable, eg X^2
  
• Has its own PDF
  
• See also pset 3, #4
  
• Several random variables and joint PDFs
• Independent random variables
• Random variables are independent if their joint PDF is the product of their individual PDFs
  
• This is best understood in terms of the relationship to independent events
  
• Examples
• Red die and blue die

Pitman, Section 3.2

• Expectation value
• Defintion: The expectation value of a random variable is the sum of its possible values weighted by their probabilities.
• See also pset 3, #4c and pset 4, #1
  
• Mean and variance of a random variable
• Intuitively, the mean is the value around which you'd expect data on the random variable to cluster. It's usually (but not always) close to the hump in the PDF of a random variable (if it has one).
  
• Intuitively, the standard deviation (which is the square root of the variance) is the width you'd expect of the cluster of data around the mean.

Pitman, Section 4.1 and Stewart, Probability Subsection of 12.5

• Continuous random variable
• An RV whose space is a continuous subset of the real numbers (like the interval from a to b) is a called a continuous RV
• Its PDF
• Definition: The function f(x) is the probability that the value of the random variable X lies in the interval [x, x+dx].
  
• Note: The probability that a continuous random variable has any given value is zero but the probability of its value lying in an interval can be nonzero. This is very much like mass. The mass of a point in a solid is zero but the mass of a tiny box in the solid is nonzero.
• Properties
  
• Always nonnegative
• Integral over outcome space is 1.
  
• Expectation value
• Now involves integral rather than sum
  
• See also problems from Stewart on pset 4
• Uniform distribution
• Spinner on a board game
  
• Determining the constant by requiring integral to be 1.
  
• Mean and variance
• Note: For this distribution, the mean doesn't have much meaning. In fact, it depends on your convention for labeling angles.
  
• Exponential distribution
• Time to failure of a light bulb
• Graph of PDF
  
• Determining the constant by requiring the integral to be 1.
  
• Mean and variance
• Calculating just requires integration by parts
  
• Interpretation
• Can now label the parameter  appearing in the PDF in terms of the mean
• Graph of family of exponential distributions with varying means
• Conditional probability in this example and the deficieny of this constant failure rate model for a light bulb
• Gaussian distribution (also known as normal distribution)
  
• "Standard" Gaussian has mean 0 and variance 1
  
• Graph of standard Gaussian
• Determining the constant by requiring integral to be 1.
  
• Doing integral requires trick using polar coordinates
  
• Mean and variance
• Calculated by u substitution
  
• Interpretation
• Can be used as labels for the parameters in the PDF again
• Graph of a family of Gaussians with varying means and standard deviations
• Examples of random variables with Gaussian distributions
  
• Birthweight
• Noise

Topic not in Pitman

• Estimation using the maximum likelihood method
  
• Heuristic explanation of its use in phylogenetics
  
• Example: A physical model suggests that a random variable of interest has an exponential distribution. The parameter appearing in the distribution is unknown, and our goal is to extract the most likely estimate for the parameter from the data.
• Definition of the likelihood function
• Maximizing likelihood equivalent to maximizing Log of likelihood
• Find that the mean of the data is the most likely estimate
• Example: Model suggests a power law distribution with an unknown power.
  
• Proceed as in the previous example
• Find an unexpected result