Paul Hewson has a home page here: www.plymouth.ac.uk/staff/phewson.

Lecture notes and problem sheets for Bayesian Statistics course

Material will be placed here as the course progresses. We've been asked to put a summary sheet together on some of the more common probability functions - ccPDFs.pdf, if your favourite distibution is missing please let us know!

Week 1

A solution sheet to the regression exercise is under RegressionSolutions.pdf.

Week 2

• (Monday) Conditional probability and Bayes Theorem. We will cover combinations and permutations very briefly (see ccPermuteComb.pdf). Most of the work will concern conditional probability - there is an interactive worksheet (ccConditional.pdf) and a short problem sheet. Most of the problems have been set for fun! They are rather thought provoking.
• (Tuesday and Wednesday) Maximum Likelihood. This method of estimation is common to both conventional and Bayesian statistics - the lecture ccLikelihood.pdf will therefore review some key properties.
• (Wednesday and Thursday) Estimating a proportion using the Binomial under Bayesian and non-Bayesian Inference. There is a (newly updated) Problem Sheet for the Lab Session on Thursday morning.
• (Friday) Further notes on the choice of prior (HERE) and a worksheet on inference for the Poisson distribution

There is a homework sheet to do to complete our work in week 2 (to be handed in by Wednesday 16th December). The typo in the Poisson log-likelihood has been corrected.)

There is also a song (sung to the tune of "Let it Be" by the Beatles) that tells us about the key properties of maximum likelihood estimator (mlesong.txt)

Week 3

• Hierarchical Models. At the moment the notes contain some illustrative calculations in R - see ccHierarchical.pdf.
• Using simulation techniques for inference.
  1. We have a lab worksheet: ccRejection.pdf.
  2. Whilst python has facilities to calculate means, variances and so on it doesn't seem to have inbuilt facilities to estimate quantiles, so you might want to use quantile.py.
  3. There is also a possible solution to the first problem in rejectbeta.py - but don't look at this too soon.
  4. Finally, as simulation is such an important technique there is a mini-Homework (this problem has been extended slightly, and is now due in on Friday 18th. It will be last homework for this course.
• Linear regression as a likelihood problem leading to linear regression as a Bayesian problem.
• Simulation based inference using McMC: the jags package.
  1. There some rather lengthy notes for the session.
  2. There is code for the two models: betabinomial.jags and binomial.jags.
  3. The data are in dump format here:rats.dump and
  4. The initial values are in dump format here betabinom.inits for the first model and rats.inits for the second.
  5. If you are curious there is a local version of the full JAGs manual although I don't intend setting any homework problems on this.
  6. If this course has left you interested but wondering exactly why Bayesian inference is useful, have a look at this paper on malaria mapping (I've put a local copy of the formulae pages HERE. You should be able to see for example that the response is assumed Binomial, there are a variety of risk factors in the model in a regression format, and there is also a spatial prior for the "random effects".

References

This course is essentially based upon

• Andrew Gelman et al (2004) "Bayesian data analysis", CUP, reference QA279.5.B386.

In order to introduce the material in this book we have added some material on probability theory and likelihood theory. You might find the following books useful:

• Geoffrey Grimmett, "Probability : an introduction" QA273.G735
• Yudi Pawitan (2001) "In all likelihood : statistical modelling and inference using likelihood" OUP QA276.P286
• Peter M Lee "Bayesian statistics : an introduction" QA279.5.L44 is a book that focuses on the theory
• John A Rice "Mathematical statistics and data analysis" QA276.12.R53 1988 is a nice book covering the fundamentals of mathematical statistics

In case you ever get involved in teaching undergraduates, could I just mention Grinstead and Snell's book. It was written by them as an American Mathematical Society project. It is a very good book and is available from the web at http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/book.html. It's very good, but I'm not sure about the computer program they used for the examples - about time these examples were rewritten in python?