FEO3230 Probability and Random Processes, Spring 2022

The aim of this course is to give a solid background on measure theoretic probability and random processes for PhD students in information theory, signal processing, learning and control.

The main motivation behind developing the course is that to appreciate some, both classic and new, work on theory for signals and systems, it is necessary to have at least some basic background on measure theory and the language used in results that build on it. One important class of problems where this holds in particular is achievability proofs in information theory based on ergodic theory. Another important area is decision and estimation theory.

The course is registered as FEO3230 and is worth 12 cu's.


The main text for the course is Robert Gray (Stanford): Probability, random processes and ergodic properties (1st edition available from Gray's web-page, 2nd edition printed by Springer). As a complement, parts of the first half of the course will be based on lecture notes only. The lecture notes essentially follow McDonald and Weiss: A course in real analysis, and students who plan to dig deeper are advised to acquire this textbook too, as a complement.

Other texts, useful as complements, are Klenke: Probability Theory, Springer 2008; Kallenberg: Foundations of Modern Probability, Springer 1997; Shiryaev: Probability, Springer 1996; Wong/Hajek: Stochastic Processes in Engineering Systems, Springer 1985; and, Aliprantis/Border: Infinite Dimensional Analysis, Springer 2006. (For the books published by Springer, note that KTH has access to Springer Link via the library website.)

All meetings, except meetings 2 and 13, are held on Fridays at 9:30-12:00.

Meeting 1 is held in
  • Room V22, Teknikringen 72, floor 4
meeting 2 in
  • Room Q2, Malvinas väg 10, floor 2: Starting at 10:00 on Monday March 7,
while the location for meetings 3 through 9 is
  • Room Harry Nyquist, Malvinas väg 10, floor 7
Meeting 10 is held in
  • Room Gustaf Dahlander, Teknikringen 31, floor 3
while for meetings 11 through 13 we return to Harry Nyquist.

Schedule 2022

  • Lecture 1 (Feb 18): Lebesgue measure on the real line
  • Lecture 2 (Mar 7): The Lebesgue integral on the real line
  • Lecture 3 (Mar 11): General measure theory
    • Measure spaces and measurable functions
    • Convergence in measure
  • Lecture 4 (Mar 18): General integration theory
    • The abstract Lebesgue integral
    • Distribution functions and the Lebesgue-Stieltjes integral
  • Lecture 5 (Mar 25): Probability and expectation
    • Probability spaces
    • Expectation
    • The law of large numbers for i.i.d. sequences
  • Lecture 6 (Apr 1): Differentiation
    • Functions of bounded variation
    • Absolutely continuous functions
    • The Radon-Nikodym derivative
    • Probability distributions and pdf's; absolutely continuous random variables
  • Lecture 7 (Apr 8): Conditional probability and expectation
    • Conditional probability/expectation
    • Decomposition of measures; continuous, mixed and discrete random variables
  • Lecture 8 (Apr 22): Topological and metric spaces
    • Topological and metric spaces
    • Completeness and separability, Polish spaces
    • Standard spaces
  • Lecture 9 (Apr 29): Extensions of measures and product measure
    • Extension theorems
    • Product measure
  • Lecture 10 (May 13): Random processes
    • Process measure, Kolmogorov's extension theorem
  • Lecture 11 (May 20): Dynamical systems and ergodicity
    • Random processes and dynamical systems
    • The ergodic theorem
  • Lecture 12 (June 3): Applications
    • Detection and estimation
    • Information and coding