Theory of Machine Learning, Spring 2021

CSCI 7000, APPM 5590, APPM 4490

University of Colorado, Boulder

---

Instructor: Bo Waggoner

Time: Tue/Thu 8:00am - 9:15am
Meets synchronously, remotely (link is on Canvas page)

Office hours (on Zoom, link in Canvas): Tuesday 9:15am - 10:30am; Thursday 4pm - 5pm

Course information and syllabus

---

Overview

Presents the underlying theory behind machine learning in proofs-based format. Answers fundamental questions about what learning means and what can be learned via formal models of statistical learning theory. Analyzes some important classes of machine learning methods. Specific topics may include the PAC framework, VC-dimension and Rademacher complexity. Recommended prerequisites: APPM 4440 and CSCI 5622.

The course will be similar to Prof. Stephen Becker's version.

---

Resources and Links

• [SSS] Understanding Machine Learning by Shalev-Shwartz and Ben-David.
• [Mohri] Foundations of Machine Learning by Mohri, Rostamizadeh, and Talwaker.
• [Hastie] The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman.
• [RW] Gaussian Processes for Machine Learning by Rasmussen and Williams.
• [Murphy] Machine Learning: A Probabilistic Perspective by Murphy.
• [SuttonBarto] Reinforcement Learning: An Introduction by Sutton and Barto.
• Canvas page: https://canvas.colorado.edu/courses/71018.

---

Schedule

• Thu, Jan 14: Intro, what is (theory of) ML? Estimating coin bias.
  Reading:[SSS] Ch1 and Ch2 for next time.
  HW: Sign up for Zulip.
• Tue, Jan 19: Statistical learning setting, ERM, PAC setting. Papaya example from [SSS], axis-aligned rectangles.
• Thu, Jan 21: PAC setting and variants; PAC-learnability of finite hypothesis classes.
  Reading:[SSS] Ch3 and Ch4 for next time.
  HW: homework 1 posted on Canvas, due Mon. March 1 at 11:59pm to gradescope.

---