Seminar - Current Topics in Deep Learning Theory

Course Overview

In this seminar, we will discuss currently influential ideas and approaches in the theory of deep learning, for example double descent, neural tangent kernel, or muP-parameterization. By the end of the seminar, students will

• have an overview of central approaches in the theory of deep learning.
• understand some of the main challenges, for example related to optimization, generalization, or feature learning.
• know one approach, typically corresponding to one publication, in detail.
• be able to present this approach in a clear and engaging oral presentation tailored to their peers.
• be able to critically examine the presented work and situate it within the broader context of deep learning theory.

Prerequisites

• Recommended: Lecture on Theory of Deep Learning by Prof. Michael Krämer.