Home

Membership

Community

Events and Awards

Media

About ISI

Resources

SCB

  • Register
Topic Data Science

Instructor

Affiliated with Technische Universität Darmstadt, Germany

Dr. Sophie Langer

  • Education

     

    • 2020: PhD in Deep Learning (Dr. rer. nat.), summa cum laude. Thesis: A contribution to the statistical theory of Deep Learning.
    • 2017: Masters in Mathematics with Economics.
    • 2015: Bachelors in Mathematics with Economics.
  • Research interest
    • Statistical aspects of Deep Learning.
  • Teaching experience
    • Summer Semester 2021: Lecture, Course: Statistical Foundations of Deep Learning, Technische Universität Darmstadt, Germany.
    • Fall Semesters 2018-2020: Basis lecture, Course: Statistics for Humanities, Technische Universität Darmstadt, Germany.
  • Certificates
    • Zertifikat Hochschullehre (Engl. University teaching certificate)

Course description

This course covers foundation and recent advances of deep neural networks (DNNs) from the point of view of statistical theory. Understanding the power of DNNs theoretically is arguably one of the greatest problems in machine learning. During the last decades DNNs have made rapid process in various machine learning tasks like image and speech recognition and game intelligence. Unfortunately, little is yet known about why this method is so successful in practical applications. Recently, there are different research topics to also prove the power of DNNs from a theoretical point of view. From an aspect of statistical theory, several results could already show good convergence results for DNNs in learning different function classes.

The course is roughly divided into two parts. In the first part, DNNs are introduced and different network architectures are discussed. In the second part, we focus on the statistical theory of DNNs. Here we will introduce frameworks addressing two key puzzles of DNNs: approximation theory, where we gain insights in the approximation properties of DNNs in terms of network depth and width for various function classes and generalization, where we analyze the rate of convergence for both, regression and classification problems.

Target audience

PhD students and researchers in the area of mathematics/statistics/computer science.

Syllabus

Course Goal

To provide participants with the theoretical background of DNNs and to survey the state of the art in statistical theory results for DNNs.

Learning outcome to be covered

By the end of the course, students will be able to:

  1. Understand and apply different network architectures
  2. Summarize the state of the art of Deep Learning theory
  3. Understand different proof techniques in this context
  4. Detect open questions for further research work

Course outline

  1. Motivation: The present great success of Deep Learning
  2. Statistical learning theory: Regression and Classification
  3. The theory behind Deep Learning:
    1. Introduction of Neural Networks
    2. Different network architectures and activation functions
    3. Approximation theory: Which function classes can be efficiently approximated by DNNs?
    4. Generalization: Convergence rates of DNNs for different regression and classification problems. When are those networks able to circumvent the curse of dimensionality?
  4. Limits and open questions of DNNs

Required software

Python.