Mathematische Grundlagen der Neuronalen Netze

What it is about

The module conveys selected mathematical foundations for the analysis of artificial neural networks. You will learn how approximation properties, stability with respect to input perturbations, and the learnability of networks are studied using various mathematical tools. In the end you will understand the central theoretical results and the analytical methods used.

What you will be able to do

Apply mathematical methods for the analysis of neural networks
Compare approximation properties of deep and shallow networks
Understand stability phenomena and countermeasures
Classify the basics of learnability and required sample sizes

What the module consists of

LecturePresentation of theory, examples and discussion; no exercises planned

Teaching method

Board lectureFor in-depth, step-by-step derivation of the theory
Discussion with studentsPromotes independent analysis and deeper understanding
Possibly slides additionalFor visualization and structuring of the content

What you should already know

MA1101 Linear Algebra and Discrete Structures 1
MA1101 Linear Algebra and Discrete Structures 2
MA1001 Analysis 1
MA1002 Analysis 2
MA3001 Functional Analysis, Approximation Theory, Fourier Analysis and Waveletsrecommended
MA4800 Foundations of Data Analysisrecommended

Assessment

Oral exam

Oral examination, duration one hour. The focus is on understanding the presented results and the details of their proofs. (No information about aids or weighting in the raw data set.)

Reading list

HN Mhaskar - Approximation properties of a multilayered feedforward artificial neural network
HN Mhaskar - Neural networks for optimal approximation of smooth and analytic functions
HN Mhaskar, T Poggio - Deep vs. shallow networks: An approximation theory perspective
CK Chui, X Li, HN Mhaskar - Limitations of the approximation capabilities of neural networks with one hidden layer
CK Chui, HN Mhaskar - Deep nets for local manifold learning
A Rajagopal, S Chandrasekaran, HN Mhaskar - Deep Algorithms: designs for networks
R Alaifari, GS Alberti, T Gauksson - ADef: an Iterative Algorithm to Construct Adversarial Deformations
J Bruna, S Mallat - Invariant scattering convolution networks
S Mallat - Group invariant scattering
T Poggio, H Mhaskar, L Rosasco, B Miranda, Q Liao - Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review
T Poggio, H Mhaskar, L Rosasco, B Miranda, Q Liao - Why and when can deep–but not shallow–networks avoid the curse of dimensionality: a review (arXiv)
T Poggio, Q Liao - Theory II: Landscape of the empirical risk in deep learning
T Poggio et al. - Theory of Deep Learning III: explaining the non-overfitting puzzle
M Fornasier, K Schnass, J Vybiral - Learning functions of few arbitrary linear parameters in high dimensions
M Fornasier, J Vybíral, I Daubechies - Identification of Shallow Neural Networks by Fewest Samples
F Bach - Breaking the curse of dimensionality with convex neural networks

Filters

Mathematische Grundlagen der Neuronalen Netze

What it is about

What you will be able to do

What the module consists of

Teaching method

Module ratings

Rate this module