Understanding Large Language Models: Interpretability, Limitations, and Social Implications

What it is about

You will learn how large language models (LLMs) work, what methodological limitations and biases (shortcuts, data biases) they have, and how to detect and evaluate them. In the end you will be able to place existing interpretation and evaluation approaches, identify weaknesses, and critically reflect on possible improvements and social implications.

What you will be able to do

• Explain the basic principles of LLMs (representation learning, training objectives, inference behavior)
• Analyze selected research contributions on LLMs and interpret model behavior using interpretation methods
• Identify limitations due to data, model design, or evaluation
• Critically reflect on assumptions in development and deployment of LLMs (bias, generalization, robustness)

What the module consists of

• SeminarIntroduction to problems of shortcuts and biases; overview of fundamentals and current techniques; student paper presentations with discussion and feedback

Teaching method

• Vorlesende EinführungProviding foundations and context for the seminar topics
• Reading-List and student paper presentationsEncourages independent engagement with current research, presentation and discussion skills