The tutorial is organized around one practical question: which LLM capabilities matter for music, and how can we deliberately use them?
Structure¶
Figure 1:Two foundation chapters lead into three applications, one for each kind of music user.
Part I · Foundations¶
1 · Overview of Large Language Models and Agents (30 min) — autoregressive modeling, pre-, mid-, and post-training, advanced capabilities, and evaluation
2 · LLM-powered Music Understanding (30 min) — music perception and reasoning, large music-language models across audio, score images, symbolic notation, and metadata, and benchmarks
Part II · Applications¶
3 · Music Listening with LLMs — conditioned music annotation, search and recommendation over an LLM-enriched music database, and open problems
4 · LLMs for Creators — music making through suggestions, real-time music co-creation, and agentic music making
5 · Feedback & Music Education — multimodal understanding of performances, assessment and coaching, and interactive music education
Wrap-up¶
Conclusion — recap, future research directions, Q&A and discussion
References — every work cited in the tutorial, with further reading
Learning goals¶
By the end of the tutorial, participants should be able to:
connect autoregressive modeling and post-training to modern LLM behavior;
compare architectures for audio, score-image, symbolic, and text inputs;
select capabilities such as instruction following, reasoning, or tool use for a musical task;
identify suitable benchmarks and test whether a system’s answer depends on the music rather than on text alone.
How to use this book¶
Each chapter has a web page, slides, and reproducible code on GitHub. Chapters in Part II can be read in any order once Part I is familiar. Every page lists the works it cites at its end, and References collects them all.