Background: LLMs and Agents

Language models are older than most of the people reading this. Claude Shannon used one in 1951 to estimate how predictable English is 1, and for the next sixty years they served as a general-purpose component wherever a machine produced text: speech recognizers, translation systems, spelling correctors, and phone keyboards all asked a language model which candidate sentence was most plausible. Around 2018, pretrained Transformers turned the language model from a component into the system itself. BERT, trained by masked language modeling to fill in hidden words, showed that one model, pretrained once and lightly tuned per task, beat designs built for each task 2. T5, a sequence-to-sequence model, cast every task, translation and classification alike, as text in and text out, so that a single model could be trained on all of them at once 3. GPT-2 and then GPT-3, trained by autoregressive language modeling to predict the next word, showed that the tuning was not even needed: given an instruction or a few examples in plain English, one model could translate, summarize, answer questions, and parse 45. Richard Sutton had called the pattern the bitter lesson: general methods that use more computation eventually beat methods built from human expertise 6.

Then came the ChatGPT moment, in November 2022 7. The model underneath was a descendant of GPT-3, which had been public for two years. What was new was the final stage of its training: people wrote example answers and ranked the model’s attempts, and the model was tuned toward what they preferred 8. For the first time, the public addressed a model in conversation and it answered like an assistant: it explained, revised when corrected, asked for clarification, and kept the thread. By one estimate, a hundred million people used it within two months, faster than any consumer application before it 9. The interface changed from “run a model on an input” to “ask, read, ask again,” and that loop is what turned a text-continuation engine into something that can be asked to plan, to explain, and before long to act.

Many tasks came together in one model, and the model acquired a conversational interface that made it an assistant. The question this page asks is what is carrying us from there into a world of assistants that act. What is in the model that lets it talk about music it has never heard? What turned it into something one can talk to? And what lets it reach beyond text to tools, including the tools MIR has spent twenty years building? The sections that follow take these in order.

The timeline below puts both stories side by side.

Read the figure from top to bottom and two patterns stand out. The first is one recipe, built up one ingredient at a time on the left side: an architecture (the Transformer, 2017), pretraining one model for many tasks (GPT-1, BERT, T5), scale (GPT-3), the discovery that some abilities emerge only at scale and that a model reasons better when it writes out its steps (2022), and post-training that turned the text predictor into an assistant (ChatGPT). The sections below take these in order.

By 2023 the same recipe was being reproduced outside OpenAI: Anthropic and Google released the closed assistants Claude and Gemini, and Meta and Alibaba released LLaMA and Qwen with downloadable weights that anyone can fine-tune 1011.

The second pattern is a shift from answering to acting. The agent papers on the left cluster in 2023, and by 2025 calling tools had become infrastructure: a shared protocol for connecting models to tools 12, reinforcement learning for reasoning and for search 1314, and open models trained for agentic work 1516. The closed models at the frontier, now GPT-6 Astra, Claude Fable 5, and Gemini 4 Argon, set the ceiling; the open ones decide what everyone else can build on.

From Language Models to LLM Agents¶

A language model predicts what comes next in a piece of text, one word at a time. What changed after 2018 was what such a model could be made to do.

2018 · Pretraining: one model, many tasks¶

Before 2018, each language task had its own model, dataset, and metric. The Transformer made it practical to train one network on far more text than before 17, and in 2018 two models showed what to do with it. GPT-1 was pretrained to predict the next word and then fine-tuned per task 18; BERT was pretrained to fill in masked words 2. Both beat models designed for each task, and T5 then cast every task as text in and text out 3. The field stopped building a model per task and started adapting one model to many.

2020 · Scale: tasks from a prompt¶

GPT-3 showed that a large enough model needs no tuning at all. Given an instruction or a few worked examples in its prompt, it performs the task on a new case, which is called in-context learning 5. This is what we mean by a large language model (LLM): a model trained to predict text across most of the public web, which absorbs a great deal of knowledge along the way. Some abilities appear only past a certain scale: a smaller model performs at chance, and a larger one suddenly does not 19.

2022 · Post-training: from model to assistant¶

A pretrained model continues text; it does not reliably answer. Post-training is the set of stages after pretraining that shape how a model behaves rather than what it knows. ChatGPT’s version had two steps: supervised fine-tuning on example answers written by people, then reinforcement learning from human feedback (RLHF), in which people rank the model’s attempts, a reward model learns their preferences, and the model is optimized to score well on it 8. The stage is comparatively cheap, and its effect is large: a 1.3-billion-parameter model tuned this way was preferred by human raters over the 175-billion-parameter GPT-3. The interface changed from “run a model on an input” to “ask, read, ask again.”

The preference step has since been simplified twice. Direct preference optimization (DPO) drops the reward model and tunes the policy on ranked pairs directly 20. Group relative policy optimization (GRPO) drops the value model: the policy samples a group of answers to each prompt and is pushed toward the ones that score above the group’s average 21; it is the algorithm behind DeepSeek-R1’s reasoning training. Music generation has adopted the same stages, with NotaGen tuned by DPO and Qwen-Music by DPO and a GRPO variant 2223.

Within a year the recipe had spread. Anthropic and Google released their own assistants, Claude and Gemini 2425, and open models such as LLaMA and Qwen put weights of comparable design in anyone’s hands 1011. The closed models set the frontier; the open ones are what research groups, including most of the music systems in this tutorial, build on.

2024 · Reasoning: thinking before answering¶

Asking a model to write out intermediate steps was found to help on multi-step problems 26. From 2024, reinforcement learning made that behavior the default: the model works through a problem at length before it answers 2713. Tasks that need several dependent steps became markedly more reliable. Open models followed: Qwen3 switches between thinking at length and answering at once within a single model 28.

2023–2025 · Agents: models that act¶

An LLM knows nothing after its training date, cannot perceive anything but text, and cannot carry out long exact computations. The fix is to let it call tools in a loop: the model chooses an action, a program executes it, and the result becomes the input to the next choice. For the three people this tutorial follows, the loop might run like this:

The model reads each result before choosing the next call: the listener’s second search asks for 150 BPM only because the tempo estimate came back at 112. Systems differ in how much of that choice is left to the model. In a workflow, the developer writes the path in advance; in an agent, the model decides at run time which tool to use next, corrects itself, and judges when the task is done or should go back to the user 2930. We write that choice as a policy $\pi_\theta(a_t \mid c_t)$ that picks the next action $a_t$ from the record so far $c_t$. The harness is the system around the policy: it decides what the model sees, which tools it can call, what it remembers, and how the results are checked 31. The harness is itself a design space, and it can be searched: Automated Design of Agentic Systems (ADAS) had a meta-agent write new agent designs as code and keep the ones that scored better 32, and Agentic Context Engineering (ACE) left the weights alone and instead grew the context, a playbook of strategies that the model revises after each task 33.

The path here ran through a handful of papers. Retrieval came first: a model that reads retrieved passages answers more accurately 34, and WebGPT let it browse on its own 35. ReAct gave the loop above, alternating reasoning and action 36; Toolformer taught a model when to call a tool 37, and ToolLLM scaled this to thousands of real APIs 38. HuggingGPT made the tools other models, the pattern most music agents follow 39, and CodeAct let agents act through short programs instead of single calls 40. Agents then gained reflection 41, memory 42, reusable skills 43, and teammates 44, and CoALA named the parts: memory, actions, and a loop that decides between them 45. SWE-bench moved evaluation to real software repositories 46, where SWE-agent showed that the design of a tool’s interface matters as much as the model 47. By 2025 the plumbing was standard: a shared protocol for tools 12, reinforcement learning for search 14, and open models trained for agentic work 1516. Instructions in English now do much of the work that glue code once did, which Karpathy (2025) calls programming in natural language. For more, see the EMNLP 2024 tutorial on language agents 49 and its slides.

Large Language Models for Music¶

Music information retrieval (MIR) was organized like NLP before 2018: by task, each with its dataset, its metric, and often its MIREX track 50. The same shifts have reached it, most of them a year or two behind, and they have opened three lines of work that this tutorial covers.

The first gave LLMs ears. LLark connected a pretrained music encoder to an LLM through a small adapter and trained it on a wide range of music tasks, from key and tempo to captioning and reasoning, with training data written by an LLM from music metadata 51. MU-LLaMA and MusiLingo used the same design to caption a track and answer open questions about it 5253, and SALMONN did the same for speech, sound, and music in one model 54. Music Flamingo scaled this to millions of full songs with step-by-step reasoning about harmony, structure, and timbre 55. Captioning itself has been rethought: a model predicts structured metadata from audio and an LLM writes the caption from it, so the style can change without retraining 56. (Chapters 2 and 3)

The second reads and writes music itself. ChatMusician read and wrote ABC notation as if it were text 57, MIDI-LLaMA follows instructions about symbolic music directly 58, and Nota taught vision-language models to read printed scores 59. On the writing side, the LLM recipe has been applied whole. NotaGen pretrained on 1.6 million pieces in ABC notation, fine-tuned on classical scores, and then ran reinforcement learning on a musicality reward 22; MIDI-LLM extended an open LLM’s vocabulary with MIDI tokens so that the same model writes text and music 60. For audio, YuE scaled a LLaMA-style model to full songs with vocals from lyrics 61; Qwen-Music plans the melody as a chain of thought before rendering the song 23, and YuE2 goes one step further by drafting a readable score first and then realizing it as audio in one model 62. (Chapters 2 and 4)

The third turns these models into partners in a conversation, and increasingly into agents. In the same year as HuggingGPT, MusicAgent used an LLM to break a music request into steps and route each step to a music model 63. TalkPlay turned recommendation into a conversation 64, and TalkPlay-Tools and LLM2Fx-Tools moved tool calling inside it, with MIR models as the tools for recommendation and for mixing 6566. For songwriters, Amuse turned a photo, a phrase, or a clip into chord progressions to build on, and won a best paper award at CHI 67. A design space drawn from 184 live music agents maps where such systems are used, how musicians interact with them, and what they are built on 68. For performers, LLaQo and similar models answer questions about a player’s recording and critique it 69707172, and MuseAgent-1 and MusicTutor call score, transcription and pitch tools on each take 7374. Other agents take teaching roles 7576, but memory across weeks and a teaching policy remain largely untested in music. (Chapters 3 to 5)

Benchmarks have followed the models, and they keep asking whether the models listen. MuChoMusic found that audio-language models often answer from the question rather than the audio 77. PianoJudge asked whether audio encoders can judge a piano performance as a teacher would 78. VocalCoachBench asked the same of singing: 18 vocal trainers annotated 515 recordings, and fewer than 7% of the models’ diagnoses strictly match the experts’ 79. CMI-Bench recast standard MIR tasks, from key detection to beat tracking, as instructions and scored LLMs with the field’s usual metrics; they still trail supervised models by a wide margin 80. RUListening rewrote music-QA questions so that the text alone no longer gives the answer away 81. Music is also one slice of the general audio benchmarks: MMAU tests 27 skills across speech, sound, and music 82, MMAR asks for multi-step reasoning over the mix of the three 83, and MMAU-Pro extends this to 49 skills, including long-form and multi-clip audio 84. Others widened the inputs: MusiQAl asks questions about the audio and video of a performance 85, and MusiXQA about images of sheet music 86. (Chapters 2 and 5)

A fixed vocabulary of labels is no longer the boundary of a music system: requests arrive in ordinary language and can be revised over several turns. The models MIR has built for twenty years do not disappear; they become the agent’s ears and hands. One gap has not closed. An LLM learned about music from text, not sound, so it knows about music far more than it hears it.

How to read what follows¶

Chapter 1 covers how LLMs and agents are built, trained, and evaluated, and Chapter 2 how audio, scores, and symbols enter them. Chapters 3 to 5 follow three people and what they ask. A listener wants “something like the Bill Evans trio, but with more energy, for a morning run” (Chapter 3). A producer says, “My chorus is louder than the verse, but it sounds smaller. Why, and what would you change?” (Chapter 4). A student brings a recording of a Bach prelude and asks where they rush, why there, and what to practise this week (Chapter 5).

Carry one question through all of them. The models now read fluently and listen poorly. Before you trust what one hands you, ask whether the answer would change if the music changed.