Pedagogy as the Agent's Policy

Pedagogy as the agent’s policy¶

The main parts of a practice agent are summarized in Figure 1. Three of them are the same in every tool-using agent: a model reached through an API that supports tool calling, a set of tool schemas with the functions behind them, and a loop that sends the conversation, runs whatever tools the model asked for, appends the results, and repeats Chu et al. (2025). Tools and memory are infrastructure. Teaching is the policy over them: which move to make now, how much help to give, and when to stop helping. A pedagogical policy decides the tutor’s next action and has been induced from student data with reinforcement learning as in Chi and colleagues Chi et al. (2011). Without a policy, a well-instrumented agent defaults to answering rather than teaching with domain knowledge.

VanLehn splits tutoring behaviour into two loops VanLehn (2006). The outer loop runs once per task and chooses what the student works on next. The inner loop runs once per step, gives hints and feedback, and updates the student model. Agent harnesses have the same split: a task loop that picks the next unit of work, and a turn loop that handles one model request and its tool calls Bui (2026), OpenAI (2026). In a practice agent, which passage to work on next is an outer loop decision, and how much to say about one take is an inner loop decision.

Domain knowledge: Centuries of pianism treatises¶

Before an agent can decide how to teach, it has to know how the piano is played. There are two centuries of treatises cover it (Figure 2): how a key is pressed and a tone produced Matthay (1903), the five basic motions of arm and hand Sándor (1981), how to practise and memorise Leimer & Gieseking (1972), what a teacher listens for in tone and rhythm Neuhaus (1973), the nervous system behind the movement Kochevitsky (1967). It’s similar as the case law Harvey trained into its legal model, and Agentic Coaching Systems outlines the two ways to put it into an agent: train it into the weights, or retrieve it at run time by scaffolding.

Ideally, the policy should sit on top of this knowledge. Given what the treatises say about playing and what memory says about this learner, the policy decides which move to make now, which tool call to examine which aspect, how much help to give, and when to stop.

Domain knowledge: Classical anchors for education¶

Zone of proximal development. Vygotsky’s ZPD is the gap between what a learner can do alone and what they can do with guidance Vygotsky (1978). For a practice agent this is a nice fit for outer loop policy: pick the next drill just past current mastery, instead of repeating owned material or jumping to performance tempo.

Scaffolding and fading. Wood, Bruner and Ross described scaffolding as support that lets a learner solve a problem they could not solve alone, withdrawn as competence grows Wood et al. (1976). Later work stresses that the withdrawal has to be deliberate, or the support becomes permanent Pol et al. (2010).

Formative feedback. Useful comments are specific, timely, and about the task or the process rather than the person Hattie & Timperley (2007), Shute (2008). Details like summary praise, note-level evidence, a diagnostic exercise and a physical cue shall be separate to address each bullet point.

Reactive or proactive. Reviews separate agents that only reply, from agents that start the conversation Li & Zheng (2026). Between lessons an agent can open the turn itself from memory, asking student in a proactive way.

Automation against learning. Systems tuned to finish the task for the user will remove the struggle that produces the learning Woollaston et al. (2026). In their manuscript, they discussed more favorable responses like withholding completion, asking a question, or handing back to a human teacher.

How recent LLM tutors enact those ideas¶

Plan then adapt. ScaffoldLM builds a stepwise plan first, then uses assessment-driven memory to decide whether to remediate the current step or advance Li et al. (2026). The music version is a plan for one passage (hands separate, slow hands together, target tempo) whose step advances when tool-assessed criteria are met.

Roles that protect autonomy. MusicScaffold frames generative AI as guide, coach and partner, explaining structure and prompting reflection without taking over Hu et al. (2025). Chat Melody has the student explain while the agent plays novice, and reports learning gains with lower cognitive load Jin et al. (2025). Acting like a teacher can mean not giving the answer.

Incorporating domain knowledge into the agent¶

Three routes, in increasing order of cost.

In the prompt, or in skills loaded on demand. A system prompt holds rules, such as a feedback rubric or the sight-reading criteria for one grade, but not a book. Agent Skills keep such knowledge in a directory whose full contents load only when a task matches Anthropic (2025).

Retrieval. Retrieval beat fine-tuning for adding knowledge to a model, both for facts seen in pretraining and for new ones Ovadia et al. (2024). A few dozen piano treatises are small enough for keyword search over chapters. A retrieved passage carries its book and page, so a teacher can check the claim; a claim recalled from the weights has no source.

Into the weights. Harvey’s 2024 case-law model, built with OpenAI, tried fine-tuning and retrieval first, then added the equivalent of 10 billion tokens of US case law to a base model; attorneys preferred its answers to GPT-4’s 97 percent of the time OpenAI (2024). A book cannot be trained in by reading it once: a model stores a fact seen once but cannot answer questions about it unless the training data presents the fact in many rewritten forms Allen-Zhu & Li (2024). EntiGraph generates those rewrites (Figure 4). From 1.3 million tokens of source text it synthesised 455 million, and training on them recovered more than 80 percent of what retrieval achieves Yang et al. (2025). Training Neuhaus into a model would follow the same recipe.

Incorporating the policy into the agent¶

A student plays bar 12 wrong for the third time. The tutor can name the note, ask whether it sounded right, set a slow drill on bars 11 to 13, or move on. The policy picks one. It can be written into the prompt, which is cheap but which the model may ignore, or trained into the weights. Figure 5 follows the bar 12 example up three rungs.

Write it. Most policies are written by hand in three places. The prompt holds a few rules, added only after observed failures Rajasekaran et al. (2025). Tool returns decide what the model can see: “accuracy 82%” lets it say only “keep practising” Aizawa (2025). The harness enforces in code what must not be skipped, through stored plan state, gated tools, and hooks that check each reply before it goes out Bui (2026), OpenAI (2026).

Optimise it. GEPA keeps the policy as prompt text but has a model rewrite it Agrawal et al. (2026). It runs the agent on 20 to 100 scored examples, has a model write down what went wrong in each run, and edits the prompt. It beat GRPO, which trains the weights, with up to 35 times fewer runs.

Learn it. Reinforcement learning trains the weights from a reward. Chi and colleagues trained a physics tutor to choose between telling a step and asking for it, rewarded by the students’ test gains Chi et al. (2011); MC-CPO limits the allowed actions by the learner’s mastery Olukola & Rahimi (2026). Self-distillation fine-tuning (SDFT) needs no reward: the model is trained so that its answer without a teacher’s demonstration matches its answer with one Shenfeld et al. (2026).

Coding agents are trained with reinforcement learning because tests pass or fail automatically. Music practice has no such check: whether a drill worked shows a week later. Distillation needs teacher demonstrations instead, a take and what the teacher said next, and no one has collected a corpus of them yet. Until then the policy is written and prompt-optimised.

Closing the loop¶

Tools diagnose, memory carries state, pedagogy picks the move. One multi-session thread shows why no single module is coaching: the same unstable bar across several weeks, moving from note-level callouts to a diagnostic drill and then to faded check-ins.

The demo notebook checks each take against note-level evidence before saying anything about it, and ranks that check above fine-tuning. Evaluation is the gap, and it has moved backwards. Chi and colleagues ran classroom studies and showed that which policy you use changes what students learn Chi et al. (2011). Most LLM tutors now write the policy by hand and report task performance instead of learning gain. DeepTutor’s TutorBench evaluates interactive, profile-conditioned tutoring Zhao et al. (2026), and nothing directly comparable exists for music practice over time.