Open Problems for Performance Data

Open problems for performance data in the feedback world¶

Feedback systems stand or fall on the data they see. MIR has spent two decades chasing clean signals, meaning well-tuned transcriptions, virtuoso recordings and curated recital sets, while filtering out everyone who is not yet good at their instrument. However, practice softwares, exam platforms, and coaching LLMs need the opposite: imperfect performance, in all its taxonomic and acquisition messiness. Closing that gap is an open problem for the field.

Challenge 1. The imperfect taxonomy¶

“Bad performance” is not a single class. Sight-reading, the longitudinal arc of one learner on one piece, the same passage at different skill levels, children versus adult beginners versus returning amateurs, and professionals starting a new work, all corrupt the intended score differently. Taxonomies inherited from expert critique corpora Matsubara et al. (2021) and graded assessments Parmar et al. (2021), Zhang et al. (2024) emphasise conservatory axes of accuracy and expression; singing and non-Western instruments require different categories entirely Sun et al. (2023), Huang et al. (2020), Li et al. (2023).

Lerch et al. Lerch et al. (2019) already separate technical correctness, expressive deviation, and aesthetic quality. Collapsing them into one “good / bad” label (or metrical scores) produces data that mis-teach any downstream LLM. Synthetic corruptions and simulated mistakes Morsi et al. (2024) can pad coverage, but they are not a substitute for a sourcing taxonomy that names which imperfect regime a recording belongs to or accounts for their underlying physiological causes.

Challenge 2. Acquisition: who holds the messy hours?¶

Research datasets remain small and professional-skewed Hawthorne et al. (2019), Foscarin et al. (2020), Zhang et al. (2022). Commercial practice apps hold millions of learner hours behind product walls, plus the coaching products discussed in Agentic Coaching Systems. Conservatory exams are vast and annotated yet often legally untouchable; social media is plentiful and barely curated. Privacy is critical when many learners are children, and teachers reasonably resist having their craft distilled into a model they do not control.

A rare counter-example is intentional release of imperfect performance. Sturm’s A Difficult Christmas Sturm (2024) and Sturm and Kanhov’s En Svår Jul Sturm & Kanhov (2025) document unpracticed, unpolished playing, violin after a forty-year gap and piano-violin Christmas repertoire at mixed proficiency, precisely because prompt-based generation systems cannot synthesise music that sounds unproduced. The albums are licensed for machine learning without further permission, and serve as a playful reminder that the musicking research filters out is exactly what learner-facing feedback needs.

Challenge 3. Processing: alignment that survives error¶

Score following, beat tracking, and DTW-style alignment assume the performer is roughly trying to play what is on the page Nakamura et al. (2017). Learners do not: they play wrong notes, restart phrases, skip bars, and sit on a fermata while thinking. Unified models such as RUMAA Chang et al. (2025) argue that alignment and mistake detection must happen at the same level: pipelines that separate “clean alignment” from “error catching” fail exactly on the recordings coaching systems receive. Until alignment is robust to imperfect play, score-grounded feedback Zhao et al. (2026) will cite the wrong measure at the worst moments.

Challenge 4. When we talks about performance, what are we really talking about?¶

How should imperfect performance be labelled? It would be quite simplified as a single audio + text → text captioning channel. The same pitch mistake can be ‘captioned’ at very different depths:

• You played smoothly, with occasional pitch error.

• You played the 4th sixteenth note in the second measure one semitone lower.

• You keep missing the sharp on G. Check by playing the A major scale again?

• Position your hand a bit towards the inside, so you can reach the F♯ at this tempo without slipping to the white key.

The first is a summary judgment; the second is evidence-linked; the third proposes a diagnostic exercise; the fourth addresses physical technique. Exam and conservatory vocabularies tend to flatten this spectrum onto a narrow axis (unclear, rushed, unmusical), and LLM-generated critiques regress to that mean even harder Zhang et al. (2025), Hattie & Timperley (2007). Play a track from either album above and ask what a useful annotation would even look like: scalar grade, note-level error list, technique cue, or something multimodal the performer can contest.