Agentic Music Post-Production
Abstract
We introduce MixAgent, an agentic pipeline for music post-production that follows the workflow of audio engineers to build interpretable and editable mixing plans guided by a reference song. A grouping agent first routes related input tracks, such as drum recordings, into subgroups. A planning agent then uses retrieved examples to choose and order audio effects, such as volume control and reverb, for tracks, subgroups, and the full mix. To refine the mix, a detector identifies what needs correction, and a critic proposes numerical parameter adjustments from tracks to the full mix. In our experiments, prompting recent multimodal models to listen and revise often worsens the mix, making reliable correction a central challenge. We therefore build MixCritic, a dataset of 45,657 repair critiques derived from parameters fitted to professional mixes, and train the critic with supervised fine-tuning followed by preference optimization on rendered audio outcomes. On MedleyDB, learned refinement reduces error by 19% relative to the initial mix, with preference optimization reducing it by 8.4% over supervised training alone. Retrieved examples further improve effects selection, and a human listening test shows that MixAgent gets higher quality ratings than mixes produced by Qwen3-Omni, Gemini 3.8 Flash and GPT-4o-audio.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
MixAgent introduces a novel agentic pipeline for music post-production that combines retrieval-augmented planning with learned numerical refinement, achieving state-of-the-art results in reference-guided mixing and providing a robust framework for interpretable audio editing.
Methodology
The paper proposes MixAgent, a multi-agent framework for reference-guided music mixing. The methodology is structured into three distinct phases: grouping (routing tracks to subgroups), planning (selecting and ordering audio effects via retrieval-augmented generation), and refinement (detecting and correcting parameter errors). A key methodological contribution is the construction of the MixCritic dataset, which pairs corrupted mixing states with repair critiques derived from professional mixes. The authors employ a two-stage training process for the critic: Supervised Fine-Tuning (SFT) followed by Direct Preference Optimization (DPO) on rendered audio outcomes. The system explicitly models the hierarchical nature of mixing (tracks -> subgroups -> full mix) and uses a bottom-up refinement strategy, which is shown to be superior to top-down or flat approaches. The use of a frozen planner with retrieved examples to guide effect selection is a practical and effective design choice that mitigates the tendency of LLMs to select suboptimal or overly short effect chains.
Results & evaluation
The experiments are rigorous and comprehensive. The authors evaluate on MedleyDB and AlbumDB, using non-overlapping excerpts to prevent data leakage. They compare against strong baselines, including recent multimodal LLMs (Qwen3-Omni, Gemini 3.8 Flash, GPT-4o-audio) under various prompting strategies (Single-pass, Self-Refine, Self-Critic). The results demonstrate that MixAgent outperforms these baselines in reconstruction error (MSS) and predicted quality metrics. A particularly insightful analysis is the comparison of feedback forms (perceptual, tool-specific, numerical), showing that numerical corrections trained via DPO yield the best performance. The human listening test, while small (11 participants), provides qualitative validation that the system produces mixes rated higher than LLM baselines, though still below professional human mixes. The ablation studies on retrieval, refinement order, and detector selection are thorough and provide clear insights into the system's components.
Reproducibility
The paper provides high reproducibility. The authors release the code and demo on GitHub. They detail the architecture of the detector and critic, the training procedures (SFT and DPO), and the construction of the MixCritic dataset. The prompts used for the agents are included in the appendix. The use of standard datasets (MedleyDB, AlbumDB) and public models (Qwen, Gemini, GPT) further enhances reproducibility. The specific hyperparameters and training settings are listed in tables, allowing for faithful replication.
Limitations
The primary limitation is the small size of the evaluation sets (20 MedleyDB clips, 14 AlbumDB clips), which limits statistical power and generalizability. The listening test with only 11 participants is also a weakness. The system relies on GRAFX-fitted parameters for supervision, which may not perfectly capture the intent of professional engineers. The audio effects library is limited to a subset of professional tools, and the plan is fixed after the planning stage, meaning the critic cannot add or remove effects. The computational cost of multiple LLM calls for mixing is also noted as a practical limitation.
Broader impact
This work has significant implications for the field of AI-assisted creative tools. By providing an interpretable and editable mixing plan, it addresses a key barrier to adoption of AI in professional audio workflows. The findings on the effectiveness of numerical feedback and preference optimization for audio parameter adjustment are broadly applicable to other domains where LLMs control numerical tools. The construction of the MixCritic dataset is a valuable resource for the community. The paper also highlights the challenges of evaluating audio quality with current metrics, pointing to the need for better perceptual evaluation methods. MixAgent introduces a novel agentic pipeline for music post-production that combines retrieval-augmented planning with learned numerical refinement, achieving state-of-the-art results in reference-guided mixing and providing a robust framework for interpretable audio editing.
Author of this paper? Send a correction.