ChoreoStudio: Planning What, When, and Who in Music-Driven Group Choreography
Abstract
Music-driven group dance generation must decide not only how each dancer moves, but also what formation the group should form, when it should change, and who should execute each transition. Existing methods can generate orderly and rhythmically synchronized group motion, but remain limited in formation evolution and inter-dancer interaction. In human choreography, formation transitions often align with changes in high-level musical structure, such as the progression from an introduction to a chorus or from a chorus to a climax. We present ChoreoStudio, which makes choreography planning explicit. A music disentangler first decomposes the audio into frame-level rhythmic features and segment-level structural features. Given the segment-level features and sparse labels, a frozen omni-modal model acts as a scriptwriter, producing a global semantic description that guides the Director's choreography planning. Based on this context, the Director produces three coupled interaction decisions: a centroid-relative point set specifies what formation to produce, a transition gate determines when it should change, and a mobility allocation indicates who should execute the transition. The transition gate and mobility allocation are further conditioned on the planned geometric change, coupling formation targets with their temporal and dancer-specific execution. At a finer temporal scale, frame-aligned music features condition the diffusion generator to control detailed body motion and preserve beat-level alignment. The segment-level interaction plan is injected in parallel through frame-aligned instruction tokens and cross-dancer attention biases, coordinating how the generator realizes each transition. Experiments on AIOZ-GDance demonstrate that ChoreoStudio follows the planned interactions while achieving state-of-the-art performance in motion quality, group organization, and formation dynamics.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
ChoreoStudio introduces a novel explicit choreography planning framework that decouples high-level formation, timing, and role decisions from low-level motion synthesis, achieving state-of-the-art performance in group dance generation by leveraging multi-scale music conditioning and frozen LLM semantic intent to guide a structured Director module, thereby addressing the critical gap in macro-level formation evolution and inter-dancer interaction control in existing methods.
Methodology
The paper proposes ChoreoStudio, a hierarchical framework that explicitly decouples choreography planning (what, when, who) from motion synthesis (how). The core novelty lies in the "Director" module, which generates a structured plan consisting of relative formation geometry, transition timing gates, and dancer mobility allocations. This plan is derived from multi-scale music features (frame-level rhythm and segment-level structure) and semantic intent from a frozen omni-modal LLM (Qwen3-Omni). The method addresses a specific gap in existing group dance generation: the lack of explicit control over macro-level formation evolution and asymmetric role assignment. The use of a frozen LLM for high-level semantic planning is a strong architectural choice, leveraging recent advances in multimodal understanding without requiring fine-tuning of the large model. The explicit plan-to-motion interface, using frame-aligned instruction tokens and cross-dancer attention biases, is technically sound and ensures that the generated motion adheres to the planned choreography. The verification-guided refinement loop adds robustness by allowing the system to iteratively correct physical or plan-consistency errors.
Results & evaluation
Experiments are conducted on the AIOZ-GDance dataset, which is a standard benchmark for group dance generation. The paper introduces novel frequency-based metrics (IDDV, Jitter, FMR) to evaluate formation dynamics, which is a significant contribution as standard metrics often miss macro-level structural issues. The quantitative results show state-of-the-the-art performance in Group Motion Realism (GMR) and Group Motion Correlation (GMC) across various group sizes (2-7 dancers). The ablation studies effectively demonstrate the contribution of each component, particularly the Director and the semantic description. The controlled intervention experiments are a strong point, providing causal evidence that the model responds to changes in the planned formation, timing, and mobility. The comparison with baselines like TCDiff++ and ST-GDance is fair and uses a shared protocol.
Reproducibility
The paper provides detailed descriptions of the architecture, training objectives, and evaluation metrics. The use of a frozen LLM (Qwen3-Omni) and standard diffusion models enhances reproducibility compared to custom large models. However, the specific implementation details of the "Director" and the exact prompts used for the LLM are not fully specified in the main text (referenced to appendices). The code availability is not explicitly stated in the provided text, but the detailed methodology suggests it is reproducible for a skilled team. The use of standard datasets and metrics further aids reproducibility.
Limitations
The method relies on a frozen LLM for semantic planning, which may limit the adaptability to novel or complex musical styles not well-represented in the LLM's training data. The planning is done at a segment level, which may not capture very rapid or micro-level choreographic changes. The evaluation is limited to the AIOZ-GDance dataset, and generalization to other datasets or real-world scenarios is not tested. The computational cost of the iterative refinement loop and the LLM inference may be significant for real-time applications.
Broader impact
This work has significant implications for the field of motion generation, particularly for applications in virtual reality, gaming, and automated video production. By making choreography planning explicit and editable, the system enables more controllable and interpretable group dance generation. The approach could be extended to other multi-agent motion tasks, such as crowd simulation or robotic swarm coordination. The integration of LLMs for high-level planning in motion generation is a promising direction that could lead to more sophisticated and semantically aware generative models. ChoreoStudio introduces a novel explicit choreography planning framework that decouples high-level formation, timing, and role decisions from low-level motion synthesis, achieving state-of-the-art performance in group dance generation by leveraging multi-scale music conditioning and frozen LLM semantic intent to guide a structured Director module, thereby addressing the critical gap in macro-level formation evolution and inter-dancer interaction control in existing methods.
Author of this paper? Send a correction.