Automatic Song Aesthetics Evaluation Challenge

A competition aim at fostering the development of automatic models that can predict human aesthetic ratings of generated songs.

Call for Participation

With the rapid growth of generative music models, such as song generation (composing melodies, lyrics, harmonies, and vocals), we are entering an exciting new era of personalized music, virtual artists, and multimedia content creation. Despite these advancements, the evaluation of the aesthetic quality of generated music remains a challenge. Traditional metrics like pitch accuracy and signal clarity fall short of capturing the complex emotional and artistic dimensions of music that matter most to listeners. This challenge aims to create a benchmark for assessing the aesthetic quality of automatically generated songs. Participants will develop models that predict human ratings of songs based on musicality, emotional engagement, vocal expressiveness, and overall enjoyment.

Join us to push the boundaries of song aesthetics evaluation and contribute to the future of generative music!

Challenge Overview

The ICASSP 2026 Automatic Song Aesthetics Evaluation Challenge is designed to foster the development of models that can predict human aesthetic ratings of full-length generated songs. We focus on generating songs that align with human perceptions of musicality, emotional depth, and vocal expressiveness. Participants will be tasked with developing models that predict subjective ratings based on audio inputs. Objective: Create models that can predict human ratings of aesthetic quality in songs, including dimensions like overall musicality, emotional engagement, and vocal expressiveness.

Track Settings

The competition consists of two tracks:

Track 1: Overall Musicality Score Prediction Participants must predict a single holistic aesthetic score for each song, representing an overall musical impression of the song’s artistic quality.

Track 2: Fine-Grained Aesthetic Dimension Prediction Participants to predict five specific aesthetic dimensions for each song.

Evaluation

Each track will use correlation-based metrics:

  • System-level SRCC and LCC
  • Utterance-level SRCC and LCC for each dimension, with macro-averaged SRCC as the primary evaluation metric.
  • More to be considered

Timeline

  • September 01, 2025: Registration Opens
  • September 10, 2025: Train set and baseline system release
  • November 10, 2025: Test set release
  • November 20, 2025: Results Submission Deadline
  • December 07, 2025: Paper submission (2-page papers)
  • January 11, 2026: Paper acceptance notification
  • January 18, 2026: Camera-ready paper submission
  • May 4-8, 2026: ICASSP 2026 in Spain

Organizers

The challenge is organized by a distinguished team of researchers:

  • Lei Xie, Northwestern Polytechnical University (China)
  • Hao Liu, Shanghai Conservatory of Music (China)
  • Wenwu Wang, University of Surrey (UK)
  • Wei Xue, Hong Kong University of Science and Technology (HK)
  • Yui Sudo, SB Intuition (Japan)
  • Ting Dang, University of Melbourne (Australia)
  • Haohe Liu, Meta (UK)
  • Hexin Liu, Nanyang Technological University (Singapore)
  • Jingyao Wu, Massachusetts Institute of Technology (USA)
  • Hao Shi, SB Intuition (Japan)
  • Jixun Yao, Northwestern Polytechnical University (China)
  • Huixin Xue, Shanghai Conservatory of Music (China)
  • Ruibin Yuan, Hong Kong University of Science and Technology(HK)
  • Guobin Ma, Northwestern Polytechnical University (China)

Contact

For any inquiries, please contact: Email: yaojx@mail.nwpu.edu.cn

Are you ready?

Get started with track 1