DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion
1. Abstract
Recent advancements in music generation have garnered significant attention, yet existing approaches face critical limitations. Some current generative models can only synthesize either the vocal track or the accompaniment track. While some models can generate combined vocal and accompaniment, they typically rely on meticulously designed multi-stage cascading architectures and intricate data pipelines, hindering scalability. Additionally, most systems are restricted to generating short musical segments rather than full-length songs. Furthermore, widely used language model-based methods suffer from slow inference speeds. To address these challenges, we propose DiffRhythm, the first latent diffusion-based song generation model capable of synthesizing complete songs with both vocal and accompaniment for durations of up to 4m45s in only ten seconds, maintaining high musicality and intelligibility. Despite its remarkable capabilities, DiffRhythm is designed to be simple and elegant: it eliminates the need for complex data preparation, employs a straightforward model structure, and requires only lyrics and a style prompt during inference. Additionally, its non-autoregressive structure ensures fast inference speeds. This simplicity guarantees the scalability of DiffRhythm.
2. Demo
2.1. English
| DiffRhythm | Lyrics | Audio Reference |
|---|---|---|
# The following lyrics are generated by Deepseek R1 [00:15.00]Thunder cracks the hourglass sky [00:18.30]Petals bleed through concrete cracks [00:21.75]We're the seeds that learned to fight [00:25.20]Roots claw up through fading light [00:28.65]Dust won't settle on our skin [00:32.10]Clocks dissolve where we begin [00:35.55]Fists clenched tight around the sun [00:38.95]Tomorrow's wars are ours to won [00:42.50]FLOWERS OF TIME! We bloom through the grind! [00:45.85]Thorns in our laughter defy the decline [00:49.30]FLAMES IN OUR VEINS rewrite the design [00:52.75]Eternity's hostage? We're breaking the crime [01:03.15]Rusty gears scream in our wake [01:06.55]Horizons bend with every take [01:10.00]Wrinkled years melt like cheap wine [01:13.45]We'll plant flags where timelines twine [01:16.90]Scars glow brighter than the moon [01:20.35]Youth's not sold we stole the tune [01:23.80]Graffiti skies with comet trails [01:27.25]Our chorus drowns out warning wails |
||
# The following lyrics are generated by Deepseek R1 [00:15.00]Golden rays melt the skyline slow [00:19.30]Horizon sighs in violet hues [00:23.85]Whispers paint the clouds with glow [00:28.20]Daylight folds its tired shoes [00:32.75]Silkened winds trace my eyelids [00:37.10]Shadows stretch like cello strings [00:41.55]Twilight hums through ocean grids [00:45.90]Time dissolves in seagull wings [00:50.55]Oh twilight's velvet lullaby [00:54.80]Cradles every scar I own [00:59.25]Crimson waves carry me home [01:03.70]Where the stars stitch wounds unknown [01:08.15]Nightshade blooms in muted tones [01:12.50]Moon dust sprinkles silent vows [01:17.00]Tides erase the day's sharp stones [01:21.35]Salt air mends my furrowed brows [01:25.90]Oh twilight's velvet lullaby |
||
[00:00.52]Abracadabra abracadabra [00:03.97]Ha [00:04.66]Abracadabra abracadabra [00:12.02]Yeah [00:15.80]Pay the toll to the angels [00:19.08]Drawin' circles in the clouds [00:23.31]Keep your mind on the distance [00:26.67]When the devil turns around [00:30.95]Hold me in your heart tonight [00:34.11]In the magic of the dark moonlight [00:38.44]Save me from this empty fight [00:43.83]In the game of life [00:45.84]Like a poem said by a lady in red [00:49.45]You hear the last few words of your life [00:53.15]With a haunting dance now you're both in a trance [00:56.90]It's time to cast your spell on the night [01:01.40]Abracadabra ama-ooh-na-na [01:04.88]Abracadabra porta-ooh-ga-ga [01:08.92]Abracadabra abra-ooh-na-na [01:12.30]In her tongue she's sayin' [01:14.76]Death or love tonight [01:18.61]Abracadabra abracadabra [01:22.18]Abracadabra abracadabra [01:26.08]Feel the beat under your feet [01:27.82]The floor's on fire [01:29.90]Abracadabra abracadabra |
||
[00:04.34]Tell me that I'm special [00:06.57]Tell me I look pretty [00:08.46]Tell me I'm a little angel [00:10.58]Sweetheart of your city [00:13.64]Say what I'm dying to hear [00:17.35]Cause I'm dying to hear you [00:20.86]Tell me I'm that new thing [00:22.93]Tell me that I'm relevant [00:24.96]Tell me that I got a big heart [00:27.04]Then back it up with evidence [00:29.94]I need it and I don't know why [00:34.28]This late at night [00:36.32]Isn't it lonely [00:39.24]I'd do anything to make you want me [00:43.40]I'd give it all up if you told me [00:47.42]That I'd be [00:49.43]The number one girl in your eyes [00:52.85]Your one and only [00:55.74]So what's it gon' take for you to want me [00:59.78]I'd give it all up if you told me [01:03.89]That I'd be [01:05.94]The number one girl in your eyes [01:11.34]Tell me I'm going real big places [01:14.32]Down to earth so friendly [01:16.30]And even through all the phases [01:18.46]Tell me you accept me [01:21.56]Well that's all I'm dying to hear [01:25.30]Yeah I'm dying to hear you [01:28.91]Tell me that you need me [01:30.85]Tell me that I'm loved [01:32.90]Tell me that I'm worth it |
||
[[00:22.19]I'm tired of being what you want me to be [00:25.87]Feeling so faithless lost under the surface [00:30.64]Don't know what you're expecting of me [00:34.38]Put under the pressure of walking in your shoes [00:39.66]Caught in the undertow [00:40.94]Just caught in the undertow [00:42.83]Every step that I take is another mistake to you [00:48.44]Caught in the undertow [00:49.55]Just caught in the undertow [00:52.23]I've become so numb I can't feel you there [00:57.23]Become so tired so much more aware [01:01.51]I'm becoming this all I want to do [01:05.83]Is be more like me and be less like you [01:10.01]Can't you see that you're smothering me [01:13.97]Holding too tightly afraid to lose control [01:18.56]'Cause everything that you thought I would be [01:22.41]Has fallen apart right in front of you [01:27.61]Caught in the undertow [01:29.00]Just caught in the undertow |
||
[00:07.23]Fight me fight me fight me [00:08.73]You made me so unlike me [00:10.34]I don't wanna talk come behind me [00:12.07]Know you ain't the one but you might be [00:13.58]Who sent you [00:14.20]Who sent you [00:14.88]Who sent you [00:15.93]Who sent you 'round again [00:20.14]I'm so I'm so shady [00:21.68]I don't really mind when you play me [00:23.46]Wanna switch it up go crazy [00:25.06]I ain't gonna leave till you hate me [00:26.41]Who sent you [00:27.09]Who sent you [00:27.81]Who sent you [00:28.69]Who sent you 'round [00:32.22]We say it's over [00:33.80]But I keep in love with you [00:35.34]And every time I do I wake up with this love hangover [00:40.44]You got me pouring for two [00:42.36]I swore I'd never do it again [00:44.91]Until you came over [00:46.88]I started lying with you [00:48.64]And every time I do I wake up with this love hangover ah oh [00:55.25]I swear I'll never do it again [00:57.84]Ah shit I did it again [01:00.07]Call me back call me back call me back [01:02.07]Oh but you know I'm gonna do it again [01:06.67]Call me back call me back call me back [01:09.84]Yeah you know I'm gonna do it again [01:11.68]I had woke up in a daze [01:12.74]My sweat was cold [01:13.59]My lip was cracked [01:14.63]I can't leave this girl I had to single double triple back [01:17.82]She gon leave me [01:18.58]But she wants to keep me on [01:19.94]What's up with that [01:20.99]I ain't even really got time for all that shit right now [01:22.99]I'm a baby father [01:23.93]And all them jokes she told at dinner last night ain't had me dead I was playing possum [01:27.57]And when the waiter brought the check she said let's head to mine I said okay awesome [01:30.97]One minute we're growing apart |
||
[00:00.61]On the first page of our story [00:05.92]The future seemed so bright [00:11.84]Then this thing turned out so evil [00:17.46]I don't know why I'm still surprised [00:23.83]Even angels have their wicked schemes [00:30.13]And you take that to new extremes [00:36.37]But you'll always be my hero [00:41.80]Even though you've lost your mind [00:47.91]Just gonna stand there and watch me burn [00:53.40]Well that's alright because I like the way it hurts [00:59.96]Just gonna stand there and hear me cry [01:05.51]Well that's alright because I love the way you lie [01:11.39]I love the way you lie [01:17.69]I love the way you lie [01:24.74]Now there's gravel in our voices |
2.2. Chinese
| DiffRhythm | Lyrics | Audio Reference |
|---|---|---|
# The following lyrics are generated by Deepseek R1 [00:15.39]沙漏漏下第一粒光斑 [00:18.82]苔痕爬上褪色的门栓 [00:22.67]你曾蜷缩在 [00:25.13]雨季的褶皱里 [00:27.41]数褪色花瓣 [00:30.09]荆棘刺穿过指缝柔软 [00:33.65]年轮吞下所有的不安 [00:37.36]霜雪落在 [00:39.72]你颤抖的睫毛时 [00:42.91]凝成花瓣 [00:46.35]当晨雾漫过山峦 [00:50.03]你握紧枯萎的勇敢 [00:53.77]裂痕里生出 [00:55.54]千万种绚烂 [00:57.06]废墟上跳动着星火阑珊 [01:00.74]当候鸟掠过云端 [01:03.11]你听见时光在呼喊 [01:07.89]盛开吧 盛开吧 [01:11.43]所有蛰伏的绚烂 [01:15.02]都是岁月给的答案 |
||
[00:16.53]我一大早 起不来 [00:19.82]我的眼睛 都没睁开 [00:22.69]蝴蝶 都飞起来 [00:26.61]畅快 [00:28.18]忽然听见 [00:29.20]好听的音乐 [00:32.35]就继续用力 [00:33.29]闭上 我的眼睛 baby [00:39.37]这音乐 太动听 我怎么办 [00:43.05]我太开心 所有烦恼 被洗干净 [00:51.61]我一大早 就起来 [00:54.85]我的声音 都没放开 [00:57.79]歌声 却飞起来 [01:26.28]一道光 两支花 三个人 四分五裂 [01:29.58]六六大顺 七上八下 |
||
[00:18.03]有女孤身 [00:23.37]策马昂扬 [00:28.68]战鼓声声 风沙铸戎装 [00:44.01]有女铮铮 [00:49.35]热血难凉 [00:54.60]问君何人 邀我入月光 [01:09.96]我只爱 爱旗鼓相当 [01:16.26]只有此刻的滚烫 [01:19.74]无来日方长 [01:25.92]我一生 未羡过鸳鸯 |
||
[00:06.79]失联了几年 [00:10.03]意外的邀约 [00:12.93]你像搁置太久 [00:15.57]忘了洗的底片 [00:19.42]深刻的从前 [00:22.33]禁不起陈列 [00:25.85]原封不动 [00:27.38]才得以保全 [00:30.42]我最终没准备 赴的约 [00:36.76]想不出该化解 的心结 [00:43.14]时间早代替我 已读已回 [00:49.45]抱歉 [00:51.29]无须 这杯 常温的咖啡 [00:57.36]和你 多难 单纯的聊天 [01:03.79]不愿让你误会 交深却言浅 [01:09.70]被遗憾美化的昨天 [01:13.19]不涉及眷恋 [01:16.39]追忆 何必 再比对细节 [01:29.02]或满桌餐点 |
||
[00:18.23]让我掉下眼泪的 [00:21.80]不止昨夜的酒 [00:26.06]让我依依不舍的 [00:29.91]不止你的温柔 [00:33.79]余路还要走多久 [00:37.92]你攥着我的手 [00:41.81]让我感到为难的 [00:45.78]是挣扎的自由 [00:51.81]分别总是在九月 [00:55.66]回忆是思念的愁 [00:59.77]深秋嫩绿的垂柳 [01:03.55]亲吻着我额头 [01:07.55]在那座阴雨的小城里 [01:11.47]我从未忘记你 [01:15.37]成都 带不走的 只有你 [01:23.51]和我在成都的街头走一走 |
||
[00:16.25]那是一座座山连着山的小村庄 [00:22.97]那有一群群孩天天念想着远方 [00:29.39]有一家 小卖部 [00:32.86]夜夜开到凌两点 [00:37.23]不知她又是思念谁 [00:44.38]那儿月亮弯弯高高挂在了天上 [00:51.28]那时外婆总是倚坐小小板凳上 [00:57.58]等天亮 去茶山 [01:01.05]一朵一朵地摘下 [01:05.48]说是给我买理想 [01:11.60]故乡啊 喃喃讲 静静唱 [01:19.55]思念的人请别来无恙 [01:25.77]他乡路 长又长 去又往 望呀望 |
3. Ethics statement
DiffRhythm enables the creation of original music across diverse genres, supporting applications in artistic creation, education, and entertainment. While designed for positive use cases, potential risks include unintentional copyright infringement through stylistic similarities, inappropriate blending of cultural musical elements, and misuse for generating harmful content. To ensure responsible deployment, users must implement verification mechanisms to confirm musical originality, disclose AI involvement in generated works, and obtain permissions when adapting protected styles.