29 Jul 2021 15:00-16:00: MA Xichu (Stan) / AI-Lyricist: Generating Music and Vocabulary Constrained Lyrics
67 views
Skip to first unread message
Min Yen KAN
Jul 28, 2021, 3:55:51 AM7/28/21
to Singapore NLP Group, Wang Ye, Ma Xichu
Dear all:
This Thursday (tomorrow) Stan, a Ph.D. student in Prof. Wang Ye's
Sound and Music Computing Group, will give a research talk (practice
talk) during his lab's meeting on the topic of “AI-Lyricist:
Generating Music and Vocabulary Constrained Lyrics” which will be
presented at ACM Multimedia 2021 as a full paper.
ABSTRACT
We propose AI-Lyricist: a system to generate novel yet meaningful
lyrics given a required vocabulary and a MIDI file as inputs. This
task involves multiple challenges, including automatically identifying
the melody and extracting a syllable template from multi-channel
music, generating creative lyrics that match the input music’s style
and syllable alignment, and satisfying vocabulary constraints. To
address these challenges, we propose an automatic lyrics generation
system consisting of four modules: (1) A music structure analyzer to
derive the musical structure and syllable template from a given MIDI
file, utilizing the concept of expected syllable number to better
identify the melody, (2) a SeqGAN-based lyrics generator optimized by
multi-adversarial training through policy gradients with twin
discriminators for text quality and syllable alignment, (3) a deep
coupled music–lyrics embedding model to project music and lyrics into
a joint space to allow fair comparison of both melody and lyric
constraints, and a module called (4) Polisher, to satisfy vocabulary
constraints by applying a mask to the generator and substituting the
words to be learned. We trained our model on a dataset of over 7,000
music–lyrics pairs, enhanced with manually annotated labels in terms
of theme, sentiment and genre. Both objective and subjective
evaluations show AI-Lyricist’s superior performance against the
state-of-the-art for the proposed tasks.
We will try to finish the seminar and Q/A within 1 hour. Please type
questions via Chat room during the presentation. We can discuss after
the presentation is completed so that we can finish the seminar in
time.
*******************
Ye Wang is inviting you to a scheduled Zoom meeting.
3-4pm, 29 July 2021
Join Zoom Meeting
https://nus-sg.zoom.us/j/89874493179?pwd=NHZuQjZyK2V2cFlhM0E5dkJPQ3VLQT09
Meeting ID: 898 7449 3179
Passcode: 940244
This Thursday (tomorrow) Stan, a Ph.D. student in Prof. Wang Ye's
Sound and Music Computing Group, will give a research talk (practice
talk) during his lab's meeting on the topic of “AI-Lyricist:
Generating Music and Vocabulary Constrained Lyrics” which will be
presented at ACM Multimedia 2021 as a full paper.
ABSTRACT
We propose AI-Lyricist: a system to generate novel yet meaningful
lyrics given a required vocabulary and a MIDI file as inputs. This
task involves multiple challenges, including automatically identifying
the melody and extracting a syllable template from multi-channel
music, generating creative lyrics that match the input music’s style
and syllable alignment, and satisfying vocabulary constraints. To
address these challenges, we propose an automatic lyrics generation
system consisting of four modules: (1) A music structure analyzer to
derive the musical structure and syllable template from a given MIDI
file, utilizing the concept of expected syllable number to better
identify the melody, (2) a SeqGAN-based lyrics generator optimized by
multi-adversarial training through policy gradients with twin
discriminators for text quality and syllable alignment, (3) a deep
coupled music–lyrics embedding model to project music and lyrics into
a joint space to allow fair comparison of both melody and lyric
constraints, and a module called (4) Polisher, to satisfy vocabulary
constraints by applying a mask to the generator and substituting the
words to be learned. We trained our model on a dataset of over 7,000
music–lyrics pairs, enhanced with manually annotated labels in terms
of theme, sentiment and genre. Both objective and subjective
evaluations show AI-Lyricist’s superior performance against the
state-of-the-art for the proposed tasks.
We will try to finish the seminar and Q/A within 1 hour. Please type
questions via Chat room during the presentation. We can discuss after
the presentation is completed so that we can finish the seminar in
time.
*******************
Ye Wang is inviting you to a scheduled Zoom meeting.
3-4pm, 29 July 2021
Join Zoom Meeting
https://nus-sg.zoom.us/j/89874493179?pwd=NHZuQjZyK2V2cFlhM0E5dkJPQ3VLQT09
Meeting ID: 898 7449 3179
Passcode: 940244
Reply all
Reply to author
Forward
0 new messages