Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music

Hao-Wen Dong¹   Chris Donahue²   Taylor Berg-Kirkpatrick¹   Julian McAuley^1
1 University of California San Diego   ² Stanford University

paper   demo   video   slides   code   reviews

---

Content

• Best Samples
• Cette année-là by Claude François (Figure 1)
• String Quartet No. 11 in F minor, Op. 95, Movement 1 by Ludwig van Beethoven (Figure 2)
• Wer nur den lieben Gott läßt walten, BWV 434 by Johann Sebastian Bach (Figure 3)
• Theme of Universe from Miracle Ropit’s Adventure in 2100 (Figure 4)
• Blame It On the Boogie by The Jacksons (Figure 5)
• Quando Quando Quando by Tony Renis (Figure 6)
• Citation

---

Important Notes

All samples are synthesized using FluidSynth with the MuseScore General soundfont.

---

Best Samples

Mixture (input)	Predicted instrumentation (output)
(guitar)	(piano, guitar, bass, strings, brass)

---

Cette année-là by Claude François (Figure 1)

Colors: piano, guitar, bass, strings, brass.

Mixture (input)
LSTM
BiLSTM
Ground truth

---

String Quartet No. 11 in F minor, Op. 95, Movement 1 by Ludwig van Beethoven (Figure 2)

Colors: first violin, second violin, viola, cello.

Mixture (input)
LSTM
BiLSTM
Ground truth

Original music score:

---

Wer nur den lieben Gott läßt walten, BWV 434 by Johann Sebastian Bach (Figure 3)

Colors: soprano, alto, tenor, bass.

Mixture (input)
LSTM
BiLSTM
Ground truth

Original music score:

---

Theme of Universe from Miracle Ropit’s Adventure in 2100 (Figure 4)

Colors: pulse wave I, pulse wave II, triangle wave.

Mixture (input)
LSTM
BiLSTM
Ground truth

---

Blame It On the Boogie by The Jacksons (Figure 5)

Colors: piano, guitar, bass, strings, brass.

Mixture (input)
LSTM
BiLSTM
Ground truth

---

Quando Quando Quando by Tony Renis (Figure 6)

Colors: piano, guitar, bass, strings, brass.

Mixture (input)
LSTM
BiLSTM
Ground truth

---

Citation

Hao-Wen Dong, Chris Donahue, Taylor Berg-Kirkpatrick, and Julian McAuley, “Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music,” Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2021.

@inproceedings{dong2021arranger,
    author = {Hao-Wen Dong and Chris Donahue and Taylor Berg-Kirkpatrick and Julian McAuley},
    title = {Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music},
    booktitle = {Proceedings of the International Society for Music Information Retrieval Conference (ISMIR)},
    year = 2021,
}