| Instructor | Hao-Wen Dong (ude.hcimu@gnodwh) |
| Room | Moore 376 (Davis) |
| Days & times | 12:30–2pm, Mondays & Wednesdays |
| Office hours | 3–4pm, Mondays & Wednesdays @ Stearns 131 & Zoom |
[Piazza] [Gradescope] [Canvas]
An introduction to generative AI and its applications to music and audio creation. Topics include music generation, audio synthesis and assistive music creation tools. Students work on a semester-long group project to gain hands-on experience on creating music using AI tools. Prior coding experience is recommended.
| Week | Date | Lecture | Project | Assignments |
|---|---|---|---|---|
| 1 | Aug 26 | Introduction | ||
| Background | ||||
| Aug 28 | ├ Intro to AI Music | |||
| 2 | ├ |
|||
| Sep 4 | ├ Intro to AI Music II | |||
| 3 | Sep 9 | ├ Deep learning fundamentals I | ||
| Sep 11 | ├ Deep learning fundamentals II | Assignment 1 | ||
| 4 | Sep 16 | ├ Guest lecture by Prof. Bryan Pardo | └ due on Sep 20 | |
| Sep 18 | ├ Optimization & CNNs | Assignment 2 | ||
| 5 | Sep 23 | ├ RNNs, LSTMs & Transformers | │ | |
| Sep 25 | ├ VAEs & GANs | │ | ||
| 6 | Sep 30 | └ Diffusion models | │ | |
| Symbolic Music Generation | │ | |||
| Oct 2 | ├ Melody, harmony & chord progression generation | └ due on Oct 4 | ||
| 7 | Oct 7 | ├ Polyphonic music generation | ||
| Oct 9 | ├ Multitrack music generation | Assignment 3 | ||
| 8 | ├ |
│ | ||
| Oct 16 | └ Multimodal music generation | │ | ||
| Audio Synthesis | │ | |||
| 9 | Oct 21 | ├ Time-domain audio synthesis I | │ | |
| Oct 23 | ├ Time-domain audio synthesis II | Team-up | └ due on Oct 25 | |
| 10 | Oct 28 | ├ Frequency-domain audio synthesis I | ||
| Oct 30 | ├ Frequency-domain audio synthesis II | Assignment 4 | ||
| 11 | Nov 4 | └ Multimodal audio synthesis | │ | |
| Nov 6 | Project pitch & discussion | Proposal | │ | |
| 12 | │ | |||
| │ | ||||
| Assistive Music Creation Tools | │ | |||
| 13 | Nov 18 | ├ Neural audio effects | │ | |
| Nov 20 | ├ Auto-mixing | └ due on Nov 22 | ||
| 14 | Nov 25 | └ Live performance & interactive systems | ||
| 15 | Dec 2 | Discussions — ethical concerns & copyright issues | ||
| Dec 4 | Review | |||
| 16 | Dec 9 | Project presentation | Presentation & final report |
All grading will be handled via Gradescope.
| Assignments | 40% | Project | 60% |
|---|---|---|---|
| ├ Assignment 1 | 10% | ├ Proposal | 10% |
| ├ Assignment 2 | 10% | ├ Final report | 20% |
| ├ Assignment 3 | 10% | └ Presentation | 30% |
| └ Assignment 4 | 10% |
The final grading scale is as follows.
| A+ | >96 | B+ | 87–89 | C+ | 77–79 | D+ | 67–69 | F | <60 |
| A | 93–96 | B | 83–86 | C | 73–76 | D | 63–66 | ||
| A− | 90–92 | B− | 80–82 | C− | 70–72 | D− | 60–62 |
| Content | Out | Due on | |
|---|---|---|---|
| Assignment 1 | AI song contest | Sep 11 | Sep 20 |
| Assignment 2 | Musical note classification using CNNs | Sep 20 | Oct 7 |
| Assignment 3 | Generating music using transformers | Oct 2 | Oct 28 |
| Assignment 4 | Synthesizing audio using diffusion models | Oct 30 | Nov 25 |
| Due on | |
|---|---|
| Team-up | Oct 25 |
| Proposal | Nov 8 |
| Presentation | Dec 9 |
| Final report | Dec 15 |
There is no required reading. Here is some good optional reading:
The University of Michigan recognizes disability as an integral part of diversity and is committed to creating an inclusive and equitable educational environment for students with disabilities. Students who are experiencing a disability-related barrier should contact Services for Students with Disabilities ((734) 763-3000 or ssdoffice@umich.edu). For students who are connected with SSD, accommodation requests can be made in Accommodate. If you have any questions or concerns please contact your SSD Coordinator or visit SSD’s Current Student webpage. SSD considers aspects of the course design, course learning objects and the individual academic and course barriers experienced by the student. Further conversation with SSD, instructors, and the student may be warranted to ensure an accessible course experience.
Title IX prohibits discrimination on the basis of sex, which includes sexual misconduct — including harassment, domestic and dating violence, sexual assault, and stalking. We understand that sexual violence can undermine students’ academic success and we encourage anyone dealing with sexual misconduct to talk to someone about their experience, so they can get the support they need. Confidential support and academic advocacy can be found with the Sexual Assault Prevention and Awareness Center (SAPAC) on their 24-hour crisis line at (734) 936-3333. Alleged violations can be non-confidentially reported to the Office for Institutional Equity (OIE).
Students may experience stressors that can impact both their academic experience and their personal well-being. These may include academic pressure and challenges associated with relationships, mental health, alcohol or other drugs, identities, finances, etc. If you are experiencing concerns, seeking help is a courageous thing to do for yourself and those who care about you. If the source of your stressors is academic, please contact me so that we can find solutions together. For personal concerns, U-M offers many resources, some of which are listed at Resources for Students on the Well-being Collective website. You can also search for additional resources on that website.