PAT 463/563: Music and AI (Fall 2025)
| Instructor | Hao-Wen (Herman) Dong (ude.hcimu@gnodwh) |
| Room | Moore 376 (Davis) or Zoom |
| Days & times | 1:30–3pm, Mondays & Wednesdays |
| Office hours | By appointment |
[Gradescope] [Last year’s course website]
Description
An introduction to the emerging field of AI music. This course introduces students to AI’s applications in music from analysis, creation, retrieval to processing. Example topics include music transcription, optical music recognition, music source separation, automatic music composition, music synthesis, music recommendation and auto-mixing. Students will gain hands-on experience on using AI tools through open-ended assignments and a final project on a relevant topic of their choice. Prior coding experience is recommended.
This course counts as a Flexible Technical Elective for CS-Eng and DS-Eng programs, and an approved course for the MIDAS GDSC program.
Objectives
- Gain an overview of the field of AI music
- Learn fundamental AI concepts and principles
- Learn representative AI tools for music analysis, creation, retrieval and processing
- Gain hands-on experience on using AI music tools
Schedule
| Week | Date | Lecture | Recording | Assignment |
|---|---|---|---|---|
| 1 | Aug 25 | Introduction | ||
| Background | ||||
| Aug 27 | ├ AI & ML Fundamentals | |||
| 2 | ├ |
|||
| Sep 3 | ├ AI & Music | |||
| 3 | Sep 8 | ├ Music Processing Fundamentals & PA 1 Walkthrough | HW 1 due | |
| Sep 10 | ├ Audio Processing Fundamentals | |||
| 4 | Sep 15 | ├ PA 2 Walkthrough & Catch-up | ||
| Sep 17 | ├ Deep Learning Fundamentals | PA 1 due | ||
| 5 | ├ |
|||
| ├ |
||||
| 6 | Sep 29 | ├ Deep Learning Fundamentals | PA 2 due | |
| Oct 1 | └ Deep Learning Fundamentals II | |||
| Analysis | ||||
| 7 | Oct 6 | ├ Source Separation | ||
| Oct 8 | ├ Convolutional Neural Networks & PA 3 Walkthrough | |||
| 8 | ├ |
|||
| Oct 15 | ├ Music Analysis | HW 2 due | ||
| 9 | Oct 20 | └ Music Classification & PA 4 Walkthrough | ||
| Creation | ||||
| Oct 22 | ├ Language-based Music Generation | PA 3 due | ||
| 10 | Oct 27 | ├ Piano Roll-based Music Generation | ||
| Oct 29 | ├ Audio-domain Music Generation | |||
| 11 | Nov 3 | └ Latent-based Music Generation | PA 4 due | |
| Nov 5 | Project Pitch + Catch-up | |||
| Retrieval & Processing | ||||
| 12 | Nov 10 | ├ Music Search & Recommendation | HW 3 due | |
| Nov 12 | └ Music Production & Editing | |||
| 13 | Nov 17 | Discussion | HW 4 due | |
| Nov 19 | Challenges & Opportunities | |||
| 14 | Nov 24 | Project Consultation | ||
| 15 | Dec 1 | Project Presentation | ||
| 16 |
All slides are licensed under CC BY 4.0 
.
Assignments
- All assignments must be completed on your own. You are welcome to exchange ideas with your peers, but this should be in the form of concepts and discussion, not in the form of writing and code.
- Provide proper citations/references for any external resources you use in your writing and code.
- Submit your work to Gradescope.
- All assignments are due at 11:59pm ET on the date specified.
- Late submissions will be accepted for up to a week with 1 point deducted per day.
| Homework | Due |
|---|---|
| HW 1: Real or Fake!? | Sep 8 |
| HW 2: Source Separation | Oct 15 |
| HW 3: AI Song Contest 2025 | Nov 10 |
| HW 4: AI Music Tools | TBD |
| Programming Assignment | Due |
|---|---|
| PA 1: Symbolic Music Processing | Sep 15 |
| PA 2: Spectral Analysis | Sep 29 |
| PA 3: Source Separation | Oct 22 |
| PA 4: Musical Note Classification | Nov 3 |
Project
- Instructions can be found here.
- Individual project is expected.
- Provide proper citations/references for any external resources you use in your writing and code.
- Submit your work to Gradescope.
- All deliverables are due at 11:59pm ET on the date specified.
- No late submissions! Submit your work early and update it later.
| Milestone | Due |
|---|---|
| Pitch | Nov 5 |
| Presentation | Dec 1 |
| Report | Dec 15 |
Grading
All grading and regrade requests will be handled on Gradescope.
- Homework (20%)
- Programming assignments (40%)
- Project (40%)
- Presentation (20%)
- Report (20%)
The final grading scale is as follows.
| A+ | >96 | B+ | 87–89 | C+ | 77–79 | D+ | 67–69 | F | <60 |
| A | 93–96 | B | 83–86 | C | 73–76 | D | 63–66 | ||
| A− | 90–92 | B− | 80–82 | C− | 70–72 | D− | 60–62 |
Computing Resources
- You will want to access a GPU machine via Google Colab or Great Lakes for faster training of deep learning models.
- Check out this 10-min YouTube tutorial to learn how to work with Google Colab. The free version is good enough for the assignments.
- Follow the instructions to learn how to access Great Lakes. You will be provided with ~560 GPU hours for assignments and final project.
Optional Reading
- “Deep and Shallow: Machine Learning in Music and Audio“ by Shlomo Dubnov and Ross Greer
- “Fundamentals of Music Processing“ by Meinard Müller (notebooks)
- “Intelligent Music Production“ by Brecht De Man, Ryan Stables, and Joshua D. Reiss
- Course materials for “Deep Learning for Music Analysis and Generation“ by Yi-Hsuan Yang
- Course materials for “Musical Applications of Machine Learning“ by Juhan Nam
- Course materials for “Machine Learning for Music“ by Julian McAuley
Policies
Attendance & Course Recordings
- On-time attendance to class is expected.
- Course lectures will be recorded and made available on the course website. If you do not wish to be recorded, please contact me to discuss alternative arrangements.
- Please attend in-person for the project pitch and project presentation.
Generative AI Usage
- Feel free to use GenAI tools (U-M GPT, ChatGPT, Gemini, Claude, GitHub Copilot, DALLE, Midjourney, Stable Diffusion, Firefly, etc.) in your workflow. However, you must disclose your usage of GenAI services in your write-ups.
- You take full responsibility for AI-generated materials as if you had produced them yourself: ideas should be attributed and facts should be true.
Plagiarism & Academic Misconduct
- Plagiarism and cheating violate SMTD’s Academic Code of Conduct. All plagiarism, cheating and other academic misconduct cases will be reported to SMTD’s Office of Academic and Student Affairs.
- All assignments must be completed on your own. You are welcome to exchange ideas with your peers, but this should be in the form of concepts and discussion, not in the form of writing and code.
- You must provide proper citations/references for any external resources you use in your writing and code.
Accommodations for Students with Disabilities/Disability Statement
The University of Michigan recognizes disability as an integral part of diversity and is committed to creating an inclusive and equitable educational environment for students with disabilities. Students who are experiencing a disability-related barrier should contact Services for Students with Disabilities ((734) 763-3000 or ssdoffice@umich.edu). For students who are connected with SSD, accommodation requests can be made in Accommodate. If you have any questions or concerns please contact your SSD Coordinator or visit SSD’s Current Student webpage. SSD considers aspects of the course design, course learning objects and the individual academic and course barriers experienced by the student. Further conversation with SSD, instructors, and the student may be warranted to ensure an accessible course experience.
Sexual Misconduct Policy
Title IX prohibits discrimination on the basis of sex, which includes sexual misconduct — including harassment, domestic and dating violence, sexual assault, and stalking. We understand that sexual violence can undermine students’ academic success and we encourage anyone dealing with sexual misconduct to talk to someone about their experience, so they can get the support they need. Confidential support and academic advocacy can be found with the Sexual Assault Prevention and Awareness Center (SAPAC) on their 24-hour crisis line at (734) 936-3333. Alleged violations can be non-confidentially reported to the Office for Institutional Equity (OIE).
Mental Health and Well-Being
Students may experience stressors that can impact both their academic experience and their personal well-being. These may include academic pressure and challenges associated with relationships, mental health, alcohol or other drugs, identities, finances, etc. If you are experiencing concerns, seeking help is a courageous thing to do for yourself and those who care about you. If the source of your stressors is academic, please contact me so that we can find solutions together. For personal concerns, U-M offers many resources, some of which are listed at Resources for Students on the Well-being Collective website. You can also search for additional resources on that website.