ARTS 4880: Interdisciplinary Research Seminar - Music & AI

Arts Department, Rensselaer Polytechnic Institute

Tuesday/Friday, 12:00-1:50am, SAGE 2510

Instructor: Rob Hamilton, West Hall 305/6

e: hamilr4 [at] rpi [dot] edu

w: http://homepages.rpi.edu/~hamilr4/arts4880

Office Hours: TBD

COURSE SYLLABUS - Spring, 2024

PREREQUISITE

ARTS 2380 or ARTS 2020, or permission of instructor

COURSE DESCRIPTION

Advances in machine learning and artificially-intelligent systems are currently being applied to creative tasks such as musical composition and performance. This course is an advanced seminar focusing on the current state of AI research and application in musical domains. Topics to be discussed include the ethics of AI's use in music, the way(s) that trained systems are being applied towards the creation and performance of music and the hands-on application of AI/ML toolsets and frameworks for musical expression.

CATALOG DESCRIPTION

This is a course introducing music majors to advanced research topics of the Rensselaer music faculty. Each semester a member of the music faculty will focus the seminar on a research topic or paradigm related to their own body of artistic and technological research. Sample topics might include Spatial music and sound, New Instrument Design, Network Music, Music Information Retrieval, Ethnomusicology, Sonification Art and Science, Music and Logic, Spectralism and Beyond, Music Herstory (feminist music composition), Experimental music and sound history. Through hands-on creative research, students will explore questions of both musical and technological significance while engaging that same topic through their own hands-on creative practice.

LEARNING OUTCOMES

    Students who successfully complete this course will demonstrate...
  1. an understanding and appreciation of the current statue of AI/ML systems for musical creation
  2. basic technical facility in the application of current tools and frameworks for AI/ML assisted composition, source separation, classification, sound synthesis and post production.
  3. creativity and resourcefulness through the creation of musical AI/ML systems and creation/composition of their own sonic projects

EVALUATION

Evaluation is based on the following:

CLASS PARTICIPATION

You will be required to present some of your assignments to the class, to show your work within the software environment you used to create it, and to engage the class in discussion of your work. When you are not presenting your own work, you need to be attentive to whoever is presenting, and to engage them in discussion of their work. Failure to participate in class will lower your grade.

ATTENDANCE

You must attend class to succeed in this course.

  1. Since much of the class is focused on listening to and discussing work in class, attendance is mandatory.
  2. ** More then two unexcused absences will affect your grade, detracting 1/2 grade each additional 2 unexcused absences. **
  3. Late arrivals are very disruptive - continued late arrival will affect your grade.
  4. It should go without saying but no use of mobile devices or personal computers during class time (except for as required by the coursework itself) is acceptable. Continued violations will be treated as an unexcused absence.

STATEMENT REGARDING ACADEMIC INTEGRITY

Collaboration between students in this course is strongly encouraged. Likewise, students are encouraged—indeed, to some extent required—to exchange ideas, opinions and information . You are also encouraged to help each other in the lab and with performance, production, and presentation of composition projects.

Plagiarism of any kind is in direct violation of University policy on Academic Dishonesty as defined in the Rensselaer Handbook, and penalties for plagiarism can be severe. In this class you will be expected to attribute due credit to the originator of any ideas, words, sounds, or music which you incorporate substantially into your own work. This applies particularly to citation of sources for sonic "samples" included in your compositions.

The use of automated technical aids (e.g. ChatBots, AI code plugins) is not strictly forbidden but if used MUST be documented in great detail and discussed with the Professor prior to use in a graded project/assignment.

Submission of any assignment that is in violation of this policy may result in a grade of F for the assignment in question. Violation of this policy will be reported, as defined in the Rensselaer Handbook

DISABILITY SERVICES FOR STUDENTS

Students requiring assistance are encouraged to contact Disability Services: http://doso.rpi.edu/dss to discuss any special accommodations or needs for this course.

OFFICE HOURS

Office Hours for Spring, 2024 will be TBD

TEXTBOOKS / REPOSITORIES / FOUNDATIONAL MATERIAL


COURSE SCHEDULE

The proposed course topics and schedule will be as follows (take note of project due dates!). Based on class progress and interests, this schedule is subject to change. Special topics, guest lectures, supplemental reading, listening and additional assignments to be announced.

Week 1:
Friday, 1/12

Algorithmic Composition: David Cope and EMI
http://artsites.ucsc.edu/faculty/cope/

Voice before, and in the age of AI

THE VODER @ Bell Labs
Long (audio + images): https://www.youtube.com/watch?v=5hyI_dM5cGo
Short (video): https://www.youtube.com/watch?v=0rAyrmm7vv0

Who owns a song?
Fogerty vs. Fantasy
Blurred Lines: Thicke/Pharrell vs. Estate of Marvin Gaye

Who owns a voice?

Midler vs Ford: https://en.wikipedia.org/wiki/Midler_v._Ford_Motor_Co.
Ford commercial: https://www.youtube.com/watch?v=hxShNrpdVRs
Midler version: https://www.youtube.com/watch?v=QGpwgSo3THE

Waits vs Frito-Lay: http://law2.umkc.edu/faculty/projects/ftrials/communications/waits.html
Commercial: https://www.youtube.com/watch?v=bNIJWCNEzTw

Week 2:
Friday, 1/19

Due: Tone Transfer Sketch

Simple interfacers/instruments for "public" use: Ocarina by Smule: https://www.youtube.com/watch?v=ZoYz4uM9RgY
Feedback loop - Alvin Lucier's "I Am Sitting In A Room": https://www.youtube.com/watch?v=fAxHlLK3Oyk

Week 3:
Tuesday, 1/23

Machine Learning and Music
George Lewis: Too Many Notes: Complexity and Culture in Voyager (2000)
George Lewis demonstrates Voyager: https://www.youtube.com/watch?v=ncy4_FHX3Jc
George Lewis "Why Do We Want Our Computers To Improvise" (Talk): https://www.youtube.com/watch?v=wDP8FsjyCaA

Friday, 1/26

Fun with ChucK: https://chuck.stanford.edu/
* Joystick Chanting: Joy of Chant
* ChucK AI: ChAI, AI Tools
* Ge Wang (2/16): CCRMA Homepage

Chuck/Markov example: smrtMarkov.ck
More on Markov Processes: Simoni: Algorithmic Composition Using Probabilistic Methods
Lisp Markov Example: markov.lisp

Rebecca Fiebrink's WEKINATOR
http://www.wekinator.org/

For class on Tuesday (1/30): Select one AIMC paper and present it to the class
* AIMC Archives: https://aimusiccreativity.org/conferences/

Week 4
Friday, 2/2

NO CLASS

Reembodied Sound Symposium @ EMPAC: https://empac.rpi.edu/events/2024/reembodied-sound/schedule-keynote

Week 5
Tuesday, 2/6

Student AIMC paper presentations pt. 1

Friday, 2/9

Student AIMC paper presentation pt. 2

Week 6
Tuesday, 2/13

ChAI (Chuck AI) Overview

Friday, 2/16

Ge Wang Visit: CCRMA Homepage

ChucK Community Discord: https://discord.gg/RGkZracy

Week 7
Tuesday, 2/20

NO CLASS (Presidant's Day Observed)

Friday, 2/23

Student Project Proposal Presentations pt. 1

Week 8
Tuesday, 2/27

Student Project Proposal Presentations pt. 2

Friday, 3/1

Preliminary Musical AI & Ethics discussion (EASE intro)

Week 9
Tuesday, 3/5

NO CLASS (SPRING BREAK)

Friday, 3/8

NO CLASS (SPRING BREAK)

Week 10
Tuesday, 3/12

EASE Demo/Workshop in class

Week 11
Friday, 3/22

Guest Speaker: Anna-Kaisa Kaila, PhD Candidate, Creative AI Researcher
KTH Royal Institute of Technology, Media Technology and Interaction Design
http://www.kth.se
https://www.creative-ai-project.se/


Week 12
Tuesday, 3/26

Prompt-interfaces for Music, Language and LLMs

Friday, 3/29

Prompt-interfaces for Music (continued)

ASSIGNMENT DUE:

Please complete the following assignment and bring your work to class on Friday to share and discuss:

Given the propensity of prompt-based interfaces for contemporary AI systems, a deeper look at how language is being used in musical AI feels important. For class on Friday, please create an account on one of the following AI Music services (you can use a "burner" email account if you wish) and investigate how that systems converts your human language prompt to criteria for Generative music creation.

- write a one or two sentence description of your goal musical output, this is not necessarily the language you will use for a prompt but instead the language you'd use to communicate to your classmates what it is you want.
- use the prompt interface of the service you're using to try to make your desired output.
- Document your successes, failures, and each step of the process (like you would do for a mathematical proof). Of particular interest are the changes you make to your prompts to make your output "better" or more in line with your target goal. So for each prompt you share with the class, there should be a generated audio output.

You can use any musical AI site/app out there, but here are a few choices that seem to have free tier usages and some level of credibility around them:
https://www.audiocipher.com/post/text-to-music

Week 13
Tuesday, 4/2

The disturbing complexities of AI voice recreation: https://gizmodo.com/mamma-mia-star-replaced-by-ai-in-bbc-project-1851372021
Suno pop song output: https://www.tiktok.com/t/ZPRTPESMJ/
Peter Jackson's Beatles Documentary: Audio Cleanup - https://www.youtube.com/watch?v=sBR5VNWJZmw
New Scientist Article: Beatles Documentary AI Cleanup

Audio Watermarking, How to tell if something was AI Generated (text, images, sound)

Friday, 4/5

One on One Project meetings: sign-up sheet


Week 14
Tuesday, 4/9

One on One Project meetings: sign-up sheet


Friday, 4/12

Guest Speaker: Kosmas Giannoutakis
Ph.D. Candidate, Rensselaer Polytechnic Institute
Film: "Nick Land's Meltdown AI-fication"
- https://www.youtube.com/watch?v=RgqFQENSRdo

Week 15
Tuesday, 4/16

Guest Speaker: Spencer Salazar, Ph.D.
Principal Engineer @Output
https://output.com


Friday, 4/19

Class Presentations

Week 16
Tuesday, 4/23

FINAL CLASS
Class Presentations (continued)