Uncategorized

Four AI composition tools easy enough to soundtrack your film masterpiece

30Views
Not only did I film this bit of dog fan fiction, I composed the soundtrack, thanks to technology. (Click here for transcript)

It used to be you could spend an afternoon drumming up a home movie with your little sister, soundtrack it with your favorite mixtape cuts, and upload it to the Internet for sharing without a care. But as bot-driven copyright tools started scanning new uploads and society generally became more concerned with proper song licensing, using whatever track you desired for some publicly available Web video became less of an option.

What's an amateur-at-best musician to do? I may have marched on a collegiate snare line (and therefore understand rhythm, phrasings, and tempo), but my ability to create melody probably stopped with middle school recorder lessons.

Luckily, we musically challenged filmmakers and podcasters now have robots. A number of high-profile AI composition initiatives have surfaced in recent years—perhaps most notably, Sony's Flow Machine released its debut album in January—and slowly but surely these tools are moving from the research labs and professional production studios into publicly available spaces.

So when I was recently left by myself for a weekend with just my dog for company and a desire for late night tacos, I decided the time had come to reenter the short film arena. The above documentary on Ernie the Shih Tzu is scored entirely by smart composition tools you can access right now. Would-be children's publishers: feel free to email me directly about his expanded universe. For everyone else, here's the lowdown on four song-generating programs you can use to help people get down.

Chrome music maker

  • This, this right here is a spy thriller jam. (Tonal notes are represented by colored blocks, snare hits are triangles and bass drum hits are circles.)
  • By default, Song Maker offers you a standard bar of music in 4/4 time with the C Major scale. But you can go ahead and funk things up within a small range of options.
  • In addition to changing your song structure, Song Maker has limited options for selecting instrument voicings and playing with tempo (40 to 240 bpm).
  • For composing, the issue with Song Maker may be that you can only do one melodic instrument at a time (here I'm creating a string part I'd like to accompany my earlier beat, but no luck within the program).

Released on March 1, Chrome's Song Maker represents the newest of these tech-y composition tools. Technically, this isn't an artificially intelligent composer. Instead, it's a tool that simplifies composition to the point where anyone can do it regardless of your familiarity with pitch or rhythm. It greets users with a grid representing beats on the X axis and tones on the Y, and filling in any individual square will generate a sound. You have five options for tonal instruments (piano, synth, strings, marimba, woodwind) and four for percussion (electronic, blocks, kit, conga). You can manually point and click; you can add through a MIDI keyboard or by singing into a mic; you can even let your keyboard keys do the songwriting if desired.

Song Maker comes from Chrome Music Lab, a Google initiative to build browser-based tools for learning basic music principles. The Music Lab has a dozen or so tools already focusing on everything from oscillators to rhythm to arpeggios. So while this may be aimed literally at the elementary school demographic most of the time, Song Maker has enough customization options for any novice composer to be effective. You can make your loop up to 16 bars, change into funky time signatures like 5/4 or 12/8, break the note subdivisions into triplets or sixteenth notes, choose any key you'd want by alternating scale and starting note, and extend your range up to three octaves.

The lone issue for my purposes? Song Maker only allows you to choose a single percussion and tonal instrument pairing for your composition. I wanted a meatier, spy thriller-inspired ditty for the moment my dog reveals his day job (roughly the 1:21 mark of the video). Ideally, I could combine the basic beat I crafted with a higher-octave string line. But even if you're cautious enough to choose the same amount of bars and the same tempo, there doesn't appear to be a way to layer multiple tracks on top of one another for fuller songs within Song Maker. That requires a basic sound-editing program and the ability to capture sound from a browser in this instance. The limitations of Song Maker therefore prove to be both positive and negative—things are simplified enough that it would be hard to create something truly unlistenable, but compositions have a complexity ceiling, too.

Google’s Magenta tools

  • Have you ever wanted to fuse a piccolo and thunder together into a single instrument? N Synth is for you.
  • The main interface of Infinite Drum, a drum machine based on everyday sounds.
  • You can limit your options by choosing a few of many different subcategories.
  • Ultimately, my beat became a case of beer + animal dog + sci-fi door + sci-fi laser.
  • AI Duet allows me to channel my inner freeform jazz pianist. I tap along however my fingers may lead, and the robots respond with sensible (in a music sense) phrases to build our song.

This being Google, of course there are multiple initiatives in progress to upgrade the music-making process. Compared to Chrome Music Labs, the Magenta initiative represents the headier approach.

Introduced back at the 2016 Moogfest (increasingly a bucket-list event for music tech fans), Magenta wants to leverage artificial intelligence and machine learning to empower anyone to be musically creative.

“The goal of Magenta is not just to develop new generative algorithms, but to 'close the creative loop,'" the team wrote when introducing N Synth last year. "We want to empower creators with tools built with machine learning that also inspire future research directions. Instead of using AI in the place of human creativity, we strive to infuse our tools with deeper understanding so that they are more intuitive and inspiring."

N Synth, a tool that allows you to essentially interweave two instruments and generate a new sound, is likely a little above my pay grade. The same goes for the goods available to anyone who really wants to tap into Magenta's work—the team releases all their tools and models in open source on GitHub. For my purposes, I instead stuck to two of Magenta's available prebuilt tools: Infinite Drum and AI Duet.

Infinite Drum is essentially a drum machine composed of everyday sounds that were organized through machine learning. "The computer wasn’t given any descriptions or tags—only the audio," the GitHub description reads. "Using a technique called t-SNE, the computer placed similar sounds closer together. You can use the map to explore neighborhoods of similar sounds and even make beats using the drum sequencer."

From a user perspective, the tool becomes incredibly easy to use. You scroll around or search for a general category of sound, ultimately settling on four to combine. You can choose to shuffle things up or change the tempo, but otherwise you simply press play from there to get a looped beat.

AI Duet, on the other hand, will help you craft a perfect melody. Tap away however you'd like with whatever skills you might have, and Google machine learning takes over to respond with a sensible melody. The AI leverages machine learning by ingesting tons of compositions, and then its neural network responds to your individual inputs (including key and rhythm). As developer Yotam Mann puts it, "It's even fun to just mash the keyboard. The neural network tries to deliver something coherent from whatever you give it."

Like the Chrome Music Labs offerings, these individual tools function more as singular instruments than full composition platforms at the moment. So coming up with a final product—at the 2:55 mark, you can hear my Homeland-inspired avant-garde jazz closing track—once again requires a separate audio editing and capture program.

Amper AI

  • Do not be alarmed by the blank sound wave in front of you, Amper makes composition as easy as a few clicks. Just choose how long you'd like this musical snippet to last.
  • Here, I'm opting for the "Pro" version of Amper's beta tool. After I've selected my length, clicking the newly created sound brings up all these options along the left rail.
  • Ideally, I want some "Gonna Fly Now"-style brass fanfare for a montage of my dog walking…
  • After choosing Cinematic, I'm prompted for even more granular style choices.
  • …here's the end of that list.
  • After selecting a style, you also get the option to add a handful of instruments from six general categories: synth, keyboards, strings, guitar, basses, and percussion.
  • Pick an instrument, and you're then able to choose what end of the pitch range you'd like this voicing to stay within and what corresponding tone you'd prefer (some instruments have more options than others, of course).
  • Amper's AI will even warn you if you make some wonky choices—like overloading the instrumentation with basses—but ultimately it's up to you if you'd like to click through and hear the results.
  • Choose from a multitude of tempos…
  • …and keys, too.
  • Click save and it's render time (much faster than rendering movies overnight as a college kid, mind you).
  • Amper allows you to export as either an .MP3 or a .wav, and then it asks about your satisfaction and future use given this is a beta.
  • If there are too many options, Amper AI also has a "Simple" version. Here, you simply choose the stylistic/genre option of your choice…
  • …and tell the bots how long you need the music to be.
  • Jukebox, a British company pushing AI composition since 2015, was another publicly available option I toyed with. Generally, it offers less customization than Amper AI and sounds slightly more artificial to these ears.
  • After choosing your genre, duration, and optional climax, next choose whether you want bigger atmospheric instrumentation or something more minimalist.
  • Like all the other song-making tools, Jukebox lets you select an exact BPM.

If the tools above sound too complicated, fear not. As Google pushes forward with Magenta and Sony puts its AI-powered FlowMachine in the hands of real musicians, there is a true AI composition option out there that's available right now to the rest of us: Amper AI.

Amper may not be the first publicly available AI composition tool (the British company behind Jukebox dates back to 2015, for instance), but it's the best combination of ease, customization, and ultimate quality. Amper made some headlines last year when it teamed up with YouTuber Taryn Southern for a single that felt indistinguishable from what you'd hear on Top 40 radio. Now, Southern will even release an entirely AI-driven album, called I Am AI, in May. Different songs will leverage Amper, IBM Watson, Aiva, and Google's Magenta.

“Our ultimate goal is to write music as well as John Williams [and] have it sound like it was recorded at Abbey Road Studios and produced by Quincy Jones,” Amper CEO Drew Silverstein told me back in December. "That’s the musical standard we hold ourselves to, but admittedly we’re not there. There’s a lot of work to be done on the music side of the project before Amper’s product is indistinguishable from human music, at which point it’d be the most valuable tool it could be for creators."

Musicians, not programmers, started Amper the company, so Silverstein and co. view their AI ultimately as a tool and not a replacement (hence the collaboration with Southern rather than promoting a single entirely of the bot's work). They trained their AI on the work of real-world composers much like how AlphaGo learned through observing real-world matches. The result is a tool that, even in its simplified beta iteration, offers a lot of customization while generating almost fool-proof usable results.

In fact, Amper currently has two separate interfaces to choose from: Simple or Pro. The former merely asks you to choose from some pre-set stylistic options and set a duration. The latter allows you to begin with similar stylistic parameters, but you can dig down to change seemingly everything: the number of instruments involved, the type of instruments involved, the tone of instruments involved, the overall key, the overall time signature, etc.

Roughly 30 seconds into my video, you can hear what Amper's simple tool does when instructed to craft "modern folk." And around the 2:00 mark, you experience my inspirational cinematic composition created via Amper Pro in order to score a montage of my dog walking. I may have envisioned "Gonna Fly Now"-style brass lines, but realistic brass sounds are something that eludes all these tools at the moment. Still, this particular composition swells and has soaring strings I can live with.

Am I going to win a best original song Oscar any time soon thanks to new AI instruments? Probably not. But even in the early stages, these tech-y layman's composition tools work. They're easy enough that I'm not spending all day on 30 seconds of sound, and I don't need to understand a lick of music theory. Yet, these tools can do a passable enough job to wash away any regrets I have that I can't get Vagabon to soundtrack my next Ernie short. Bands like that will likely always get called before the bots for projects of substance, but things for the weekend auteurs will only get better from here.

Original Article

Leave a Reply