Making visual content respond to audio input is a common task for TouchDesigner developers of all skill levels. Fortunately, TouchDesigner has a built-in tool that makes this job a lot easier, regardless of the type of audio: audioAnalysis! In this post, we’ll dive into the audioAnalysis tool and its features, and wrap up with a number of additional resources that use the tool as the core of their functionality.
Make Anything Audio Reactive with audioAnalysis
In TouchDesigner, there’s a built-in tool that makes creating audio reactive content really straightforward: audioAnalysis, found in the Tools folder within the Palette.
audioAnalysis can be used to analyze input audio waveform for a variety of properties including low, mid, and high frequency levels, along with kick, snare and rhythm detection. Though some of its features are clearly designed for musical application, it can be used with any audio stream and still provide useful results.
audioAnalysis Features and Controls
audioAnalysis includes three distinct feature sets that are useful for creating audio reactive works: levels for specific frequency bands (low, mid, high), percussion detection (kick, snare, rhythm), and spectral tools (density, centroid). All tools can be toggled on and off depending on what’s needed for the particular effect.
All three feature sets can be useful for creating audio reactive visuals, but we’re going to specifically focus on the frequency bands and percussion detection, as they provide the most instantaneous response to the incoming audio.
For the next few examples, we’ve set up a custom Container COMP that shows the audioAnalysis UI on the left, the output of the Mid channel output on the right, and the Mid channel’s contents displayed over a period of four seconds via a Trail CHOP on the far right. This is helpful for visualizing what impact the controls for the frequency bands actually have on the CHOP channels.
Low-, Mid-, High-Level Settings: Threshold
The threshold setting is useful for setting a value or level that the incoming signal has to reach in order for it to pass through. This means that you can use the threshold setting to catch peaks or really loud transients in the frequency band. With the dial turned all the way to the left, the threshold is set to zero, which allows all of the signal to pass through. As you move the dial to the right, more and more of the signal is filtered out until only the loudest peaks remain. Depending on the audio, you may hit a point where nothing passes through at all.
Low-, Mid-, High-Level Settings: Gain
The gain dial adjusts the gain of the signal. With the dial all the way to the left, the signal is multiplied by zero, so no signal passes through. With the dial turned all the way to the right, the signal is multiplied by 5. You can use this setting to help get the output control signal in a useful range for your effect.
Low-, Mid-, High-Level Settings: Add
The Add setting is used to add or subtract a constant value to the signal, which can sometimes be useful as a part of the process of “tuning” the data for your particular visualization. With the dial turned all the way to the left, a constant value of -1 is added to the signal, and on the right +1 is added.
Low-, Mid-, High-Level Settings: Smooth
The Smooth setting is used for smoothing changes in the signal. With the dial all the way to the left, no smoothing is applied. Increasing the dial adds incrementally more smoothing, which can be useful for filtering out abrupt changes or dealing with “noisy” data.
Percussion Detection Settings: Threshold
The percussion detection tools only have one control: threshold. This functions in a very similar way to the threshold we looked at for the frequency bands. With the dial set to the left, the threshold is set to zero and so all of the signal is allowed to pass through and the snare detection constantly triggers. With the dial increased slightly, the detection starts to work better, picking out the snare and some other percussive elements. For this particular audio once the dial hits about 75% the triggers occur only when the snare hits in the track. As with any of the previous settings, you’ll have to experiment with the settings for your particular track.
Get Our 7 Core TouchDesigner Templates, FREE
We’re making our 7 core project file templates available – for free.
These templates shed light into the most useful and sometimes obtuse features of TouchDesigner.
They’re designed to be immediately applicable for the complete TouchDesigner beginner, while also providing inspiration for the advanced user.
Different Types of Audio Content
As briefly mentioned above, audioAnalysis is a useful tool for analyzing audio regardless of its contents. Some of the functions, specifically the percussion detection, will likely not give you any useful data if there is no percussion in the audio you’re trying to analyze, but the other Low/Mid/High channels can still be used.
Here’s a look at a couple of different example audio files and the type of data they produce.
Example Audio: Sonar
In this first example, a demo file of a sonar-like sound is played back. Even though the file doesn’t contain what we would typically define as “musical” content, audioAnalysis is able to turn the pinging sound in the middle frequency band into a usable control signal.
Example Audio: Electronic Music
In this second example, electronic music is played with the Audio File In CHOP. In this case, there are clearly audible drums in the track, which makes it easy for audioAnalysis to track the bass drum, snare drum, and rhythmic percussion parts independently.
Example Audio: Voiceover
In this example, a voiceover is played back. The voiceover contains no musical content or a detectable rhythmic element in the audio, but audioAnalysis still gives a useful signal from the middle and high frequency bands.
Using Audio to Control Visual Content
Regardless of your audio source, once you have made the necessary adjustments to the settings within audioAnalysis, it’s often possible to use the CHOP channels it outputs to directly control parts of your TouchDesigner network and thus make them audio reactive! In the example below, after fine tuning the settings, the mid channel is used to control the Uniform Scale parameter of a Transform POP.
In conjunction with a Trail POP that has a transformation applied, we now have a visual history of the mid channel over a period of four seconds. Any time the mid channel increases, the scale of the circle will increase, creating a direct visual connection to the incoming audio stream.
But this is of course just a single simple example of how audioAnalysis can be used. There are a ton of examples in the TouchDesigner world of audioAnalysis in use.
Here are just a couple of great examples found on The Interactive & Immersive HQ YouTube channel, all of which utilize audioAnalysis to drive the visual effect. As you’ll see in a number of these examples, it’s also possible to continue processing the CHOP data that audioAnalysis produces into even more useful control sources for animation.
Wrap-Up
That’s it for this one! We hope this has given you some ideas for working audio reactivity into projects of your own using the audioAnalysis component.
