Building Instruments with TouchDesigner and Max

If you’ve spent a lot of time reading through the blog, you know firsthand that TouchDesigner is an immensely powerful tool with an endless scope of possible applications. TouchDesigner pairs nicely with Max/MSP, Cycling ‘74’s premiere visual programming platform for experimental music and audio, creating an unstoppable tag team of industry standard-setting visuals and audio. Just like TouchDesigner, Max excels in quick implementation of complex concepts with a simple node-based visual system that places a focus on interactivity. 

With that being said, I just recently dug my Kinect 2 out of a pile of old tech left untouched since 2013. Exploring its fantastically in-depth implementation in TouchDesigner connected a few audio-nerd dots and left me wondering what it would take to get a Kinect to turn into an instrument. The perfect idea hit me: a TouchDesigner and Max theremin! With a Kinect, some Max programming, a bit of TouchDesigner magic and a sprinkle of OSC, a Kinect theremin is a fun and easy project anyone can do in half an hour’s time!

A Question to OSC…

To properly get numbers out of TouchDesigner and into Max, you’ll have to do a bit of fiddling with OSC. OSC, which stands for Open Sound Control, is a means through which a software can communicate with another independent software through a message system (a lot like MIDI). When working on your own local computer, you’ll only need to specify the port number to send and receive messages through since the server is your local system!

OSC implementation in TouchDesigner is robust and intuitive: all you’ll need for this project is a single OSC Out CHOP. In Max, however, OSC is a bit more difficult to parse. To remedy this, while inside the Max application, go to the File dropdown menu and click on Package Manager. You’ll want to look up CNMAT Externals, a very sturdy and easy-to-use OSC library, and download it. Once you do that, plug in your Kinect and you’re ready to get programming!

Touch and Go

First things first, you’ll want to get everything sorted in TouchDesigner with the Kinect before moving to Max. Start by dropping a Kinect TOP and Kinect CHOP into your project, then double check that World Space Positions in the Kinect CHOP’s parameters is toggled On. Once your Kinect warms up and starts showing data in the Kinect CHOP, you’ll notice there’s a ton of data to parse through; for this short tutorial, you’ll only be needing the coordinates of your left and right hands. To do that, connect the output of the Kinect CHOP to a Select CHOP and in the Channel Names field under the Select CHOP’s parameters, enter this strings:

p1/hand_l:tx, p1/hand_l:ty, p1/hand_r:tx, and p1/hand_r:ty 

Each of these should be entered as is without any separating commas. 

Each of these strings gets you the Kinect’s readout on your hands’ X and Y positions in the Kinect’s world space from -1 to 1. Underneath Channel Names is Rename from, where you can put the same four strings, and Rename to, where you’ll need to rename the strings to:

leftX leftY rightX rightY

Though we’ll only be using leftY and rightX to get an authentic theremin controller, you can use the other two unused parameters for something fun like filtering or changing the tone!

Next, connect your Select CHOP to a Math CHOP, go to the Range tab in the parameters, and set From Range to be -1 to 1 and theTo Range to be 0 to 1. This makes the numbers easier to scale properly in Max, an important step that comes a bit later. Route that into a OSC Out CHOP where you’ll need to do some things before TouchDesigner will start sending information to Max. In the OSC Out CHOPs parameter window, be sure Network Address reads localhost, and Network Port is 5000. 

With that, the TouchDesigner side of things is done, leaving a bit of programming left to do inside Max!

To the Max!

In a new Max project, place a udpreceive object (tip: the N key is a handy shortcut for creating new objects). You’ll be adding two arguments to it to make it work properly, so before moving on, double click the udprecieve and add 5000 cnmat.

This gets Max to listen to port 5000 for any incoming messages, which it then prints every millisecond. Create an OpenSoundControl object and connect its input port to the udprecieve output, then create an OSC-route object. Connect the OpenSoundControl’s secondoutput to the OSC-route’s input to start automatically parsing through incoming OSC messages with just a few more arguments. In the OSC-route object, add the arguments “/leftX /leftY /rightX /rightY” and hit enter – you’ll notice that the OSC-route object now has four outputs, all set to output messages from those four channels. Connect a float object (hit the F key) to each of the outputs, move your hands around, and watch Max spit out Kinect data in real-time!

The last few steps do the same sort of scaling in Max that the Math CHOP does in TouchDesigner. Connect the OSC-route outlet leftY to a scale object, and do the same for the rightX outlet. For the leftY’s scale, you’ll want to enter “0.3 0.75 0. 128.” This scales the OSC input numbers into a number more palatable for Max’s gain~ object, which you can then create and connect directly to the scale object’s output. 

Moving your left hand vertically now raises and lowers the value of the gain~ object, which will soon control the volume of the DIY theremin. Change the rightX’s scale to “0.5 0.77 40. 2000.” to be able to control the theremin’s frequency, from 40 to 2000 Hz, with your right hand. Connect a slide object with the arguments “5 5” to achieve an authentic theremin glissando effect, then connect that to a new cycle~ object, a simple sine oscillator.

Connect the cycle~ to the gain~, and for the final addition, add an ezdac~ object and connect both of its inlets to the gain~’s outlets. Press the ezdac’s toggle and voila, you’ve got a brand new theremin!

Example project

You can download the example projects below for your future reference:

Wrap Up

TouchDesigner’s power as a visual arts platform is incredibly vast, and complemented by Max’s audio prowess, the two make an incredibly versatile and creative pair. This tutorial is just a fun, brief dive into the capabilities of connecting TouchDesigner and Max – do your own exploring, get crafty, and you’ll come up with brilliant audio-visual pieces fit for museums!