One of the most sought-after and exciting techniques used by immersive and audiovisual artists nowadays is the ability to connect sonic and visual creations. This, perhaps due to the fact that a vast majority of what we consume, whether it is commercially or artistically, is undeniably a bond of auditory and visual stimulation. But have you ever wondered how the pros achieve this connection between the audiovisual realms?
The truth is there are many techniques, some decades old and some fresh and new. Today I’d like to introduce you to one of the most popular and versatile techniques available: the OSC Protocol.
What is OSC?
Open Sound Control is a protocol created exactly 20 years ago at the University of Berkeley by Adrian Freed and Matt Wright. It was initially devised as a tool that would allow musicians to exchange performance data relating to their practice. This means that parameters such as gesture and sequencing could be easily shared, extending the possibilities for artists and their communities.
Needless to say, this technology quickly spread to realms other than music. Now, two decades later, it is one of the most popular content formats along with MIDI.
Think of OSC as a refined version of MIDI, where parameters are translated and shared in a much higher resolution, a finer grain, which allows for a more precise language to work with.
This protocol bridges virtually any software, including: TouchDesigner, Ableton Live, Max/MSP, VCV rack, Unreal Engine, Processing, and just so many more. Exciting!
OSC, TouchDesigner and Audio Reactivity
Thanks to the OSC CHOP operators we are able to both receive and send OSC messages, and it is so simple!
First let’s understand how this works, then we’ll move on to some examples.
OSC is a peer-to-peer network, which allows sharing information through ports, or I.P. addresses if connected to a network. This means you can easily connect your phone to your computer, one computer to another, or a network of computers with another network of computers. You can really extend its capabilities.
In the case of TouchDesigner you’ll be using the OSC In operator to receive messages and the OSC Out operator to send messages.
Whatever external module or operator you are using in other software you will come across Ports available for sending and receiving data.
Here’s an example of VCV rack, a virtual Eurorack modular synthesizer studio, and its OSC module (operator) configured to send and receive messages. In this case the idea is to connect Audio data from VCV rack and Visual data in TouchDesigner. Audio reactivity! Note the use of Out and In ports in both software.
Here is the complete and delightful tutorial by Omri Cohen, in case you want to sync sound and visuals between VCV rack and TouchDesigner.
Get Our 7 Core TouchDesigner Templates, FREE
We’re making our 7 core project file templates available – for free.
These templates shed light into the most useful and sometimes obtuse features of TouchDesigner.
They’re designed to be immediately applicable for the complete TouchDesigner beginner, while also providing inspiration for the advanced user.
TouchOSC and TouchDesigner
When learning about OSC and its capabilities I came across the TouchOSC app. What a fitting name, isn’t it? Fun fact: it’s not designed by Derivative, just a coincidence.
Now, if you have never experienced surpassing the use of your mouse to control your TouchDesigner projects and visuals, buckle up.
This app provides the opportunity to use controller templates, customizable of course, in order to interact with your projects. These can be triggered using your mobile phone, a tablet, a touch-screen workstation, and even a Raspberry Pi. The possibilities are vast here and expand way beyond our initial use for audio reactivity in TouchDesigner. And once again, it’s an app that bridges all kinds of software like Notch, Ableton Live/Max, MadMapper, Traktor, Resolume and many more.
Through OSC, the app designates a control surface to control parameters in the software we desire. For instance, you could have a set of sliders on your smartphone’s screen and use them to control operator parameters, and it’s smooth.
Here’s an example of an iPad being used to control TouchDesigner parameters:
Wrap Up
OSC is remarkable. It’s as simple as that. The possibilities this protocol offers are virtually endless. Once you’ve gotten an initial idea of its potential it is only a matter of tailoring to the need of the software from which you wish to extract data from, or send into.
Be sure to check out the resources we’ve left you. They are clear and like I said before, inspiring. I’m sure you’ll enjoy exploring this. Please share your results with us!
