3D Projection Mapping in TouchDesigner

Of the techniques available for projection mapping, 3D projection mapping is one of the more interesting and accurate options. Instead of relying on drawing masks and other stuff, 3D projection mapping uses a 3D model of the projection surface, a virtual camera and some math to calibrate the projection. Although it takes a little more upfront effort (you need an accurate 3D model of the surface), it makes calibration a snap. You’re probably thinking: “Great, sounds expensive!”. But actually, TouchDesigner includes amazing 3D mapping tools and functionality right out of the box! We’ll take a look at this more further down.

In this article, we’ll dive into how this technique works, review the benefits and the downsides of the technique, and finally look at how you can use 3D projection mapping tools in TouchDesigner.

3D Projection Mapping and How It’s Different from 2D

To start, let’s put together a basic definition. Projection mapping is the ability to map a projection (whether it’s video mapping, still, interactive, etc.) onto real-world, three dimensional surfaces. 2D and 3D mapping are different techniques used to map the projection to the final surface.

2D projection mapping is probably what you think of when you think of the technique. It usually involves careful drawing and positioning of many different 2D masks in the software to perfectly match the contours of the projection surface. It can be a bit labor intensive, but pretty simple to do (especially if your surface isn’t too complex).

3D projection mapping technology takes a different approach, doing away with the need to draw or manipulate masks. You do need an accurate 3D model of the surface, whether that is an actual space that you’re projecting onto (say, a stage with props, or projected images on a building) or a geometric sculpture you’ve constructed out of paper. Once the model has been loaded into the software, you’ll click on the screen to define a couple of points, which are used to correctly align a virtual camera with the real world projector (or multiple projectors). Once complete, you should have a very accurately mapped canvas for displaying all kinds of cool graphics on!

The Benefits

While 2D can be pretty fast to set up in certain situations, it can also lead to a lot of headaches if the positioning of the projector or surface changes (you might be in for some heavy editing of masks or even starting from scratch). 3D on the other hand can be very quick to set up and calibrate, once the initial 3D modelling of the surface is complete.

Even with positioning changes to the display surface itself, it can be easier to adjust the positioning of components of a 3D model than having to redraw/reposition a large number of masks in a 2D system. 3D can also make it easier to create seamless textures and avoid geometric distortions.

The Downsides

The main downside of 3D projection is the heavy lifting required before you start: creating an accurate 3D model of the projection surface. The complexity and length of this task are going to vary greatly depending on how complex your surface is and your own ability to create 3D models. That’s assuming you try to go the DIY route and don’t enlist a friend to help or pay someone else to do it for you, which still could mean added time or expense. But, as mentioned above, in a lot of cases this makes for a much easier process down the road.

TouchDesigner Projection Mapping Tools: camSchnappr

As we touched on briefly, the ability to create these techniques are built in to TouchDesigner! camSchnappr is TouchDesigner’s 3D projection mapping tool, and can be found under the Mapping category in the palette.

camSchnappr is a TouchDesigner-based mapping tool inspired by mapamok (https://github.com/YCAMInterlab/mapamok), a tool created by artist Kyle McDonald back in 2012 at the YCAM Interlab. mapamok was developed using openFrameworks and OpenCV, and runs as a standalone app. It needs about 8-12 user defined points for a good calibration. This video from 2012 gives a great demonstration of mapamok’s functionality. Pretty impressive, even today!

Fortunately, both mapamok and its underlying OpenCV-based functionality are open source, meaning that it is possible to bring the functionality to other platforms — which is where camSchnappr comes into play.

camSchnappr implements the same OpenCV cameraCalibrate functionality used by mapamok into TouchDesigner via Python, creating a powerful standalone mapping tool built in to the program. It uses 6 or more alignment points to calculate the position, rotation, scale, and viewing angle of a TouchDesigner Camera component, which is then used for rendering. With camSchnappr built in TouchDesigner ecosystem, it’s super fast to get accurate calibration and start experimenting with projection content!

For some inspiration for putting camSchnappr into use and more information on how to use it, check out:

• Exploring Possibilities for Speed and Immersion in Spherical Space with Maotik | Derivative. This article gives an in-depth look at some of Maotik’s installation/performance work, which utilizes camSchnappr for projection mapping.
• TouchDesigner – CamSchnappr – usage and simple cube – YouTube. This video covers the basics of camSchnappr as well as mapping onto a simple paper cube.

Wrap Up: 3D Projection Mapping in TouchDesigner

A 3D projection mapping project is a super useful tool to add to your toolbox as an aspiring projection mapper, and one that you can get started exploring right away via TouchDesigner’s built in camSchnappr functionality! Hopefully this article has helped give you a better understanding of 3D projection mapping.