The Interactive & Immersive HQ

3D Projection Mapping in TouchDesigner

Of the techniques available for projection mapping, 3D projection mapping is one of the more interesting and accurate options available. Instead of relying on drawing masks, 3D projection mapping uses a 3D model of the projection surface, a virtual camera and some math to calibrate the projection. Although it takes a little more upfront effort (you need an accurate 3D model of the surface), it makes calibration a snap.

You’re probably thinking: “Great, sounds expensive!”. Actually, TouchDesigner includes 3D projection mapping functionality right out of the box! We’ll take a look at this more later.

In this article, we’ll dive into 3D projection mapping, review the benefits and the downsides of the technique, and finally look at how you can use this mapping technique in TouchDesigner.

3D Projection Mapping and How It’s Different from 2D Projection Mapping

To start, let’s put together a basic definition of projection mapping. Projection mapping is the ability to map a projection (whether a video, still, interactive, etc.) onto real-world, three dimensional surfaces. 2D and 3D projection mapping are different techniques used to map the projection to the final projection surface.

2D projection mapping is probably what you think of when you think “projection mapping”. It usually involves careful drawing and positioning of many different 2D masks in the projection mapping software to perfectly match the contours of the projection surface. It can be a bit labor intensive, but pretty simple to do (especially if your projection surface isn’t too complex).

3D projection mapping takes a different approach, doing away with the need to draw or manipulate masks. You do need an accurate 3D model of the projection surface, whether that is an actual space that you’re projecting onto (say, a stage with props) or a geometric sculpture you’ve constructed out of paper. Once the model has been loaded into the software, you’ll click on the screen to define a couple of points, which are used to correctly align a virtual camera with the real world projector. Once complete, you should have a very accurately mapped canvas for displaying all kinds of cool graphics on!

The Benefits of 3D Projection Mapping

While 2D projection mapping can be pretty fast to set up in certain situations, it can also lead to a lot of headaches if the positioning of the projector or projection surface changes (you might be in for some heavy editing of masks or even starting from scratch). 3D projection mapping on the other hand can be very quick to set up and calibrate, once the initial 3D modelling of the projection surface is complete.

Even with positioning changes to the surface itself, it can be easier to adjust the positioning of components of a 3D model than having to redraw/reposition a large number of masks in a 2D system. 3D projection mapping can also make it easier to create seamless textures and avoid geometric distortions.

The Downsides of 3D Projection Mapping

The main downside of 3D projection mapping is the heavy lifting required before you start: creating an accurate 3D model of the projection surface. The complexity and length of this task are going to vary greatly depending on how complex your projection surface is and your own ability to create 3D models. That’s assuming you try to go the DIY route and don’t enlist a friend to help or pay someone else to do it for you, which still could mean added time or expense. But, as mentioned above, in a lot of cases this makes for a much easier process down the road.

Get Our 7 Core TouchDesigner Templates, FREE

We’re making our 7 core project file templates available – for free.

These templates shed light into the most useful and sometimes obtuse features of TouchDesigner.

They’re designed to be immediately applicable for the complete TouchDesigner beginner, while also providing inspiration for the advanced user.

camSchnappr: 3D Projection Mapping in TouchDesigner

As we touched on briefly, 3D projection mapping is built in to TouchDesigner! camSchnappr is TouchDesigner’s 3D projection mapping tool, and can be found under the Mapping category in the palette.

camSchnappr is a TouchDesigner-based mapping tool inspired by mapamok (https://github.com/YCAMInterlab/mapamok), a 3D projection mapping tool created by artist Kyle McDonald back in 2012 at the YCAM Interlab. mapamok was developed using openFrameworks and OpenCV, and runs as a standalone app. It needs about 8-12 user defined points for a good calibration. This video from 2012 gives a great demonstration of mapamok’s functionality. Pretty impressive, even today!

Fortunately, both mapamok and its underlying OpenCV-based functionality are open source, meaning that it is possible to bring the functionality to other platforms — which is where camSchnappr comes into play.

camSchnappr implements the same OpenCV cameraCalibrate functionality used by mapamok into TouchDesigner via Python, creating a powerful standalone mapping tool built in to the program. It uses 6 or more alignment points to calculate the position, rotation, scale, and viewing angle of a TouchDesigner Camera component, which is then used for rendering. With camSchnappr built in TouchDesigner ecosystem, it’s super fast to get accurate calibration and start experimenting with projection content!

For some inspiration for putting camSchnappr into use and more information on how to use it, check out:

Wrap Up

3D projection mapping is a super useful tool to add to any aspiring projection mapper’s toolbox, and one that you can get started exploring right away via TouchDesigner’s built in camSchnappr functionality! Hopefully this article has helped give you a better understanding of 3D projection mapping.