Engine COMP Pro Tricks

Engine COMP is shaping up in a way that will truly change the way we make TouchDesigner projects. The ability to control what aspects of our projects are on the main CPU thread or on a different one without having to go through the headache of creating totally new TouchDesigner projects is one of the most exciting features for pros. But the trouble with these new features is that there’s no real precedence or best practices created yet, let alone examples of people using it in the wild that you can look at it. In this blog post, I wanted to talk about a few of the tricks and advanced usages of Engine COMP.

Getting started

We already wrote an introduction to Engine COMP that you should read before diving into this post.

And we have this beginner tutorial about Engine COMP:

If you’ve already worked with Engine COMP a bit, then you should be good to go.

DAT Communication

One of the tricky things with Engine COMP is that while it has TOP and CHOP inputs/outputs built into the system, it doesn’t yet have this same capability for DATs. This is kind of a buzz kill that I’m sure will be added as the Engine COMP is developed further, but for now we need a workaround! The most direct setup we’ve found to be reliable is using Touch In DATs and Touch Out DATs to send DAT information over the network between the processes. Inside of the Engine COMP it could look as simple as this:

And then what you’ll see on the outside of the Engine COMP is:

The nice thing about using the Touch In/Out DATs are that we don’t really need to worry about playing with too many settings. The only thing we need to do is make sure we change the port number every time we make a new set of communication pipes. By default, it is 9500, which works fine for the first set of Touch In/Out DATs you create, but after the first two, you should start incrementing that number. So if you had a second engine and you wanted to communicate DAT data with it, set both the Touch In DAT and Touch Out DAT’s Port to be 9501.

The nice thing about this method is that it can work for both inputs and outputs of the Engine. Above, we showed an example of getting data out of the Engine, but you can also do the reverse to get data into it. You could create a Touch Out DAT in the main project and create a Touch In DAT inside of the engine to catch that data. But just as we mentioned above, you’d want those 2 new Touch In/Out DATs to have a different port number. If this starts to get confusing, make a quick spreadsheet to help you keep track of all the ports in use. Something as simple as this:

At the end of the day, you just want to know where the operators are that are creating network connections, which ports their using, and what they’re being used for. We make spreadsheets like this all the time on projects that have network communication going on.

Engine-ception

One of the coolest tricks we figured out recently is that you can actually nest Engine COMPs inside each other. This can create some seriously impressive workflows all in a single project. We discovered this on our recent Twitch streams with Dylan Roscover when we were examining his Nodeo platform and different ways of optimizing the project and de-coupling the user interface from the main workload. Dylan was already using Engine COMPs to ensure that the four media slots in the media server were able to load without impacting the overall performance of the server. Because the user interface of Nodeo is using a lot of more complex interactions (drag and drops, scroll wheel gestures, etc) we couldn’t easily put the UI into it’s own Engine COMP, because it would be very difficult to interact with it with any prebuilt tools (RemotePanel, etc).

So we decided to see if it was possible to take the whole project, including the 4 existing Engine COMPs, and put it into another Engine COMP that would separate any other system logic and compositing from the UI. Well, it worked! It wasn’t that hard either! The main thing you need is a Container COMP “in between” each Engine COMP. When you’re getting ready to use Engine COMP for the first time, you know that you have to take the part of the project, put it in a Container COMP, then save that as an external tox file which the Engine COMP loads. So if you wanted to nest a single Engine COMP inside of another Engine COMP, instead if trying to right click and save the Engine COMP out as a tox, we recommend taking that Engine COMP, put it inside of a Container COMP, create your inputs/outputs/custom parameters, and then save that Container COMP as a tox to load into the next Engine COMP.

This would look something like this if we took it in steps. First we’ll make our network of a movie file and an Out TOP:

Then we go up a level and save the Container holding those operators:

Then we’ll make a new Container COMP to hold our Engine COMP.

Next step, we load our Engine COMP to look at the container1.tox we saved previously, and we can see the output of it.

Since we want to now put this in another Engine COMP, we need to make sure we create an Out TOP that takes the output from the first engine and will expose it on the next Engine COMP (just like we did above!). Once that’s ready, we can save the Container COMP holding this Engine COMP!

Then the final step is to load this new container2.tox into yet another Engine COMP!

The cool thing you’ll see here that we now have our one Engine COMP loading in the container2.tox, that has an Engine COMP inside of it which loads the movie and outputs it, and all of that data trickles up to our main project area.

Now this example is overly simple just to show you how to approach it, but this is really powerful, especially as you create projects that have lots of heavy areas of processing that you might want to isolate from each other but then move together into another isolated area where their outputs can be dealt with. It’s a really exciting feature and approach to architecture of a project.

Wrap up

We’re loving Engine COMP, plain and simple. I truly feel like it’s going to revolutionize how projects are made as more and more people start to put it in their main tool kit. It’s flexible, powerful, and easy to use. And it’s still being heavily developed. I can’t wait until you can select which GPU the processes will run on, get native DAT inputs and outputs, and more. I think it’s the feature I’ve been most excited about in recent years and I hope these tricks help you get excited and using the Engine COMP too.