The Interactive & Immersive HQ

Get Off The Timeline #1: Systems Thinking

There’s nothing inherently wrong with the timeline. Let’s get that out of the way. The problem is trying to apply the workflows and programming paradigms of working with a timeline to an environment like TouchDesigner (Or Max MSP, Processing, oF or anything similar for that matter). All the things you might want to do inside of TouchDesigner if you come from a background of heavy timeline-reliance will only bind you, slow down your workflow, and ultimately limit your creativity and frustrate you until you quit using the software all together. “It’s not for me! They don’t even have a working timeline!” How many times have I heard that? Tons. How many of those cases were they using TouchDesigner incorrectly? Almost all of them. And yes you can use TouchDesigner incorrectly, just like you can use a hammer incorrectly or how you can try to use a phillip’s screwdriver on a flat head screw.

Why get off the timeline?

There’s a few reasons you’ll want to get off the timeline (and maybe even keyframed data in general):

  • The timeline in TouchDesigner is linked to the full system rendering, so you can’t just “start and stop” it like you would a normal application without impacting everything else in the project, which effectively makes the application timeline unusable in a final installation deployment
  • Using local timelines could work but then you lose a lot of the native UI elements of the timeline, to the point where you might as well just not use a timeline
  • Hard to control complex and dynamic systems of many different things happening at different times / in parallel if everything has to be linked to a single timeline of events
  • Animation COMP is very limiting in terms of flexibility for scripting and dynamic updating
  • Easier to intercept, control, and redirect individual control values in real-time if they aren’t just keyframed sequences
  • Much easier to build interactive UIs for parts of your project that just aren’t being driven by keyframed data
  • Most of the time you won’t really have the ability to use keyframed data when interacting with unpredictable user inputs

These are just a few reasons, but there are more where that came from when you start digging deeper. So what do we do?

Two approaches

So I’m making big claims here. I’m telling you there are ways you can use TouchDesigner incorrectly. Instead of focusing time and energy on all those things, I’ll provide you the two approaches and states of mind you should be in that will naturally lead to you using TouchDesigner optimally. The two approaches are systems thinking and signals thinking. The first is a real thing that already exists as it’s own discipline and the second is something I made up (maybe a similar thing exists?) because it sounds good next to systems thinking. Luckily for you, in this post we’ll focus on systems thinking, and in next weeks post I’ll write about signals thinking.

Systems thinking

Just to get it out of the way, yes – it is plural systems thinking, not system thinking. Don’t ask me why, I’m just telling you. Regardless, systems thinking is a field of study that revolves around approach problems not from a linear perspective (A -> B -> C) but from a holistic and dynamic perspective, where many different actors, entities, and elements comprise complex webs of interconnection and interdependence. Even if you’re making the simplest movie player in TouchDesigner, systems thinking can help.

For a good introduction to systems thinking in general, this page has a quick preface and links to the PDF that is a good read.

Why systems?

Thinking about our systems as lots of interconnected pieces with their own inputs, outputs, motives, and behaviours is a way of thinking that starts to break you away from the timeline. Timelines are linear, they start in one place and end in another. Everything along the way is dictated and pre-planned. You might even think to yourself “ya, but once my experience starts, it’s like that also!” But it only is at a high level. At a low level, developing a linear system is still a complex system of micro actions that happen in rapid succession to create what appears to be a linear experience. We’re developers not end users. What appears to the end user as a 3-step linear show will in reality probably be about 20 steps of preparation and asset management, that probably have to happen in a specific order, 8 steps of playback management, 4 steps of show control, 4 steps of external hardware control, 6 steps of transitions, 6 steps of timing control, and maybe even more! All of these things will be happening at specific times and almost certainly will be reliant on each other to set things up correctly and ensure the 3-step linear show is properly executed.You can see what I mean when I say that even simpler developments are still complex systems.

Causal loop diagrams

One way you can begin to get away from timeline thinking is force yourself to draw and visualize your program as a system and not a linear series of events and mini loops. One of the tools of systems thinkers is causal loop diagrams. Check out this one as an example:

Causal Loop1

This example I pulled from the web was in regards to employee work schedules and how small changes in the system might effect things in the bigger picture. Does it look familiar? Maybe similar to how your installation show flows might look?

Your shows might have a starting point, some options for the user that branch in different directions, some interactive content, and then maybe another option for the user that branches again before returning back to the starting point. Even an installation as simple as that would spawn a Causal Loop Diagram similar to the above with branches and eventually closes the loop with the beginning.

But that’s still a high level. I’m talking about development. I’m talking about making causal loop diagrams on a really low level. So maybe there’s a starting point I mentioned. What does that starting point entail? Maybe it entails picking one of 5 random attract video loops and it’s corresponding audio. Maybe there are 5 initialization scripts that need to run at the starting point that will prepare elements of the content happening later on. Then maybe it activates the Kinect camera that will detect when a user approaches, and it’s inside a small detection loop for a while as it’s polling the depth values for something close to it. Then a person appears and you leave the polling loop. Then some menu options need to appear for the user, and the previous attract audio and video are crossfaded to something else, which are all their own steps in the diagram. From here we’re in a loop where we’re calculating how long the users hand has hovered over the different menu options to confirm one option or another, and maybe there’s even smaller micro loops where the hand hovering over the option start and LFO that pulses the menu option. So the user picks an element, then maybe from there we have to pick a movie associated with the option and it needed to be preloaded, etc etc.

You can see how this could build up quite quickly into a pretty complex chart with every minute detail and action of the program laid out with all the interconnectedness of actions and actors visualized as a more full and complete system. This may sound like a waste of time but it makes managing complex systems a hell of a lot easier. What happens when you build a really complex system of tons of timed elements and interdependent script chains and then the client asks if you can put a button in the UI that can reset the system back to any other state? You’ve probably heard that before and groaned loudly because of how much it would wreck your existing development. But if you approached things from a systems thinking perspective, these just become yet another small step where some loops can escape back to others. Nothing major, nothing new, no struggle, just a few extra lines of codes to go from one loop point to another.

In reality, speaking from experience, this is what our projects feel like in our heads when we’re trying to think through all the nitty gritty steps and make sure everything connects appropriately to the right place. We could just go ahead and start putting those thoughts into diagrams like this:

https://expandedknowing.files.wordpress.com/2015/05/cldexample.png

Get Our 7 Core TouchDesigner Templates, FREE

We’re making our 7 core project file templates available – for free.

These templates shed light into the most useful and sometimes obtuse features of TouchDesigner.

They’re designed to be immediately applicable for the complete TouchDesigner beginner, while also providing inspiration for the advanced user.

Why so much thinking??

You might reading this thinking “Oh ya, ok that sounds cool, but it’s all just talk, what about actually getting off the timeline?” You’re right, it’s a lot of talk! Because the first step to getting off the timeline isn’t learning how to program without one, it’s learning to think without a timeline. In all the little example projects I talked about above and looking at the diagrams above, none of them would make you think “I really need a timeline to make this possible.”

So some practical steps for you to do to start getting off the timeline?

  • Map every little element and piece of your project as a diagram of some sort, whether you want to look into causal loop diagrams, or use more common block and arrow diagrams that are used for network diagrams. Make sure to close loops whenever possible (movie is playing while waiting for user detection but maybe movie needs to change after 5 minutes of no user, etc etc)
  • Whenever you feel like using a timeline, just make a list of all the different things you’re trying to control, where their signal comes from, and what actions in the system are related to that control signal (in preparation for next weeks blog post!)

Wrap up

With that said, next week’s blog post will focus on some more practical implementations of systems thinking in TouchDesigner, or what I call, signals thinking. For now, even just putting yourself in the habit of thinking about your installations as complex systems (or even just little micro systems) of actions that all work together is a big step forward. If you really commit to this kind of thinking, you’ll soon find yourself not even reaching for timeline functionality in TouchDesigner, you’ll just start building your systems in place, which is the optimal way to work in TouchDesigner.