Useful TouchDesigner Operators: DATs

In the previous posts in this series, we looked at useful TOPs, SOPs, and CHOPs. As you continue working through TouchDesigner’s operator families, you might be looking to start working with Python within the program or to process large amounts of data, and DATs can do both! DATs, short for Data Operators, are TouchDesigner’s operator family for holding text data like strings, scripts, and XML. DATs can contain multiple lines of text (as in a script), or a table of cells (each containing a string) organized in rows and columns. In this post, we’ll take a look at a number of useful DATs in TouchDesigner, including operators that are used for generating and processing data.

Useful Generator DATs

Generator DATs cover a wide variety of functionality, from manually entering text or tables of data, to retrieving data from the web, running Python scripts based on the behavior of other operators, or enabling interaction with 3D rendered scenes.

We’ll start by looking at the Text and Table DATs, which are often the first two DATs you’ll encounter in TouchDesigner.

Text DAT

The Text DAT allows you to edit free-form, multi-line ASCII text. Text DATs can also load their content from external files (either from a .txt or a .dat file). You can use the Text DAT to write Python scripts, GLSL shaders, XML/HTML, notes, and more. As you can see in the example above, it’s possible to run the Text DAT as a script by right clicking on the operator and choosing “Run Script” (or using the keyboard shortcut of Ctrl+R/Cmd+R).

Table DAT

The Table DAT is another common starting point for the DAT family: it lets you hand-edit or create a table of cells. Each cell can contain a string of text, and you can either pre-define the number of rows and columns in the table, or add/remove rows and columns ad hoc. Cells can be manually filled (as in the example GIF above), procedurally filled using the Table DAT’s parameters, or can be filled externally using Python scripts. Like the Text DAT, the Table DAT can also load from external files. You can load a table from a .csv, .txt or .dat file.

CHOP Execute DAT

The CHOP Execute DAT allows you to create custom Python scripts which will be run when certain conditions are met in a specific CHOP’s channels. You’re able to specify the CHOP and the specific channels to watch, and choose from a number of options of what conditions in the CHOP channel will cause the script to trigger. It’s a great tool for building complex functionality stemming from changing CHOP data. There are a number of “Execute” DATs in TouchDesigner, which allow for running scripts based on certain conditions or changes in a specific operator or parameter. These include the DAT Execute DAT, OP Execute DAT, Panel Execute DAT, Parameter Execute DAT and more.

CHOP To DAT

The CHOP To DAT lets you convert CHOP channel values into a DAT table format. As seen in the GIF above, this is a live connection that updates in real-time. The CHOP To DAT can be a useful tool for processing data in conjunction with the Filter DATs that we’ll be looking at later in this post.

Folder DAT

The Folder DAT lists the files and subfolders found in a file system folder and monitors any changes. It allows you to filter for specific filetypes, including images, movies, audio, and more, and has a callbacks DAT included for creating/running custom Python scripts when folder contents change or specific files are found.

MIDI In DAT

The MIDI In DAT is a counterpart to the MIDI In CHOP we looked at in our previous article. Like the MIDI In CHOP, it keeps track of all MIDI messages coming into TouchDesigner from a specified MIDI device. Unlike the MIDI In CHOP, it outputs this data in a table format. Each message received creates a new row in the table, with columns of data showcasing the message type, MIDI channel, index (MIDI note number, controller number, etc.) and value (velocity, controller value, etc.). The MIDI In DAT also includes a very useful callbacks DAT which allows you to build custom Python functionality based on the MIDI information received. All of the data contained in each MIDI message is routed through the callbacks function, allowing for filtering and creating of functions in Python based on message type, MIDI channel, note number, control value, and more.

OSC In DAT

The OSC In DAT is used for receiving and parsing Open Sound Control packets using UDP. Each packet of data is parsed and appended to the DAT’s table. Each row of the table represents one OSC message or an OSC bundle. It’s a great way of receiving communication from other software, such as Max MSP as shown in the example above. Like some of the other DATs we’ve looked at, the OSC In DAT also includes a callbacks DAT for building custom Python functionality based on the data received.

Render Pick DAT

The Render Pick DAT lets you get information about the 3D surface at any pixel of a 3D render, which allows you to implement multi-touch interactivity to a 3D rendered scene. It samples from a rendering and returns 3D information from the geometry that has been “picked”. Typically, you’ll use the Render Pick DAT in conjunction with the Multi-Touch In DAT, which receives messages/events from the Windows multi-touch API. Check out the multiTouch example in the palette under the Techniques folder for an advanced example of a multi-touch/render pick implementation!

Web Client DAT

The Web Client DAT allows you to send HTTP request to web servers from TouchDesigner. Since it includes a callbacks DAT (a common theme among the generator DATs), it’s a great tool for requesting and then parsing through data from the web. You’ll commonly see it used for retrieving data from a web API, and then used in conjunction with Python or additional DATs for parsing the resulting data.

Useful Filter DATs

As in other operator families, the filter DATs are used to modify incoming DAT data. That said, they vary widely as to their actual function, sometimes working to modify table contents, and in other cases converting between different types of data.

Insert DAT

The Insert DAT allows you to insert a row or column into an existing table. In the example above, the initial table of data is missing the header row, which gives context to the data. The Insert DAT is used to insert the header row before the original contents of the table.

JSON DAT

The JSON DAT is a somewhat newer operator to the DAT family (first appearing in the 2021.10330 release), and a welcome one at that! It allows for the formatting and filtering of JSON text without the need to resort to code. In the example above, JSON data has been received from a web API with a Web Client DAT as a raw text string. The JSON DAT is used to first filter through the results for hourly weather statistics, and then output that data in table form, rather than as text.

Merge DAT

The Merge DAT is a multi-input DAT which can be used to merge text or tables together. As seen in the example above, there are a number of options for merging the content of tables together, including appending rows or columns, collapsing rows or columns, and more!

Reorder DAT

Although at first glance the Reorder DAT looks very similar in function to the Sort DAT (seen below), it offers a couple of notable differences. You can use the Reorder DAT to create a specific ordering of the contents of the table, which can be defined by the row/column index or contents. In the second example in the image above, we swap the fifth, sixth, and seventh rows of the table with the first, second, and third. Although not shown in the examples above, you can also repeat the contents of the input table as well — say you wanted to repeat the eighth row three times, it’s as simple as modifying the Order parameter!

Sort DAT

Unlike the Reorder DAT, the Sort DAT can be used to sort table content in a number of ways, including numerically, alphabetically, or randomly. The first example above shows a set of random integers that is sorted numerically, then a set strings that is sorted alphabetically, and finally a numerically arranged set of integers that is sorted randomly.

Substitute DAT

The Substitute DAT is used to change the contents of an incoming DAT using pattern matching and substitution strings. As seen in the example above, this can be used to process both text and tables of data. Using the parameters, it’s possible to only replace the first instance of the matched string, or define a limited scope of rows/columns to replace the contents of.

Wrap-Up

As usual, this is just the tip of the iceberg for the DAT family! There are many more operators worth mentioning here, but, well, we had to stop somewhere. We hope that this post has given you a sense of some of the great functionality that the DAT family has to offer.