Artificial intelligence is paving the way for promising explorations. We made a list of some of the most interesting AI tools for immersive media projects.
We know, artificial intelligence is everywhere. From chatbots to TV headlines, AI technologies are part of our everyday lives and will play a key role in shaping our future.
As fas as our work is concerned, artificial intelligence is radically changing the creative realm, thus opening new opportunities for experimentation and creative disruption.
There are literally thousands of AI creative tools we can make use of. We’ve made a selection of some of the most interesting ones.
AI Tools for Immersive Media Projects: Visual Tools
It is no doubt that text-to-image systems are one of the most widespread tools for integrating artificial intelligence in the creative process. R&D in this field is running rapidly and new opportunities arise. There are several platforms that are worth to mention. Let’s have a look.
DALL·E 3
DALL·E 3 is currently one of the most popular text-to-image systems. Developed by OpenAI, the third release of the system features a significant increase in realism and coherence to user prompts. DALL·E 3 is natively built on chatGPT and is marked by a further integration with the chat engine.
Currently the web version is available only for ChatGPT Plus, Team and Enterprise users with paid subscription. Developers can however access all the functionalities of DALL·E through the official Python library for the OpenAI API on GitHub.
Stability AI
Stability AI is the company behind Stable Diffusion, one of the leading systems for generative artificial intelligence. Compared to other competitors, the company released open source models that can be easily integrated into personal projects for creative and artistic purposes.
Stable Diffusion features state of the art text-to-image generation algorithms and AI-based solutions for image upscaling, editing and control.
The developer platform offers extensive documentation and examples available in Javascript and Python.
Get Our 7 Core TouchDesigner Templates, FREE
We’re making our 7 core project file templates available – for free.
These templates shed light into the most useful and sometimes obtuse features of TouchDesigner.
They’re designed to be immediately applicable for the complete TouchDesigner beginner, while also providing inspiration for the advanced user.
AI Tools for Immersive Media Projects: Data-Focused Tools
According to forecasts, about 147 zettabytes of data will be created in 2024. One zettabyte is equal to 1021 bytes, so it is no surprise that data is a big business. Aside from that, we can integrate data tools in our creative systems and programming environments. Let’s go.
Pathway
Pathway is a framework for real time operational analytics. It makes use of AI and machine learning to analyze multiple data streams, thus extracting valuable information for business intelligence purposes.
The framework allows for connecting multiple data sources – APIs, local files, cloud folders and databases – to perform operations such as time series analysis, anomaly detection and graph exploration to name a few.
Pathway is a Python-based environment with easy-to-use APIs. The developer website includes several examples as well as comprehensive documentation. It can represent a good starting point for developing AI-data projects in TouchDesigner.
Kedro
Kedro is an open source framework for data analysis and data science. Machine learning algorithms are employed to simplify complex data-based tasks with the ultimate goal of strengthening decision making processes and workflows.
It is a fully open source project – hosted by the Linux Foundation – and it is available as a standalone node-based interactive environment. Kedro is based on a modular system for wide integration with external databases and data pipelines.
The GitHub repository includes extensive documentation and examples. Kedro runs in Python and Javascript, a standalone React component is available as well.
AI Tools for Immersive Media Projects: Tracking Tools
It has been a long time since Microsoft launched the first release of the Kinect, back in 2010. Nowadays, computer vision and image processing tools are common applications, in our daily lives as well as in our creative practice. Let’s have a look at quite few examples.
Google MediaPipe
MediaPipe is the Google machine learning framework for desktop, mobile, web, edge and IoT devices. It is based on TensorFlow, the Google flagship machine learning library. MediaPipe supply users with easy-to-deploy low-code and no-code solutions.
MediaPipe offers several tracking features such as object detection, image classification, hand gesture recognition, interactive segmentation and face/pose detection to name a few. For each feature, code examples are available in Python, Javascript (Typescript), Android and iOS.
api4ai
Api4ai is a company specialized in developing AI-based image processing and computer vision solutions. As the name suggests, api4ai offers ready-to-use APIs that are available on the RapidAPI marketplace.
Tracking features include object detection, background removal, image anonymization, face analysis, image labelling, furniture and household item recognition and much more.
The GitLab repository includes documentation and examples in Python. Api4ai is available both as free version (max 25 credits per month) and as paid/enterprise subscription.
OpenCV and Camera Tracking in Touchdesigner
It is worth to remind that there are already easy-to-implement components in TouchDesigner for image processing and tracking, even if not strictly based on artificial intelligence.
OpenCV is one of the most important computer vision library and is already pre-installed on TouchDesigner. OpenCV is the swiss knife of image processing and can be easily implemented in TD projects with few lines of code.
TouchDesigner is already equipped as well with built-in components dedicated to Kinect Azure, Orbbec, Intel RealSense and ZED cameras.
Wrap Up
When speaking about AI creative tools, the sky is the limit. Integrating artificial intelligence in the creative process opens up new unbounded opportunities.
But before the journey begins, it is important to have clear in mind what the obstacles could be and how to face them.
AI is a complex technology, requiring a good understanding of the technical foundations. Most of all, AI is a set of different technologies, ranging from machine learning to predictive analysis to synthetic data. And each AI is different from the others.
So remember, it is important to focus on the design goals, which requires defining your objectives, embracing the right intelligence and starting to explore.
