Virtual Production Techniques

We recently discussed the rise of Virtual Production as one of the most promising fields in terms of audiovisual creation. Without a doubt it’s been a game-changer, and because it’s so deeply technology-driven we could dare to say that its state of the art is ever evolving. So today, in order to keep up with its evolution we will be having a look at the main types of Virtual Production tools and methods being implemented in the industry.

The Core Technology

Virtual Production is the result of practices that originated in the gaming and virtual reality fields. This means that software such as Notch, Unreal Engine and TouchDesigner, are used to deploy techniques we’ve been using in video games for decades now, just in an updated, high-end, incredibly mathematical manner. 

The applications of these systems and techniques have made a significant impact in most of the range of content we consume nowadays. The use of Virtual Productions has permeated the fields of interactive Film, TV, Live Events and Broadcasting, Video games, Architecture and Engineering, just to name a few.

Now, let’s have a look at the different technological configurations commonly used in Virtual Production.

LED Volumes

This is perhaps the most common approach to Virtual Production design nowadays. It is characterized by the use of a large, high-definition LED screen that replaces a scene’s background.

By placing the LED volume in a large, comfortable and safe space, studios are now able to create hybrid scenes. Actors and props are placed in front of the LED screen and, through camera tracking, are captured in a mixed reality environment.

In such a creative space filmmakers can achieve shots that would’ve been almost impossible to capture in location shooting, and certainly at a greatly reduced price.

Previs / Techvis / Postvis

These are the three stages of visualization that are transited through during a virtual production project.

Previs

Refers to the pre-visualization process where artists begin to develop 3D storyboards based on scripts and ideas that have been provided by the director and writing team. 

This is a huge advantage since artists are able to start designing a scene way before it is actually shot with actors and props, which in turn allows for deeper idea exploration. And, even though a large amount of the content that is created doesn’t make the final cut, it is through this process of pre-visualization that directors and artists refine the development of complex and challenging scenes.

Techvis

Once a scene has been developed in previs, and shortly before the scene is shot with talent and props, there is a joint action between the virtual art department and the real-world art department. This is the stage at which the integration between the physical world and digital begins.

Since artists and directors already have information about the scene, it is only a matter of fine-tuning both digital and physical components to accomplish a unified scene in interactive cinema.

Postvis

At this stage the captured footage is processed and adjusted by the postvis team. This can range from light to heavy intervention depending on the captured footage, but since Virtual Production can be such an accurate form of creation, the process of postvis is not as dramatic as traditional post-production is. Think about it, all the information about the scene is readily available, so in case there’s a need to introduce further props, CGI characters or adjust mood and lighting settings, it can be executed effortlessly in real time and then captured by the same camera capturing the actors.

Green Screen in Virtual Environments

Even though it was the use of LED volumes that displaced the green screen, there is still space for its implementation in Virtual Production. It, mainly, serves the purpose of integrating “actors” in fully virtual and immersive settings. 

For example, let’s say there is a virtual conference with speakers in different parts of the world. A virtual conference room is created, and the actors are remotely integrated into the scene through the use of a green screen and virtual reality techniques. 

This is a technique that allows implementation and access to public services where a host, agent, or human resource is needed such as virtual art galleries, virtual doctor’s appointments etc. or simply for the realization of global live virtual events. There are many new technologies that might further displace the green screen such as AI/ML based background subtraction and GPU accelerator background subtraction found in Nvidia RTX pipelines.

XR Virtual Production

Extended Reality is a form of Virtual Production that emphasizes on the expanding of sets and virtual studios. The most common arrangements for this expansion are LED “caves” and the use of back plate, camera plate, and front plates. This is a technique used to give further layering to a scene, dividing it into several stages that are composited in order to create a unified scene. 

The back plate refers to the content placed behind actors and props, the camera plate is the area that contains talent, and the front plate provides an area in front of the talent to further develop our scene.

Another form of expansion is the use of LED volumes in select locations for hybrid reality performances and broadcasts. In this case all possibilities are taken into account: the digital content that will be incorporated in the LED projection, the setting where the performance is taking place, and VFX added on top of the scene.

Here is an incredible example by Volvox Studios and Dubfire.

Wrap Up

The world of Virtual Production is a large and varied one, and is taking over the interactive films workflow. Many technologies work in synergy to develop a highly sophisticated product, and it takes some time to get familiar with all the practices involved in this creative process, like user interactivity. I hope today’s information has made the topic more accessible to you TouchDesigner pros, and encourages you to research the areas you’re more interested in! Until next time!