Fulldome 101 – Part #2

In the last post in this series, we talked about fulldome theatres and their inputs and outputs in general and explored a few types of projections. Now that’s just the beginning of working with the fulldome! Before starting to create proper content for your dome project, we need to know what are the options and explore some tools required to make different content types. That is the focus of this post. If you missed the first post in this series, click here.

Recap

Just a quick note here, in the last post, we saw examples of cub maps and equirectangular(spherical) projections to understand how they can capture the whole scene and can be used as our project inputs, but they require extra steps to be converted to fisheye output at some stage of production. We add to the list a third type called ‘Fisheye’ that is the most straightforward projection we can use as both input and output. The reason is that if you consider the entire world around you as a 360 degree environment in all directions, it means you are imagining yourself in the center of a sphere, Fisheye projection captures half of that sphere, a hemisphere that can be easily wrapped around another hemisphere which is the dome screen.

In the third part of this series we will compare all mentioned projections and see how we can convert them to one another using TouchDesigner.

Content Types

Here we are going to categorize fulldome content creation into 3 different pipelines and talk about them in more depth. These are:

  • Photography
  • Film
  • Computer Generated Visuals 
    • Pre Rendered content
    •  Real-time/Interactive content
Great content creation ideas in the article on fisheye photography: https://www.nikonusa.com/en/learn-and-explore/a/tips-and-techniques/something-different-a-new-lens-for-new-views.html
DARK, a 20 minutes Fulldome movie about dark matter in fisheye format
https://www.petermorse.com.au/dark-the-fulldome-movie/
Computer generated visual in fisheye format designed by the author

Photography

Fisheye Photography

Fisheye lenses let you capture a wider range of the scenes. If you’ve ever seen an ultrawide lens, you’ve likely seen a fisheye lens.

There are  two common types of distorted fisheye lenses that photographers use. full-frame fisheye and Circular fisheye.

Full-frame fisheye, also called Diagonal fisheye, covers the entire frame. It captures 180 degrees of the world(or less) diagonally, while circular fisheye offers 180 degree of view horizontally, vertically and diagonally.

Depending on the sensor size there are cameras that support both types. The following short video which the above photos are taken from, depicts both effects using the same camera.

https://www.youtube.com/watch?v=n6q951swsnQ

You can start to see that the circular fisheye lens is perfect for the dome content as it falls in line with the standard we set for the output. So, when investing in cameras for dome photographic content it is important to be mindful of the differences between fisheye lenses and pick the right one.

Followings are examples of fisheye lenses available in the market.

https://en.nikon.ca/nikon-products/product/camera-lenses/af-s-fisheye-nikkor-8-15mm-f%252f3.5-4.5e-ed.html

https://www.canon.ca/en/product?name=EF_8-15mm_f/4L_Fisheye_USM&category=/en/products/Lenses/EF-Lenses/Ultra-Wide-Zoom

Equirectangular imagery (panoramic 360)

Today, most smartphone cameras offer this feature which is achievable with as few as three or four photos. Used for VR content creation, this imagery captures 360 degrees horizontally and 180 vertically. The 360 degree aspect in particular can be very interesting to fulldome content creators, as if you wrap these images around a sphere and then position yourself in the center of that environment and start turning 360 degrees around, you will see the image covers the whole surface or screen around you. 

Later we elaborate on this subject and practice it inside TouchDesigner. But if you are interested and want to use them, the good news is that there are many free sources, from which you can download examples of this type of imagery called HDRI (High Dynamic Range Image). You can use them as background in your projects or texture of your lighting and create great looking ambience and reflections in TouchDesigner. Two great sources sources where you can practice these techniques are:

 Elburz’s blog on improving the quality of your renders in TouchDesigner

https://interactiveimmersive.io/blog/3d/how-to-up-your-touchdesigner-render-quality/

and Matthew Ragan’s Pristine Rendering & Post Processing course  In The HQ PRO.

https://share.substance3d.com/libraries/284

This website offers free daily VR tours through panoramic photography:

https://roundme.com/explore

Expectedly, there are many cameras in the market that offer this option, but one model which is greatly priced is the KODAK PIXPRO 360° VR family.

https://kodakpixpro.com/cameras/360-vr

Film

Given the cost of film production, this is not the most common type of content you see in fulldome projects. The common bottleneck is the output resolution. To record the content in correct format, we can use a direct fisheye lens on the camera such as Sigma 4.5mm f/2.8 EX DC HSM Circular Fisheye Lens:

https://www.sigma-global.com/en/lenses/45_28/

 or we can use panoramic 35mm film cameras that capture spherical (Equirectangular) footage such as Spinner 360 and NOBLEX PRO 6/150. For higher resolution spherical footage and to capture the scene perfectly, motorized camera rigs can make the work much easier.

https://microsites.lomography.com/spinner-360/

https://www.noblexcanada.com/noblexproducts.htm

This movie has been filmed with spherical video technique and is from the first person perspective, resulting in a truly immersive footage.

https://www.youtube.com/watch?v=96EChBYVFhU

Another alternative for shooting 360 movies is to use cameras with multiple lenses. The 360 Ladybug camera is a great example. It’s equipped with 6 lenses, captures 90 percent of the entire world (its drawback for fulldome content) and offers 8K or 4K resolutions at 30 and 60 fps respectively.

https://360rumors.com/flir-ladybug5-8k-360-camera-has-six-23-inch-sony-sensors/

Stereoscopic fisheye method uses more than one camera and simulates human visual perception, but the workflow is complex and requires multiple-step to warp and calibrate the content on the fulldome 360 screen.

If you want to know more about this method check the link below:

http://paulbourke.net/papers/cgat09/

As said before, output resolution will be determinant of the quality of your fulldome project when investing in film cameras. Also, another bottleneck that should be taken into account is the space you need for storing your recorded high resolution footage. Here, we focus on the video cameras that offer 4K and can provide you with satisfying output. A few examples are listed below ranging from medium to high end. For all of them you need to use a fisheye lens when recording the scenes.

RED Scarlet video Cameras:

https://www.4kshooters.net/2017/04/27/red-scarlet-w-gets-4k-prores-ipp2-overview-and-new-firmware/

Canon 5D cameras:

https://canon.ca/en/product?name=EOS_5D_Mark_III&category=/en/products/Cameras/DSLR-Cameras/Professional

Panasonic Lumix GH5:

https://shop.panasonic.com/cameras-and-camcorders/cameras/lumix-interchangeable-lens-ilc-cameras/DC-GH5KBODY.html

Computer generated

The power of GPU and CPU’s calculations makes artists capable of producing great pieces that are unique and magnificent right inside their computers. We can use all the available engines such as Unreal, Unity, Blender, and C4D to create content for our fulldome projects. To do this we need to render our project in fisheye format and for that we need a dome camera in our 3D scene. Same types of projections that we reviewed before like cube map and spherical, exist in the default rendering settings of those engines, but for most of them you need 3rd party plugins or extra steps to create a dome camera that gives the fisheye output. 

Some commonly used workflows for rendering in fisheye on different engines are listed below:

Blender:

https://docs.blender.org/manual/en/2.79/game_engine/camera/dome.html#fisheye-mode

Unreal

https://unrealengine.com/marketplace/en-US/product/camera/questions

Cinema 4D

http://blendy360cam.com/

Unity 3D

https://github.com/rsodre/FulldomeCameraForUnity

Interestingly, in TouchDesigner this feature is accessible and you do not need to go through a complicated process to output your content in fisheye format. In the next post we will explore further the great feature of TouchDesigner that allows this to be accomplished.

Now in terms of content, aside from pre rendered material, we can also use all the data visualization tools and sensors like Microsoft Kinect and be as creative as possible and leverage their power in our fulldome project. But when talking about designing your content for the dome, you should draw a bold distinction between your playback and real-time content in terms of TouchDesigner workflow.

For playback content you may use all the features like heavy lightings, detailed texturing, insane geometries, and effects that your engine offers you like global illumination and raytracing. This way you can have your output in 4096X4096 resolution without having to worry about the rendering time and post-processing stage. Of course, you can totally rely on the dome’s projections system and its powerful computer to play your project smoothly. Below are some examples of immersive films and documentaries (pre rendered content) in fulldome format.

https://www.youtube.com/watch?v=toIqNAZeVi8
https://www.youtube.com/watch?v=TyXn4e_ukaI&t=146s
https://www.youtube.com/watch?v=PhiBCmMm7Mc

But for real-time content the case is not the same. A reasonable resolution for your content could even be 2048X2048 unless your device is powerful enough to handle higher resolutions with no or minimum frame drops. You possibly have to make many compromises along the way to make your content prepared for real-time interactivity.

https://www.youtube.com/watch?v=MVjPGlXTWKA

Kinetarium, an interactive fulldome project on: what’s orbiting above our heads? And what does space debris have to do with lasers?

Real-time Fulldome project developed by the author running at 50fps demonstrates that even through a small setup we can expect frame drops due to the size of the input and output

Wrap up

In the end, we talked about many aspects of fulldome content and introduced gears and methods that can be utilized in your dome projects. If you really like to dive into more details and investigate these areas more deeply, check the work of Paul Bourke on his website, including his research and his deep knowledge with respect to fulldome theatre and the science behind it.

http://paulbourke.net/dome/

Here, we assume most readers of this blog work within the boundaries of computer generated content and in the next post, we are going to maneuver around this area in terms of design, tips, techniques, and TouchDesigner’s workflow(s) to enable you to get the decent results for your Fulldome project.