The Project

The use of LED as a backdrop for virtual production is well documented. The in-camera effects that can be created have opened up a host of new opportunities for filming and for productions of all sizes. One of the limitations of lighting a scene with LED video walls is that it only casts a soft light. It is not possible to create a hard edge. This soft light is great for ambient lighting, casting reflections, and setting the tone and mood of the scene. However, if the scene requires that hard light, ie. a spotlight, beams of sunlight coming through a window or a hard shadow from a real object, etc. then the addition of more traditional lighting fixtures are required.Depending on the scene that has been built within Unreal Engine, this solution can be limiting. It can be difficult to match the colour temperature and intensity with traditional lighting fixtures to the lighting generated in Unreal Engine, especially in dynamic scenes. It also takes a lot more time to change any of the “real” lighting characteristics if the lighting within UR4 is changed. This slows down the production which can have severe budget implications.The lighting functionality within Unreal Engine 4 is outstanding and from the previews that have been released, it looks like this capability is being greatly enhanced in Unreal Engine 5. We want to investigate the use of high output digital projectors to be used as a light source throwing the exact light generated in UE4.We expect that by tracking the subject in a space we can mask the projectors to only cast light on the subject or a predetermined area. We expect that this will create the possibility for more realistic dynamic lighting effects to be achieved in-camera.

The Potential Applications

Using digital projectors offers the opportunity to produce soft lighting elements with a hard lighting element through it. We can create the movement, colour, and dynamics of the light generated within Unreal Engine and output the light exactly as designed. 

We believe that there are several real-world applications for this type of use. It is a strong tool to be used in film and drama. It creates a number of creative possibilities in the advertising and content marketing space.

Live theatre and opera designers are always looking for new technology to help tell their stories on stage and to help enthral live audiences.

The use of LED as backdrops for theatre shows is not unusual and several productions have used projection as a light source in their productions

The image is generally driven as a static image with limited ability for interaction and dynamic movement. Productions also have to rely on complex media servers to manipulate and control these images. 

We believe that there is a great opportunity to develop a workflow for live theatre that allows the creative teams the ability to create, backdrops, lighting (both soft and hard), and interactivity within the Unreal Engine universe.

The Team

Reverist is an independent production studio based in Dublin, Ireland. 

Our team of creatives, designers, artists, engineers, and makers use technology to help fuel creativity and bring imagination to life. 

We work in television, advertising, film, theatre, and live events.

Our sister company Eventco is a leading production design company working on live events and spectaculars across Europe. 

We have been working on virtual production workflows for the past year. While talking to other collaborators in the television, theatre, and corporate markets we have discovered several roadblocks that might inhibit us from using Unreal Engine in these industries. The progress being made by companies similar to ours around the world has been amazing. This free sharing of information and learning has allowed us to find solutions to many workflow challenges. 

We have discovered that the issue of hard light is a recurring conversation that has come up. Many people exploring virtual production work in the film and television industries, as stated we want to find an intelligent solution to help us in live events as well as film and television.. 

We are a partner with N-Cam. We are the first independent production studio to take delivery of the new mk II camera tracking system. We will be using the N-Cam system to generate our spatial and tracking data.

We expect to be pushing the boundaries of this technology and look forward to working with the N-Cam engineers to find hardware solutions when needed. 

For this project, we are working very closely with a lighting designer called Adam Silverman. https://www.adamsilverman.co.uk/  He has a long history of working on cutting edge theatre, opera, and live performance events. 

His insight into the requirements of live theatre and opera productions is invaluable in working on providing a workable solution to the industry

We are also working with Peter Jordan. Peter is one of the foremost theatre production managers in Ireland and the UK.

His knowledge of theatre production, venue limitations, and the realities of touring shows is important in developing a solution that will work for theatre companies.

The outcome of this project

Our goal is to develop a process that allows us to use projection as a light source. in film and broadcast, We want to be able to create hard lighting elements that are generated from UE4 but do not interfere with the camera frustum if shooting against LED or projection surfaces. We also want to promote the use of UE4 as a creative production tool in theatre and opera productions. We would like to have a set of tools that are available to our clients and collaborators that show the power of Unreal Engine and the ease of use to incorporate it into live productions with ease and reliability.