How NVIZ and Ncam brought The Irregulars’ RIP character to life on set.
NVIZ and Ncam used AR, virtual production and real-time camera tracking to visualise a RIP in the world, for Netflix’s Sherlock Holmes-inspired series, The Irregulars.
The script called for the Linen Man character to open a tear in the space-time continuum, so during the PreViz sequence, NVIZ was asked to visualise something as abstract as a rip in the wall of a dark cave. This feature became a constant gateway theme in the cave. And this is where lighting became an intricate dance as the scenes progressed. The Rip wasn’t just some magical effect. It was an essential part of the story: a tear between the natural and supernatural worlds that functioned as a character in its own right. As the storyline goes, the barrier between the worlds has begun to collapse. This RIP grows, pulses, changing the effects, strength and direction of the light. This needed to be coordinated and directed, with VFX studios DNEG and UPP at the reigns, all during the COVID-19 pandemic.
“The Rip continuously expands and contracts to reflect key turning points throughout the series,” says Eduardo Schmidek, the Virtual Camera Operator at NVIZ. “These include when the woman Sherlock loves, sacrifices herself to the Rip in order to save the world; or when the show’s villain, The Linen Man, plans to absorb the Rip’s powers to become a God.”
Each of the Rip’s many movements affected both environment lighting and talent interactions in the final shot. This couldn’t be accurately depicted through concept art alone. Instead, extensive previsualisation and AR virtual production from NVIZ was required to give the on-set team full control over the final result.
First, NVIZ artists worked with the Director of episodes 3, 4, 7 & 8 for the series, Joss Agnew, Director of Photography Nick Dance and VFX Supervisor Richard Briscoe to generate extensive CG previz sequences involving the RIP, via remote interactive sessions in Unreal Engine’s Editor mode.
The on-set visualisation enabled the Director and Camera Operator to match the framing established in the Previs sessions by overlaying live, looping animation of an early look-dev of the Rip.
Because they could see a real-time preview of how the Rip would affect lighting, characters and environments as it moved, Agnew and Dance could easily make creative decisions on elements such as framing or camera rigs during principal photography rather than having to rely on imagination alone.
“Ncam was very helpful for integrating the Rip into the environment the way it was required,” Schmidek says. “With Unreal and our proprietary augmented reality tools, we were able to display the Rip floating on the set, even for shots without green screens.”
This was incredibly useful for visualising a sequence where the Rip opens inside a dark cave, illuminating the environment with an electric blue glow. “In the story, the Rip grows, causing the cave to be further bathed in the changing blue light as the scene progresses,” Schmidek continues. “Without AR, you would just see a plain disk and soft light. With AR, we could fully understand and prep the lighting on set, and the effects that were based around it.”
The carefully planned workflow also enabled The Irregulars team to be far more efficient during principal photography, helping them budget how much CG the series would need and showcase the overall pace and rhythm of the scenes before post production.
This was crucial to ensure the series could logistically be completed despite COVID-19, with minimal crew and on-set hours. By the time the project was over, the NVIZ team had used AR powered by Ncam tracking to deliver 23 CGI-heavy shots from just four days of shooting – far more than would have been possible with a traditional workflow.
“Planning each sequence meant there was far less time wasted setting up different camera rigs,” Schmidek explains. “For example, there were some scenes looking down at the Rip from above and it looks like a thin membrane. Being able to see the AR version of the shot was important to not just ensure lighting was accurate, but also to place the camera at the right angle.”
The workflow was such a success that the NVIZ team sent the composited AR versions of the shot straight to post production. This allowed final post and VFX teams – including DNEG and UPP – to work from the Rip’s lookdev as inspiration, speeding up their delivery.
Relate links:
VFX SCIENCE 
awesome