First, a check
Baillie informed Cartoon Brew that, within the early levels of pre-production, he and Zemeckis thought-about a number of methods through which the dolls and their faces could possibly be achieved. These included truly capturing live-action actors and augmenting them with digital doll-like joints, after which utilizing compositing strategies to give them a last ‘heroic’ motion determine physique. Other strategies thought-about have been full movement seize – physique and face – for a fully-cg doll mannequin. To assist them make a choice, a fake shoot with Steve Carell was deliberate and undertaken like a display check.
“It was filmed about a year before we actually started principal photography on the film, with Steve Carell in costume under lighting as if he was doing a makeup test for a live-action movie,” stated Baillie. “At the time, we thought that the most likely methodology that we were going to use was going to be digitally giving him a heroic doll physique, using compositing based methods and augmenting him with digital 3d tracked in doll joints.”
The outcomes from that preliminary check have been disastrous.
“The first time we had a work in progress of that test for Bob to look at, we all agreed that it looked absolutely horrific,” recalled Baillie. “It looked like Steve Carell in a very high-end Halloween costume. There’s just something about the way that humans move and how skin stretches between joints that just didn’t look like a doll at all, even if his joints themselves looked like doll joints.”
The group moved on to think about full movement seize, however utilizing the live-action check as enter. Artists hand roto-mated and hand-tracked about 2000 frames value of Carell’s efficiency, in addition to that of Leslie Zemeckis (who performs Suzette within the movie), as if the actors’ our bodies had been movement captured. The faces have been hand-animated, as if there had been head-cam movement seize.
Again, the outcomes have been thought-about sub-optimal. “Bob instantly responded to it as if he was seeing an animation take from his mocap movie, A Christmas Carol,” stated Baillie. “I had worked with him on that film back in 2007 and 2008. I could just see him just going back into the same place of like, ‘Oh my gosh, we’re going to have to do so much work to get the actors’ faces to really read the soul of the actor.’ There’s something in that motion capture-driven process where there’s a little bit of soul of the actor that’s lost. He just said, ‘I can’t suffer that again on this movie. If this is all we can do, I just can’t do it like this.’”
Baillie went again to the drafting board, however quickly the filmmakers had what he calls an ‘a-ha moment.’ Having realized there was now in existence primarily a movement seize model of Carell’s doll (referred to as ‘Hoggie’) when it comes to physique movement, it was additionally obvious to the staff that live-action footage of Carell from the check additionally existed that matched precisely to the physique movement – in motion and in lighting.
“So we thought, ‘Why don’t we just use little pieces of Steve Carell? Let’s use his eyes and his mouth from the footage, combine it with this perfectly believable cg doll body and see if we can get the soul of Steve Carell to show through in his Hoggie character that way.’ It was, however, a long road from that a-ha moment to actually presenting a final proof of concept.”
To get there, vfx artists from Atomic Fiction had to work out methods to re-project the actual footage of Carell’s face onto a cg proxy face, and particularly have the footage of the eyes and mouth line up correctly with the cg physique and head. Then this wanted to appear to be a plastic doll and never have a human pores and skin texture. “Once we got there,” stated Baillie, “we had developed a process that not only produced a test to help get the movie greenlit, but also served as a baseline for establishing how we would do all 17 doll characters that ended up in the movie.”
Attention then turned to crafting remaining photographs for the movie, a course of that drew immediately upon the advantages of virtual production. This concerned movement capturing the actors in a devoted seize quantity arrange in Vancouver by Profile Studios. At the identical time, the actors have been lit as in the event that they have been being filmed for a actual shot, after which additionally captured with cinema cameras (the identical Arri Alexas used for the live-action parts of the movie).
“At the end of the day,” stated Baillie, “from the motion capture stage, we had motion data for the bodies of the actors, we had information for where the cameras were and what they were seeing, and we had footage of the actors’ faces that was ‘beauty lit’ that was filmed from the exact camera angles that the final shots from the movie were going to be filmed on.”
In order to mild the movement seize stage correctly – and match it to what can be executed in live-action – the cinematographer, C. Kim Miles, went by means of a means of pre-lighting each scene within the movie earlier than any movement seize had been filmed. This was made attainable by producing a real-time cg asset of the Marwen city through which the dolls exist, which based mostly on the precise miniature of the Marwen city that had been constructed by the manufacturing designer, Stefan Dechant, and his workforce in addition to Dave Asling from miniatures home Creation Consultants. Through an iPad app and engineering by way of Unreal Engine, Miles might interactively mild the city, regulate solar positions, and make different digital cinematography selections.
“That enabled us,” stated Baillie, “to know that when we showed up on the motion capture stage, C. Kim Miles could light the characters exactly as he planned. He basically had a blueprint for how the lighting was going to be set up. So that enabled him and his team to work quicker on the mocap stage, because they already knew what they were going to be doing before they got there that day. And what it also allowed us to do was show Kim and Bob in real-time with moving doll characters, in a world that is representative of what the final Marwen town was going to look like, how his lighting actually looks in the town, rather than just guessing at it in a gray void of a mocap stage.”
Armed with that live-action footage and the motion-captured actor our bodies, the visible results group might then get to work on re-projecting the actual faces onto their cg doll counterparts. Each actor had been beforehand laser scanned, and the ensuing scan used to assist sculpt a digital doll head. These scans additionally knowledgeable the precise bodily dolls used on set. Animators hand key-framed facial performances onto the cg doll characters utilizing the reside motion as reference (physique animation would contain arising with doll-like actions).
Vfx artists, moreover, carried out exact object and physique tracks of the live-action footage. The live-action footage of the actual actor’s face and head was projected onto the cg doll’s head. A proprietary course of was then utilized to ‘plasticize’ the top, which primarily de-aged and smoothed out the look to make it appear to be a figurine with an applicable plastic sheen. “The end result,” stated Baillie, “was that the facial performances have every nuance of the actors’ real performances, but they look perfectly integrated, as if they are part of the rest of this plastic doll.”
VFX that works, and on a finances
One cause Baillie and his staff selected that notably methodology for Welcome to Marwen’s dolls was the comparatively restrained price range for the movie. “If we had had $150 million to make this movie,” says the visible results supervisor, “we would’ve been very tempted just to dump tens of thousands of man hours into animating faces that ultimately wouldn’t have looked as good as this methodology that we came up with. But we only had a fraction of that budget.”
Instead, benefiting from live-action footage and the digital manufacturing strategy was in a position to accommodate each price range and authenticity. “I feel Welcome to Marwen and the method we used to make it’s a poster baby for what digital manufacturing may be and the way it might help to contain each division on the bodily manufacturing of the movie,” stated Baillie. “It was very, very cool to see how well it all worked.”