Director Jon Favreau’s The Jungle Book won this year’s Oscar for Best Visual Effects. A Walt Disney Picture, the movie is unique because the only live action character, the boy Mowgli, was shot on bluescreen while everything else was computer-generated. The visual effects team included key talent from Technicolor and its subsidiary, the Moving Picture Company. In part one of our five-part series, Adam Valdez, visual effects supervisor with MPC explains how teamwork, talent and a pursuit of excellence combined to create a movie that has been widely acclaimed for its artistic and technological accompaniments.
Creating The Jungle Book with one live actor and digitally created animals in a totally digital environment required a level of synchronization between greatly redefined and fully integrated processes that radically redefine the traditional notions of pre-production, production and post. To support the development complex photo-realistic animal characters and rich jungle environments of The Jungle Book it was critical to develop user interfaces that masked the technical complexities of the collaboration and communications systems so artists could focus on their creative activities.
Hundreds of artists and VFX specialists spread between London, Bangalore and Los Angeles generated many terabytes of data. The movie was color finished by Technicolor in both 2D, 3D and a multitude of high dynamic range deliverables and set the standard for global virtual productions and photorealism work. Valdez coordinated the VFX contributions of the various artists.
Digital Cinema Report: What were some of the unique challenges you faced as you approached this unique movie, The Jungle Book, and what were the resources you brought to bear on the project?
Adam Valdez: When we were first contacted by Disney about the film they described it as a largely virtual film. We would shoot a live action boy but mostly on a blue screen stage. That was something of a head-scratcher. Many films these days have synthetic sequences, or actors, or set pieces, but it was pretty daunting to contemplate doing an entire film that way.
In some ways, it meant we would be tackling an animated feature-scale problem: generating an entire world. So, it was pretty intimidating at first. I had to do some very careful thinking around the practicalities of even shooting something like that.
The great thing is there are a lot of visual effects supervisors at MPC who have lots of different kinds of experience. They are a great resource and I was able to send them emails, make phone calls, have lunch with them and chat with them about ways of shooting this film.
Across the group, we had lots of different examples of how to shoot things that have worked out well, or sometimes not so well. We just spent some time figuring it out and then we started testing, taking each problem as it came, doing a proof of concept and working our way through issue-by-issue.
The Jungle Book was definitely the most challenging movie I have ever worked on in terms of its shooting style and the overall ambition for the film. And it was collaboration with all the other filmmakers that enabled us to pull it off in the end. It was a really big team effort.
DCR: There were many partners involved in this project yet it looks like a really seamless film. The story hangs together; the visuals hang together; everything seems to have come together very nicely. How did you manage this with such a VFX-intensive project that required so much complex input?
AV: Jon Favreau, the director, had spent some time at Pixar and seen their feature animation process and he had noticed that the way they work is very collaborative.
The traditional live action department heads — director of photography, production designer and so forth — are fairly established in live action. He had to do some work to take those roles, a role like mine of visual effects supervisor, the group that was going to deliver the final images and bring us all together at the very beginning and form a team out of us.
That is not the traditional way and it took a bit of getting used to for everybody. The nuts and bolts of it required a lot of planning and scheduling together and then working out how to involve live action filmmakers in what I would call the pre-visualization process, or the layout process.
A lot of movies these days use pre-visualization but they hire a pre-visualization company, specialists who do their work on computers. That does not always allow the live action folks to get their hands directly on the material.
At the end of our process the pre-visualization felt very different to me from the typical pre-visualization I usually see, just because it was live action filmmakers using their sensibilities in the process.
That was the first big step. The next was getting buy-in from those same people with us at MPC to design lighting and sculpt sets in the computer in a post pre-visualization mode that would inform the very specific things we did on set.
For example, there are scenes at the opening of the film where Mowgli and Bagheera the panther walk along tree branches together and have a conversation. To build a branch on set for Neel Sethi, the actor, to walk along we needed that branch to be planned ahead. We wanted to make sure the branch he was walking along on set would mesh neatly into the digital branch we were going to create later, because the Jungle Book was an entirely synthetic world most of the time.
That required commitments to the shape of that branch before we started shooting. Live action folks are used to that: they build a set and then they shoot on it. But in the digital world a lot of things are plastic and because you are in pre-visualization mode you think ‘I can design what I want and we will figure things out.’ Here we had to mix the two and we saw great collaborative energy and spirit between the live action folks and the digital folks.
That was the foundation on which the final work was done at MPC London and at our Bangalore India office where we created the final photorealistic version of the branch and brought the photography and the digital world together.
It was really about Jon Favreau initiating that process and all of us working through those steps of pre-visualization, shooting and final. If you had been there throughout all the previous days and you understood what those choices were and why they had been made you would execute to the plan. That was the way we glued it all together.
DCR: How were you able to leverage the different technologies and talent in other parts of Technicolor for a project like this?
AV: One great example of how we came together with the other Technicolor folks was on the color pipeline for the movie. This was a movie where we wanted to really take advantage of high dynamic range (HDR) imagery.
I started consulting with the color scientists at Technicolor Hollywood and Steve Scott, our colorist who Jon Favreau had worked with several times before. And we worked with our overall visual effects supervisor for The Jungle Book, Rob Legato, and our director of photography, Bill Pope.
We all started talking out what would be the best color pipeline for the show, and I started working on a way to do color for the show that was familiar to myself and Rob and Bill, who had worked in film and wanted to have some of that familiarity. Jon Favreau the director was also quite interested in the look of film.
We looked at a lot of films like Black Beauty and Never Cry Wolf, which had beautiful natural photography and beautiful color work that felt analog. We all collaborated on that really closely and made sure that, while we were in the design phase, we were setting things up not only for the technical rigors of high dynamic range projection but also to make sure that colorist Steve Scott would not have material that failed him in the digital intermediate processes.
I wanted to provide him with the most robust imagery possible and even though we have a lot of advances in technology — computers are faster and we are storing more bits to represent color in the computer — it is still really challenging to make synthetic colors and synthetic pictures that will withstand the color grading process as robustly as live action footage does.
By setting things up properly with our Technicolor cousins in Hollywood we knew we had a good set up that was robust and that would allow me to proof the color at the end of the chain.
DCR: Everything in The Jungle Book looks so real. It is regarded by many to in the vanguard of photorealism in a virtual production. Talk about the research phase of this project and what you and the team did to create such a real-world reference environment.
AV: There were a few main concepts we used. The first one was that any time you shoot photography and create synthetic imagery and bring the two together you run the risk of degrading the color information and the photographic information for the sake of bringing the two images towards each other.
For example, if I handed you a black and white photo and then I made a full color photo-digital painting for a background and I said ‘bring these two images together’ what would you do? If you painted color into the black and white photo it would not look natural. Your only solution might be to make the background black and white so at least the two images would mesh and look like one photo.
That is the most extreme example I can give you. It is hypothetical but we face this problem on every project we do. We are trying to combine different visual sources and make them look like one photograph.
Avoiding any sort of color degradation in that process requires a lot of study as to what attribute of the original photography you are matching to...what to replicate in the background...what is it that will provide an image that does not require degradation to achieve a match?
So, you have to look at the basics of your key-to-fill ratio. You have to look at how light is behaving on different materials. It requires quite complex computer software to handle all those problems.
We have teams of people who every year are improving the way our materials software that we call “Shaders,” who represent the way light interacts with different kinds of materials and different surfaces.
It is a huge team effort. We have worked with Renderman software from Pixar for over a decade. Our internal teams develop new things and the lighting department – the people who actually have to produce the work shot-by-shot have to continually up their game because that is where the rubber hits the road.
You can have all the latest technology but when you are doing the shot, you have to be visually sensitive and technically accurate about what you are doing. That is the foundation. If you can have good renders that match photographically to the thing you are trying to match to. If the material definitions and the software that renders do a thorough job of approximating how light and materials really interact; that is the foundation.
The next thing is the content itself, and this is where our assets teams and our environments teams came into play because The Jungle Book has so much intricate detail. The individual characters have millions of hairs that build up their fur coats, very detailed modeled irises, little eyelashes and everything you can imagine.
Then the world around them is composed of literally millions of individual leaves and the detritus, twigs, dead leaves and pebbles that are scattered all over the floor of the jungle.
When you build that level of detail and you combine it with photo-realistic rendering and photo-realistic lighting you are going to approach something that the eye and the brain start to accept as real.
But I can tell you that if one or the other side is not firing on all cylinders you are going to lose out because you are going to have a very beautiful object that is not lit well and that does not look real. Or you can have beautiful lighting on objects that don’t look real and you will be in trouble. I would say that you have to have those two main ingredients.
The Jungle Book was a very ambitious project, but MPC’s teams have built up a lot of expertise in this area. We had a partner in Disney that really believes strongly in quality as part of their identity and what they want to offer audiences. We had a director in Jon Favreau who has one of the most discriminating eyes I have worked with.
We had everybody agreeing that we are going to push this to another level and then of course we had the MPC teams of artists and technicians that had to actually deliver it.
We had all the right technology, the right show, the right studio, the right attitude What then happened was that our artists — and we had 800 people involved in this show — gave their best because they could feel this to be something extraordinary.
DCR: Photorealism is the term that has become associated with the genre that you have created with The Jungle Book. How do you feel the movie contributes to taking this concept further, and what impact do you think it will have on the storytelling process?
AV: When I first met with Jon Favreau in Los Angeles a few years ago to discuss the issues around this film and what would make it work we were all a bit scared of the animal movies we had seen before. Jon had primal ideas in his mind about how people would connect to him.
I talked to him about my own pets and people’s general connection to animals and how when you are with your pets at home you have an emotional special bond with them and you talk to them. Many kids have the fantasy that they can talk to their animals and live with them like friends.
So, for this film to transcend the talking animal genre and the family film genre it required what Jon would call a magic trick. Meaning ‘We fooled you’. You thought you saw something that you did not really see: an absolutely believable animal talking. Not just an animal with its mouth moving like a dog food commercial but a living, breathing, emotional being that never looks like a cartoon or behaves like a cartoon.
If we could find that intersection it would make people suspend their disbelief and result in a transcendent piece of work that would create a unique experience.
The trick was that The Jungle Book had to be photo real. If there is ever a moment where your brain pops out of the movie and says ‘I am not actually seeing what I am seeing’ it falls over and the movie fails.
I think what the Jungle Book does is to remind people there are still new things out there. It reminds me of when Avatar came out, or some of the first Pixar movies, and people had a fresh experience.
Cinema can do so much and it can connect with audiences in a non-verbal way. The imagery and the execution of the performances and all that stuff really need to guide you through moment by moment for that to work. The Jungle Book is a reminder that there is undiscovered territory out there, that there is still stuff to innovate, stuff to try.
The Jungle Book is one of those films that makes filmmakers be brave and be bold in what they want to undertake, and it's the same for the studios. They see the rewards of something that is well received.
We feel honored and privileged to have been part of it. It is so rare when you get a great movie and people go and like it and review it well. Now we are getting recognized for our work, and that is amazing.