With four Academy Awards and six other nominations under his belt, there are few individuals more seasoned, and deservedly celebrated, in the visual effects and postproduction community than Joe Letteri. His work on the Lord of the Rings franchise, Avatar and Planet of the Apes films not only heralded industry-wide changes in the way that filmmakers create CGI characters, but transformed the way audiences experience and relate to them. And the Director of Weta Digital, he has overseen dozens of other productions as they implemented the technology developed and unveiled to make fantasy not just come alive, but become real on screen.
Letteri is currently deep in production on James Cameron’s forthcoming Avatar sequels, but the visual effects artist and supervisor recently visited Film Fest 919, currently celebrating its second year in Chapel Hill, North Carolina, for a presentation of materials from his vast and eclectic career. Afterward, the filmmaker sat down with Nerdist for an exclusive conversation about the progress he and Cameron are making on Avatar; in addition to talking about the opportunities and challenges they’re dealing with on that eight-year production, Letteri revealed some of the secrets, and foundations, to creating memorable and believable CGI characters, and reflected on his own past, present and future in the industry as other titans like Martin Scorsese begin to utilize the technology he and his collaborators pioneered to make it possible to tell different kinds of stories than he, or they, ever imagined.
Film Fest 919We’re not finished figuring this stuff out yet. There’s still a lot to learn. A lot of it is story-driven. And that’s one of the great things about the Avatar sequels, especially number two coming up. It has the story and it also has the scope. So I think you’ll see a lot of changes coming out of that, but I can’t talk about them for two more years.
How far along are you in the process of making Avatar 2?
We’re still shooting in live action, but we’re in full production on the post production and visual effects with the two years left to go. December 2021 is the release date. So we’ll be finished in about two years from today.
What is it about the Avatar sequels that will rekindle interest in this world after more than a decade of audiences waiting?
Jim does not disappoint when it comes to story. The characters are well-written. That’s really all you need. And then the rest of it just expands outward from there. And it’s a big world that he wants us to create, but the foundations are there.
How much did you find yourself drawing on the original film is as a reference point or foundation for the sequels?
We’re definitely going back to the first one as a reference. We don’t want to forget what we learned. You need to be true to making sure that it’s the same world, and it was a long time ago. And there’s a lot of this stuff that we’re all tending to forget because it was that long ago.
How tough or easy is it to develop those ideas with multiple films in mind, being put into production simultaneously?
It’s not really any more difficult. These are all things that we know we have to do at one point or another. So I’m happy to work on them in whatever order makes sense. Some of them depends on the order that Jim has shot scenes in. Some of it depends on things that we’re developing when it’s ready. But we just look at it as one big project.
Is there an element of the original Avatar that you personally were eager to explore in more depth in these sequels?
All of it, really. Again, there was so much to see on that planet the first time, and it’s a big planet, I think there’ll be a lot of interesting things for the audiences to see that they haven’t seen before.
James is a one-man studio to some extent, and a visionary. But people are skeptical about his dedication to exploring this universe in multiple movies for decades. What is his disposition as he’s making these films, since he clearly hasn’t felt rushed to try to duplicate the success of the original movie, or even to do anything else?
I mean with Jim it’s all about, again, make sure the story is there. And once he was happy with that, it’s planning for the movie. This whole virtual production concept that we pioneered on the first film, we’ve been using it extensively. Jim is very disciplined. He does a lot of planning. So it takes a long time to get to the point where you’re ready to go. But once you’re ready to go, the execution is generally pretty good because he has thought of all that.
And that’s really the key. He’s spent so much time getting it to where he thinks it needs to be that you know what you need to do to help him finish that off as soon as possible. Once that’s there as sort of baseline, if there’s other ideas that you might want to pitch him, that are more interesting, genuinely, he’s pretty open to them. He’s always open to making it better. But he knows the minimum that he needs it to be, and that’s where we all start from. So that discipline is actually really helpful in a lot of ways.
20th Century FoxMost of what we’re doing now is looking at rewriting and changing for the first film. Like the way we did Alita: Battle Angel and the face and taking that off of Rosa [Salazar], we’re learning so much about how to do facial animation that we’re changing once again how we do it. And that’s just the nature of the business, but especially on a project like Avatar. It’s so big that we have to take some time to step back and rethink it and make sure that we really understand what we’re doing. So that’s kind of where we are right now, just rolling out the new changes and seeing how they work in shots.
How much have you been using the technology you developed to render scenes immediately during production?
We created that on Avatar and we’ve been developing it ever since. And a lot of filmmakers are picking it up in different ways, using game engines and things like that. Jon Favreau used that a lot on Jungle Book and Lion King, but we developed that for the first Avatar. We’ve continued to push it, but the difference is we’ve actually built our own end-to-end system for filmmakers to work in an environment that completely embraces that. That’s why we call it virtual production, because it integrates fully with live action.
We did a lot of that with Steven Spielberg on BFG, doing that all while we were shooting. You’re doing it while you’re doing the photography. The whole point is to allow the camera operators to see the non-real elements, the digital elements while they’re shooting so they’re including that as part of their framing. The other actors are working with it as part of their performance. Even though we’re shooting on a set, not a mo-cap stage. The idea is to capture the performance in the moment. So we’ve really pushed hard on developing those tools.
How tough is it to sustain enthusiasm for something you know you’ll be working on for eight years?
I think there’s enough in the story that that’s not really a problem. It’s like making four movies, and each movie brings its own set of challenges to it, but also its own set of familiarities. I’ve done at least three trilogies now, so you kind of get used to doing that longer form storytelling.
Does that mean that you actively don’t take anything else on during that time?
Personally, I’m going to be focused just on Avatar. The show is big enough that I won’t be having as much of a role in other films as I used to. Also because our teams, our supervisors and artists have been with us for so long that they know what they’re doing, I don’t think they need much of my help these days.
What have you learned over the years that storytellers need to come to your team with in order to make sure that the execution of their storytelling and world-building is as effective as possible?
That the story is complete in itself. If there are gaps that you’re hoping will be filled in with visual effects, you’re likely to be disappointed. We can add ideas, we can help in whatever way that we can, but you want to make sure that when you read it, it reads well.
What do you consider your main responsibility as on these movies? Not just Avatar, but more broadly in your career?
I think first and foremost to engage the audience. Everything that we do has to be part of the audience wanting to sit there and watch that movie and see what happens next. And it’s a combination of things. It’s the drama of the characters. It’s maybe what you can do to a scene to make it compelling to look at, the realism that you might need to get people drawn into that moment. It could be any number of things, but it’s really about just making sure that you’re always in mind of how the audience is experiencing what they’re seeing.
20th Century FoxWell, Apes was a good example because when I read the script, the first thing that popped into my mind is, “Andy Serkis would be great for this.” They actually weren’t considering Andy at the time because he’d just done Kong and they thought he might not want to do another ape movie. So I just talked to him and said, “Andy, I just read this great script. Would you be interested? It’s got another ape in it.” And he said, “well, what’s the difference? I do two movies where I play humans. What’s the difference in doing two movies where I play apes?” And so I got them together and they went with Andy for the lead.
Because of the constant changes in creative processes and the industry itself, do you ever feel like you reach a point where you’re like, “okay, I completely understand this.”
Not yet. There’s still a lot that we don’t know. So I can’t say we’ve figured it out yet really. We may hopefully make it look like we have, but we really haven’t.
What’s the secret for you in terms of facilitating this incredible work from the people that you work with?
As I said earlier, so many of the people at Weta have been there for so long, we’ve really been working together enough to start to see the problems before they develop. Everyone has that sense of, okay, this is not going to work. We’ve seen this. We’ve been asked to do this several times before. It never worked. Here’s what we did instead. And that’s what made it into the movie. So that helps short circuit a lot of the pain. But also there’s just some general principles about realism, whether it’s in the modeling and geometry side of it, whether it’s in the cinematography, whatever aspect it needs to be that people are familiar with and they know that that’s the goal.
There’s that realistic ideal, and there’s also keeping in mind how it serves the story. And I think pretty much everyone on our teams these days at Weta and on all the films know that and are expected to know how whatever they do fits the story as well. And all of those things are really good to have at a core level. So where all the artists working on the film have that in mind and bring that to the work really helps it all kind of gel a little bit better.
Have you heard any plans for James and Robert to continue with Alita?
We’ve asked every once in a while because we all enjoyed working on that. But I just don’t know what’s going to happen with it. Because Robert’s off doing other films, and it was a Lightstorm project which Jim produced and we’re all pretty busy with Avatar right now. So I don’t know if anything will come with that. But I’d love to see another one because there was that whole world that you could build off of. I think you could [do a sequel] if the people were available to do it. That would be the hard thing, assembling the team. But they could easily go right into it. There’s a lot of material there. And I think the film gave it enough of a grounding that as a character you could take her other places.
Is directing something that ever intrigued you?
Not in that way. I have been interested ever since we did Tintin in just doing like more just animated films where you can break some of the rules. You’re not so bound by the physicality of it. That could just be fun. But I’ve been too busy to really look at anything beyond that. We were looking at developing an animation and I thought, okay, I think I know what needs to be done here. So maybe I’d stick with it if it happened. But right now with Avatar, I’m tied up for the next eight years. So I won’t be doing anything like that.
Paramount PicturesYeah, for sure. But again, that’s all story-driven – like you work on that and then figure out when you can use it. When we did the third Planet of the Apes film, for the whole avalanche sequence at the end we grew that forest so that we could destroy it. The idea of actually growing the forest rather than doing it the traditional CG way, which would be to take a bunch of models and place them around the landscape, just came out of the experience with Avatar where [we asked], what really happens, how, what makes it look natural, how does it develop?
So we started looking at software to figure out how do ecosystems grow. And we worked on it for a few years not knowing exactly when we would need it, but knowing that we would need it. And it turned out to be ready about the time that we had to do that last sequence, and we said, okay, let’s roll it out. So yeah, some of these longer-term projects do dovetail like that. But again, a lot of that experience came out of Avatar knowing that we could be asked to do anything in the world – why don’t we just start preparing for it? But the final focus really depends on the film.
When you see filmmakers like Martin Scorsese doing using de-aging technology on The Irishman, do you feel a sense of vindication? Because there does seem to be this persistent distinction drawn between spectacle and special effects and classical, dramatic storytelling.
I think it’s interesting seeing that acceptance and how filmmakers think that they can use it because we’ve always brought that to the films. Like with Gollum and Lord of the Rings, one of the things I really loved about those films was that Peter could do the big scenes, but he could also bring it down to the drama level. And that wasn’t just Gollum, but he did have dramatic scenes with Gollum that needed to work as well as the dramatic scenes between Aragorn and Gandalf or anything else. It was all part of the fabric of it.
So it’s always been there. It may have just been hiding in plain sight. So the more other filmmakers start to pick up on that and see the possibilities to me, that’s great. Because that’s what was always interesting about this as well, was expanding the horizons of the kinds of films that you could make. Like you’ve talked to Andy Serkis about this and everyone talks to him about doing performance capture. He’s always said, “I could play anything with this, so why wouldn’t you want to do that as an actor?” So it’s really great seeing even other actors picking up on that idea as well and not being afraid of it.
How far down the road do you think we are from people recognizing it with the same kind of awards we give traditional performances?
I don’t know. I mean, in Gemini Man we’ve got Will Smith playing himself twice, and he gives a good performance on both sides. So maybe now that actors like that are starting to embrace those roles, maybe the Academy or SAG will start to look at it differently. But it’s just hard to say.
It feels like you guys have had to sort of repeatedly teach audiences how to empathize with non-human characters.
I always liked science fiction because kind of force you to look at the world in a new way. But you have audiences that will immediately say, well, that’s not my kind of movie, so I won’t see it. It probably is your kind of movie. But one of the first things we had in mind when we did Gollum, is that he can’t be a monster. He has to have gotten this way for a reason that we can empathize with, so that in the end you’re almost like “I’m not sure if I want him to die.”
That’s the edge you want the audiences to be on, because that’s where the real engagement comes from, and we wanted to get people to the point where that’s what they’re thinking about. And that was Caesar’s experience as well. Did he go too far? Did he lose his humanity? It’s a funny thing to say for an ape, but in those movies, humanity lost its humanity, and humanity got carried on in the apes. And we all know from the original films, that that will get corrupted down the line as well. But that’s the story.
Featured Image: 20th Century Fox