Today we’re introducing Jean Moreno, the Shading and Lighting Tools Lead for Sonder. Jean chatted with us from his home in Bretagne, France about how he got involved with Sonder, the custom tools and techniques he used on the film, and how he helped give the film its distinctive look.
Q: Thanks for chatting with us today, Jean! To start, can you tell us where you’re from originally, and where you live now?
Jean: I was born in the southwest of France, but lived most of my life in a suburb of Paris. Now I know that I don't want to live in such a crowded area, so I'm back in the countryside in the west, in Bretagne (where they make very good crêpes and salted caramel!).
Q: How did you get involved with Sonder?
J: The team bought one of the packages I sell on the Unity Asset Store, Toony Colors Pro 2, which is used to give that non-photorealistic look to the film. [Technical Supervisor] Andrea Goh contacted me with some support questions, and asked if I'd be interested to meet the director and producer (Neth and Sara) to know more about the film. I said yes, and a few weeks later I was officially part of the team! That was in April 2016.
Q: You’re Sonder’s Shading and Lighting Tools Lead. What are some of your main responsibilities in that role?
J: I have to make sure that the tools and shaders we use in Unity work as expected, and enable our lighters to meet the visual goals they have. I also help in other areas whenever needed, such as managing our server or working on the pipeline itself.
Q: Let’s get technical. Tell us about some of the tools you used in working on the film.
J: Well as I explained, the team chose me for my work on Toony Colors Pro 2, which is the set of shaders that give this very stylistic, non-photorealistic look to the film. I've been expanding it greatly with all the requests I've had from our creative people, and some of those changes actually made it back into the official product!
Usually when animated films have a stylistic tone to them, it is done through exaggerated shapes or colors, but the lighting calculations are based on photorealism (you might have heard of "physically-based shading," for example). Here we are entirely customizing the lighting, and the response of the different surfaces to the virtual lights in the scene is manually edited by the lighters. Visually we want to steer away from that computer-generated feeling and try to mimic traditional 2D animation, but still using 3D geometry. This is the biggest concept behind Sonder's unique look.
The other unique choice is the usage of Unity, a real-time game engine, to produce the film, and the biggest advantage there is that we get near-instant results. Lighters see how the final frame they're working on will look like without having to wait for a long rendering process. This allows much faster iterations, and thus in the end, higher lighting quality.
Q: Did you have to build anything custom just for Sonder? If so, why? And how did it turn out?
J: We use a lot of custom tools, which have been built thanks to Unity's flexibility to create in-editor tools that look like built-in ones. One of them is the cascading materials system, which is used to assign the correct materials to all the geometry pieces in a scene, with a priority-based system: you've got 4 priority levels for a single material: LookDev, Foundation Lighting, Master Lighting, and finally Shot Lighting. The tool will pick the most relevant material for you depending on the files that exist in the project.
Say that a lighter needs to tweak some specific values for their shot, without affecting all the other shots and sequences. They'll just create a new material with the Shot Lighting priority and that will be picked by the tool—no need for manual assignment and handling of that material.
We also made some tools to render directly from within Unity. I said that you didn't have long rendering time, which is true for a single frame, but we still have to output frames as individual images, then assemble them into movie files, and then upload that to our server for easy review (as everyone works in a different physical location). All that can be done within Unity with a few buttons, with automatic organization in the server file-structure. On their end, our director Neth Nom and director of photography Farhez Rayani can then get the latest versions with a single click, and compare them to the previous versions very easily.
These kinds of tools are major time-savers compared to uploading and finding your way through by hand, which in turn allows for fast iteration. Whenever you have notes for your work, it doesn't feel painful to have to go back there and do your changes, because it's a very smooth and non-frustrating experience overall.
Finally there's our custom "Negative Lights" system that I built. You can place 3D volumes (spheres, cubes, tubes, etc.) in your scene so that they block lighting (direct, ambient or both). It is widely used in the film, and is a great help for lighters to shape their lighting. Shadow maps—the built-in technique in Unity to get real-time shadows based on textures—is rather limited, so this system completes them very well.
Q: You’re also on the FX team for Sonder. What are some FX techniques you used in your work on the film? Are there any visual effects in Sonder that you are particularly proud of or excited about?
J: We are mostly sticking to techniques you'd find in a game, because they work well with real-time engines. One very nice effect is the ability to have "ramped fog": we can create some very stylistic fog that has a different color depending on its depth, using a simple color gradient.
I'm also very proud of the water shader, which is a collaboration with [FX Artist] Jason Johnston and [Lighting Artist] Chad Orr: it really feels like it was hand-painted, and knowing that it could run in a game is very impressive.
Q: Did you encounter any specific technical challenges in working on Sonder? If so, how did you overcome them?
J: In the beginning of the project, we didn't know exactly how we would share files and keep their history safe. We tried a few things, and ultimately settled on using Subversion (SVN). While primarily aimed at code projects, some of its features really came in handy, such as the ability to only partially download a project (e.g. a lighter only downloads the Unity project they're interested in), and good compression for binary assets, which is 90% of our files. The downside is that it can get quite technical, especially when things don't go as expected, and SVN needs you to be very explicit about the steps you take to solve issues. For non-technical people, this can be very frustrating—especially when they need someone technical to help them get out of a tricky situation. But I still have to find an artist-friendly tool that "just works", and is not prohibitively expensive!
Q: What are some other animation or game projects you have worked on, other than Sonder?
J: I had only worked on games prior to Sonder. The whole animation field is very new to me!
Notable games I've worked on include CounterSpy, a stylish game set in a ‘60s spy-inspired theme; Furi, a fast-paced boss rush game that is part close combat and part "bullet hell"; and Seasons after Fall, a charming hand-drawn 2D platforming game where you play as a fox that can change the seasons at will!
Q: How does working on an animated film compare to working on games? Do you prefer one over the other?
J: The main difference is that with a film all you care about is the final images that will make the movie, whereas in a game you have that interactivity layer where you can't predict every single thing a player will do, coupled with the fact that you have to keep performances in mind (i.e. make sure the game runs at 30 or 60 frames per second and still looks good at any time). Thus not worrying about optimization or gameplay bugs is really liberating!
Now of course the end result isn't the same kind of experience, and I don't think I'm ready to entirely leave games behind just yet!
Q: Finally, how would you describe your experience working on Sonder, and being part of the Soba crew overall?
J: Very good! Sonder will be the longest I've been on a project (almost two years now!) and the biggest team I've worked with too, and everyone has always been very supportive of each other. It's also been a very interesting journey to learn about the production of a movie. The only downside for me is that I'm the only person based in Europe, and I have only been able to meet just one person from the team in real life when our producer, Sara Sampson, was visiting Paris! But that will change soon as I have planned to come to the US for our official crew wrap party and meet the people there!