Although no Swiss were in the running for Oscar glory last night, two behind-the-scenes heroes from Zurich – whose work has contributed to several blockbusters over the years – have been awarded a “Technical Oscar”.
Markus Gross, professor of computer graphics at the Federal Institute of technology in Zurich (ETHZ) and director of Disney Research Zurich, tells swissinfo.ch about mixing science, business and art – and why he didn’t patent his award-winning software.
On February 9, Gross, his former post-doc student Nils Thürey and two other scientists from Cornell University attended a ceremony in Beverly Hills where they received a Technical Achievement Award.
Their success is down to two words: wavelet turbulence. In 2008, they developed software that could calculate smoke and explosions in films quickly and recreate them realistically (see video).
Their software caught the attention of DreamWorks, the film studio co-founded by Steven Spielberg, which put it to use on Monsters vs Aliens in 2009. Since then, it’s become the industry standard and has been used in about 20 major Hollywood productions including Avatar, Battleship, Sherlock Holmes, Alice in Wonderland, Hugo, Kung Fu Panda, The Amazing Spider-Man and the forthcoming Iron Man 3 and Man of Steel.
swissinfo.ch: You didn’t start your work on wavelet turbulence in order to win an Oscar. How did your work on this subject start?
M.G.: We are academics, researchers – and often researchers meet at conferences, develop ideas and then start collaborating on the ideas with a goal to publish a paper eventually, to come up with an innovation.
And this is what we did. We met at the annual ACM SIGRAPH conference [the leading conference for computer graphics], where the idea was born for this particular technology. We then worked out this idea, figured out the concept, did the implementations and finally submitted our research to the ACM SIGRAPH conference. It was accepted and published the subsequent year.
We then put the code online free for everybody to use. We never patented it or filed any IP [intellectual property] on it.
swissinfo.ch: Why not?
M.G.: We thought that if we made it available to the public and made it free for everybody to use, it would be picked up more quickly and it would migrate more quickly into applications.
Wavelet turbulence is a very good example of how essential open access to innovations is for the rapid transfer of technology into industry. From an academic perspective, this can make a lot of sense and you can get a lot of rewards for that – the Tech Oscar is an example. Of course, from an industrial perspective, things are different: there you might want to patent and protect it.
I also think that had we patented it, the effects studios, which are under a lot of financial pressure and operate at a very low margin, would not have picked it up. The burden of clarifying IPs, of paying special fees for licences and so forth would have been prohibitive. But with this open access and open source code, there was basically no hurdle for them, and it’s a technology that really provides a tangible improvement to the state of the art – it reduces costs and improves the visual quality. So they just picked up the code and integrated it into their own proprietary workflows. This is why it migrated so quickly into many Hollywood movies.
swissinfo.ch: How did the collaboration between ETHZ and Disney begin?
M.G.: This started four years ago. Disney Research is part of the company’s strategy to focus on technology. As part of this strategy, the decision was taken within Disney to create external research laboratories close to world-class research institutions. Since we at ETH had been working on visual computing and computer animation special effects for a long time, they identified us as one of two places to create these labs and build them up. The second place is at Carnegie Mellon University [Pennsylvania, United States].
swissinfo.ch: What are the advantages for both sides?
M.G.: For the ETH it’s wonderful. First of all we gain access to a spectrum of very interesting applied problems in the field of entertainment. We can work on novel technologies that in principle can be seen by billions of people in films or in theme parks. In particular for the ETH it’s interesting because we create IP a lot – we are filing patents and these patents belong to both partners, so the ETH gets a lot of patents. Also, Disney sponsors a significant number of PhD students at the ETH who work on Disney topics.
On the other hand, for Disney, the ETH provides an intellectual and physical infrastructure – the labs were designed to permit a high degree of permeability between the two institutions. So the senior researchers who we hire and work for Disney can get involved in graduate teaching, work with PhD students or co-supervise Masters students – it’s almost like a university environment, but at the same time we are very cautious in the way we protect intellectual property and so on.
So this high degree of permeability gives Disney access to a category of researchers which would otherwise be difficult to get in an industrial research lab.
swissinfo.ch: As you say, billions of people have seen your work, but often the aim of good visual effects is not to be noticed. Doesn’t that bother you after all your hard work?
M.G.: That’s exactly the aim. We want to mimic reality with a lot of special effects. With all the physics simulations we’re doing, we really want to mimic physical effects. But in our case we want to create methods and effects that are as natural and as visually indistinguishable from reality as possible.
swissinfo.ch: Are you happy with the results on the screen? Do you go to the cinema often?
M.G.: Oh yes – maybe once a week. I watch two different genres: one is driven by special effects, so fantasy movies and sci-fi movies. Also lots of 3D movies because we also do a lot of work with stereoscopic 3D. But I watch these films essentially to judge the quality of the visual effects – it’s less about the story.
And then I like story-driven movies where the actors are in the foreground and which are not so heavy on visual effects, like Carnage or The King’s Speech [see related article] – really good movies.
swissinfo.ch: What are you working on at the moment? What will you win your next Oscar for?
M.G.: Right now we’re focusing on facial animation – we’ve developed a very nice ‘markerless’ facial motion capture system that allows us to track and reconstruct facial geometry down to pore level, down to the micro-geometry. This is a fundamental technology that is essential for all sorts of visual effects on human faces – which is still some of the most difficult visual effects because of the high sensitivity of the human visual system with respect to human motion…
swissinfo.ch: Uncanny Valley…
M.G.: Yes, it’s the Uncanny Valley Effect [see link] – we’re very much focused on that. I hope that some years down the road this will all migrate into major productions and maybe be considered by the Academy for an award.