In amongst all the hubbub over the New Aesthetic was a very interesting post by Dan Catt about the difference between the very NA-friendly concept of computer vision and the not-so-interesting computer graphics. This is what’s got me thinking about my experience of CGI. More specifically, about how computers ‘see’ a CG scene and how CG image data is represented. My film 'Z’ is a visual exploration of these concepts, but I’d like to break it down a bit more for the benefit of those who don’t know much about CG.
Embedded into the CG image are multiple 'passes’ - versions of the same image. Imagine if you could see multiple interpretations of the world all at once: you look at a bunny rabbit and you see heat maps, Xrays and MRIs, as well as a colourful Pixar version of it. Well, that’s a bit like render passes in CG. Each render pass is used by the CG artist to tweak how the final image looks.
When we build something in a 3D program, it starts as a shell, then we paint it up, move a camera around it and light it. Then the computer calculates (renders) the image. Rendering one frame takes a relatively long time - usually it’s a matter of minutes but it can be hours or days. If that frame is out of focus or too dark or too motion blurred, we don’t want to wait another week to render the frame again. So before we render, we make sure to ask the computer to sample as much data as possible from the scene. This data comes in multiple forms and allows us more control in cleaning up whatever the computer spat out.
Below are some of the more interesting passes that can be embedded in a CG image. I love the names of each pass. The bunnies are made of milk by the way.
1) The Beauty Pass.
This is what you’ll see in the magazine, on the TV or on your computer screen. This is the 'final’ image.
2) The Matte Pass.
This pass is used like a stencil. It’s comprised from areas of flat, primary colours that are easy to distinguish from each other. This allows the artist to 'key’ only the red, for example, and use that as a mask to tweak one specific object in the scene without affecting the others.
3) The Occlusion Pass.
The occlusion pass is a black and white pass that shows where objects 'occlude’ or block light from hitting each other. It essentially simulates the effect of ambient shadow, like in dark crannies, corners and cavities. This can be layered in with the final image to add extra 'realism’ to a CG scene.
4) The Velocity Pass.
This pass allows artists to add fake motion blur onto CG animation. The velocity pass looks at the moving objects in the 3D scene and colours them according to the direction and velocity at which they are travelling. In this image, the pink-yellow bunny is travelling left to right - the opposite direction to the other three. The colour differences are a shorthand that tell the computer how to apply the motion blur effect - to what degree and in which direction.
5) The Normal Pass.
The normal pass looks similar to the Velocity Pass but instead of showing where the object is moving, it shows us where it’s surfaces face. So a cube would have a different colour for each of it’s six faces. The bunny is more complex so the surface shows gradients between colors. This pass can be used in post-production to specify how lights affect an object. If I wanted to simulate a light placed in the top left corner of the scene, the computer could use the orangey parts in the image as a guide for where to apply the light effect.
5) The Depth Pass.
The depth pass represents distance according to colour, or rather shades of grey. Objects close to are black, and objects far away are white. Or vice versa. Rendering focal blur straight from a 3D program is possible but rarely done (at least for quick-turnaround projects) as it’s very expensive to compute. So the z-depth image is like an app for focussing in post-production. It’s the
of the CG 'eye’ (can I get away with that pun? I think I just did).
So those are some of the unusual image formats that never get seen or used outside the CG industry. They’re far more significant than they seem. Each uses a distinct visual code. Each offers a radically singular 'mode’ of seeing. Each is as aesthetically unique as Impressionism or Cinema Verité, but all are informed by what I would consider to be 'computer vision’. In a sense, they are hybrid images: they can be read both by a computer digitally and by the human eye aesthetically. This alone doesn’t make them artistic or significant, per se, but it does add a richness to CGI that perhaps would otherwise go unnoticed. Plus, it’s valuable to consider what happens behind the scenes of the grand magic trick that is CG image production.