This article is about how motion painting evolved in relation to ‘What Dreams May Come’ which won an Academy Award for visual effects and breakthroughs in using this technique.
Here we cover process and aesthetics but see the section on Optical Flow for a more technical view.
In this film I decided to make the afterlife a world of paintings: The afterlife images come from one man’s memory of his wife’s art work. In order to remember her, he envisages paradise as if he is actually living within a series of her paintings.
The paintings needed to be moving and alive – paintings in motion.
In solving this I felt we came across a new aesthetic – especially in the way we created a sense of moving visceral paint on film.
The practical question became, ‘how on earth can we create motion painting that is convincing?’ As far as I could see up until now no one had really come up with a way to do this credibly so that it would seamlessly combine with living footage and move with it.
I approached several visual effects houses and one house proposed the use of some trial Kodak software that was still in the early stages of development. We began testing – building our toolkit, and designing and improving it. But while we felt it had potential it clearly was not yet working for our needs.
So we put together a unique team for this section of the film – one that combined , programmers, producers and technicians, with those that could work with digital imaging yet had an active background in oil painting.
In technical terms our aim was to track individual pixels frame by frame and attach colors and scanned paint strokes to them as the pixels moved and changed. Using an image processing software adapted from the military, (where I believe it had been used for tracking missiles), we were able to achieve this. For our purposes it has been given the name optical flow.
For ease and speed I wanted to be able to shoot a regular scene – say in the mountains of Montana with actors. This would give me a solid basis in reality from which to depart. It would never be an issue of mimicking reality since that was the starting point, instead we could concentrate on the style of changing and reinterpreting it. Once filmed we would go back to the quiet of the studio and rebuild the shots by adding scanned paint strokes and then tracking their movement through vectors of light, shade, colour and movement.
Our breakthrough came when we realized the paint strokes could really attach to parts of the live action image and move naturally with it. This worked even in a shot with a lot of movement and produced a very exciting result, quite unlike anything that had been achieved before.
Much more than a technical break through it was about trying to find an aesthetic that could shape these techniques into a compelling moving painting. Since then I have been experimenting with printing individual sequences of frames, sometimes slowing them down by up to 20,000 percent. These frames, each one of them, one by one, became a painted artwork in its own right.
It had struck me for some time how beautiful each of these individual frames were, or could be, in certain shots. “Planned accidents”: compositions formed from a frozen moment in the original moving camera footage and then the gestural instant of the layered paint strokes we had added. They seemed somehow more alive than many actual gestural paintings and more spontaneous. And yet, and yet . . . they actually moved.
The clips above and images in the gallery show Motion Painting. This is a fresh approach and distinct from work seen in traditional forms of animation, the film canvas paintings and more recent digital paintings that are used within shots as scenic backgrounds by Matte painters. It is also a different approach to rotoscope animation painting, or to those painting on physical film like the scratch films of Len Lye.