Rolling-Shutter Skew Can Make Motion Tracking a Drag

Digital motion-picture imaging has come so far, so fast, that it’s easy to forget, sometimes, that we still have a long way to go. A case in point is one of the many bits of cool software engineering being showcased at last month’s NAB show in Las Vegas – The Foundry is experimenting with technology that would remove the rolling-shutter artifacts often associated with images captured by CMOS sensors.
Rolling-shutter artifacts occur because CMOS chips don’t update an entire frame at exactly the same time. The image is captured line by line, moving across the sensor in one direction, meaning that the picture may suffer from temporal artifacts. For example, a straight horizontal line may appear skewed as the camera pans past it, since the camera has actually swiveled some distance between the time that the top lines in the frame were captured and the time that the bottom lines are recorded. The effect can be seen to varying degrees on everything from cheap cell phone cameras to footage captured with pro cameras.

It’s a bit of a hassle for the VFX business because the CMOS artifacts can introduce a kind of wobbling effect to elements in the image, making it hard to do good motion tracking. That’s why The Foundry, which makes motion-tracking plug-ins like Furnace and Ocula, has been working on a tool that leverages the company’s understanding of motion to analyze an image and then shift the appropriate portions to correct those artifacts in the course of tracking.

At NAB, the company was showing it as a compelling aspect of its future motion-tracking systems and not necessarily as a standalone technology. But, in the course of running demos during the show, The Foundry found that there is keen interest in using such a system simply to get rid of the Jell-o-like effect that can plague CMOS-acquired footage.

The Foundry isn’t estimating a date when such a tool might be available, but the company is aware that the technology will likely have an expiration date. As in-camera hardware and software improves, manufacturers are getting better at minimizing rolling-shutter artifacts, especially in their high-end gear. As technology marches on, there may be only a two- or three-year window in which filmmakers find it necessary to correct CMOS artifacts before the camera vendors build the fix into their own systems.

For more information: www.thefoundry.co.uk