Disney Research scientists are adapting low-cost depth-sensing cameras for use underwater, with the goal of capturing 3-D models of marine flora and fauna with a high degree of accuracy.

The scientists developed a method that corrects for refraction effects that occur when infrared light used by the depth camera's sensor passes through the waterproof housing of the underwater system. This experiment marks the first successful demonstration of applying a commercial low-cost depth sensor for underwater capture of depth images.

The next stage of work will use the captured depth images to create 3-D reconstructions of objects. This technique has the potential to build models with millimeter accuracy.

The researchers will present their method at the IEEE Winter Conference on Computer Vision, WACV 2016, March 7-9 in Lake Placid, NY.

The depth camera research supports the Coral Reef Restoration Project at Castaway Cay, a Disney-owned island near Great Abaco Island in the Bahamas. The goal was to develop a handheld camera system that divers could use to monitor the growth of coral pieces suspended in an underway coral nursery, helping to determine when corals are ready for transplanting into a reef.

The research team created a prototype system consisting of a depth sensor and mini-computer in an underwater housing, suitable for handheld use by a diver. Initial evaluation on the Castaway Cay reef was successful and has spurred a new round of improvements to the design.

"It's possible to use ordinary cameras to perform 3-D scene capture underwater, but depth cameras produce much denser and more reliable depth measurements," said Markus Gross, vice president of research at Disney Research. "This will be a boon to our Disney colleagues at Castaway Cay and also to the broader marine science and oceanography communities, which increasingly are embracing 3-D scene capture."

Previously, the coral restoration researchers had to measure coral size by manually estimating the volume of an elliptical cylinder that would enclose the coral segment. The new system will make it possible to get an automatic estimate of the true volume of the coral.

Depth cameras, including those popularized as videogame controllers, work by projecting infrared light onto a scene; by measuring how long it takes a point of light to reflect back to the camera (time of flight, or TOF, cameras) or the degree to which a pattern projected on the scene is distorted (structured light cameras), precise 3-D contours can be calculated.

The team used relatively low-cost, commercially available depth cameras for the study, designing a waterproof housing with magnetic switches that enabled divers to activate or deactivate a limited number of functions.

One challenge to using depth cameras underwater is that water heavily attenuates infrared light, said Paul Beardsley, a principal research scientist at Disney Research. But the research team determined that commercially available depth cameras can accurately scan at ranges of up to tens of centimeters in water, which is sufficient for the coral growth monitoring application.

Likewise, the problem of refraction underwater is known and techniques exist for correcting this distortion for ordinary cameras. The Disney researchers developed a mathematical model to account for the refraction in depth cameras. Their system requires a single calibration step of scanning a planar surface.

In addition to the field tests at Castaway Cay, they demonstrated the method using both TOF and structured light depth cameras, scanning objects in an aquarium tank both with and without water.

"Low-cost depth cameras have proved a game changer in recent years in making 3-D models of terrestrial scenes," Beardsley said. "We are building on the exciting results within this area of research by taking it underwater."

In addition to Beardsley, the research team included Sundara Tejaswi Digumarti, Aparna Taneja and Gaurav Chaurasia of Disney Research, Roland Siegwart of ETH Zurich, Amber Thomas, a marine biologist at Walt Disney World, and Larry Boles, researcher. For more information, including a video, please visit the project web site here.