Deep-learning technique reveals 'invisible' objects in the dark
by Jennifer Chu, MIT News Office
Boston MA (SPX) Dec 13, 2018
Small imperfections in a wine glass or tiny creases in a contact lens can be tricky to make out, even in good light. In almost total darkness, images of such transparent features or objects are nearly impossible to decipher. But now, engineers at MIT have developed a technique that can reveal these "invisible" objects, in the dark.
In a study published in Physical Review Letters, the researchers reconstructed transparent objects from images of those objects, taken in almost pitch-black conditions. They did this using a "deep neural network," a machine-learning technique that involves training a computer to associate certain inputs with specific outputs - in this case, dark, grainy images of transparent objects and the objects themselves.
The team trained a computer to recognize more than 10,000 transparent glass-like etchings, based on extremely grainy images of those patterns. The images were taken in very low lighting conditions, with about one photon per pixel - far less light than a camera would register in a dark, sealed room. They then showed the computer a new grainy image, not included in the training data, and found that it learned to reconstruct the transparent object that the darkness had obscured.
The results demonstrate that deep neural networks may be used to illuminate transparent features such as biological tissues and cells, in images taken with very little light.
"In the lab, if you blast biological cells with light, you burn them, and there is nothing left to image," says George Barbastathis, professor of mechanical engineering at MIT. "When it comes to X-ray imaging, if you expose a patient to X-rays, you increase the danger they may get cancer. What we're doing here is, you can get the same image quality, but with a lower exposure to the patient. And in biology, you can reduce the damage to biological specimens when you want to sample them."
Barbastathis' co-authors on the paper are lead author Alexandre Goy, Kwabena Arthur, and Shuai Li.
Deep dark learning
Each computational layer calculates the probability for a given output, based on an initial input. For instance, given an image of a dog, a neural network may identify features reminiscent first of an animal, then more specifically a dog, and ultimately, a beagle. A "deep" neural network encompasses many, much more detailed layers of computation between input and output.
A researcher can "train" such a network to perform computations faster and more accurately, by feeding it hundreds or thousands of images, not just of dogs, but other animals, objects, and people, along with the correct label for each image. Given enough data to learn from, the neural network should be able to correctly classify completely new images.
Deep neural networks have been widely applied in the field of computer vision and image recognition, and recently, Barbastathis and others developed neural networks to reconstruct transparent objects in images taken with plenty of light. Now his team is the first to use deep neural networks in experiments to reveal invisible objects in images taken in the dark.
"Invisible objects can be revealed in different ways, but it usually requires you to use ample light," Barbastathis says. "What we're doing now is visualizing the invisible objects, in the dark. So it's like two difficulties combined. And yet we can still do the same amount of revelation."
The law of light
"When we look with the naked eye, we don't see much - they each look like a transparent piece of glass," Goy says. "But there are actually very fine and shallow structures that still have an effect on light."
Instead of etching each of the 10,000 patterns onto as many glass slides, the researchers used a "phase spatial light modulator," an instrument that displays the pattern on a single glass slide in a way that recreates the same optical effect that an actual etched slide would have.
The researchers set up an experiment in which they pointed a camera at a small aluminum frame containing the light modulator. They then used the device to reproduce each of the 10,000 IC patterns from the database. The researchers covered the entire experiment so it was shielded from light, and then used the light modulator to rapidly rotate through each pattern, similarly to a slide carousel. They took images of each transparent pattern, in near total darkness, producing "salt-and-pepper" images that resembled little more than static on a television screen.
The team developed a deep neural network to identify transparent patterns from dark images, then fed the network each of the 10,000 grainy photographs taken by the camera, along with their corresponding patterns, or what the researchers called "ground-truths."
"You tell the computer, 'If I put this in, you get this out,'" Goy says. "You do this 10,000 times, and after the training, you hope that if you give it a new input, it can tell you what it sees."
"It's a little worse than a baby," Barbastathis quips. "Usually babies learn a bit faster."
The researchers set their camera to take images slightly out of focus. As counterintuitive as it seems, this actually works to bring a transparent object into focus. Or, more precisely, defocusing provides some evidence, in the form of ripples in the detected light, that a transparent object may be present. Such ripples are a visual flag that a neural network can detect as a first sign that an object is somewhere in an image's graininess.
But defocusing also creates blur, which can muddy a neural network's computations. To deal with this, the researchers incorporated into the neural network a law in physics that describes the behavior of light, and how it creates a blurring effect when a camera is defocused.
"What we know is the physical law of light propagation between the sample and the camera," Barbastathis says. "It's better to include this knowledge in the model, so the neural network doesn't waste time learning something that we already know."
They found that both methods reconstructed the original transparent pattern reasonably well, but the "physics-informed reconstruction" produced a sharper, more accurate image. What's more, this reconstructed pattern, from an image taken in near total darkness, was more defined than a physics-informed reconstruction of the same pattern, imaged in light that was more than 1,000 times brighter.
The team repeated their experiments with a totally new dataset, consisting of more than 10,000 images of more general and varied objects, including people, places, and animals. After training, the researchers fed the neural network a completely new image, taken in the dark, of a transparent etching of a scene with gondolas docked at a pier. Again, they found that the physics-informed reconstruction produced a more accurate image of the original, compared to reproductions without the physical law embedded.
"We have shown that deep learning can reveal invisible objects in the dark," Goy says. "This result is of practical importance for medical imaging to lower the exposure of the patient to harmful radiation, and for astronomical imaging."
Research Report: "Low Photon Count Phase Retrieval Using Deep Learning."
DRS to provide power modules for the Air and Missile Defense Radar
Washington (UPI) Dec 10, 2018
The Navy has awarded DRS Power & Control Technologies $13.3 million for Arleigh Burke-class power conversion modules for Air and Missile Defense Radar production and support. The contract, announced Friday by the Department of Defense, provides for up to 12 ship sets for the Arleigh Burke-class under the Flight III destroyer construction program for modernized versions of the vessels. The sets will provide power from the ship's electrical system. The program is expected to be completed by 2022. ... read more
|The content herein, unless otherwise known to be public domain, are Copyright 1995-2024 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. All articles labeled "by Staff Writers" include reports supplied to Space Media Network by industry news wires, PR agencies, corporate press officers and the like. Such articles are individually curated and edited by Space Media Network staff on the basis of the report's information value to our industry and professional readership. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. General Data Protection Regulation (GDPR) Statement Our advertisers use various cookies and the like to deliver the best ad banner available at one time. All network advertising suppliers have GDPR policies (Legitimate Interest) that conform with EU regulations for data collection. By using our websites you consent to cookie based advertising. If you do not agree with this then you must stop using the websites from May 25, 2018. Privacy Statement. Additional information can be found here at About Us.