. 24/7 Space News .
Deep-learning technique reveals 'invisible' objects in the dark
by Jennifer Chu, MIT News Office
Boston MA (SPX) Dec 13, 2018

From an original transparent etching (far right), engineers produced a photograph in the dark (top left), then attempted to reconstruct the object using first a physics-based algorithm (top right), then a trained neural network (bottom left), before combining both the neural network with the physics-based algorithm to produce the clearest, most accurate reproduction (bottom right) of the original object.

Small imperfections in a wine glass or tiny creases in a contact lens can be tricky to make out, even in good light. In almost total darkness, images of such transparent features or objects are nearly impossible to decipher. But now, engineers at MIT have developed a technique that can reveal these "invisible" objects, in the dark.

In a study published in Physical Review Letters, the researchers reconstructed transparent objects from images of those objects, taken in almost pitch-black conditions. They did this using a "deep neural network," a machine-learning technique that involves training a computer to associate certain inputs with specific outputs - in this case, dark, grainy images of transparent objects and the objects themselves.

The team trained a computer to recognize more than 10,000 transparent glass-like etchings, based on extremely grainy images of those patterns. The images were taken in very low lighting conditions, with about one photon per pixel - far less light than a camera would register in a dark, sealed room. They then showed the computer a new grainy image, not included in the training data, and found that it learned to reconstruct the transparent object that the darkness had obscured.

The results demonstrate that deep neural networks may be used to illuminate transparent features such as biological tissues and cells, in images taken with very little light.

"In the lab, if you blast biological cells with light, you burn them, and there is nothing left to image," says George Barbastathis, professor of mechanical engineering at MIT. "When it comes to X-ray imaging, if you expose a patient to X-rays, you increase the danger they may get cancer. What we're doing here is, you can get the same image quality, but with a lower exposure to the patient. And in biology, you can reduce the damage to biological specimens when you want to sample them."

Barbastathis' co-authors on the paper are lead author Alexandre Goy, Kwabena Arthur, and Shuai Li.

Deep dark learning
Neural networks are computational schemes that are designed to loosely emulate the way the brain's neurons work together to process complex data inputs. A neural network works by performing successive "layers" of mathematical manipulations.

Each computational layer calculates the probability for a given output, based on an initial input. For instance, given an image of a dog, a neural network may identify features reminiscent first of an animal, then more specifically a dog, and ultimately, a beagle. A "deep" neural network encompasses many, much more detailed layers of computation between input and output.

A researcher can "train" such a network to perform computations faster and more accurately, by feeding it hundreds or thousands of images, not just of dogs, but other animals, objects, and people, along with the correct label for each image. Given enough data to learn from, the neural network should be able to correctly classify completely new images.

Deep neural networks have been widely applied in the field of computer vision and image recognition, and recently, Barbastathis and others developed neural networks to reconstruct transparent objects in images taken with plenty of light. Now his team is the first to use deep neural networks in experiments to reveal invisible objects in images taken in the dark.

"Invisible objects can be revealed in different ways, but it usually requires you to use ample light," Barbastathis says. "What we're doing now is visualizing the invisible objects, in the dark. So it's like two difficulties combined. And yet we can still do the same amount of revelation."

The law of light
The team consulted a database of 10,000 integrated circuits (IC), each of which is etched with a different intricate pattern of horizontal and vertical bars.

"When we look with the naked eye, we don't see much - they each look like a transparent piece of glass," Goy says. "But there are actually very fine and shallow structures that still have an effect on light."

Instead of etching each of the 10,000 patterns onto as many glass slides, the researchers used a "phase spatial light modulator," an instrument that displays the pattern on a single glass slide in a way that recreates the same optical effect that an actual etched slide would have.

The researchers set up an experiment in which they pointed a camera at a small aluminum frame containing the light modulator. They then used the device to reproduce each of the 10,000 IC patterns from the database. The researchers covered the entire experiment so it was shielded from light, and then used the light modulator to rapidly rotate through each pattern, similarly to a slide carousel. They took images of each transparent pattern, in near total darkness, producing "salt-and-pepper" images that resembled little more than static on a television screen.

The team developed a deep neural network to identify transparent patterns from dark images, then fed the network each of the 10,000 grainy photographs taken by the camera, along with their corresponding patterns, or what the researchers called "ground-truths."

"You tell the computer, 'If I put this in, you get this out,'" Goy says. "You do this 10,000 times, and after the training, you hope that if you give it a new input, it can tell you what it sees."

"It's a little worse than a baby," Barbastathis quips. "Usually babies learn a bit faster."

The researchers set their camera to take images slightly out of focus. As counterintuitive as it seems, this actually works to bring a transparent object into focus. Or, more precisely, defocusing provides some evidence, in the form of ripples in the detected light, that a transparent object may be present. Such ripples are a visual flag that a neural network can detect as a first sign that an object is somewhere in an image's graininess.

But defocusing also creates blur, which can muddy a neural network's computations. To deal with this, the researchers incorporated into the neural network a law in physics that describes the behavior of light, and how it creates a blurring effect when a camera is defocused.

"What we know is the physical law of light propagation between the sample and the camera," Barbastathis says. "It's better to include this knowledge in the model, so the neural network doesn't waste time learning something that we already know."

Sharper image
After training the neural network on 10,000 images of different IC patterns, the team created a completely new pattern, not included in the original training set. When they took an image of the pattern, again in darkness, and fed this image into the neural network, they compared the patterns that the neural network reconstructed, both with and without the physical law embedded in the network.

They found that both methods reconstructed the original transparent pattern reasonably well, but the "physics-informed reconstruction" produced a sharper, more accurate image. What's more, this reconstructed pattern, from an image taken in near total darkness, was more defined than a physics-informed reconstruction of the same pattern, imaged in light that was more than 1,000 times brighter.

The team repeated their experiments with a totally new dataset, consisting of more than 10,000 images of more general and varied objects, including people, places, and animals. After training, the researchers fed the neural network a completely new image, taken in the dark, of a transparent etching of a scene with gondolas docked at a pier. Again, they found that the physics-informed reconstruction produced a more accurate image of the original, compared to reproductions without the physical law embedded.

"We have shown that deep learning can reveal invisible objects in the dark," Goy says. "This result is of practical importance for medical imaging to lower the exposure of the patient to harmful radiation, and for astronomical imaging."

Research Report: "Low Photon Count Phase Retrieval Using Deep Learning."

Related Links
Massachusetts Institute of Technology
Space Technology News - Applications and Research

Thanks for being there;
We need your help. The SpaceDaily news network continues to grow but revenues have never been harder to maintain.

With the rise of Ad Blockers, and Facebook - our traditional revenue sources via quality network advertising continues to decline. And unlike so many other news sites, we don't have a paywall - with those annoying usernames and passwords.

Our news coverage takes time and effort to publish 365 days a year.

If you find our news sites informative and useful then please consider becoming a regular supporter or for now make a one off contribution.
SpaceDaily Monthly Supporter
$5+ Billed Monthly

paypal only
SpaceDaily Contributor
$5 Billed Once

credit card or paypal

DRS to provide power modules for the Air and Missile Defense Radar
Washington (UPI) Dec 10, 2018
The Navy has awarded DRS Power & Control Technologies $13.3 million for Arleigh Burke-class power conversion modules for Air and Missile Defense Radar production and support. The contract, announced Friday by the Department of Defense, provides for up to 12 ship sets for the Arleigh Burke-class under the Flight III destroyer construction program for modernized versions of the vessels. The sets will provide power from the ship's electrical system. The program is expected to be completed by 2022. ... read more

Comment using your Disqus, Facebook, Google or Twitter login.

Share this article via these popular social media networks
del.icio.usdel.icio.us DiggDigg RedditReddit GoogleGoogle

Russian spacewalkers take sample of mystery hole at space station

George H.W. Bush's overlooked legacy in space exploration

UConn Research Project Heading to International Space Station

NASA sends new research, hardware to Space Station on SpaceX mission

China puts 2 Saudi satellites into orbit

Aerojet Rocketdyne awarded DARPA contract to design advanced opfires propulsion system

Tesla CEO Elon Musk taunts US financial regulatory agency

Rocket Lab prepares to launch historic CubeSat mission for NASA

InSight's robotic arm ready for some lifting on Mars

NASA's InSight lander 'hears' wind on Mars

NASA's Mars InSight Flexes Its Arm

Mars 2020 rover mission camera system 'Mastcam-Z' testing begins at ASU

Evolving Chinese Space Ecosystem To Foster Innovative Environment

China sends 5 satellites into orbit via single rocket

China releases smart solution for verifying reliability of space equipment components

China unveils new 'Heavenly Palace' space station as ISS days numbered

CAT rules in favour of Ofcom's EAN authorisation decision

Fleet Space Technologies' Centauri launched aboard SpaceX Falcon 9

Roscosmos Targeted by Info Attack to Hamper Revival of Space Industry in Russia

SAS Signs Distribution Agreement with GlobalSat Group

Deep-learning technique reveals 'invisible' objects in the dark

DRS to provide power modules for the Air and Missile Defense Radar

Researchers develop mathematical solver for analog computers

Terahertz laser for sensing and imaging outperforms its predecessors

Life in Deep Earth totals 15 to 23 billion tons of carbon

An exoplanet loses its atmosphere in the form of a tail

Unknown treasure trove of planets found hiding in dust

Radio Search for Artificial Emissions from 'Oumuamua

Radio JOVE From NASA: Tuning In to Your Local Celestial Radio Show

The PI's Perspective: Share the News - The Farthest Exploration of Worlds in History is Beginning

Encouraging prospects for moon hunters

Evidence for ancient glaciation on Pluto

The content herein, unless otherwise known to be public domain, are Copyright 1995-2024 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. All articles labeled "by Staff Writers" include reports supplied to Space Media Network by industry news wires, PR agencies, corporate press officers and the like. Such articles are individually curated and edited by Space Media Network staff on the basis of the report's information value to our industry and professional readership. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. General Data Protection Regulation (GDPR) Statement Our advertisers use various cookies and the like to deliver the best ad banner available at one time. All network advertising suppliers have GDPR policies (Legitimate Interest) that conform with EU regulations for data collection. By using our websites you consent to cookie based advertising. If you do not agree with this then you must stop using the websites from May 25, 2018. Privacy Statement. Additional information can be found here at About Us.