. 24/7 Space News .
ORNL researchers design novel method for energy-efficient deep neural networks
by Staff Writers
Oak Ridge TN (SPX) Mar 16, 2018

ORNL's Hong-Jun Yoon, Mohammed Alawad and Gina Tourassi have developed a novel method for more efficiently training large numbers of networks capable of solving complex science problems. Image courtesy Jason Richards/Oak Ridge National Laboratory, US Department of Energy.

An Oak Ridge National Laboratory method to improve the energy efficiency of scientific artificial intelligence is showing early promise in efforts to parse insights from volumes of cancer data.

Researchers are realizing the potential of deep learning to rapidly advance science, but "training" the underlying neural networks with large volumes of data to tackle the task at hand can require large amounts of energy. These networks also require complex connectivity and enormous amounts of storage, both of which further reduce their energy efficiency and potential in real-world applications.

To address this issue, ORNL's Mohammed Alawad, Hong-Jun Yoon, and Georgia Tourassi developed a novel method for the development of energy-efficient deep neural networks capable of solving complex science problems. They presented their research at the 2017 IEEE Conference on Big Data in Boston.

The researchers demonstrated that by converting deep learning neural networks (DNNs) to "deep spiking" neural networks (DSNNs), they can improve the energy efficiency of network design and realization.

DSNNs imitate neurons in the human brain via pulses or "spikes" in the place of actual signals, with the individual spikes indicating where to perform the computations. This process minimizes the necessary calculations and maximizes the network's energy efficiency. However, energy efficiency comes at the cost of task performance, and the authors' novel stochastic method for implementing DSNNs overcomes this tradeoff.

The results were impressive: the team's approach achieved nearly the same accuracy as the original DNN and performed better than a state-of-the-art spiking neural network. The team's stochastic-based DSNN, which distributes spikes uniformly over time, consumed 38 times less energy than the original DNN and almost 2 times less energy than a conventional DSNN while delivering markedly better task performance.

The researchers trained their network on clinical text data from the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) Program, which provides cancer statistics such as incidence, prevalence, and mortality across the population associated by age, sex, race, year of diagnosis, and geographic areas.

The ORNL team applied the newly trained networks to clinical pathology reports, the main source of information for the national cancer surveillance program. These reports contain vast amounts of unstructured text, said Yoon, and researchers are developing intelligent language understanding systems to extract the most relevant clinical concepts in the sea of text.

The clinical reports represent a "sparse" dataset, which typically pose unique challenges to spiking networks. Most DSNN techniques have focused on computer vision tasks such as the MNIST dataset, which consist of a series of handwritten digits to train image processing networks. These datasets are typically "dense," meaning all variables in the dataset are populated with values, a characteristic that often simplifies analyses.

Traditional techniques for improving the performance and energy efficiency of spiking networks often preserve the structures of conventional neural networks, a practice that sacrifices accuracy and performance. These shortcomings drove the team to develop a new methodology, which relies on simple circuit hardware to perform complex calculations.

"Spiking the network lowers energy consumption because we disregard the unnecessary computations and we look only for the relevant nodes of the network," said Yoon, "and this is one way we get energy efficiency improvements while identifying important clinical information with high accuracy."

The team's technique will help ORNL researchers in the CANcer Distributed Learning Environment (CANDLE) project, which aims to use the lab's world-class big data expertise and computing facilities to scan millions of clinical reports in search for insights on causes of cancer, best courses of treatment, and improved outcomes. They will soon attempt to parallelize the algorithm for more computational efficiency.

The spiking networks were optimized on graphics processing units (GPUs), the processors of choice for artificial intelligent applications, particularly those that utilize machine learning and deep learning. However, the methodology can be extended for training spiking networks, further increasing the energy efficiency of these novel networks while expediting scientific discovery via deep learning.

Related Links
Oak Ridge National Laboratory
Space Technology News - Applications and Research

Thanks for being there;
We need your help. The SpaceDaily news network continues to grow but revenues have never been harder to maintain.

With the rise of Ad Blockers, and Facebook - our traditional revenue sources via quality network advertising continues to decline. And unlike so many other news sites, we don't have a paywall - with those annoying usernames and passwords.

Our news coverage takes time and effort to publish 365 days a year.

If you find our news sites informative and useful then please consider becoming a regular supporter or for now make a one off contribution.
SpaceDaily Monthly Supporter
$5+ Billed Monthly

paypal only
SpaceDaily Contributor
$5 Billed Once

credit card or paypal

Researchers demonstrate promising method for improving quantum information processing
Oak Ridge TN (SPX) Feb 20, 2018
A team of researchers led by the Department of Energy's Oak Ridge National Laboratory has demonstrated a new method for splitting light beams into their frequency modes. The scientists can then choose the frequencies they want to work with and encode photons with quantum information. Their work could spur advancements in quantum information processing and distributed quantum computing. The frequency of light determines its color. When the frequencies are separated, as in a rainbow, each color phot ... read more

Comment using your Disqus, Facebook, Google or Twitter login.

Share this article via these popular social media networks
del.icio.usdel.icio.us DiggDigg RedditReddit GoogleGoogle

Astronaut Scott Kelly weighs in on the 'State of Science'

Knowledge matters for Year of Education on Station

NASA, partners seek input on standards for deep space technologies

Goddard licenses gear bearing tech to Bahari Energy for urban wind power

SpaceX carries out 50th launch of Falcon 9 rocket

NASA team outfits Orion for abort test with lean approach

World-first firing of air-breathing electric thruster

GOES-S marks 100th launch of Rocketdyne AJ-60A solid rocket booster

The Case of the Martian Boulder Piles

Opportunity collects more 'Selfie' frames

Dyes for 'live' extremophile labeling will help discover life on Mars

Mars Express views moons set against Saturn's rings

China moving ahead with plans for next-generation X-ray observatory

China to launch Long March-5B rocket in 2019

Satellite will test plan for global China led satcom network

China plans rocket sea-launch

Lockheed Martin Begins Assembly of JCSAT-17 Commercial Communications Satellite

ESA Astronaut will test CIMON aboard the ISS Watson AI

Iridium Certus readies for takeoff with aviation service providers

ESA incubators ranked among world's best

Researchers use 'flying focus' to better control lasers over long distances

Technique to see objects hidden around corners

New imaging technology shows laser pulses are formed from chaos

Latest Updates from NASA on IMAGE Recovery

Heat shock system helps bug come back to life after drying up

Rare mineral discovered in plants for first time

Hubble observes exoplanet atmosphere in more detail than ever before

Chemical sleuthing unravels possible path to forming life's building blocks in space

Jupiter's turmoil more than skin deep: researchers

Jupiter's Jet-Streams Are Unearthly

You are entering the Jovian Twilight Zone

The PI's Perspective: Why Didn't Voyager Explore the Kuiper Belt?

The content herein, unless otherwise known to be public domain, are Copyright 1995-2024 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. All articles labeled "by Staff Writers" include reports supplied to Space Media Network by industry news wires, PR agencies, corporate press officers and the like. Such articles are individually curated and edited by Space Media Network staff on the basis of the report's information value to our industry and professional readership. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. General Data Protection Regulation (GDPR) Statement Our advertisers use various cookies and the like to deliver the best ad banner available at one time. All network advertising suppliers have GDPR policies (Legitimate Interest) that conform with EU regulations for data collection. By using our websites you consent to cookie based advertising. If you do not agree with this then you must stop using the websites from May 25, 2018. Privacy Statement. Additional information can be found here at About Us.