. 24/7 Space News .
Data scientists mine government data for evidence of historical events
by Brooks Hays
Washington (UPI) Dec 29, 2017

Researchers have developed a way to mine electronic government records for evidence of significant historical events.

For journalists and historians, sorting through government records is a tedious but important job. Though many government records aren't released until after a news event has come and gone, once published, the documents can shed new light historical happenings.

Data scientists at Columbia University in New York have developed a way to more quickly and efficiently identify insightful and important government documents.

Researchers tested their new data mining methodology on newly available classified records dated between 1973 and 1977. The records include 1.4 million declassified cables, as well as metadata related to 400,000 documents delivered via diplomatic pouch.

Each document is tagged with codes relevant to its subject matter. UNGA marks documents related to the United Nations General Assembly, for example. Cables related to South Vietnam are coded VS. Finland-related messages are tagged FI.

When plotted along a timeline, these codes reveal spikes corresponding with world events. UNGA spikes correspond with the annual assembly, while a large uptick in VS messages corresponds with the fall of Saigon in April of 1975.

These spikes are rather obvious. For more useful insights, scientists compared the messaging timeline with corresponding background activity -- the news as it was being reported at the time.

The technique allowed researchers to rank the significance of world events, and to identify important events that were missed by journalists at the time.

For example, scientists identified several spikes to government records related to administrative problems, such as transport issues and changes to the visa records system.

"The ranking finds a wide range of other important events, such as the Carter administration's prioritization of human rights, the president of Egypt, Anwar Sadat's surprise visit to Israel in 1977, the Southeast Asian 'Boat People' crisis of 1975-76, the 1973 Yom Kippur War and Portugal's withdrawal from Angola in 1975-76 and so on," as reported by the MIT Technology Review.

The researchers published the results of their data mining analysis online at arXiv.org.

'Magnetoelectric' material shows promise as memory for electronics
Madison WI (SPX) Dec 08, 2017
Smartphones and computers wouldn't be nearly as useful without room for lots of apps, music and videos. Devices tend to store that information in two ways: through electric fields (think of a flash drive) or through magnetic fields (like a computer's spinning hard disk). Each method has advantages and disadvantages. However, in the future, our electronics could benefit from the best of eac ... read more

Related Links
Space Technology News - Applications and Research

Thanks for being there;
We need your help. The SpaceDaily news network continues to grow but revenues have never been harder to maintain.

With the rise of Ad Blockers, and Facebook - our traditional revenue sources via quality network advertising continues to decline. And unlike so many other news sites, we don't have a paywall - with those annoying usernames and passwords.

Our news coverage takes time and effort to publish 365 days a year.

If you find our news sites informative and useful then please consider becoming a regular supporter or for now make a one off contribution.
SpaceDaily Monthly Supporter
$5+ Billed Monthly

paypal only
SpaceDaily Contributor
$5 Billed Once

credit card or paypal

Comment using your Disqus, Facebook, Google or Twitter login.

Share this article via these popular social media networks
del.icio.usdel.icio.us DiggDigg RedditReddit GoogleGoogle

Race for 'smart' hits fever pitch at electronics show

JPL sketches out a trip to the Alpha Centauri system in 2069

The Russian Progress MS-06 cargo freighter undocks and takes a final Pacific dive

NASA picks finalists to explore comet, Saturn's moon

Triumph expands contract for Dream Chaser spacecraft landing gear system

China tests new ballistic missiles with hypersonic glide vehicles

One Small Step: Massive Stratolaunch Aircraft Conducts First Taxi Tests

Space Launch System solid rocket booster avionics complete key testing

Opportunity takes extensive imagery to decide where to go next

Mars: Not as dry as it seems

Mars' surface water - the truth is out there

Thirsty rocks may contain the missing water of Mars

Nation 'leads world' in remote sensing technology

China plans for nuclear-powered interplanetary capacity by 2040

China plans first sea based launch by 2018

China's reusable spacecraft to be launched in 2020

Russia restores contact with Angolan satellite

Fourth set of Iridium NEXT satellites arrive in orbit and provide telemetry

SpaceX launches 10 more satellites for Iridium

Green Light for Continued Operations of ESA Science Missions

Data scientists mine government data for evidence of historical events

Nature's smallest rainbows, created by peacock spiders, may inspire new optical technology

New lensless camera creates detailed 3-D images without scanning

Accelerated analysis of the stability of complex alloys

Which came first: Complex life or high atmospheric oxygen?

NASA Invests in Concept Development for Missions to Comet and Titan

Genes in Space-3 successfully identifies unknown microbes in space

Powerful new tool for looking for life beyond Earth

Study explains why Jupiter's jet stream reverses course on a predictable schedule

New Horizons Corrects Its Course in the Kuiper Belt

Does New Horizons' Next Target Have a Moon?

Juno probes the depths of Jupiter's Great Red Spot

The content herein, unless otherwise known to be public domain, are Copyright 1995-2024 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. All articles labeled "by Staff Writers" include reports supplied to Space Media Network by industry news wires, PR agencies, corporate press officers and the like. Such articles are individually curated and edited by Space Media Network staff on the basis of the report's information value to our industry and professional readership. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. General Data Protection Regulation (GDPR) Statement Our advertisers use various cookies and the like to deliver the best ad banner available at one time. All network advertising suppliers have GDPR policies (Legitimate Interest) that conform with EU regulations for data collection. By using our websites you consent to cookie based advertising. If you do not agree with this then you must stop using the websites from May 25, 2018. Privacy Statement. Additional information can be found here at About Us.