24/7 Space News
AI models struggle to identify nonsense, says study
AI models struggle to identify nonsense, says study
by AFP Staff Writers
Paris (AFP) Sept 14, 2023

The AI models that power chatbots and other applications still have difficulty distinguishing between nonsense and natural language, according to a study released on Thursday.

The researchers at Columbia University in the United States said their work revealed the limitations of current AI models and suggested it was too early to let them loose in legal or medical settings.

They put nine AI models through their paces, firing hundreds of pairs of sentences at them and asking which were likely to be heard in everyday speech.

They asked 100 people to make the same judgement on pairs of sentences like: "A buyer can own a genuine product also / One versed in circumference of highschool I rambled."

The research, published in the Nature Machine Intelligence journal, then weighed the AI answers against the human answers and found dramatic differences.

Sophisticated models like GPT-2, an earlier version of the model that powers viral chatbot ChatGPT, generally matched the human answers.

Other simpler models did less well.

But the researchers highlighted that all the models made mistakes.

"Every model exhibited blind spots, labelling some sentences as meaningful that human participants thought were gibberish," said psychology professor Christopher Baldassano, an author of the report.

"That should give us pause about the extent to which we want AI systems making important decisions, at least for now."

Tal Golan, another of the paper's authors, told AFP that the models were "an exciting technology that can complement human productivity dramatically".

However, he argued that "letting these models replace human decision-making in domains such as law, medicine, or student evaluation may be premature".

Among the pitfalls, he said, was the possibility that people might intentionally exploit the blind spots to manipulate the models.

AI models burst into public consciousness with the release of ChatGPT last year, which has since been credited with passing various exams and has been touted as a possible aide to doctors, lawyers and other professionals.

Related Links
All about the robots on Earth and beyond!

Subscribe Free To Our Daily Newsletters

The following news reports may link to other Space Media Network websites.
Musk, Zuckerberg visit US congress to discuss AI
Washington (AFP) Sept 13, 2023
Big tech bigwigs including Elon Musk and Mark Zuckerberg traveled to Capitol Hill on Wednesday to share their plans for artificial intelligence as the US prepares to draw up legislation to better control the technology. Senator Chuck Schumer, the Democratic majority leader of the US Senate, has planned a series of so-called AI Innovation forums, closed door meetings where lawmakers can quiz tech leaders about the technology that has taken the world by storm since the release of ChatGPT last year. ... read more

Kayhan Space Raises $7 million, Unveils First-Ever Autonomous Space Traffic Coordination Service

Two Russians, American reach space station

Rockets and Porsches: rich Russians flock to Baikonur spaceport

Soyuz hatch opens, Expedition 69 expands to 10 crewmates

'Anomaly' ends Rocket Lab launch mid-flight

SpaceX deploys another 22 Starlink satellites

Third Subscale Booster for future Artemis missions fires up at Marshall

Mini space thruster that runs on water

New milestones despite tricky boulders

Curiosity reaches Mars ridge where water left debris pileup

Reading the Rocks: The Importance of the Margin Carbonate Unit on Mars

New Mars gravity analysis improves understanding of possible ancient ocean

Tianzhou 5 spacecraft burns up on Earth reentry

Crew of Shenzhou XV mission honored for six-month space odyssey

China solicits names for manned lunar exploration vehicles

From rice to quantum gas: China's targets pioneering space research

Intelsat delivers new reliable broadcast connectivity service

Terran Orbital announces pricing of Public Offering

Terran Orbital announces Proposed Public Offering

Sidus Space secures position on upcoming Bandwagon Mission

FAA proposes rule to reduce space debris as SpaceX launches 22 satellites into orbit

China builds new radio telescope to support lunar, deep-space missions

AFRL'S newest supercomputer 'Raider' promises to compute years' worth of data in days

Skyloom and Satellogic sign agreement for Multipath Optical Comms Data Transmission

Tiny sea creatures reveal the ancient origins of neurons

Exoplanet with a large iron core adds to puzzle of how planets form

New recipes for origin of life may point way to distant, inhabited planets

On the road to spotting alien life

Webb finds carbon source on surface of Jupiter's moon Europa

Juice: why's it taking sooo long

Hidden ocean the source of CO2 on Jupiter moon

Possible existence of Earth-like planet predicted in Outskirts of Solar System

Subscribe Free To Our Daily Newsletters


The content herein, unless otherwise known to be public domain, are Copyright 1995-2023 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. All articles labeled "by Staff Writers" include reports supplied to Space Media Network by industry news wires, PR agencies, corporate press officers and the like. Such articles are individually curated and edited by Space Media Network staff on the basis of the report's information value to our industry and professional readership. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. General Data Protection Regulation (GDPR) Statement Our advertisers use various cookies and the like to deliver the best ad banner available at one time. All network advertising suppliers have GDPR policies (Legitimate Interest) that conform with EU regulations for data collection. By using our websites you consent to cookie based advertising. If you do not agree with this then you must stop using the websites from May 25, 2018. Privacy Statement. Additional information can be found here at About Us.