24/7 Space News
A system to keep cloud-based gamers in sync
stock illustration only
A system to keep cloud-based gamers in sync
by Adam Zewe for MIT News
Boston MA (SPX) Sep 04, 2023

Cloud gaming, which involves playing a video game remotely from the cloud, witnessed unprecedented growth during the lockdowns and gaming hardware shortages that occurred during the heart of the Covid-19 pandemic. Today, the burgeoning industry encompasses a $6 billion global market and more than 23 million players worldwide.

However, interdevice synchronization remains a persistent problem in cloud gaming and the broader field of networking. In cloud gaming, video, audio, and haptic feedback are streamed from one central source to multiple devices, such as a player's screen and controller, which typically operate on separate networks. These networks aren't synchronized, leading to a lag between these two separate streams. A player might see something happen on the screen and then hear it on their controller a half second later.

Inspired by this problem, scientists from MIT and Microsoft Research took a unique approach to synchronizing streams transmitted to two devices. Their system, called Ekho, adds inaudible white noise sequences to the game audio streamed from the cloud server. Then it listens for those sequences in the audio recorded by the player's controller.

Ekho uses the mismatch between these noise sequences to continuously measure and compensate for the interstream delay.

In real cloud gaming sessions, the researchers showed that Ekho is highly reliable. The system can keep streams synchronized to within less than 10 milliseconds of each other, most of the time. Other synchronization methods resulted in consistent delays of more than 50 milliseconds.

And while Ekho was designed for cloud gaming, this technique could be used more broadly to synchronize media streams traveling to different devices, such as in training situations that utilize multiple augmented or virtual reality headsets.

"Sometimes, all it takes for a good solution to come out is to think outside what has been defined for you. The entire community has been fixed on how to solve this problem by synchronizing through the network. Synchronizing two streams by listening to the audio in the room sounded crazy, but it turned out to be a very good solution," says Pouya Hamadanian, an electrical engineering and computer science (EECS) graduate student and lead author of a paper describing Ekho.

Hamadanian is joined on the paper by Doug Gallatin, a software developer at Microsoft; Mohammad Alizadeh, an associate professor of electrical engineering and computer science and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); and senior author Krishna Chintalapudi, a principal researcher at Microsoft Research. The paper will be presented at the ACM SIGCOMM conference.

Off the clock
At the heart of interstream delay in cloud gaming is a fundamental problem in networking known as clock synchronization.

"If the controller and the screen could look at their watches and at the same time see the same thing, then we could synchronize everything to the clock. But a lot of theoretical work on clock synchronization shows that there are certain bounds you can never overcome," Hamadanian says.

Many approaches attempt clock synchronization by ping-pong messaging, where a device sends a ping message to the server, which sends a pong message back. The device counts how long it takes the message to return, and cuts that value in half to calculate the network delay.

But the path over the network is likely asymmetric, so it may take more time for the message to reach the server than it does for the return message. Therefore, this method is unreliable and can introduce hundreds of milliseconds of error. Humans can typically perceive interstream delay once it reaches 10 milliseconds.

"So if something happens on the screen, we want it to happen within 10 milliseconds on the controller, as well," Hamadanian explains.

He and his collaborators decided to try listening to game audio to synchronize these separate streams.

In cloud gaming, the microphone on the player's controller records audio in the room, including game audio played by the speakers on the screen, which it sends back to the server. But using this for synchronization is unreliable because the room audio contains background noise.

So they designed Ekho to add identical sequences of extremely low-volume white noise, known as pseudo noise, to the game audio before it is streamed to the player's screen. It uses these pseudo-noise segments for synchronization.

Before building Ekho, the researchers conducted a user study to prove that players could not hear the pseudo noise in the game audio. These noise sequences are also resilient to compression, which is important because audio sent from the controller is highly compressed to speed the data transfer.

Pseudo noise, real success
The Ekho-Estimator module adds pseudo-noise sequences to the game audio. When it receives the recorded game audio from the controller, it listens for those markers and tries to line up the streams. This enables it to precisely calculate the inter-stream delay.

The Ekho-Estimator sends that information to the Ekho-Compensator module, which either skips a few milliseconds of sound or adds a few milliseconds of silence to the game audio being sent by the server, which synchronizes the streams.

They tested Ekho on real cloud streaming sessions and found that it was superior to other synchronization methods, even when the microphone quality was poor or background noise was picked up by the recording.

Ekho limited interstream delay to less than 10 milliseconds for nearly 87 percent of the time during streams. No other method the team tested was able to cut that delay to less than 50 milliseconds.

"The traditional way of doing this, which involves trying to measure the synchronization error using the underlying network, the errors are significantly larger. When we started this project, were weren't sure whether this could even be done. But the accuracy we can get down to with Ekho, at sub-millisecond levels, it is unheard of," says Chintalapudi.

Impressed by these results, the researchers want to see how well Ekho performs in more complex situations, such as synchronizing five controllers to the same screen device. Also, since Ekho was targeted for cloud gaming, it has range limitations. Future work could seek to enhance Ekho so it can synchronize devices at either end of a very large room, like a concert hall.

"Using inaudible white noise as a sort of 'timekeeper' is a great example of how out-of-the-box thinking can produce unexpected results," says Alizadeh. "The technique could improve user experience, not just in cloud gaming but potentially in any multidevice streaming scenario."

Research Report:"Ekho: Synchronizing Cloud Gaming Media Across Multiple Endpoints"

Related Links
Computer Science and Artificial Intelligence Laboratory
Space Technology News - Applications and Research

Subscribe Free To Our Daily Newsletters

The following news reports may link to other Space Media Network websites.
AI revolution in video games has industry players treading warily
Cologne, Germany (AFP) Aug 27, 2023
From generating story lines to coding entire games to turning ideas into animation, artificial intelligence is front and centre at Gamescom, one of the video game industry's biggest fairs. But even the ultra-connected industry is eyeing the innovation warily, with fears growing that jobs could be made redundant and artistic creations usurped. "AI is really a turning point," according to Julien Millet, an AI engineer and founder of United Bits Game studio, who attended the industry fair this wee ... read more

Station Hosts 11 Crewmates from Five Countries

A multinational crew blasts off from Florida, heading for the International Space Station

NASA challenges students to fly Earth and Space experiments

US seeks to extend China science accord, but only briefly for now

Musk threatens to sue anti-defamation group for falling revenue

Benchmark Space Systems cracks code for viable ASCENT propellant

SpaceX sends crew of four to ISS

Rocket Lab Launches 40th Electron Mission, Successfully Flies Reused Engine

NASA, Partners study ancient life in Australia to inform Mars search

Martian Tapas With a View: Sols 3926-3927

Delight at Dream Lake

Approaching the Ridgetop - "Bermuda Triangle" Ahead: Sols 3923-3925

From rice to quantum gas: China's targets pioneering space research

China to launch "Innovation X Scientific Flight" program, applications open worldwide

Scientists reveal blueprint of China's lunar water-ice probe mission

Shenzhou 15 crew share memorable moments from Tiangong Station mission

LeoStella and Hera Systems Establish Strategic Alliance

Viasat provides status update on Inmarsat-6 F2

Momentus announces reverse stock split

Pentagon awards contracts for next 'swarm' of tiny missile defense satellites

A system to keep cloud-based gamers in sync

From art squat to Berlin gentrification lightning rod

US envoy feasts on Fukushima fish, slams China water 'dumps'

Japan PM eats 'safe and delicious' Fukushima fish

Scientists detect and validate the longest-period exoplanet found with TESS

Accretion disks: How big are they really?

Study explains how part of the nucleolus evolved

Size dependence and the collisional dynamics of protoplanetary dust growth

In the service of planetary science, astrophysics and heliophysics

Mysterious Neptune dark spot detected from Earth for the first time

Neptune's Disappearing Clouds Linked to the Solar Cycle

The Road to Jupiter: Two decades of trajectory optimization

Subscribe Free To Our Daily Newsletters


The content herein, unless otherwise known to be public domain, are Copyright 1995-2023 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. All articles labeled "by Staff Writers" include reports supplied to Space Media Network by industry news wires, PR agencies, corporate press officers and the like. Such articles are individually curated and edited by Space Media Network staff on the basis of the report's information value to our industry and professional readership. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. General Data Protection Regulation (GDPR) Statement Our advertisers use various cookies and the like to deliver the best ad banner available at one time. All network advertising suppliers have GDPR policies (Legitimate Interest) that conform with EU regulations for data collection. By using our websites you consent to cookie based advertising. If you do not agree with this then you must stop using the websites from May 25, 2018. Privacy Statement. Additional information can be found here at About Us.