X-ray street vision

Scientists from the Division of Sustainable Energy and Environmental Engineering at Osaka University used generative adversarial networks trained on a custom dataset to virtually remove obstructions from building facade images. This work may assist in civic planning as well as computer vision applications.

The ability to digitally "erase" unwanted occluding objects from a cityscape is highly useful but requires a great deal of computing power. Previous methods used standard image datasets to train machine learning algorithms.

Now, a team of researchers at Osaka University have built a custom dataset as part of a general framework for the automatic removal of unwanted objects - such as pedestrians, riders, vegetation, or cars - from an image of a building's facade. The removed region was replaced using digital inpainting to efficiently restore a complete view.

The researchers used data from the Kansai region of Japan in an open-source street view service, as opposed to the conventional building image sets often used in machine learning for urban landscapes. Then they constructed a dataset to train an adversarial generative network (GAN) for inpainting the occluded regions with high accuracy.

"For the task of facade inpainting in street-level scenes, we adopted an end-to-end deep learning-based image inpainting model by training with our customized datasets," first author Jiaxin Zhang explains.

The team used semantic segmentation to detect several types of obstructing objects, including pedestrians, vegetation, and cars, as well as using GANs for filling the detected regions with background textures and patching information from street-level imagery. They also proposed a workflow to automatically filter unblocked building facades from street view images and customized the dataset to contain both original and masked images to train additional machine learning algorithms.

This visualization technology offers a communication tool for experts and non-experts, which can help develop a consensus on future urban environmental designs.

"Our system was shown to be more efficient compared with previously employed methods when dealing with urban landscape projects for which background information was not available in advance," senior author Tomohiro Fukuda explains. In the future, this approach may be used to help design augmented reality systems that can automatically remove existing buildings and instead show proposed renovations.

Research Report: "Automatic object removal with obstructed facades completion using semantic segmentation and generative adversarial inpainting"

Related Links
Division of Sustainable Energy and Environmental Engineering at Osaka University
Satellite-based Internet technologies

Tweet

Thanks for being there;
We need your help. The SpaceDaily news network continues to grow but revenues have never been harder to maintain.

With the rise of Ad Blockers, and Facebook - our traditional revenue sources via quality network advertising continues to decline. And unlike so many other news sites, we don't have a paywall - with those annoying usernames and passwords.

Our news coverage takes time and effort to publish 365 days a year.

If you find our news sites informative and useful then please consider becoming a regular supporter or for now make a one off contribution.

SpaceDaily Monthly Supporter
$5+ Billed Monthly

paypal only

SpaceDaily Contributor
$5 Billed Once

credit card or paypal

'Dune' brings galactic-scale blockbuster to Venice
Venice (AFP) Sept 3, 2021
Giant worms and inter-planetary battles rocked the Venice Film Festival on Friday as "Dune", one of the most hotly anticipated blockbusters in years, was finally set for its world premiere. It brought a cavalcade of A-listers to the city's glitzy Lido island, along with fans packing the waterfront for the arrival of Timothee Chalamet, Zendaya, Oscar Isaac, Josh Brolin and Javier Bardem. Based on a landmark of sci-fi literature about warring clans fighting for control of a desert planet, the film ... read more