City digital twins help train deep learning m

Osaka, Japan – Game engines were originally developed for building fictional worlds for entertainment. However, you can use these same engines to build a copy of your real environment. in short, digital twinResearchers at Osaka University used images automatically generated by the Digital City Twin to efficiently analyze images of real cities and train deep learning models that can accurately separate the buildings displayed in them. i found a way to do it.

A convolutional neural network is a deep learning neural network designed to process structured data arrays such as images. These advances in deep learning have fundamentally changed how tasks such as architectural segmentation are performed. However, accurate deep convolutional neural network (DCNN) models require large amounts of labeled training data, and labeling this data can be time consuming and very expensive manually. there is.

To create the synthetic digital city twin data, the researchers used 3D city models from the PLATEAU platform. The PLATEAU platform contains his 3D models of most Japanese cities at a very high level. They loaded this model into the Unity game engine and created a camera setup in a virtual car. The virtual car drove around the city and acquired virtual data images under different lighting and weather conditions. We then used the Google Maps API to obtain actual street-level imagery of the same experimental study area.

Researchers have found that digital city twin data yields better results than pure virtual data with no real-world counterpart. Additionally, adding the synthetic data to the real dataset improves the segmentation accuracy. Most importantly, however, researchers found that DCNN’s segmentation accuracy was significantly improved when certain parts of the real-world data were included in the digital city twin synthetic dataset. In fact, its performance competes with that of DCNNs trained on 100% real data. “These results demonstrate that the proposed synthetic dataset has the potential to replace all real images in the training set,” said Tomohiro Fukuda, corresponding author of the paper.

The automatic isolation of individual building façades displayed in an image is useful for construction management and architectural design, large-scale measurements for retrofits and energy analysis, and even visualization of demolished building façades. increase. The system was tested in multiple cities to demonstrate the transferability of the proposed framework. Hybrid datasets of real and synthetic data provide promising predictive results for most modern architectural styles. This makes it a promising approach for training her DCNN for future architectural segmentation tasks without the need for costly manual data annotation.

###

The article “Automatic Generation of Synthetic Datasets from City Digital Twins for Use in Instance Segmentation of Building Facades” Journal of Computational Design and Engineering DOI: https://doi.org/10.1093/jcde/qwac086.

About Osaka University

Established in 1931 as one of Japan’s seven imperial universities, Osaka University is now one of Japan’s leading comprehensive universities with a wide range of academic disciplines. This strength is combined with an extraordinary drive for innovation across the entire scientific process, from basic research to creating applied technologies that have a positive impact on the economy. Its commitment to innovation has been recognized in Japan and around the world. 1 (Innovative Universities and the Nature Index Innovation 2017). Currently, Osaka University is making use of its role as a designated national university corporation selected by the Ministry of Education, Culture, Sports, Science and Technology to contribute to the innovation of human welfare, the sustainable development of society, and the transformation of society.

Website: https://resou.osaka-u.ac.jp/