TrueCity: Real and Simulated Urban Data for Cross-Domain 3D Scene Understanding

Abstract

We introduce TrueCity, the first urban semantic segmentation benchmark with cm-accurate annotated real-world point clouds, semantic 3D city models, and annotated simulated point clouds representing the same city. TrueCity proposes segmentation classes aligned with international 3D city modeling standards, enabling consistent evaluation of synthetic-to-real gap. Our extensive experiments on common baselines quantify domain shift and highlight strategies for exploiting synthetic data to enhance real-world 3D scene understanding.

TrueCity Benchmark Dataset

Real-world point cloud (2nd row), which was manually labeled according to the class list, used for manual modeling of semantic 3D models (3rd row), which in turn were used to simulate and auto-label synthetic point clouds (4th row).

Key Features

• Real-world point clouds: 113 million points with densities up to 3,000 pts/m² and 1-3 cm accuracy
• Semantic 3D city models: CityGML 2.0 compliant models of Ingolstadt city
• Simulated point clouds: CARLA-based synthetic data representing the same city
• 12 semantic classes: Harmonized with CityGML 2.0 and OpenDRIVE 1.4 standards

Experiments

Top-down schematic of S--R mixtures along a continuous streetscape. Solid lines mark train/validation/test splits; dashed lines mark boundaries between contiguous synthetic and real segments for each mixture ratio.

Evaluation Protocol

We evaluate on TrueCity under controlled synthetic-real (S--R) mixtures: 100S--0R, 75S--25R, 50S--50R, 25S--75R, and 0S--100R. We test on point-based (PointNet, PointNet++, RandLA-Net), kernel-based (KPConv), and transformer-based (Point Transformer v1/v3, Superpoint Transformer, OctFormer) methods.

Results

Qualitative impact of the synthetic–real (S--R) training mix on models from different methods (Point-based, Kernel-based and Transformer-based).

Key Findings

• Point Transformer v3 improves mIoU from 14.13% at 100S--0R to 25.30% at 50S--50R
• Point Transformer v1 benefits primarily from real data, reaching 28.89% at 0S--100R
• PointNet++ peaks at 25.38% with 25S--75R mixture
• Balanced mixes (around 50S--50R or 25S--75R) work best for transformer-based models
• Locality-driven models (RandLA-Net, Superpoint Transformer) rely heavily on real data

Citation

If you use TrueCity in your research, please cite our work:

Acknowledgments

This work was developed by the TUM Chair of Geoinformatics. We thank all contributors and reviewers for their valuable feedback.