Self-training Room Layout Estimation
via Geometry-aware Ray-casting
ECCV 2024

  • 1National Tsing Hua University
  • 2Industrial Technology Research Institute, Taiwan
  • 3Google

Abstract

overview

In this paper, we introduce a novel geometry-aware self- training framework for room layout estimation models on unseen scenes with unlabeled data. Our approach utilizes a ray-casting formulation to aggregate multiple estimates from different viewing positions, enabling the computation of reliable pseudo-labels for self-training. In particular, our ray-casting approach enforces multi-view consistency along all ray directions and prioritizes spatial proximity to the camera view for geom- etry reasoning. As a result, our geometry-aware pseudo-labels effectively handle complex room geometries and occluded walls without relying on assumptions such as Manhattan World or planar room walls. Evaluation on publicly available datasets, including synthetic and real-world sce- narios, demonstrates significant improvements in current state-of-the-art layout models without using any human annotation.

Video

Citation

AخA
 
@misc{solarte2024ray_casting_mlc,
        title={Self-training Room Layout Estimation via Geometry-aware Ray-casting}, 
        author={Bolivar Solarte and Chin-Hsuan Wu and Jin-Cheng Jhang and Jonathan Lee and Yi-Hsuan Tsai and Min Sun},
        year={2024},
        eprint={2407.15041},
        archivePrefix={arXiv},
        primaryClass={cs.CV},
        url={https://arxiv.org/abs/2407.15041}, 
}
                

Acknowledgements

This project is supported by The National Science and Technology Council NSTC and The Taiwan Computing Cloud TWCC under the project NSTC 112- 2634-F-002-006. We also thanks to the Industrial Technology Research Institute ITRI, Taiwan.
The website template was borrowed from Michaël Gharbi.