Selected Publications


(* equal contribution, † corresponding author.)
Scene-agnostic Pose Regression for Visual Localization
J. Zheng, R. Liu, Y. Chen, Z. Chen, K. Yang, J. Zhang†, R. Stiefelhagen
CVPR 2025 Project page Paper Code
SAMBLE: Shape-Specific Point Cloud Sampling for an Optimal Trade-Off Between Local Detail and Global Uniformity
C. Wu, Y. Wan*, H. Fu*, J. Pfrommer, Z. Zhong, J. Zheng†, J. Zhang, J. Beyerer
CVPR 2025 Project page Paper Code
GraphDoc: A Graph-based Document Structure Analysis
Y. Chen, R. Liu, J. Zheng, D. Wen, K. Peng, J. Zhang†, R. Stiefelhagen
ICLR 2025 Project page Paper Code
@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology
X. Jiang*, J. Zheng*, R. Liu, J. Li, J. Zhang†, S. Matthiesen, R. Stiefelhagen
WACV 2025 Project page Paper Code
OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mapping
J. Wei, J. Zheng, R. Liu, J. Hu, J. Zhang†, R. Stiefelhagen
ACCV 2024 ( Best paper finalist) Paper Code
Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation
J. Zhang, K. Yang, H. Shi, S. Reiß, K. Peng, C. Ma, H. Fu, P. Torr, K. Wang, R. Stiefelhagen.
IEEE T-PAMI 2024 Paper Code
CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity
H. Shi*, C. Peng*, J. Zhang*, K. Yang, Y. Wu, H. Ni, Y, Lin, R. Stiefelhagen, K. Wang.
IEEE T-IP 2024 Paper Code
Open Panoramic Segmentation
J. Zheng, R. Liu, Y. Chen, K. Peng, C. Wu, K. Yang, J. Zhang†, R. Stiefelhagen.
ECCV 2024 Project page Paper Code
Occlusion-Aware Seamless Segmentation
Y. Cao*, J. Zhang*, H. Shi, K. Peng, Y. Zhang, H. Zhang, R. Stiefelhagen, K. Yang.
ECCV 2024 Paper Code
Referring Atomic Video Action Recognition
K. Peng, J. Fu, K. Yang, D. Wen, Y. Chen, R. Liu, J. Zheng, J. Zhang, S. Sarfraz, R. Stiefelhagen, A. Roitberg.
ECCV 2024 Paper Code
RoDLA: Benchmarking the Robustness of Document Layout Analysis Models
Y. Chen, J. Zhang†, K. Peng, J. Zheng, R. Liu, P. Torr, R. Stiefelhagen.
CVPR 2024 Project page Paper Code
MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments
J. Zheng*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen.
ICRA 2024 ( Best paper finalist on HRI) Project page Paper Code
Delivering Arbitrary-Modal Semantic Segmentation
J. Zhang*, R. Liu*, S. Hao, K. Yang, S. Reiß, K. Peng, H. Fu, K. Wang, R. Stiefelhagen.
CVPR 2023 Project page Paper Code Dataset
Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation
J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, R. Stiefelhagen.
CVPR 2022 Paper Code
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
J. Zhang*, H. Liu*, K. Yang*, X. Hu, R. Liu, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2023 Paper Code
Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance
J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2022 Paper Code
Exploring Event-Driven Dynamic Context for Accident Scene Segmentation
J. Zhang, K. Yang, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2021 Paper Code Dataset
Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
J. Zhang, C. Ma, K. Yang, A. Roitberg, K. Peng, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2021 Paper Code Dataset
360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View
Z. Teng*, J. Zhang*†, K. Yang, K. Peng, H. Shi, S. Reiß, K. Cao, R. Stiefelhagen.
WACV 2024 Project page Paper Code Dataset
Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers
C. Chen, J. Zhang†, K. Yang, K. Peng, R. Stiefelhagen.
WACV 2023 Paper Code
MatchFormer: Interleaving Attention in Transformers for Feature Matching
Q. Wang*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen.
ACCV 2023 Paper Code
Capturing Omni-Range Context for Omnidirectional Segmentation
K. Yang, J. Zhang, S. Reiß, X. Hu, R. Stiefelhagen
CVPR 2021 Paper Code
ISSAFE: Improving semantic segmentation in accidents by fusing event-based data
J. Zhang, K. Yang, R. Stiefelhagen
IROS 2021 Paper Code Dataset
Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation
H. Tan, C. Chen, X. Luo, J. Zhang, C. Seibold, K. Yang, R. Stiefelhagen.
IEEE ROBIO 2021 Paper Code Video
HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor
H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen
ICCV Workshop on Assistive Computer Vision and Robotics ( ACVR) 2021 Paper
Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World
J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen
ICCV Workshop on Assistive Computer Vision and Robotics ( ACVR) 2021 Paper Code
DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
C. Ma, J. Zhang, K. Yang, A. Roitberg, R. Stiefelhagen
IEEE ITSC 2021 Paper Code Dataset
Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction
Z. Marinov, S. Vasileva, Q. Wang, C. Seibold, J. Zhang, R. Stiefelhagen
IEEE EUSIPCO 2021 Paper Code
Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired
Wei Mao*, J. Zhang*, K. Yang, R. Stiefelhagen
IEEE RCAR 2021 Paper Code
Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired
Y. Zhang, H. Chen, K. Yang, J. Zhang, R. Stiefelhagen
IEEE RCAR 2021 Paper

Teaching

Lectures

Supervision

  • Since 2023 Junwei Zheng. PhD student at KIT. Vision-Language Navigation. Co-supervised with Prof. Rainer Stiefelhagen.
  • Since 2023 Ruiping Liu. PhD student at KIT. Multimodal Scene Understanding. Co-supervised with Prof. Rainer Stiefelhagen.
  • Since 2023 Yufan Chen. PhD student at KIT. Document Analysis. Co-supervised with Prof. Rainer Stiefelhagen.
  • Since 2024 Fei Teng. PhD student at HNU. Panoramic Understanding. Co-supervised with Prof. Kailun Yang.
  • 2025 Mar. Sebastian Tewes. Master student (co-supervised). Source-free Document Layout Analysis. Paper Code
  • 2025 Jan. Alexander Vogel. Master student (co-supervised). RefChartQA: Grounding Reasoning on Chart Images through Instruction-tuning.
  • 2025 Jan. Jie Hu. Master student (co-supervised). Deformable Mamba for Wide Field of View Segmentation. Paper Code
  • 2024 Nov. Qihao Yuan. Master student. Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problems. Paper Code
  • 2024 Jul. Jiale Wei. Master student. OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mapping. Paper Code
  • 2024 Feb. Xin Jiang. Master student. Unified Vision-Language Models for Assistive Technology. Paper Code
  • 2024 Feb. Jonas Schmitt. Master student. Global Hessian-Based Importance Pruning of Neural Networks in Combination with Knowledge Distillation. Paper Code
  • 2023 Nov. Daniel Bucher. Master student (co-supervised). Improving Robustness of 3D Semantic Sementation with Transformer-based Fusion and Knowledge Distillation. Paper
  • 2023 Aug. Leon Kanstinger. Bachelor student. Improving Accessibility of User Interface in Mobility Assistance Systems.
  • 2023 Juli. Fei Teng. Master student (co-supervised). OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation. Paper Code
  • 2023 Apr. Zhifeng Teng. Master student. 360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View. Paper Code Page
  • 2022 Aug. Chang Chen. Master student. Transformer-based Mapping from Egocentric Images to Top-view Semantics for Scene Understanding. Paper Code Video
  • 2022 Feb. Qing Wang. Master student. MatchFormer: Interleaving Attention in Transformers for Feature Matching. Paper Code
  • 2021 Oct. Chaoxiang Ma. Master student (co-supervised). Unsupervised Domain Adaptation for Panoramic Semantic Segmentation. Paper Code

Invited Talks

  • 2025 Jan. Multimodal Scene Understanding for Mobility Assistance HKUST (Guangzhou)
  • 2024 Dec. Towards Holistic and Robust Visual Assistive Systems Jihua Lab
  • 2024 Dec. Multimodal Scene Understanding for Inclusive Mobility ETH Zurich
  • 2024 Dec. Towards Holistic and Robust Visual Assistive Systems SCUT
  • 2024 Dec. Intelligent Visual Assistance Southeast University
  • 2024 Nov. Multimodal Scene Understanding for Mobility Assistance HNU
  • 2024 Oct. Intelligent Assistance System Based on Scene Understanding GCI Germany
  • 2023 Oct. Vision4Blind : Assistance Systems for People with Visual Impairments ICCV Demo
  • 2023 Sep. Scene Understanding for Intelligent Transportation Systems University of Oxford
  • 2022 Jul. Scene Understanding for Mobility Assistance Helmholtz Workshop, KIT
  • 2021 Oct. Efficient Transformer for Transparent Object Segmentation ACVR
  • 2021 Sep. Improving Semantic Segmentation in Accidents by Fusing Event-based Data IROS

Awards

  • ITSS Germany Dissertation Award (The First Price), 2024.
  • KIT Doctoral Award, 2024.
  • ACCV 2024 Best Paper Finalist, 2024.
  • KIT KHYS Research Travel Grant, 2024.
  • ICM Future Mobility Grants, 2024.
  • ICRA 2024 HRI Best Paper Finalist, 2024.
  • IFI Program Fellowship of the German Academic Exchange Service (DAAD), 2023.
  • The Best Practical Course, Teaching Award, KIT, Computer Science Faculty, 2021.

    • Services

    • Associate Editor: IEEE RA-L, IEEE IV 2024, IEEE IV 2025
    • Journal Reviewer: T-PAMI, T-RO, T-IP, IJCV, CVIU, TNNLS, T-ITS, RA-L, T-IV, TCSVT, IJHCI, TGRS, T-ASE
    • Conference Reviewer: CVPR, ICCV, ECCV, ICML, ICLR, NeurIPS, AAAI, IJCAI, SIGGRAPH, ACMMM, ACCV, WACV, BMVC, ICRA, IROS, ITSC, IV
    • Workshop Co-organizer: IEEE IV 2022 Workshop on Beyond Supervised Learning.

    Contact