Jiaming Zhang

Professor

Hunan University (HNU)

I am a Professor at the School of AI and Robotics at Hunan University (HNU), where I lead the Integrated Sensing and Assistive Intelligence (InSAI) Lab. Our research focuses on developing inclusive and intelligent systems at the intersection of computer vision and assistive technology, with the goal of creating solutions that assist people, especially those with visual impairments.

I collaborate with Prof. Rainer Stiefelhagen at CV:HCI Lab and ACCESS@KIT Lab at Karlsruhe Institute of Technology (KIT). I am happy to work with Prof. Philip Torr at TVG at University of Oxford, and with Prof. Marc Pollefeys at CVG Group at ETH Zurich. Previously, I was a Postdoc at KIT. I received my Ph.D. (summa cum laude) and M.Sc. from KIT and my B.Sc. from Shenzhen University.

📣[News] We have several positions (start from 2026/2027) for Postdoc, PhD, Master Student, RA/TA, Visiting Scholar, and Internship, please email me and join us!

Interest

Robotics
Computer Vision
Assistive Technology
Human-Computer Interaction
2D/3D Scene Understanding
Document Understanding
Embodied Intelligence
Autonomous Driving

Experience

Professor, 2025 - Present
School of AI and Robotics, HNU
Academic Guest, 2024 - 2025
CVG, ETH Zurich
Postdoc, 2023 - 2025
CV:HCI, KIT
Research Assistant, 2020 - 2023
CV:HCI, KIT

Education

Ph.D. in Computer Science, 2023
KIT, Germany
Visiting Ph.D. Student, 2023
University of Oxford, UK
M.Sc. in Computer Science, 2020
KIT, Germany
B.Sc. in Computer Science, 2015
Shenzhen University, China

Selected Publications

(* equal contribution, † corresponding author.)

OneBEV++: Towards Unifying Bird’s-Eye-View Semantic Mapping with Panoramas
J. Wei, Z. Teng, F. Teng, J. Zheng, R. Liu, Y. Chen, J. Hu, K. Yang, J. Zhang, and R. Stiefelhagen
IEEE T-PAMI 2026 Paper Code

X²Localizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization
Z. Zeng, W. Fan, Y. Chen, JM Goo, J. Zheng†, R. Liu, K. Peng, J. Zhang†, R. Stiefelhagen, J. Boehm
BMVC 2026 (🏆 Oral) Project page Paper Code

Position: Assistive Agents Need Accessibility Alignment
J. Hu, C. Yan, Y. Zheng, Z. Wang, J. Zhang
ICML 2026 (🏆 Spotlight) Paper

More than the Sum: Panorama-Language Models for Adverse Omni-Scenes
W. Fan, R. Liu, J. Wei, Y. Chen, J. Zheng†, Z. Zeng, J. Zhang†, Q. Li, L. Shen, R. Stiefelhagen
CVPR 2026 Project page Paper Code

RHO: Robust Holistic OSM-Based Metric Cross-View Geo-Localization
J. Zheng*†, R. Dai*, R. Liu, Z. Zeng, Y. Chen, F. Wang, K. Peng, K. Yang, J. Zhang†, R. Stiefelhagen
CVPR 2026 Project page Paper Code

HybriDLA: Hybrid Generation for Document Layout Analysis
Y. Chen, O. Moured, R. Liu, J. Zheng, K. Peng, J. Zhang†, R. Stiefelhagen
AAAI 2026 (🏆 Oral) Project page Paper Code

mmWalk: Towards Multi-modal Multi-view Walking Assistance
K. Ying*, R. Liu*†, C. Chen, M. Tao, H. Shi, K. Yang, J. Zhang†, R. Stiefelhagen
NeurIPS 2025 Paper Code

Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model
R. Liu, J. Zheng, Y. Chen, Z. Wang, K. Peng, K. Yang, J. Zhang†, M. Pollefeys, R. Stiefelhagen
NeurIPS 2025 Paper Code

Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation
Y. Cao*, J. Zhang*, X. Zheng, H. Shi, K. Peng, H. Liu, K. Yang, H. Zhang
ICCV 2025 Paper Code

Scene-agnostic Pose Regression for Visual Localization
J. Zheng, R. Liu, Y. Chen, Z. Chen, K. Yang, J. Zhang†, R. Stiefelhagen
CVPR 2025 Project page Paper Code

SAMBLE: Shape-Specific Point Cloud Sampling for an Optimal Trade-Off Between Local Detail and Global Uniformity
C. Wu, Y. Wan*, H. Fu*, J. Pfrommer, Z. Zhong, J. Zheng†, J. Zhang, J. Beyerer
CVPR 2025 Project page Paper Code

GraphDoc: A Graph-based Document Structure Analysis
Y. Chen, R. Liu, J. Zheng, D. Wen, K. Peng, J. Zhang†, R. Stiefelhagen
ICLR 2025 Project page Paper Code

@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology
X. Jiang*, J. Zheng*, R. Liu, J. Li, J. Zhang†, S. Matthiesen, R. Stiefelhagen
WACV 2025 Project page Paper Code

OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mapping
J. Wei, J. Zheng, R. Liu, J. Hu, J. Zhang†, R. Stiefelhagen
ACCV 2024 (🏆 Best paper finalist) Paper Code

Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation
J. Zhang, K. Yang, H. Shi, S. Reiß, K. Peng, C. Ma, H. Fu, P. Torr, K. Wang, R. Stiefelhagen.
IEEE T-PAMI 2024 Paper Code

CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity
H. Shi*, C. Peng*, J. Zhang*, K. Yang, Y. Wu, H. Ni, Y, Lin, R. Stiefelhagen, K. Wang.
IEEE T-IP 2024 Paper Code

Open Panoramic Segmentation
J. Zheng, R. Liu, Y. Chen, K. Peng, C. Wu, K. Yang, J. Zhang†, R. Stiefelhagen.
ECCV 2024 Project page Paper Code

Occlusion-Aware Seamless Segmentation
Y. Cao*, J. Zhang*, H. Shi, K. Peng, Y. Zhang, H. Zhang, R. Stiefelhagen, K. Yang.
ECCV 2024 Paper Code

Referring Atomic Video Action Recognition
K. Peng, J. Fu, K. Yang, D. Wen, Y. Chen, R. Liu, J. Zheng, J. Zhang, S. Sarfraz, R. Stiefelhagen, A. Roitberg.
ECCV 2024 Paper Code

RoDLA: Benchmarking the Robustness of Document Layout Analysis Models
Y. Chen, J. Zhang†, K. Peng, J. Zheng, R. Liu, P. Torr, R. Stiefelhagen.
CVPR 2024 Project page Paper Code

MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments
J. Zheng*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen.
ICRA 2024 (🏆 Best paper finalist on HRI) Project page Paper Code

360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View
Z. Teng*, J. Zhang*†, K. Yang, K. Peng, H. Shi, S. Reiß, K. Cao, R. Stiefelhagen.
WACV 2024 Project page Paper Code Dataset

Delivering Arbitrary-Modal Semantic Segmentation
J. Zhang*, R. Liu*, S. Hao, K. Yang, S. Reiß, K. Peng, H. Fu, K. Wang, R. Stiefelhagen.
CVPR 2023 Project page Paper Code Dataset

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
J. Zhang*, H. Liu*, K. Yang*, X. Hu, R. Liu, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2023 Paper Code

Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers
C. Chen, J. Zhang†, K. Yang, K. Peng, R. Stiefelhagen.
WACV 2023 Paper Code

MatchFormer: Interleaving Attention in Transformers for Feature Matching
Q. Wang*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen.
ACCV 2023 Paper Code

Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation
J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, R. Stiefelhagen.
CVPR 2022 Paper Code

Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance
J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2022 Paper Code

Exploring Event-Driven Dynamic Context for Accident Scene Segmentation
J. Zhang, K. Yang, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2021 Paper Code Dataset

Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
J. Zhang, C. Ma, K. Yang, A. Roitberg, K. Peng, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2021 Paper Code Dataset

Capturing Omni-Range Context for Omnidirectional Segmentation
K. Yang, J. Zhang, S. Reiß, X. Hu, R. Stiefelhagen
CVPR 2021 Paper Code

ISSAFE: Improving semantic segmentation in accidents by fusing event-based data
J. Zhang, K. Yang, R. Stiefelhagen
IROS 2021 Paper Code Dataset

Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation
H. Tan, C. Chen, X. Luo, J. Zhang, C. Seibold, K. Yang, R. Stiefelhagen.
IEEE ROBIO 2021 Paper Code Video

HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor
H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen
ICCV Workshop on Assistive Computer Vision and Robotics ( ACVR) 2021 Paper

Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World
J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen
ICCV Workshop on Assistive Computer Vision and Robotics ( ACVR) 2021 Paper Code

DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
C. Ma, J. Zhang, K. Yang, A. Roitberg, R. Stiefelhagen
IEEE ITSC 2021 Paper Code Dataset

Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction
Z. Marinov, S. Vasileva, Q. Wang, C. Seibold, J. Zhang, R. Stiefelhagen
IEEE EUSIPCO 2021 Paper Code

Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired
Wei Mao*, J. Zhang*, K. Yang, R. Stiefelhagen
IEEE RCAR 2021 Paper Code

Teaching

Multimodal Perception and Natural Interaction, Since 2025 Fall

Introduction to Computational Imaging, Since 2025 Fall

Introduction to Computational Imaging Technologies, Since 2025 Fall

Deep Learning for Computer Vision I: Basics, SS 2024, SS 2025

Deep Learning for Computer Vision II: Advanced Topics, WS 21/22, WS 22/23, WS 23/24

Practical Course: Computer Vision for HCI, WS 20/21, SS 2021, SS 2022, SS 2023, SS 2024, SS 2025

Flying Guide Dog, Paper Code Video
Pose2Drone, Paper Code

Seminar: Computer Vision for HCI, WS 20/21, WS 21/22, WS 23/24, WS 24/25

Seminar: Multimodal Large Language Models, SS 2024

Team

PhD Students

Since 2023 Junwei Zheng. PhD student at KIT. Co-supervised with Prof. Rainer Stiefelhagen.
Since 2023 Ruiping Liu. PhD student at KIT. Co-supervised with Prof. Rainer Stiefelhagen.
Since 2023 Yufan Chen. PhD student at KIT. Co-supervised with Prof. Rainer Stiefelhagen.
Since 2024 Fei Teng. PhD student at HNU. Co-supervised with Prof. Kailun Yang.
Since 2025 Shicheng Li. PhD student at HNU.
Since 2026 Jie Hu. PhD student at HNU.
Since 2026 Wenzhi Wu. PhD student at HNU.
Since 2026 Han Xu. PhD student at HNU. Co-supervised with Prof. Zhiyong Li.
Since 2026 Conghao Huang. PhD student at HNU. Co-supervised with Prof. Zhiyong Li.

Master Students

2026 Mar. Zirui Wang. 3D Scene Graph Generation. Publication Code
2025 Dec. Jialiang Zhang. Document Anomaly Detection. Publication Code
2025 Nov. Ruize Dai. Robust OSM-Based Cross-View Geo-Localization. Publication Code
2025 Nov. Wenzhi Wu. Indoor Navigation. Publication Code
2025 Sep. Mingzhe Tao. MLLMs for Autonomous Driving Scene Understanding and QA. Publication Code
2025 Aug. Kedi Ying. mmWalk: Towards Multi-modal Multi-view Walking Assistance. Publication Code
2025 Mar. Sebastian Tewes. Source-free Document Layout Analysis. Publication Code
2025 Jan. Alexander Vogel. RefChartQA: Grounding Reasoning on Chart Images through Instruction-tuning.
2025 Jan. Jie Hu. Deformable Mamba for Wide Field of View Segmentation. Publication Code
2024 Nov. Qihao Yuan. Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problems. Publication Code
2024 Jul. Jiale Wei. OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mapping. Publication Code
2024 Feb. Xin Jiang. Unified Vision-Language Models for Assistive Technology. Publication Code
2024 Feb. Jonas Schmitt. Global Hessian-Based Importance Pruning of Neural Networks in Combination with Knowledge Distillation. Publication Code
2023 Nov. Daniel Bucher. Improving Robustness of 3D Semantic Sementation with Transformer-based Fusion and Knowledge Distillation. Publication
2023 Juli. Fei Teng. OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation. Publication Code
2023 Apr. Zhifeng Teng. 360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View. Publication Code Page
2022 Aug. Chang Chen. Transformer-based Mapping from Egocentric Images to Top-view Semantics for Scene Understanding. Publication Code Video
2022 Feb. Qing Wang. MatchFormer: Interleaving Attention in Transformers for Feature Matching. Publication Code
2021 Oct. Chaoxiang Ma. Unsupervised Domain Adaptation for Panoramic Semantic Segmentation. Publication Code

Bachelor Students

2025 Aug. Cong Liu. OpenVision4Blind. Publication Code
2023 Aug. Leon Kanstinger. Improving Accessibility of User Interface in Mobility Assistance Systems.