Jiaming Zhang

Postdoctoral Researcher

KIT, Germany

I am a postdoctoral researcher at the CV:HCI Lab at Karlsruhe Institute of Technology (KIT), working with Prof. Rainer Stiefelhagen. I also work very closely with the ACCESS@KIT Lab and with Prof. Kailun Yang. I am lucky to have the opportunity to work with Prof. Philip Torr at Torr Vision Group (TVG) at the University of Oxford. Before that, I received the B.Sc. degree at Shenzhen University (SZU). I received the M.Sc. and Ph.D. degree at KIT.

My research interests lie at the intersection of computer vision and assistive technology, with a focus on building inclusive and user-friendly systems for all users, including those with visual impairments. The topics included are vision-language model, scene understanding, visual localization, and their applications in mobility assistive systems and intelligent vehicles.

[New] For students who are interested in a bachelor/master thesis, please feel free to send me an email.

Interest

Computer Vision
Assistive Technology
Human-Computer Interaction
Intelligent Systems
Vision-Language Models
2D/3D Scene Understanding
Autonomous Driving

Experience

PostDoc, 11/2023 - Present
CVHCI, KIT
Research Assistant, 11/2020 - 11/2023
CVHCI, KIT
Student Assistant, 05/2020 - 10/2020
ISPE, FZI
Student Assistant, 09/2019 - 03/2020
ISAS Lab, KIT

Education

Ph.D. in Computer Science, 2023
KIT, Germany
Visiting Ph.D. Student, 2023
University of Oxford, UK
M.Sc. in Computer Science, 2020
KIT, Germany
B.Sc. in Computer Science, 2015
Shenzhen University, China

Selected Publications

(* equal contribution, † corresponding author.)

RoDLA: Benchmarking the Robustness of Document Layout Analysis Models
Y. Chen, J. Zhang†, K. Peng, J. Zheng, R. Liu, P. Torr, R. Stiefelhagen.
CVPR 2024 Project page Paper Code

MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments
J. Zheng*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen.
ICRA 2024 ( Best paper finalist on HRI) Project page Paper Code

Delivering Arbitrary-Modal Semantic Segmentation
J. Zhang*, R. Liu*, S. Hao, K. Yang, S. Reiß, K. Peng, H. Fu, K. Wang, R. Stiefelhagen.
CVPR 2023 Project page Paper Code Dataset

Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation
J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, R. Stiefelhagen.
CVPR 2022 Paper Code

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
J. Zhang*, H. Liu*, K. Yang*, X. Hu, R. Liu, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2023 Paper Code

Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance
J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2022 Paper Code

Exploring Event-Driven Dynamic Context for Accident Scene Segmentation
J. Zhang, K. Yang, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2021 Paper Code Dataset

Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
J. Zhang, C. Ma, K. Yang, A. Roitberg, K. Peng, R. Stiefelhagen.
IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2021 Paper Code Dataset

360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View
Z. Teng*, J. Zhang*†, K. Yang, K. Peng, H. Shi, S. Reiß, K. Cao, R. Stiefelhagen.
WACV 2024 Project page Paper Code Dataset

Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers
C. Chen, J. Zhang†, K. Yang, K. Peng, R. Stiefelhagen.
WACV 2023 Paper Code

MatchFormer: Interleaving Attention in Transformers for Feature Matching
Q. Wang*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen.
ACCV 2023 Paper Code

Capturing Omni-Range Context for Omnidirectional Segmentation
K. Yang, J. Zhang, S. Reiß, X. Hu, R. Stiefelhagen
CVPR 2021 Paper Code

ISSAFE: Improving semantic segmentation in accidents by fusing event-based data
J. Zhang, K. Yang, R. Stiefelhagen
IROS 2021 Paper Code Dataset

Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation
H. Tan, C. Chen, X. Luo, J. Zhang, C. Seibold, K. Yang, R. Stiefelhagen.
IEEE ROBIO 2021 Paper Code Video

HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor
H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen
ICCV Workshop on Assistive Computer Vision and Robotics ( ACVR) 2021 Paper

Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World
J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen
ICCV Workshop on Assistive Computer Vision and Robotics ( ACVR) 2021 Paper Code

DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
C. Ma, J. Zhang, K. Yang, A. Roitberg, R. Stiefelhagen
IEEE ITSC 2021 Paper Code Dataset

Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction
Z. Marinov, S. Vasileva, Q. Wang, C. Seibold, J. Zhang, R. Stiefelhagen
IEEE EUSIPCO 2021 Paper Code

Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired
Wei Mao*, J. Zhang*, K. Yang, R. Stiefelhagen
IEEE RCAR 2021 Paper Code

Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired
Y. Zhang, H. Chen, K. Yang, J. Zhang, R. Stiefelhagen
IEEE RCAR 2021 Paper

Teaching

Lectures

Teaching Assistant, Deep Learning for Computer Vision II: Advanced Topics, WS 21/22, WS 22/23, WS 23/24

Teaching Assistant, Practical Course Computer Vision for Human-Computer Interaction, WS 20/21, SS 2021, SS 2022

Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation, Paper Code Video
Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction, Paper Code

Teaching Assistant, Seminar Computer Vision for Human-Computer Interaction, WS 20/21, WS 21/22, WS 23/24

Supervised Theses

Junwei Zheng. PhD student at KIT. Vision-Language Navigation. Co-supervised with Prof. Rainer Stiefelhagen.
Ruiping Liu. PhD student at KIT. Seamless Efficiency for Visual Perception. Co-supervised with Prof. Rainer Stiefelhagen.
Yufan Chen. PhD student at KIT. Document Analysis. Co-supervised with Prof. Rainer Stiefelhagen.
Fei Teng. PhD student at HNU. Scene Understanding. Co-supervised with Prof. Kailun Yang.
Xin Jiang. Master student. Unified Vision-Language Models for Assistive Technology. 2024. Paper Code
Jonas Schmitt. Master student. Global Hessian-Based Importance Pruning of Neural Networks in Combination with Knowledge Distillation. 2024. Paper Code
Daniel Bucher. Master student (co-supervised). Improving Robustness of 3D Semantic Sementation with Transformer-based Fusion and Knowledge Distillation. 2023. Paper
Leon Kanstinger. Bachelor student. Improving Accessibility of User Interface in Mobility Assistance Systems. 2023.
Fei Teng. Master student (co-supervised). OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation. 2023. Paper Code
Zhifeng Teng. Master student. 360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View. 2023. Paper Code Page
Chang Chen. Master student. Transformer-based Mapping from Egocentric Images to Top-view Semantics for Scene Understanding. 2022. Paper Code Video
Qing Wang. Master student. MatchFormer: Interleaving Attention in Transformers for Feature Matching. 2022. Paper Code
Chaoxiang Ma. Master student (co-supervised). Unsupervised Domain Adaptation for Panoramic Semantic Segmentation. 2021. Paper Code

Awards

IFI Program Fellowship of the German Academic Exchange Service (DAAD), 2023.

The Best Practical Course, Teaching Award, KIT, Computer Science Faculty, 2021.

Services

Associate Editor: IEEE RA-L, IEEE IV 2024
Journal Reviewer: T-RO, IJCV, TNNLS, T-ITS, RA-L, T-IV, TCSVT, IJHCI, TGRS
Conference Reviewer: CVPR, ICCV, ECCV, ICLR, NeurIPS, AAAI, ACMMM, ACCV, WACV, BMVC, ICRA, IROS
Workshop Co-organizer: IEEE IV 2022 Workshop on Beyond Supervised Learning.

Contact

jiaming.zhang@kit.edu
Adenauerring 10,
Building 50.28,
76131 Karlsruhe, Germany
Visitor traffic