2026

OccStress: Stress-Testing the 4D Occupancy Forecasting Chain

Y. Zheng, J. Hu, J. Xiong, R. Liu, J. Zheng, K. Yang, J. Zhang†

NeurIPS 2026

Paper Project page Code

Image coming soon

VLAN: Vision-Language Accessible Navigation

J. Hu, Y. Zheng, W. Wu, W. Feng, C. Cheng, J. Zhang†

NeurIPS 2026

Conference page

Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation

Y. Chen, Y. Chen, Z. Cai, R. Liu, J. Zheng, J. Wei, D. Wen, W. Fan, K. Peng, J. Zhang, R. Stiefelhagen

NeurIPS 2026

Paper Code

VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval

S. Ott, Y. Chen, R. Liu, J. Zheng, J. Wei, D. Wen, K. Peng, J. Zhang, R. Stiefelhagen

NeurIPS 2026

Paper Project page

OneBEV++: Towards Unifying Bird’s-Eye-View Semantic Mapping with Panoramas

J. Wei, Z. Teng, F. Teng, J. Zheng, R. Liu, Y. Chen, J. Hu, K. Yang, J. Zhang, R. Stiefelhagen

IEEE T-PAMI 2026

Paper Code

X²Localizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization

Z. Zeng, W. Fan, Y. Chen, J. M. Goo, J. Zheng†, R. Liu, K. Peng, J. Zhang†, R. Stiefelhagen, J. Boehm

BMVC 2026 (🏆 Oral)

Project page Paper Code

Position: Assistive Agents Need Accessibility Alignment

J. Hu, C. Yan, Y. Zheng, Z. Wang, J. Zhang

ICML 2026 (🏆 Spotlight)

Paper

More than the Sum: Panorama-Language Models for Adverse Omni-Scenes

W. Fan, R. Liu, J. Wei, Y. Chen, J. Zheng†, Z. Zeng, J. Zhang†, Q. Li, L. Shen, R. Stiefelhagen

CVPR 2026

Paper Code

RHO: Robust Holistic OSM-Based Metric Cross-View Geo-Localization

J. Zheng*†, R. Dai*, R. Liu, Z. Zeng, Y. Chen, F. Wang, K. Peng, K. Yang, J. Zhang†, R. Stiefelhagen

CVPR 2026

Paper Code

HybriDLA: Hybrid Generation for Document Layout Analysis

Y. Chen, O. Moured, R. Liu, J. Zheng, K. Peng, J. Zhang†, R. Stiefelhagen

AAAI 2026 (🏆 Oral)

Paper Proceedings

EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos

R. Liu, J. Zheng, Y. Chen, D. Wen, S. Quan, C. Wu, J. Zhang, K. Yang, K. Peng, R. Stiefelhagen

arXiv preprint 2026

Paper

What if? Emulative Simulation with World Models for Situated Reasoning

R. Liu, Y. Chen, Y. Zhang, J. Zheng, K. Peng, C. Wu, C. Huang, D. Wen, J. Zhang, K. Yang, R. Stiefelhagen

ECCV 2026

Paper

ObjectFinder: An Open-Vocabulary Assistive System for Interactive Object Search by People Who Are Blind

R. Liu, J. Zhang, A. Schön, K. Müller, J. Zheng, K. Yang, A. Guo, K. Gerling, R. Stiefelhagen

International Journal of Human–Computer Interaction 2026

Paper arXiv

SeqLoc: Beyond the Single Frame for Cross-View Geo-Localization in Feature-Sparse Scenes

J. Zheng, Y. Huang, R. Dai, R. Liu, Y. Chen, K. Peng, K. Yang, J. Zhang, G. Wang, O. Wysocki, R. Stiefelhagen

arXiv preprint 2026

Paper

Faster or Stronger: Towards Flexible Visual Place Recognition via Weighted Aggregation and Token Pruning

Z. Zeng, J. M. Goo, J. Zheng, W. Fan, J. Zhang, R. Stiefelhagen, J. Boehm

arXiv preprint 2026

Paper

Not an Obstacle for Dog, but a Hazard for Human: A Co-Ego Navigation System for Guide Dog Robots

R. Liu, Jingqi Zhang, J. Zheng, Y. Chen, P. S. Lee, D. Wen, K. Peng, J. Zhang, K. Yang, K. Mombaur, R. Stiefelhagen

arXiv preprint 2026

Paper

DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding

M. Tao, R. Liu, J. Zheng, Y. Chen, K. Ying, M. S. Sarfraz, K. Yang, J. Zhang, R. Stiefelhagen

CVPR Workshops (DriveX) 2026

Paper

SGR3 Model: Scene Graph Retrieval-Reasoning Model in 3D

Z. Wang, R. Liu, Y. Chen, J. Zheng, W. Fan, K. Peng, D. Wen, J. Wei, J. Zhang, R. Stiefelhagen

arXiv preprint 2026

Paper

CHAOS: Chart Analysis with Outlier Samples

Y. Chen, O. Moured, R. Liu, S. Reiß, P. Torr, J. Zhang, R. Stiefelhagen

IEEE ICASSP 2026

Paper arXiv Code

PASI: Panoramic Adaptive Sliced Inference for Open-Vocabulary Instance Segmentation in Equirectangular Panoramas

T. Kirilov, J. Zheng, R. Liu, Y. Chen, D. Wen, J. Wei, K. Peng, K. Yang, J. Zhang, R. Stiefelhagen

ECCV Workshops (HSI) 2026

Paper Code

2025

mmWalk: Towards Multi-modal Multi-view Walking Assistance

K. Ying*, R. Liu*†, C. Chen, M. Tao, H. Shi, K. Yang, J. Zhang†, R. Stiefelhagen

NeurIPS 2025

Paper Code

Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model

R. Liu, J. Zheng, Y. Chen, Z. Wang, K. Peng, K. Yang, J. Zhang†, M. Pollefeys, R. Stiefelhagen

NeurIPS 2025

Paper Code

Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation

Y. Cao*, J. Zhang*, X. Zheng, H. Shi, K. Peng, H. Liu, K. Yang, H. Zhang

ICCV 2025

Paper Code

Scene-agnostic Pose Regression for Visual Localization

J. Zheng, R. Liu, Y. Chen, Z. Chen, K. Yang, J. Zhang†, R. Stiefelhagen

CVPR 2025

Project page Paper Code

SAMBLE: Shape-Specific Point Cloud Sampling for an Optimal Trade-Off Between Local Detail and Global Uniformity

C. Wu, Y. Wan*, H. Fu*, J. Pfrommer, Z. Zhong, J. Zheng†, J. Zhang, J. Beyerer

CVPR 2025

Project page Paper Code

Graph-based Document Structure Analysis

Y. Chen, R. Liu, J. Zheng, D. Wen, K. Peng, J. Zhang†, R. Stiefelhagen

ICLR 2025

Project page Paper Code

@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology

X. Jiang*, J. Zheng*, R. Liu, J. Li, J. Zhang†, S. Matthiesen, R. Stiefelhagen

WACV 2025

Project page Paper Code

Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problems

Q. Yuan, K. Li†, J. Zhang†

BMVC 2025

Paper arXiv Code

RefChartQA: Grounding Visual Answer on Chart Images Through Instruction Tuning

A. Vogel, O. Moured, Y. Chen, J. Zhang, R. Stiefelhagen

ICDAR 2025

Paper Code

Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Driving

H. Shi, S. Wang, J. Zhang, X. Yin, G. Wang, J. Zhu, K. Yang, K. Wang

IEEE T-ITS 2025

Paper arXiv

SFDLA: Source-Free Document Layout Analysis

S. Tewes*, Y. Chen*, O. Moured, J. Zhang†, R. Stiefelhagen

ICDAR 2025

Paper Code

Exploring Video-Based Driver Activity Recognition under Noisy Labels

L. Fan, D. Wen, K. Peng, K. Yang, J. Zhang, R. Liu, Y. Chen, J. Zheng, J. Wu, X. Han, R. Stiefelhagen

IEEE SMC 2025

Paper

Exploring Self-supervised Skeleton-based Action Recognition in Occluded Environments

Y. Chen, K. Peng, A. Roitberg, D. Schneider, J. Zhang, J. Zheng, Y. Chen, R. Liu, K. Yang, R. Stiefelhagen

IEEE IJCNN 2025

Paper

DishDetect: What’s on My Plate? – Co-Designing a Mobile App for Clockwise Food Description

K. Ying*, M. Tao*, R. Dai*, R. Liu, K. Müller, G. Jaworek, J. Zhang, R. Stiefelhagen

CHI Extended Abstracts 2025

Paper

2024

OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mapping

J. Wei, J. Zheng, R. Liu, J. Hu, J. Zhang†, R. Stiefelhagen

ACCV 2024 (🏆 Best Paper Finalist)

Paper Code

Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation

J. Zhang, K. Yang, H. Shi, S. Reiß, K. Peng, C. Ma, H. Fu, P. Torr, K. Wang, R. Stiefelhagen

IEEE T-PAMI 2024

Paper Code

CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity

H. Shi*, C. Peng*, J. Zhang*, K. Yang, Y. Wu, H. Ni, Y. Lin, R. Stiefelhagen, K. Wang

IEEE T-IP 2024

Paper Code

Open Panoramic Segmentation

J. Zheng, R. Liu, Y. Chen, K. Peng, C. Wu, K. Yang, J. Zhang†, R. Stiefelhagen

ECCV 2024

Project page Paper Code

Occlusion-Aware Seamless Segmentation

Y. Cao*, J. Zhang*, H. Shi, K. Peng, Y. Zhang, H. Zhang, R. Stiefelhagen, K. Yang

ECCV 2024

Paper Code

Referring Atomic Video Action Recognition

K. Peng, J. Fu, K. Yang, D. Wen, Y. Chen, R. Liu, J. Zheng, J. Zhang, M. S. Sarfraz, R. Stiefelhagen, A. Roitberg

ECCV 2024

Paper Code

RoDLA: Benchmarking the Robustness of Document Layout Analysis Models

Y. Chen, J. Zhang†, K. Peng, J. Zheng, R. Liu, P. Torr, R. Stiefelhagen

CVPR 2024

Project page Paper Code

MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments

J. Zheng*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen

IEEE ICRA 2024 (🏆 Best Paper Finalist on HRI)

Project page Paper Code

360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View

Z. Teng*, J. Zhang*†, K. Yang, K. Peng, H. Shi, S. Reiß, K. Cao, R. Stiefelhagen

WACV 2024

Project page Paper Code & Data

TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation

R. Liu, K. Yang, A. Roitberg, J. Zhang, K. Peng, H. Liu, Y. Wang, R. Stiefelhagen

IEEE T-ITS 2024

Paper

Navigating Open Set Scenarios for Skeleton-Based Action Recognition

K. Peng, C. Yin, J. Zheng, R. Liu, D. Schneider, J. Zhang, K. Yang, M. S. Sarfraz, R. Stiefelhagen, A. Roitberg

AAAI 2024

Paper

Skeleton-Based Human Action Recognition with Noisy Labels

Y. Xu, K. Peng, D. Wen, R. Liu, J. Zheng, Y. Chen, J. Zhang, A. Roitberg, K. Yang, R. Stiefelhagen

IEEE/RSJ IROS 2024

Paper

Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation

R. Liu, J. Zhang, K. Peng, Y. Chen, K. Cao, J. Zheng, M. S. Sarfraz, K. Yang, R. Stiefelhagen

IEEE IV 2024

Paper

OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation

F. Teng, J. Zhang, K. Peng, Y. Wang, R. Stiefelhagen, K. Yang

IEEE T-AI 2024

Paper Code

Elevating Skeleton-Based Action Recognition with Efficient Multi-Modality Self-Supervision

Y. Wei, K. Peng, A. Roitberg, J. Zhang, J. Zheng, R. Liu, Y. Chen, K. Yang, R. Stiefelhagen

IEEE ICASSP 2024

Paper

Deformable Mamba for Wide Field of View Segmentation

J. Hu, J. Zheng, J. Wei, J. Zhang†, R. Stiefelhagen

arXiv preprint 2024

Paper Code

LF Tracy: A Unified Single-Pipeline Paradigm for Salient Object Detection in Light Field Cameras

F. Teng, J. Zhang, J. Liu, K. Peng, X. Cheng, Z. Li, K. Yang

ICPR 2024

Paper

Computational Imaging for Machine Perception: Transferring Semantic Segmentation Beyond Aberrations

Q. Jiang, H. Shi, S. Gao, J. Zhang, K. Yang, L. Sun, H. Ni, K. Wang

IEEE T-CI 2024

Paper

Towards Video-based Activated Muscle Group Estimation in the Wild

K. Peng, D. Schneider, A. Roitberg, K. Yang, J. Zhang, C. Deng, K. Zhang, M. S. Sarfraz, R. Stiefelhagen

ACM MM 2024

Paper arXiv

Comb, Prune, Distill: Towards Unified Pruning for Vision Model Compression

J. Schmitt, R. Liu, J. Zheng, J. Zhang, R. Stiefelhagen

IEEE ITSC 2024

Paper

Rethinking Attention Module Design for Point Cloud Analysis

C. Wu, K. Wang, Z. Zhong, H. Fu, J. Zheng, J. Zhang, J. Pfrommer, J. Beyerer

ICPR 2024

Paper

2023

Delivering Arbitrary-Modal Semantic Segmentation

J. Zhang*, R. Liu*, H. Shi, K. Yang, S. Reiß, K. Peng, H. Fu, K. Wang, R. Stiefelhagen

CVPR 2023

Project page Paper Code & Data

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

J. Zhang*, H. Liu*, K. Yang*, X. Hu, R. Liu, R. Stiefelhagen

IEEE T-ITS 2023

Paper Code

Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers

C. Chen, J. Zhang†, K. Yang, K. Peng, R. Stiefelhagen

WACV 2023

Paper Code

Delving Deep into One-Shot Skeleton-Based Action Recognition with Diverse Occlusions

K. Peng, A. Roitberg, K. Yang, J. Zhang, R. Stiefelhagen

IEEE T-MM 2023

Paper

Open Scene Understanding: Grounded Situation Recognition Meets Segment Anything for Helping People with Visual Impairments

R. Liu, J. Zhang, K. Peng, J. Zheng, K. Cao, Y. Chen, K. Yang, R. Stiefelhagen

ICCV Workshops 2023

Paper

Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Driving

S. Li, K. Yang, H. Shi, J. Zhang, J. Lin, Z. Teng, Z. Li

IEEE RA-L 2023

Paper

FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation

H. Shi, Y. Li, K. Yang, J. Zhang, K. Peng, A. Roitberg, Y. Ye, H. Ni, K. Wang, R. Stiefelhagen

CVPR Workshops (OmniCV) 2023

Paper Code

Tightly-Coupled LiDAR-Visual SLAM Based on Geometric Features for Mobile Agents

K. Cao, R. Liu, Z. Wang, K. Peng, J. Zhang, J. Zheng, Z. Teng, K. Yang, R. Stiefelhagen

IEEE ROBIO 2023

Paper

Exploring Few-Shot Adaptation for Activity Recognition on Diverse Domains

K. Peng, D. Wen, D. Schneider, J. Zhang, K. Yang, M. S. Sarfraz, R. Stiefelhagen, A. Roitberg

arXiv preprint 2023

Paper

Line Graphics Digitization: A Step Towards Full Automation

O. Moured, J. Zhang, A. Roitberg, T. Schwarz, R. Stiefelhagen

ICDAR 2023

Paper

2022

MatchFormer: Interleaving Attention in Transformers for Feature Matching

Q. Wang*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen

ACCV 2022

Paper Code

Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation

J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, R. Stiefelhagen

CVPR 2022

Paper Code

Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance

J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen

IEEE T-ITS 2022

Paper Code

Multi-modal Depression Estimation Based on Sub-attentional Fusion

P.-C. Wei, K. Peng, A. Roitberg, K. Yang, J. Zhang, R. Stiefelhagen

ECCV Workshops (ACVR) 2022

Paper

MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding

K. Peng, J. Fei, K. Yang, A. Roitberg, J. Zhang, F. Bieder, P. Heidenreich, C. Stiller, R. Stiefelhagen

IEEE T-ITS 2022

Paper

TransDARC: Transformer-Based Driver Activity Recognition with Latent Space Feature Calibration

K. Peng, A. Roitberg, K. Yang, J. Zhang, R. Stiefelhagen

IEEE/RSJ IROS 2022

Paper

Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensor

W. Ou, J. Zhang, K. Peng, K. Yang, G. Jaworek, K. Müller, R. Stiefelhagen

ICCHP 2022

Paper

Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning

X. Luo, J. Zhang, K. Yang, A. Roitberg, K. Peng, R. Stiefelhagen

CVPR Workshops 2022

Paper

Should I Take a Walk? Estimating Energy Expenditure from Video Data

K. Peng, A. Roitberg, K. Yang, J. Zhang, R. Stiefelhagen

CVPR Workshops (CVPM) 2022

Paper

2021

Exploring Event-Driven Dynamic Context for Accident Scene Segmentation

J. Zhang, K. Yang, R. Stiefelhagen

IEEE T-ITS 2021

Paper Code Dataset

Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation

J. Zhang, C. Ma, K. Yang, A. Roitberg, K. Peng, R. Stiefelhagen

IEEE T-ITS 2021

Paper Code Dataset

Capturing Omni-Range Context for Omnidirectional Segmentation

K. Yang, J. Zhang, S. Reiß, X. Hu, R. Stiefelhagen

CVPR 2021

Paper Code

ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data

J. Zhang, K. Yang, R. Stiefelhagen

IEEE/RSJ IROS 2021

Paper Code Dataset

Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation

H. Tan, C. Chen, X. Luo, J. Zhang, C. Seibold, K. Yang, R. Stiefelhagen

IEEE ROBIO 2021

Paper Code Video

HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor

H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen

ICCV Workshops (ACVR) 2021

Paper

Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World

J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen

ICCV Workshops (ACVR) 2021

Paper Code

DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange

C. Ma, J. Zhang, K. Yang, A. Roitberg, R. Stiefelhagen

IEEE ITSC 2021

Paper Code Dataset

Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction

Z. Marinov, S. Vasileva, Q. Wang, C. Seibold, J. Zhang, R. Stiefelhagen

IEEE EUSIPCO 2021

Paper Code

Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired

W. Mao*, J. Zhang*, K. Yang, R. Stiefelhagen

IEEE RCAR 2021

Paper Code

Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired

Y. Zhang, H. Chen, K. Yang, J. Zhang, R. Stiefelhagen

IEEE RCAR 2021

Paper

2020

Can We Cover Navigational Perception Needs of the Visually Impaired by Panoptic Segmentation?

W. Mao, J. Zhang, K. Yang, R. Stiefelhagen

arXiv preprint 2020

Paper