Yongming Rao

I am currently a principal researcher at Tencent Hunyuan, working on large multimodal models and physical AI foundation models. I obtained my Ph.D degree from Tsinghua University in 2023, advised by Prof. Jiwen Lu . Before that, I received B.Eng. degree from the Department of Electronic Engineering, Tsinghua University in 2018.

We are hiring interns. Please feel free to drop me an email if you are interested in working with us.

Email  /  Google Scholar  /  Github

Publications

* indicates equal contribution, # indicates project lead / corresponding author

HY-Embodied-0.5 is the first version of our embodied foundation models for real-world agents, attaining best performance on 16 out of 22 widely used benchmarks of visual perception, spatial intelligence and embodied reasoning.

Ola is our exploration for omni-model language models supporting vision, audio and text inputs. We develop a progressive alignment method to effectively learn multimodal knowledge and help the model to surpass existing open omni-modal LLMs of similar sizes across all modalities.

Insight-V is a early solution for long-chain reasoning of multimodal large language models. We developed a method to create multimodal long CoT data from basic instruct models, showing performance gains from deeper reasoning.

Oryx is a unified multimodal architecture for the spatial-temporal understanding of images, videos, and multi-view 3D scenes, offering an on-demand solution to seamlessly and efficiently process visual inputs with arbitrary spatial sizes and temporal lengths.

VPD (Visual Perception with Pre-trained Diffusion Models) is a framework that leverages the high-level and low-level knowledge of a pre-trained text-to-image diffusion model to downstream visual perception tasks.

HorNet is a family of generic vision backbones that perform explicit high-order spatial interactions based on Recursive Gated Convolution.

DenseCLIP is a new framework for dense prediction by implicitly and explicitly leveraging the pre-trained knowledge from CLIP.

Point-BERT is a new paradigm for learning Transformers in an unsupervised manner by generalizing the concept of BERT onto 3D point cloud data.

Global Filter Networks is a transformer-style architecture that learns long-term spatial dependencies in the frequency domain with log-linear complexity.

We present a dynamic token sparsification framework to prune redundant tokens in vision transformers progressively and dynamically based on the input.

PoinTr is a transformer-based framework that reformulates point cloud completion as a set-to-set translation problem.

We present an unsupervised point cloud representation learning method based on global-local bidirectional reasoning, which largely advances the state-of-the-art of unsupervised point cloud understanding and outperforms recent supervised methods.

We propose a generic Runtime Network Routing (RNR) framework for efficient image classification, which selects an optimal path inside the network. Our method can be applied to off-the-shelf neural network structures and easily extended to various application scenarios.

Website Template