Yufei (Judy) Ye 叶雨菲

I am a PhD/MS student in Robotics Institute at Carnegie Mellon University, advised by Abhinav Gupta and Shubham Tulsiani. I spent a wonderful year at FAIR as a visiting researcher. Prior to CMU, I received my Bachelor degree majoring in Computer Science from Tsinghua Universtiy where I worked with Shi-Min Hu.
My research mainly focuses on 3D computer vision, especially perceiving everyday's interactions. I recently received NVIDIA Graduate Fellowship (2022-23).
I will spend the upcoming summer in Learning & Perception Research team at NVIDIA, working with Sifei Liu and Xueting Li. I am really excited about it!

Email  /  CV  /  Google Scholar  /  Github  

profile photo
I currently work in 3D vision. In a very broad sense, my research interests lie at the intersection of computer vision, robotics and machine learning. My long-term goal is to enable artificial intelligent agents to better understand the interaction of the world - how to perceive the interaction, how to actively interact with the world, how to learn efficiently with consideration to the redundant structure.

Recently, I am curious about correspondence problem in the context of robot manipulation at different abstraction levels, from pixel-wise correspondence of the same instance in classic multi-view geometry to functional correspondence of different objects at the level of parts or implicit object states. I believe that intrinsic 3D understanding would play an important role because 1) 3D could serve as the underlying representation invariant to viewpoints and appearance in order to establish robust correspondence; 2) it could help robots to understand better about spatial 3D relations for complicated manipulation.
What's in your hands? 3D Reconstruction of Generic Objects in Hands [New]
Yufei Ye, Abhinav Gupta, Shubham Tulsiani
CVPR, 2022
project / video / arxiv / code / colab demo / slides / bibtex

Shelf-Supervised Mesh Prediction in the Wild
Yufei Ye, Shubham Tulsiani, Abhinav Gupta
CVPR, 2021
project / video / arxiv / code / poster / bibtex

Object-centric Forward Modeling for Model Predictive Control
Yufei Ye, Dhiraj Gandhi, Abhinav Gupta, Shubham Tulsiani
CoRL, 2019  
project / arxiv / poster / bibtex

Compositional Video Prediction
Yufei Ye, Maneesh Singh, Abhinav Gupta*, Shubham Tulsiani*
ICCV, 2019   (* indicates equal contribution)
project / arxiv / code / poster / bibtex

Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs
Xiaolong Wang* Yufei Ye*, Abhinav Gupta
CVPR, 2018   (* indicates equal contribution)
arXiv / code / poster / bibtex

Practical automatic background substitution for live video
Haozhi Huang, Xiaonan Fang, Yufei Ye, Songhai Zhang, and Paul L. Rosin.
Computational Visual Media (CVM), 2017  
paper / bibtex

Locating Abnormality in OCT Images with Image-level Annotation
Yufei Ye, Yankui Sun
Graduate Thesis, 2017  

Reading: Differential Geometry
Erwin Kreyszig
notes / summary, 2020

Reading: Multiple View Geometry in Computer Vision
Richard Hartley, Andrew Zisserman
notes / exercise

'TP-Link': a data explorer of the Hacker News data
Yufei Ye*, Dongyu Jiang*, Lin Luo*, Xi Luo*, and Haoyue Shi*.
Google Girl Hackathon, 2017   (User Experience Award)

Somatic Game with RGB Hand Detector from scratch.
Yufei Ye*, Binghong Chen*, and Jianxin Ma*.
Digital Logic Design, 2016   (Best Course Project)

Media Player on Cubieboard: Transparenting Android Bluetooth Protocols Stack for Car
Yufei Ye*, Binghong Chen*, and Jianxin Ma*.
Intro to Software Engineering, 2016   (Best Course Project)

  • 16-726 Learning-Based Image Synthesis, Spring 2021.
  • 16-824 Visual Learning and Recognition, Spring 2020.
  • Education and Experience
    I have spent enjoyable time in multiple great research groups.
  • Visual robot Learning Lab, CMU.  August 2017 - present, Abhinav Gupta.
  • Visualization & Image Computing Group, Tsinghua.  November 2016 - June 2017, Yankui Sun.
  • Yitu Technology, Shanghai.  December 2015 - February 2016, Yuanhao Chen.
  • Graphics & Geometric Computing Group, Tsinghua.   2015 - 2016, Shi-Min Hu.
  • Selcted Awards

    I am very fortunate to receive recognition along the way:

  • NVIDIA Graduate Fellowship (2022-2023);
  • Excellent Undergraduate Student by Tsinghua University (2017);
  • Outstanding Undergraduate Student by Tsinghua Computer Science Dept (2017);
  • TP-Link Scholarship (2017);
  • Singapore Technologies Engineering China Scholarship (2015, 2016, and 2017);
  • National Scholarship, by the Ministry of Education of China (2014);
  • Sports Outstanding Award, by Tsinghua Computer Science Dept (2014, 2016, and 2017);
  • Student service Excellence Award, by Tsinghua University (2016);

  • This guy makes a nice webpage.