Yufei (Judy) Ye Home Page

Yufei (Judy) Ye 叶雨菲

I am a final-year PhD/MS student at Carnegie Mellon University. I am fortunate to be advised by Shubham Tulsiani and Abhinav Gupta. In general, I am intersted in understanding the physical world from sensors (e.g. images/videos, wearable devices), especially how human interact with their surroudings in everyday life -- how to accurately perceive the interaction, how to discover patterns of interaction, how to actively interact with the world. My reserach is partially supported by NVIDIA Graduate Fellowship.

Prior to CMU, I recieved my B.S. in Computer Science from Tsinghua Universtiy where I worked with Shi-Min Hu, Yankui Sun. I have also spent a great summer at NVIDIA LPR with Sifei Liu and Xueting Li, and FAIR with Kris Kitani.

I will join Stanford as a PostDoc in fall, advised by Karen Liu. Looking forward to it!

Email / CV / Google Scholar / Github / Twitter

News

June 2024: Co-organize a workshop MANGO at CVPR2024. Come and check out our amazing program!

June 2024: I gave a talk at Visual Computing and Artificial Intelligence, MPI.

April 2024: Started visiting at MPI-PS, Tuebingen with Michael Black.

Dec 2023: I gave a talk at GRASP Seminar, UPenn.

Nov 2023: Selected into EECS Rising Stars workshop.

June 2023: Presented at 3 workshops (4DHOI, 3D Scene Understanding, and RHOBIN) in CVPR2023.

News History

Oct 2023: I gave a talk at Scene from Video workshop, MIT.

May 2023: I gave a talk at VGG, Oxford.

April 2023: Proposed my thesis! PhD student life counting down.

April 2023: I gave a talk at Scene Representation Group .

Sept 2022: Received NVIDIA Graduate Fellowship (2022-2023).

June 2022: Presented at workshop EGO4D-EPIC in CVPR2022.

June 2022: I gave a talk at IVG@SZ, Tsinghua.

Nov 2019: I gave a talk at Energy Efficient Computing Group , Tsinghua.

Research

<	G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis Yufei Ye, Abhinav Gupta, Kris Kitani, Shubham Tulsiani CVPR 2024 Invited to CVPR workshop RHOBIN project / video / arxiv / slides / bibtex
<	Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clips [New] Yufei Ye, Poorvi Hebbar, Abhinav Gupta, Shubham Tulsiani ICCV (Oral) , 2023 project / arxiv / bibtex
	Affordance Diffusion: Synthesizing Hand-Object Interactions Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, Sifei Liu CVPR, 2023 Invited to 3 CVPR workshops (4DHOI, 3D Scene Understanding, and RHOBIN) Appearance in NVIDIA GTC project / video / arxiv / code / slides / bibtex
	What's in your hands? 3D Reconstruction of Generic Objects in Hands Yufei Ye, Abhinav Gupta, Shubham Tulsiani CVPR, 2022 Invited to CVPR workshop EGO4D-EPIC project / video / arxiv / code / colab demo / slides / bibtex
	Shelf-Supervised Mesh Prediction in the Wild Yufei Ye, Shubham Tulsiani, Abhinav Gupta CVPR, 2021 project / video / arxiv / code / poster / bibtex
	Object-centric Forward Modeling for Model Predictive Control Yufei Ye, Dhiraj Gandhi, Abhinav Gupta, Shubham Tulsiani CoRL, 2019 project / arxiv / poster / bibtex
	Compositional Video Prediction Yufei Ye, Maneesh Singh, Abhinav Gupta, Shubham Tulsiani ICCV, 2019 (* indicates equal contribution) project / arxiv / code / poster / bibtex
	Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs Xiaolong Wang* Yufei Ye, Abhinav Gupta CVPR, 2018 ( indicates equal contribution) arXiv / code / poster / bibtex
	Practical automatic background substitution for live video Haozhi Huang, Xiaonan Fang, Yufei Ye, Songhai Zhang, and Paul L. Rosin. Computational Visual Media (CVM), 2017 paper / bibtex
	Locating Abnormality in OCT Images with Image-level Annotation Yufei Ye, Yankui Sun Graduate Thesis, 2017

Others

	Reading: Differential Geometry Erwin Kreyszig notes / summary, 2020
	Reading: Multiple View Geometry in Computer Vision Richard Hartley, Andrew Zisserman notes / exercise
	'TP-Link': a data explorer of the Hacker News data Yufei Ye, Dongyu Jiang, Lin Luo, Xi Luo, and Haoyue Shi. Google Girl Hackathon, 2017 (User Experience Award)*
	Somatic Game with RGB Hand Detector from scratch. Yufei Ye, Binghong Chen, and Jianxin Ma. Digital Logic Design, 2016 (Best Course Project)*
	Media Player on Cubieboard: Transparenting Android Bluetooth Protocols Stack for Car Yufei Ye, Binghong Chen, and Jianxin Ma. Intro to Software Engineering, 2016 (Best Course Project)*

Teaching

A guest lecture in 16-895 Understanding and Critiquing Generative Computer Vision, Fall 2023.

16-726 Learning-Based Image Synthesis, Spring 2021.

16-824 Visual Learning and Recognition, Spring 2020.

Education and Experience

I had wonderful time in several great research groups.

2023.5-2023.8: FAIR, w/ Kris Kitani

2022.5-2022.8: NVIIDA LPR, w/ Sifei Liu

2021.4-2022.4: Meta AI Mentorship (previously known as FRAIM), w/ Shubham Tulsiani

2016.11-2017.6: Visualization & Image Computing Group, w/ Yankui Sun.

2015.12 - 2016.2: Yitu Technology, w/ Yuanhao Chen.

2015 - 2016.11: Graphics & Geometric Computing Group, w/ Shi-Min Hu.

Selected Awards

I am very fortunate to receive recognition along the way:

EECS Rising Stars (2023);

NVIDIA Graduate Fellowship (2022-2023);

Excellent Undergraduate Student by Tsinghua University (2017);

Outstanding Undergraduate Student by Tsinghua Computer Science Dept (2017);

TP-Link Scholarship (2017);

Singapore Technologies Engineering China Scholarship (2015, 2016, and 2017);

National Scholarship, by the Ministry of Education of China (2014);

Sports Outstanding Award, by Tsinghua Computer Science Dept (2014, 2016, and 2017);

Student service Excellence Award, by Tsinghua University (2016);

Misc

Besides research, I'm also passionate about music and sports! I currently play at All University Orchestra at CMU. I am also an (inactive) member at CMU badminton team. Here is the fun CV about my other slices of life.

This guy makes a nice webpage.