About me

Hi, I am Yilun Xu, a research scientist in NVIDIA Research, fundamental GenAI research team.

I received my PhD in MIT EECS, advised by Tommi Jaakkola. I obtained my Bachelor’s degree from Turing Class in EECS dept, Peking University. I had the privilege of working with Yizhou Wang in PKU, and Stefano Ermon in Stanford. I also worked with Karsten Kreis and Arash Vahdat in NVIDIA Research, during summer 2023.

My research interests are machine learning, with a current emphasis on generative modeling: (i) new family of generative models [13, 12, 10]; (ii) improving the training, sampling and controllability of generative models, e.g. PFGM and diffusion models [16, 15, 14, 11, 9, 5]. Previously, I worked on bridging machine learning and information theory [4, 3, 2, 1].

Contact: yilunx@nvidia.com, ~~ylxu@mit.edu~~ , ~~xuyilun@pku.edu.cn~~

Publications

(*) denotes equal contribution

Hamiltonian Score Matching and Generative Flows
Peter Holderrieth, Yilun Xu, Tommi Jaakkola.
In Neural Information Processing Systems (NeurIPS), 2024.
[PDF]
On Physics-Inspired Generative Models
Yilun Xu.
PhD Thesis 🎓, Massachusetts Institute of Technology.
[PDF]
DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents
Yilun Xu, Gabriele Corso, Tommi Jaakkola, Arash Vahdat, Karsten Kreis.
In International Conference on Machine Learning (ICML), 2024.
[PDF] ,[Project Page]
Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models
Gabriele Corso, Yilun Xu, Valentin De Bortoli, Regina Barzilay, Tommi Jaakkola.
In International Conference on Learning Representations (ICLR), 2024; Deep Inverse Workshop, Neural Information Processing Systems (NeurIPS), 2023 (Oral); Workshop on Diffusion Models, Neural Information Processing Systems (NeurIPS), 2023.
[PDF] ,[Code]
Star
Restart Sampling for Improving Generative Processes
Yilun Xu*, Mingyang Deng*, Xiang Cheng*, Yonglong Tian, Ziming Liu, Tommi Jaakkola.
In Neural Information Processing Systems (NeurIPS), 2023.
[PDF] ,[Code]
News Coverage: [MarkTechPost],
Star
PFGM++: Unlocking the Potential of Physics-Inspired Generative Models
Yilun Xu, Ziming Liu, Yonglong Tian, Shangyuan Tong, Max Tegmark, Tommi Jaakkola.
In International Conference on Machine Learning (ICML), 2023.
[PDF] ,[Code] ,[Slide]
News Coverage: [MIT News], [Quanta Magazine], [Jiangmen Venture (CN)]
Star
GenPhys: From Physical Processes to Generative Models
Ziming Liu, Di Luo, Yilun Xu, Tommi Jaakkola, Max Tegmark.
In preprint, 2023
[PDF]
News Coverage: [MIT News], [Quanta Magazine],
Stable Target Field for Reduced Variance Score Estimation in Diffusion Models
Yilun Xu*, Shangyuan Tong*, Tommi Jaakkola.
In International Conference on Learning Representations (ICLR), 2023
[PDF] ,[Code] ,[Slide] ,[Poster]
News Coverage: [MarkTechPost],
Star
Poisson Flow Generative Models
Yilun Xu*, Ziming Liu*, Max Tegmark, Tommi Jaakkola.
In Neural Information Processing Systems (NeurIPS), 2022. (Spotlight)
[PDF] ,[Code] ,[Slide] ,[Poster]
News Coverage: [MIT News], [Quanta Magazine], [AssemblyAI Blog], [MarkTechPost], [Synced (CN)], [PaperWeekly (CN)], [QbitAI (CN)],
Star
Controlling Directions Orthogonal to a Classifier
Yilun Xu, Hao He, Tianxiao Shen, Tommi Jaakkola.
In International Conference on Learning Representations (ICLR), 2022. (Spotlight)
[PDF] ,[Code] ,[Slide] ,[Poster]
Star
A Survey on Generative Diffusion Model
Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, Stan Z. Li.
In IEEE Transactions on Knowledge and Data Engineering (TKDE), 2023.
[PDF] ,[Code]
Star
Learning Representations that Support Robust Transfer of Predictors
Yilun Xu, Tommi Jaakkola.
In preprint, 2022
[PDF] ,[Code]
Can Subnetwork Structure Be the Key to Out-of-Distribution Generalization?
Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, Aaron Courville.
In International Conference on Machine Learning (ICML), 2021. (Long talk)
[PDF]
Anytime Sampling for Autoregressive Models via Ordered Autoencoding
Yilun Xu, Yang Song, Sahaj Garg, Linyuan Gong, Rui Shu, Aditya Grover, Stefano Ermon.
In International Conference on Learning Representations (ICLR), 2021
[PDF] ,[Code] ,[Slide] ,[Poster]
Star
A Theory of Usable Information under Computational Constraints
Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, Stefano Ermon.
In International Conference on Learning Representations (ICLR), 2020. (Oral)
[PDF] ,[Code] ,[Slide] ,[Video]
News Coverage: [Synced (CN)],
TCGM: An Information-Theoretic Framework for Semi-Supervised Multi-Modality Learning
Xinwei Sun*, Yilun Xu*, Peng Cao, Yuqing Kong, Lingjing Hu, Shanghang Zhang, Yizhou Wang.
In European Conference on Computer Vision (ECCV), 2020. (Oral)
[PDF] ,[Code]
L_{DMI} : A Novel Information-theoretic Loss Function for Training Deep Nets Robust to Label Noise
Yilun Xu*, Peng Cao*, Yuqing Kong, Yizhou Wang.
In Neural Information Processing Systems (NeurIPS), 2019
[PDF] ,[Code] ,[Slide]
News Coverage: [Synced (CN)], [PKU-CFCS News (CN)]
Star
Max-MIG: an Information Theoretic Approach for Joint Learning from Crowds
Peng Cao*, Yilun Xu*, Yuqing Kong, Yizhou Wang.
In International Conference on Learning Representations (ICLR), 2019
[PDF] ,[Code] ,[Slide]
News Coverage: [PKU-CFCS News (CN)]
Star

Education

Sep. 2021* - May. 2024: Massachusetts Institute of Technology

Ph.D. in Computer Science

Advisor: Tommi Jaakkola

*: remote during Sept 2020 - Aug 2021

Sep. 2016 - July. 2020: Peking University

B.S. in Turing Class, Computer Science (summa cum laude)

Advisor: Yizhou Wang

Jun. 2019 - Sep. 2019: Stanford University

Visiting reseacher

Advisor: Stefano Ermon

Work Experience

Jun. 2023 - May, 2024: NVIDIA

Research Intern

Advisor: Karsten Kreis and Arash Vahdat

Talks

On Physics-Inspired Generative Models
- Peking University, hosted by Prof. Yizhou Wang, June 2024 [Video (CN)], [Slide]
- CUHK, hosted by Prof. Pheng Ann Heng, June 2024
Generative Models & Physical Processes
- UCLA, hosted by Prof. Yingnian Wu, Oct 2023 [Slide]
Unlocking the Potential of Physics-Inspired Generative Models
- Learning on Graphs and Geometry seminar. Oct, 2023 [Video]
- Zhejiang University, hosted by Prof. Chao Xu, Oct 2023
- Peking University, CFCS, Aug 2023
- NVIDIA Research, July 2023
- ByteDance, AI for Science Team, July 2023
- Swarma Club, May 2023, [Slide] [Video (CN)]
- TechBeat/Jiangmen Ventures, April 2023, [Slide] [Video (CN)]
- MLTea Talk, MIT, April 2023, [Slide]
- Stanford University, hosted by Prof. Mert Pilanci, Feb 2023, [Slide]
Conditional and Controllable Generation
- Guest lecturer at 6.S052/6.S952: Modeling with Machine learning for CS, MIT, April 2023, [Slide]
Poisson Flow Generative Models
- AssemblyAI AI Hackathon, Dec 2022, [Slide]
- Princeton University, hosted by Prof. Mengdi Wang, Nov 2022, [Slide]
- MIT NetMIT Group, hosted by Prof. Dina Katabi, Nov 2022
Controlling Directions Orthogonal to a Classifier,
- AI TIME, June 2022, [Slide]
Anytime Sampling for Autoregressive Models via Ordered Autoencoding
- AI TIME, June 2021, [Slide]

Service

Conference Reviewer: NeurIPS 2020, 2021, 2022, 2023; ICLR 2021, 2022, 2023, 2024; ICML 2021, 2022, 2023

Workshop Reviewer: ICML-AI4Science 2022

Journal Reviewer: TMLR, TPAMI

PC member: ICLR 2022 PAIR2Struct Workshop

Panelist: ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling

Teaching Assistant for 6.S052/6.S952: Modeling with Machine learning for CS, MIT, Spring 2023 (co-design the problem set and lectures for this new course, and receive an outstanding TA rating of 6.9 out of 7.0 from students)

Miscellaneous

I am a national second-class table tennis player in China. Men’s single champion in PKU Freshman’s Cup (2016). I also played for MIT table tennis team, winning the 3rd place (group) in the 2022 NCTTA Upper New England Championship.

I have been playing PUBG (mobile), LoL a lot. Find me out!