I’m a 3rd-year Ph.D. student in Conversational AI Group, Department of Computer Science and Technology, Tsinghua University. I’m advised by Prof. Minlie Huang. I was an intern at Microsoft Research Asia mentored by Dr. Li Dong. My research interests include Efficient Pre-trained Language Models and Dialogue Systems.

Education

• 2021.9 - Present: Ph.D. Student, Department of Computer Science and Technology, Tsinghua University
• 2017.9 - 2021.6: B.Eng., Department of Computer Science and Technology, Tsinghua University

Publications

Conference Papers

• Yuxian Gu, Li Dong, Furu Wei, Minlie Huang. MiniLLM: Knowledge Distillation of Large Language Models. ICLR 2024 (Poster). [pdf] [code]

• Yuxian Gu, Li Dong, Furu Wei, Minlie Huang. Pre-Training to Learn in Context. ACL 2023 (Long Paper, Oral). [pdf] [code]

• Yuxian Gu, Pei Ke, Xiaoyan Zhu, Minlie Huang. Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization. EMNLP 2022 (Long Paper, Oral). [pdf] [code]

• Yuxian Gu*, Xu Han*, Zhiyuan Liu, Minlie Huang. PPT: Pre-Trained Prompt Tuning for Few-Shot Learning. ACL 2022 (Long Paper). [pdf] [code]

• Qi Zhu, Yuxian Gu, Lingxiao Luo, Bing Li, Cheng Li, Wei Peng, Minlie Huang, Xiaoyan Zhu. When does Further Pre-Training MLM Help? An Empirical Study on Task-Oriented Dialog Pre-Training. EMNLP 2021 Workshop (Best Paper). [pdf] [code]

• Yuxian Gu, Zhengyan Zhang, Xiaozhi Wang, Zhiyuan Liu, Maosong Sun. Train No Evil: Selective Masking for Task-Guided Pre-Training. EMNLP 2020 (Short Paper). [pdf] [code]

• Xin Lv, Yuxian Gu, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu. Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations. EMNLP 2019 (Short Paper). [pdf] [code]

Journal Papers

• Yuxian Gu*, Jiaxin Wen*, Hao Sun*, Yi Song, Pei Ke, Chujie Zheng, Zheng Zhang, Jianzhu Yao, Lei Liu, Xiaoyan Zhu, Minlie Huang. EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training. 2022. Machine Intelligence Research [pdf] [code]

• Zhengyan Zhang*, Yuxian Gu*, Xu Han*, Shengqi Chen*, Chaojun Xiao*, Zhenbo Sun, Yuan Yao, Fanchao Qi, Jian Guan, Pei Ke, Yanzheng Cai, Guoyang Zeng, Zhixing Tan, Zhiyuan Liu, Minlie Huang, Wentao Han, Yang Liu, Xiaoyan Zhu, Maosong Sun. CPM-2: Large-Scale Cost-Effective Pre-Trained Language Models. 2022. AI Open. [pdf] [pre-train code] [fine-tune code]

• Xu Han*, Zhengyan Zhang*, Ning Ding*, Yuxian Gu*, Xiao Liu*, Yuqi Huo*, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang, Wentao Han, Minlie Huang, Qin Jin, Yanyan Lan, Yang Liu, Zhiyuan Liu, Zhiwu Lu, Xipeng Qiu, Ruihua Song, Jie Tang, Ji-Rong Wen, Jinhui Yuan, Wayne Xin Zhao, Jun Zhu. Pre-Trained Models: Past, Present and Future. 2022. AI Open. [pdf]

• Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, Maosong Sun. CPM: A Large-Scale Generative Chinese Pre-Trained Language Model. 2021. AI Open. [pdf] [pre-train code] [fine-tune code] [inference code]

Preprints

• Yuxian Gu, Li Dong, Yaru Hao, Qingxiu Dong, Minlie Huang, Furu Wei. Towards Optimal Learning of Language Models. arXiv preprint 2024. [pdf] [code]

• Yi Song*, Yuxian Gu*, Minlie Huang. Many-Class Text Classification with Matching. arXiv preprint 2022. [pdf]

• Hao Zhou, Pei Ke, Zheng Zhang, Yuxian Gu, Yinhe Zheng, Chujie Zheng, Yida Wang, Chen Henry Wu, Hao Sun, Xiaocong Yang, Bosi Wen, Xiaoyan Zhu, Minlie Huang, Jie Tang. EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training. arXiv preprint 2021. [pdf] [code]

• Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, Furu Wei. Structured prompting: Scaling in-context learning to 1,000 examples. [pdf] [code]

Services

• Program Committee Member (Conference Reviewer) EMNLP 2022, ACL 2023, EMNLP 2023, ARR 2023, ARR 2024

Teaching

I was a TA for the following undergraduate courses:

• Artificial Neural Network (2020 Fall, 2021 Fall, 2022 Fall, 2023 Fall)
• Object-Oriented Programming (2021 Spring, 2022 Spring, 2023 Spring)

Selected Honors and Awards

• Excellent Graduate, Tsinghua University, 2021
• Outstanding Graduate, Dept. CST, Tsinghua University, 2021
• Outstanding Undergraduate Dissertation, Tsinghua University, 2021
• Overall Scholarship, Dept. CST, Tsinghua University, 2020
• Science and Technology Innovation Excellence Scholarship, Dept. CST, Tsinghua University, 2019
• Silver Prize, Asia Student Supercomputer Challenge, Asia Supercomputer Community, 2019
• Overall Scholarship, Dept. CST, Tsinghua University, 2018

Miscellaneous

I like to play Jianzi, Jianqiu, a competitive sport similar to volleyball. I am the captain of Tsinghua Jianqiu Team and was the captain of the Beijing Youth Jianqiu Team. I once won the championship of the Jianqiu Competition in Beijing and represented Beijing in National Youth Olympic Games.