I am a third-year Ph.D. candidate at Fudan University, advised by Prof. Xipeng Qiu. Currently, I am also interning at Shanghai AI Laboratory.

My research interests lie in the field of Machine Learning and Natural Language Processing, with a particular focus on large language models and methods to enhance their efficiency and efficacy.

News

• [Aug. 2024] InternLM2.5 with 1M context window was released!
• [May 2024] LOMO and AdaLomo were integrated in transformers and accelerate!
• [Jan. 2024] InternLM2 with perfect performance on 200K Needle-in-a-Haystack test was released!

Education

• Fudan University
  Ph.D. candidate in Computer Science, 2021 - 2026 (expected)
  Advisor: Prof. Xipeng Qiu
• Fudan University
  B.S. in Computer Science, 2017 - 2021

Experience

• Shanghai AI Laboratory
  Advisor: Dr. Hang Yan
  July 2023 - Present

Publications

* denotes co-first authors

InternLM2 Technical Report
InternLM Team
Tenical Report. [paper] [github]

LongWanjuan: Towards Systematic Measurement for Long Text Quality
Kai Lv*, Xiaoran Liu*, Qipeng Guo, Hang Yan, Conghui He, Xipeng Qiu, Dahua Lin
arXiv 2024. [paper] [code]

AdaLomo: Low-memory Optimization with Adaptive Learning Rate
Kai Lv, Hang Yan, Qipeng Guo, Haijun Lv, Xipeng Qiu
ACL 2024 Findings. [paper] [code]

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
Kai Lv*, Shuo Zhang*, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu
EMNLP 2023 Demo. [paper] [code]

Full Parameter Fine-tuning for Large Language Models with Limited Resources
Kai Lv, Yuqing Yang, Tengxiao Liu, Qinghui Gao, Qipeng Guo, Xipeng Qiu
ACL 2024 (oral). [paper] [code]

Unified Demonstration Retriever for In-Context Learning
Xiaonan Li*, Kai Lv*, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, Xipeng Qiu
ACL 2023 (oral). [paper] [code]

CoNT: Contrastive Neural Text Generation
Chenxin An, Jiangtao Feng, Kai Lv, Lingpeng Kong, Xipeng Qiu, Xuanjing Huang
NeurIPS 2022. [paper] [code]

Service

Reviewer: CoNLL