Zheng Li (李政)
PhD Student, Nankai University
zhengli97 [AT] nankai.edu.cn
Hangzhou, Zhejiang, China
ResearchGate Github Google Scholar
Bio
I am currently a third-year Ph.D. student at Tianjin Key Laboratory of Visual Computing and Intelligent Perception (VCIP), Nankai University, advised by Prof. Xiang Li and Prof. Jian Yang. I am also a research intern at Alibaba DAMO Academy, mentored by Dr. Yibing Song. My research mainly focuses on vision-language models, multi-modal learning and efficient model computing.
The code of my research work will be open-source, and I will also attach a detailed Chinese interpretation of the paper. Although the interpretation may be somewhat fragmented, I will do my best to present the insights and ideas behind the paper.
The journey of scientific research is challenging, but I'm passionate about my work. If you're interested in my research or encounter any problems, feel free to contact me via email (zhengli97 [at] qq.com).
News
- Dec.2024. Our new work ATPrompt is now avavilable on arXiv. [Link] [Code] [Project Page].
- Oct.2024. I was invited to gave a talk on Jiangmen(将门) about prompt learning methods. [Link]
- Mar.2024. We released a curated list of prompt/adapter learning methods for VLMs. Feel free to check it out~
- Feb.2024. PromptKD was accepted by CVPR 2024. Code is public avavilable at GitHub.
Publications
Most recent publications are on Google Scholar.
# indicates corresponding author.
ATPrompt: Textual Prompt Learning with Embedded Attributes.
Zheng Li, Yibing Song, Penghai Zhao, Ming-Ming Cheng, Xiang Li#, Jian Yang#.
arxiv 2024
PromptKD: Unsupervised Prompt Distillatino for Vision-Language Models.
Zheng Li, Xiang Li#, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, Jian Yang#
CVPR 2024
PromptKD is a simple and effective prompt-driven unsupervised distillation framework for VLMs (e.g., CLIP), with state-of-the-art performance.
Dual Teachers for Self-Knowledge Distillation.
Zheng Li, Xiang Li, Lingfeng Yang, Renjie Song, Jian Yang#, Zhigeng Pan.
PR 2024
DTSKD explores a new self-KD framework where the student network receives self-supervisions by dual teachers from two dramatically distinct fields.
Curriculum Temperature for Knowledge Distillation.
Zheng Li, Xiang Li#, Lingfeng Yang, Borui Zhao, Renjie Song, Lei Luo, Jun Li, Jian Yang#.
AAAI 2023
CTKD organizes the distillation task from easy to hard through a dynamic and learnable temperature.
Online Knowledge Distillation for Efficient Pose Estimation.
Zheng Li, Jingwen Ye, Mingli Song, Ying Huang, Zhigeng Pan#.
ICCV 2021
OKDHP first proposes to distill the pose structure knowledge in a one-stage manner.
Online Knowledge Distillation via Multi-branch Diversity Enhancement.
Zheng Li, Ying Huang, Defang Chen, Tianren Luo, Ning Cai, Zhigeng Pan#.
ACCV 2020
OKDMDE is a simple and effective technique to enhance model diversity in online knowledge distillation.
ATPrompt: Textual Prompt Learning with Embedded Attributes.
Zheng Li, Yibing Song, Penghai Zhao, Ming-Ming Cheng, Xiang Li#, Jian Yang#.
arxiv 2024
Cascade Prompt Learning for Vision-Language Model Adaptation.
Ge Wu, Xin Zhang, Zheng Li, Zhaowei Chen, Jiajun Liang, Jian Yang, Xiang Li#.
ECCV 2024
PromptKD: Unsupervised Prompt Distillatino for Vision-Language Models.
Zheng Li, Xiang Li#, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, Jian Yang#
CVPR 2024
Dual Teachers for Self-Knowledge Distillation.
Zheng Li, Xiang Li, Lingfeng Yang, Renjie Song, Jian Yang#, Zhigeng Pan.
PR 2024
GEIKD: Self-knowledge Distillation based on Gated Ensemble Networks and Influences-based Label Noise Removal.
Fuchang Liu, Yu Wang, Zheng Li, Zhigeng Pan#.
CVIU 2023
Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?
Zheng Li, Yuxuan Li, Penghai Zhao, Renjie Song, Xiang Li#, Jian Yang#.
Arxiv Preprint 2023
Curriculum Temperature for Knowledge Distillation.
Zheng Li, Xiang Li#, Lingfeng Yang, Borui Zhao, Renjie Song, Lei Luo, Jun Li, Jian Yang#.
AAAI 2023
Online Knowledge Distillation for Efficient Pose Estimation.
Zheng Li, Jingwen Ye, Mingli Song, Ying Huang, Zhigeng Pan#.
ICCV 2021
Online Knowledge Distillation via Multi-branch Diversity Enhancement.
Zheng Li, Ying Huang, Defang Chen, Tianren Luo, Ning Cai, Zhigeng Pan#.
ACCV 2020
Dream-experiment: a MR user interface with natural multi-channel interaction for virtual experiments.
Tianren Luo, Mingmin Zhang#, Zhigeng Pan#, Zheng Li, Ning Cai, Jinda Miao, Youbin Chen, Mingxi Xu.
TVCG 2020
Experiences
Education
- Sep.2022 - Present. Ph.D., Computer Science and Technology, Nankai University.
- Sep.2019 - June.2022. M.Eng., Computer Applied Technology, Hangzhou Normal University.
- Sep.2015 - June.2019. B.Eng., Telecommunication Engineering, North China University of Science and Technology.
Internship
- May.2024 - Present. Alibaba DAMO Academy, Hangzhou. Research Intern. Mentored by Dr. Yibing Song.
- Sep.2023 - May.2024 Ant Group, Hangzhou. Research Intern. Mentored by Xinyi Fu and Dr. Xing Fu.
- Oct.2021 - Aug.2023 Megvii Research, Nanjing. Research Intern. Mentored by Renjie Song.
Competition
Kaggle Competition Master. Profile
- Nov.2019. "Understanding Clouds from Satellite Images." Rank: 7th/1538 (Top 1%). Gold Medal. [Link] [Solution]
- Apr.2018. "2018 Data Science Bowl." Rank: 8th/3634 (Top 1%). Solo Gold Medal. [Link] [Solution]
Misc
Personal Hobbies
- Photography 📸: I am a contracted photographer for 500px Gallery. Here are some photos I took while traveling and mountaineering.
- Mountaineering 🗻: Summit: Haba Snow Mountain (5396m).
- Trail Running 🏃♂️ [iTRA]: TNF100 Ultra Trail Challenge Moganshan - 30km Group. Finish: 5h28min (33km).