Zhangchi Feng
LLaMA Factory Core Developer
Zhangchi Feng, the core developer of LLaMA Factory, has a master degree from Beijing University of Aeronautics and Astronautics. He has published many papers in ACM MM, KDD and other conferences. He is the recipient of National Scholarship and Outstanding Graduate of Beihang University. He has been invited to deliver keynote speeches at KDD, China Networking Conference and other top academic conferences.
Topic
Principles and Practice of Efficient Fine-Tuning of Multimodal LLM in LLaMA Factory
INTRODUCTION: LLaMA Factory is a unified large model fine-tuning framework designed to simplify and accelerate the fine-tuning process for over 100 LLMs. We will introduce its core features related to multimodality, its technology stack, and its performance in real-world applications to help attendees understand how to utilize this framework to improve the training efficiency and inference performance of multimodal LLMs. Outline: 1. Introduction: background of multimodal LLMs 2. Overview of LLaMA Factory: project background and goals Overview of supported models 3. Fine-tuning Methods: Multimodal Pre-training and Supervised Fine-tuning Multimodal Reward Modeling and Reinforcement Learning Quantification Techniques 4. Advanced Algorithms: GaLore, BAdam, APOLLO, Adam-mini, etc. 5. Task support: multi-round dialog, image understanding, video recognition, speech understanding and other application scenarios 6. Practical cases: Qwen2-VL cultural travel robot MiniCPM-o-2.6 voice assistant Virgo multimodal deep thinking model 7.Summary and Prospect: The Future of Multimodal LLM
