Welcome to the LLM Compression course! This comprehensive course covers the fundamental theories and practical techniques for compressing large language models, with a focus on:
Attention Mechanisms and Transformer architectures
SVD (Singular Value Decomposition) for matrix approximation
Tensor Decomposition methods for model compression
Advanced topics including tensor networks and quantum-inspired methods
本课程系统讲解大语言模型压缩的基础理论与实践技术，涵盖：
注意力机制与 Transformer 架构
奇异值分解 (SVD) 矩阵近似方法
张量分解模型压缩技术
前沿主题：张量网络与量子启发方法
1.3 Course Structure / 课程结构
The course is divided into three parts:
Part
Content
内容
Part I
Foundations (Lectures 1-5)
基础（第1-5讲）
Part II
Compression Techniques (Lectures 6-10)
压缩技术（第6-10讲）
Part III
Advanced Topics (Lectures 11-12)
前沿方向（第11-12讲）
1.4 Prerequisites / 预备知识
Linear Algebra (矩阵论)
Machine Learning (机器学习)
Deep Learning (深度学习)
Python Programming (Python编程)
1.5 Resources / 资源
Lectures: 12 comprehensive lectures in LaTeX/Quarto format
Notebooks: Interactive Jupyter notebooks for hands-on practice
Problems: Exercise sets for each topic
Papers: Key research papers in the field
1.6 License / 许可证
This course materials are available for educational purposes.