List

Music
Album
Videos

Life is not a race, but a journey

List

Music
Album
Videos

tags

Natural Language Processing CMU Study Notes 11711 AI Multimodal Vision Diffusion Distributed Training Parallelism Scaling Quantization Inference Efficiency Retrieval RAG LLM Systems GPU Programming 11868 ML Systems Deep Learning Auto Differentiation 15642 Decoding Speculative Decoding DDP NCCL AllReduce Pipeline Parallelism Tensor Parallelism Model Parallelism GPipe Megatron-LM Gradient Checkpointing Mixture of Experts MoE Expert Parallelism Switch Transformer DeepSeek GShard DeepSpeed-MoE Tokenization NLP 15618 Parallel Programming Assignment SIMD Pthreads Systems Performance Optimization Data Parallelism ZeRO Database Systems Database 15645 Transformer Attention FlashAttention GPU Optimization Hash Tables Concurrency Control Latches B+Tree Hash Table Index Filter Bloom Filter Join Hash Join Sort-Merge Join Sorting Aggregation Query Execution Query Optimization Cost Model Cardinality Estimation Storage B-Tree Blog Writing Tips AI Agent LLM Engineering Design Claude Code OpenAI Codex Memory System Context Management Technical Reading C++ Programming C++ Language Learning TinyKV Distributed Systems KV Storage Raft Consensus UI Programming CS336 Stanford PyTorch Architectures Hyperparameters CNN CMU 11-785 Course Experiments Operating System Technical Blog Code Agent Engineering Experience 个人反思生活 Computer Networks - Campus Network - Routing 面经 git CS Tools Study Computer Networks Makefile Basics Evaluation Benchmarks

Xiang Li

Xiang Li's Blog

Announcement

Talk is cheap. Show me the code.

Recent Posts

11711 Advanced NLP: Parallelism and Scaling

11711 Advanced NLP: Parallelism and Scaling2026-04-06

11711 Advanced NLP: Quantization

11711 Advanced NLP: Quantization2026-04-06

11868 LLM Sys: Systems for Mixture-of-Experts Models

11868 LLM Sys: Systems for Mixture-of-Experts Models2026-03-21

15645 Database systems: Query Optimization

15645 Database systems: Query Optimization2026-03-20

15645 Database systems: Query Execution

15645 Database systems: Query Execution2026-03-15

Categories

AI Agent1
Blog Setup1
C/C++ Programming10
CMU 11-711 Advanced NLP8
CMU 11-868 LLM Systems6
CMU 15-618 Parallel Programming2
CMU 15-642 ML Systems2
CMU 15-645 Database Systems9

Tags

Efficiency Performance Optimization Aggregation FlashAttention GShard DeepSeek Parallelism Hyperparameters Filter B+Tree Systems MoE Query Execution RAG AllReduce Gradient Checkpointing GPipe Technical Reading Claude Code Cardinality Estimation 个人反思 Megatron-LM Study Notes Assignment Attention Pipeline Parallelism Diffusion Expert Parallelism Cost Model Sorting Architectures Query Optimization Quantization SIMD Evaluation Hash Table Retrieval DeepSpeed-MoE 面经 11868

Archives

April 2026 2
March 2026 10
February 2026 9
January 2026 10
December 2025 5
October 2025 2
September 2024 3
December 2023 1

Website Info

Article Count :

62

Unique Visitors :

Page Views :

Last Update :

© 2025 - 2026 By Xiang LiFramework Hexo 8.1.1|Theme Butterfly 5.4.3