Training & algorithms#
Understand policy objectives, precision, parallelism, and memory.
- From policy gradients to train–rollout consistency
- GPUs, memory, and parallelism
- Low Precision Training and Rollout
- Reproducibility
- On-Policy Distillation
- Megatron Config: Role-Based Training Overrides
- Supporting Model Architectures Beyond Megatron-LM
- Model Converter Test Harness
- HF→Megatron Load Modes (builtin / mbridge)
- DSA Kernels (DeepSeek Sparse Attention)
- Day-0 Model Plugins