ContextQMD
Back to Megatron Lm

Advanced Features

docs/user-guide/features/index.md

23.06726 B
Original Source
<!--- Copyright (c) 2022-2026, NVIDIA CORPORATION. All rights reserved. NVIDIA CORPORATION and its licensors retain all intellectual property and proprietary rights in and to this software, related documentation and any modifications thereto. Any use, reproduction, disclosure or distribution of this software and related documentation without an express license agreement from NVIDIA CORPORATION is strictly prohibited. -->

Advanced Features

Advanced feature guides for key Megatron Core capabilities.

{toctree}
:maxdepth: 2

fine_grained_activation_offloading
moe
context_parallel
custom_fsdp
dist_optimizer
optimizer_cpu_offload
pipeline_parallel_layout
tokenizers
megatron_energon
megatron_rl