Level 19: Train models with billions of parameters

:orphan:

################################################## Level 19: Train models with billions of parameters ##################################################

Scale to billions of parameters with multiple distributed strategies.

.. raw:: html

<div class="display-card-container">
    <div class="row">

.. Add callout items below this line

.. displayitem:: :header: Scale with distributed strategies :description: Learn about different distributed strategies to reach bigger model parameter sizes. :col_css: col-md-6 :button_link: ../accelerators/gpu_intermediate.html :height: 150 :tag: intermediate

.. displayitem:: :header: Train models with billions of parameters :description: Scale to billions of params on GPUs with FSDP, TP or Deepspeed. :col_css: col-md-6 :button_link: ../advanced/model_parallel/index.html :height: 150 :tag: advanced

.. raw:: html

    </div>
</div>