:orphan:

.. _gpu_advanced:

GPU training (Advanced)

Audience: Users looking to scale massive models (ie: 1 Trillion parameters).

For experts pushing the state-of-the-art in model development, Lightning offers various techniques to enable Trillion+ parameter-scale models.

.. raw:: html

<div class="display-card-container">
    <div class="row">

.. displayitem:: :header: Train models with billions of parameters :description: :col_css: col-md-4 :button_link: ../advanced/model_parallel/index.html :height: 150 :tag: advanced

.. raw:: html

    </div>
</div>