Inference API

:func:deepspeed.init_inference returns an inference engine of type :class:InferenceEngine.

.. code-block:: python

for step, batch in enumerate(data_loader):
    #forward() method
    loss = engine(batch)

Forward Propagation

.. autofunction:: deepspeed.InferenceEngine.forward