Loss Functions

Loss functions measure how well the model's predictions match the targets. The choice of loss function depends on your task type and data characteristics.

Regression losses:

L1Loss/MSELoss: Basic regression losses (MAE vs MSE)
SmoothL1Loss/HuberLoss: Robust to outliers

Classification losses:

CrossEntropyLoss: Multi-class classification (combines LogSoftmax + NLLLoss)
NLLLoss: Negative log likelihood (use with LogSoftmax output)
BCELoss/BCEWithLogitsLoss: Binary classification

Specialized losses:

CTCLoss: Sequence-to-sequence without alignment (speech recognition)
TripletMarginLoss: Metric learning (similarity/embedding tasks)
CosineEmbeddingLoss: Similarity learning with cosine distance

L1Loss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

MSELoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

Example:

cpp

auto loss_fn = torch::nn::MSELoss();
auto loss = loss_fn->forward(predictions, targets);

CrossEntropyLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

Example:

cpp

auto loss_fn = torch::nn::CrossEntropyLoss();
auto logits = torch::randn({32, 10});  // [batch, num_classes]
auto targets = torch::randint(0, 10, {32});  // [batch]
auto loss = loss_fn->forward(logits, targets);

NLLLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

BCELoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

BCEWithLogitsLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

HuberLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

SmoothL1Loss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

KLDivLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

CTCLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

PoissonNLLLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

MarginRankingLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

HingeEmbeddingLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

CosineEmbeddingLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

MultiMarginLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

MultiLabelMarginLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

MultiLabelSoftMarginLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

SoftMarginLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

TripletMarginLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

TripletMarginWithDistanceLoss

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members: