Utilities

Additional utilities for building neural networks: parameter initialization, module cloning, type-erased containers, padding layers, and vision utilities.

Parameter Initialization

The torch::nn::init namespace provides functions for initializing module parameters:

cpp

#include <torch/nn/init.h>

// Xavier/Glorot initialization
torch::nn::init::xavier_uniform_(linear->weight);
torch::nn::init::xavier_normal_(linear->weight);

// Kaiming/He initialization
torch::nn::init::kaiming_uniform_(conv->weight, /*a=*/0, torch::kFanIn, torch::kReLU);
torch::nn::init::kaiming_normal_(conv->weight);

// Other initializations
torch::nn::init::zeros_(linear->bias);
torch::nn::init::ones_(bn->weight);
torch::nn::init::constant_(linear->bias, 0.1);
torch::nn::init::normal_(linear->weight, /*mean=*/0, /*std=*/0.01);
torch::nn::init::uniform_(linear->weight, /*a=*/-0.1, /*b=*/0.1);
torch::nn::init::orthogonal_(rnn->weight_hh);

Cloneable

{doxygenclass}

All torch::nn modules inherit from Cloneable, enabling deep copies:

cpp

auto model = torch::nn::Linear(10, 5);
auto model_copy = std::dynamic_pointer_cast<torch::nn::LinearImpl>(model->clone());

AnyModule

AnyModule provides type-erased storage for any module, allowing you to store heterogeneous modules in a single container.

{doxygenclass}

:members:
:undoc-members:

Example:

cpp

torch::nn::AnyModule any_module(torch::nn::Linear(10, 5));
auto output = any_module.forward(input);

Functional

Wraps a function or callable as a module, useful for inserting arbitrary functions into a Sequential container.

{doxygenclass}

:members:
:undoc-members:

ModuleHolder

{doxygenclass}

:members:
:undoc-members:

CosineSimilarity

{doxygenclass}

:members:
:undoc-members:

PairwiseDistance

{doxygenclass}

:members:
:undoc-members:

PackedSequence

{cpp:class}


Holds the data and list of `batch_sizes` of a packed sequence.
All RNN modules accept packed sequences as inputs.

{cpp:function}


Returns the packed tensor containing all sequence elements.

{cpp:function}


Returns a 1D tensor of batch sizes at each time step.

{cpp:function}


Returns indices used to sort sequences by length (descending).

{cpp:function}


Returns indices to restore the original sequence order.

{cpp:function}


Moves the packed sequence to the specified device.

See also: torch::nn::utils::rnn::pack_padded_sequence and torch::nn::utils::rnn::pad_packed_sequence.

Padding Layers

ReflectionPad1d / ReflectionPad2d / ReflectionPad3d

{doxygenclass}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

ReplicationPad1d / ReplicationPad2d / ReplicationPad3d

{doxygenclass}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

ZeroPad1d / ZeroPad2d / ZeroPad3d

{doxygenclass}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

ConstantPad1d / ConstantPad2d / ConstantPad3d

{doxygenclass}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

Vision Layers

PixelShuffle

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

PixelUnshuffle

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

Upsample

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

Fold / Unfold

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members:

{doxygenclass}

:members:
:undoc-members:

{doxygenstruct}

:members:
:undoc-members: