ContextQMD
Back to Annotated Deep Learning Paper Implementations

Weight Standardization

labml_nn/normalization/weight_standardization/readme.md

latest452 B
Original Source

Weight Standardization

This is a PyTorch implementation of Weight Standardization from the paper Micro-Batch Training with Batch-Channel Normalization and Weight Standardization. We also have an annotated implementation of Batch-Channel Normalization.