Skip to main content

复现 ResNet

· 2 min read
Gavin Gong

ResNet 算得上是超经典的backbone了,其网络提出了残差结构,可有效缓解随网络层数的加深而导致的梯度消失和梯度爆炸现象。结构和设计在这里有讨论。这里主要尝试一下复现。ResNet的常见形式有:

NameTop-1 errorTop-5 error
ResNet-1830.4310.76
ResNet-3426.738.74
ResNet-5024.017.02
ResNet-10122.446.21
ResNet-15222.166.16

除了上述常见形式,也有将channel数减半的half形式,以及使用膨胀卷积的形式等。

note

本文的代码部分主要使用paddlepaddle实现。可能使用的包有:

import paddle
import paddle.nn as nn
from paddle.nn import *
import paddle.nn.functional as F

ResNet主要由两种基础结构组成:

  • Basic Block(resnet18,resnet34)
class BasicBlock(nn.Layer):
def __init__(self,
in_channels,
out_channels,
stride,
dilation=1,
shortcut=True,
if_first=False,
data_format='NCHW'):
super(BasicBlock, self).__init__()
self.conv0 = ConvBNLayer(
in_channels=in_channels,
out_channels=out_channels,
kernel_size=3,
stride=stride,
dilation=dilation,
act='relu',
data_format=data_format)
self.conv1 = ConvBNLayer(
in_channels=out_channels,
out_channels=out_channels,
kernel_size=3,
dilation=dilation,
act=None,
data_format=data_format)

if not shortcut:
self.short = ConvBNLayer(
in_channels=in_channels,
out_channels=out_channels,
kernel_size=1,
stride=1,
is_vd_mode=False if if_first or stride == 1 else True,
data_format=data_format)

self.shortcut = shortcut
self.dilation = dilation
self.data_format = data_format
self.add = layers.Add()
self.relu = layers.Activation(act="relu")

def forward(self, inputs):
y = self.conv0(inputs)
conv1 = self.conv1(y)

if self.shortcut:
short = inputs
else:
short = self.short(inputs)
y = self.add(short, conv1)
y = self.relu(y)

return y
  • Bottleneck Block(resnet50,resnet101,resnet152)