神经网络与深度学习 课件 第6章-深度卷积神经网络

神经网络与深度学习第6章深度卷积神经网络6.1VGG及其应用6.1.1VGG的结构6.1.2利用VGG-16进行CIFAR-100分类6.2GoogLeNet及其应用6.2.11*1卷积6.2.2Inception6.2.3GoogleNet的结构6.2.4利用GoogleNet进行TinyImageNet分类6.3ResNet及其应用6.3.1残差块6.3.2ResNet的结构6.3.3基于迁移学习的ResNet分类应用6.1VGG及其应用牛津大学视觉几何组（VisualGeometryGroup，VGG）的西蒙尼杨（KarenSimonyan）和西塞曼（AndrewZisserman）在2014年提出。通过使用一系列大小为3*3的小尺寸卷积核和汇聚层构造深度卷积神经网络。VGG模型结构简单、应用性强。其网络结构设计方法，为构建深度神经网络提供了方向。6.1VGG及其应用VGG的结构VGG的核心思想是通过使用小的卷积核，即采用3*3的卷积核，增加卷积神经网络的深度，从而提升卷积神经网络的表达能力。根据层数和层的配置的不同，VGG模型主要有5种不同的结构。6.1VGG及其应用VGG的结构ABCDE11个带权层13个带权层16个带权层16个带权层19个带权层输入（224*224RGB图像）cov3-64

cov3-64cov3-64cov3-64cov3-64cov3-64cov3-64cov3-64cov3-64最大汇聚（maxpool）cov3-128

cov3-128cov3-128cov3-128cov3-128cov3-128cov3-128cov3-128cov3-128最大汇聚（maxpool）cov3-256cov3-256cov3-256cov3-256cov3-256cov3-256cov1-256cov3-256cov3-256cov3-256cov3-256cov3-256cov3-256cov3-256最大汇聚（maxpool）cov3-512cov3-512cov3-512cov3-512cov3-512cov3-512cov1-512cov3-512cov3-512cov3-512cov3-512cov3-512cov3-512cov3-512最大汇聚（maxpool）cov3-512cov3-512cov3-512cov3-512cov3-512cov3-512cov1-512cov3-512cov3-512cov3-512cov3-512cov3-512cov3-512cov3-512最大汇聚（maxpool）全连接层（FC-4096）全连接层（FC-4096）全连接层（FC-1000）softmax6.1VGG及其应用VGG的结构根据包含可学习参数的层数不同，VGG模型一般命名为VGG-11、VGG-13、VGG-16和VGG-19，其中VGG-16和VGG-19更为常用。VGG-16拥有13个卷积层，卷积核大小均为3*3，输出特征映射从64到512个，5个最大汇聚层，3个全连接层，节点数分别为4096、4096和1000个。VGG-19比VGG-16多了3个卷积层，其他配置基本相同。6.1VGG及其应用VGG的结构由于层数深，可训练参数多，所以VGG模型规模很大。VGG模型参数数量（单位：百万）模型ABCDE参数数量1331331341381446.1VGG及其应用VGG的结构利用VGG-16进行CIFAR-100分类CIFAR-100数据集与CIFAR-10类似，不同之处在于它有100个类别，每个类包含600幅图像，分为500幅训练图像和100幅测试图像。CIFAR-100中100个类分为20个超类。每个图像都带有一个“精细”标签（它所属的类）和一个“粗”标签（它所属的超类）。本例使用“精细”标签。超类（Superclass）类（Classes）aquaticmammalsbeaver,dolphin,otter,seal,whalefishaquariumfish,flatfish,ray,shark,troutflowersorchids,poppies,roses,sunflowers,tulipsfoodcontainersbottles,bowls,cans,cups,platesfruitandvegetablesapples,mushrooms,oranges,pears,sweetpeppers…………6.1VGG及其应用VGG的结构利用VGG-16进行CIFAR-100分类classVGG16(nn.Module):

def__init__(self,num_classes=10):super(VGG16,self).__init__()#声明16个层：layer1~layer13、fc、fc1、fc2

self.layer1=nn.Sequential(nn.Conv2d(3,64,kernel_size=3,stride=1,padding=1),nn.BatchNorm2d(64),nn.ReLU())

6.1VGG及其应用VGG的结构利用VGG-16进行CIFAR-100分类self.layer2=nn.Sequential(nn.Conv2d(64,64,kernel_size=3,stride=1,padding=1),nn.BatchNorm2d(64),nn.ReLU(),

nn.MaxPool2d(kernel_size=2,stride=2))

self.layer3=nn.Sequential(nn.Conv2d(64,128,kernel_size=3,stride=1,padding=1),nn.BatchNorm2d(128),nn.ReLU())

####中间层略

6.1VGG及其应用VGG的结构利用VGG-16进行CIFAR-100分类

self.layer13=nn.Sequential(nn.Conv2d(512,512,kernel_size=3,stride=1,padding=1),nn.BatchNorm2d(512),nn.ReLU(),nn.MaxPool2d(kernel_size=2,stride=2))

self.fc=nn.Sequential(nn.Dropout(0.5),nn.Linear(7*7*512,4096),nn.ReLU())

self.fc1=nn.Sequential(nn.Dropout(0.5),nn.Linear(4096,4096),nn.ReLU())

self.fc2=nn.Sequential(nn.Linear(4096,num_classes))6.1VGG及其应用VGG的结构利用VGG-16进行CIFAR-100分类#定义前馈函数

defforward(self,x):out=self.layer1(x)out=self.layer2(out)out=self.layer3(out)out=self.layer4(out)out=self.layer5(out)out=self.layer6(out)out=self.layer7(out)out=self.layer8(out)out=self.layer9(out)out=self.layer10(out)out=self.layer11(out)out=self.layer12(out)out=self.layer13(out)out=out.reshape(out.size(0),-1)out=self.fc(out)out=self.fc1(out)out=self.fc2(out)

returnout6.2GoogLeNet及其应用VGG参数数量巨大，训练非常耗时。人类的视觉系统在信息处理时，信息在多个尺度上并行处理，然后再进行局部聚合。GoogLeNet通过多个尺度上并行处理，在增加网络深度和宽度、提升网络表达能力的同时，将计算保持在一个相对恒定的水平。GoogLeNet在多尺度信息处理时，为了减少内存消耗，采用了1*1大小的卷积运算。6.2GoogLeNet及其应用1*1卷积作用：降低维数，减少模型的参数（包括权重和偏置）数量。通过减少参数，还增加了模型的深度。假设要执行有48个卷积核的5*5卷积14*14*48014*14*485*548个卷积核（a）不含1*1卷积14*14*48014*14*485*548个卷积核（b）增加1*1卷积1*116个卷积核14*14*166.2GoogLeNet及其应用1*1卷积直接进行5*5卷积操作，参数数量为48*(5*5*480)=576000,卷积操作的操作总数为(14*14*48)*(5*5*480)=112.9M;若先进行1*1的卷积，再进行5*5卷积操作，参数数量为16*(1*1*480)+48*(5*5*16)=7680+19200=26880,卷积操作的操作总数为(14*14*16)*(1*1*480)+(14*14*48)*(5*5*16)=1.5M+3.8M=5.3M.显然，参数数量大幅减少，相应的，卷积操作的次数得到也大大减少。14*14*48014*14*485*548个卷积核（a）不含1*1卷积14*14*48014*14*485*548个卷积核（b）增加1*1卷积1*116个卷积核14*14*166.2GoogLeNet及其应用InceptionGoogLeNet使用Inception模块来实现多个尺度信息的并行处理，同时为了减少参数，在其改进版中使用了1*1卷积。堆叠1*1卷积3*3卷积5*5卷积1*1卷积1*1卷积1*1卷积3*3最大汇聚前一层堆叠28*28*641*1卷积28*28*1283*3卷积28*28*325*5卷积28*28*321*1卷积28*28*961*1卷积28*28*161*1卷积

28*28*1923*3最大汇聚64卷积核数量9616192128卷积核数量3232

28*28*(64+128+32+32)=28*28*25628*28*192前一层6.2GoogLeNet及其应用InceptionInception模块定义importtorch.nnasnn#卷积块定义classConvBlock(nn.Module):def__init__(self,in_channels,out_channels,kernel_size,**kwargs):super(ConvBlock,self).__init__()#

in_channels：输入通道；out_channels：输出信号通道；kerner_size：卷积核大小

self.conv=nn.Conv2d(in_channels,out_channels,kernel_size,**kwargs)#批量正则化：将不同批次数据拉回到均值为0，方差为1的正态度布上#一方面使得数据分布一致，另一方面避免梯度消失

self.bn=nn.BatchNorm2d(out_channels)

self.relu=nn.ReLU()defforward(self,x):

return

self.relu(self.bn(self.conv(x)))6.2GoogLeNet及其应用InceptionInception模块定义#Inception块定义classInceptionBlock(nn.Module):

def__init__(self,im_channels,num_1x1,num_3x3_red,num_3x3,num_5x5_red,num_5x5,num_pool_proj):super(InceptionBlock,self).__init__()#1*1卷积

self.one_by_one=ConvBlock(im_channels,num_1x1,kernel_size=1)#3*3卷积

self.three_by_three_red=ConvBlock(im_channels,num_3x3_red,kernel_size=1)

self.three_by_three=ConvBlock(num_3x3_red,num_3x3,kernel_size=3,padding=1)

6.2GoogLeNet及其应用InceptionInception模块定义#5*5卷积

self.five_by_five_red=ConvBlock(im_channels,num_5x5_red,kernel_size=1)

self.five_by_five=ConvBlock(num_5x5_red,num_5x5,kernel_size=5,padding=2)#3*3最大汇聚

self.maxpool=nn.MaxPool2d(kernel_size=3,stride=1,padding=1)

self.pool_proj=ConvBlock(im_channels,num_pool_proj,kernel_size=1)6.2GoogLeNet及其应用InceptionInception模块定义

defforward(self,x):x1=self.one_by_one(x)x2=self.three_by_three_red(x)x2=self.three_by_three(x2)x3=self.five_by_five_red(x)x3=self.five_by_five(x3)x4=self.maxpool(x)x4=self.pool_proj(x4)#堆叠x=torch.cat([x1,x2,x3,x4],1)

returnx6.2GoogLeNet及其应用InceptionInception模块的结构有多个改进版本InceptionV2用两个3*3的卷积代替5*5的卷积，用以降低参数量并减轻过拟合，这类似于VGG模型；批量正则化方法（batchnormalization）也是在InceptionV2中提出的。通过批量正则化，网络将不同批次数据拉回到均值为0，方差为1的正态度布上，并避免梯度消失。InceptionV3引入了卷积分解的思想，将一个较大的二维卷积拆成两个较小的一维卷积，比如将7*7卷积拆成1*7卷积和7*1卷积，或者将3*3卷积拆成1*3卷积和3*1卷积。卷积分解节约了大量参数，加速运算并减轻了过拟合，同时增加了一层非线性扩展模型表达能力。这种非对称的卷积结构拆分，其结果比对称地拆为几个相同的小卷积核效果更明显，可以处理更多、更丰富的空间特征，增加特征多样性。InceptionV4主要是结合了ResNet的主要思想，极大地加速了训练，同时性能也有所提升。6.2GoogLeNet及其应用GoogLeNet的结构共有22层网络包括9个Inception模块、5个汇聚层、两个卷积层、全连接层、分类器，以及两个辅助分类器。Inception模块被组织成3组分别是Inception3a、3b，Inception4a到4e，以及Inception5a、5b。输入最大汇聚3*3S=2卷积7*7S=2最大汇聚3*3S=2卷积3*3S=1Inceptions2ⅹ(3b)(3a)最大汇聚3*3S=2Inceptions5ⅹ(4e)(4d)(4c)(4b)(4a)最大汇聚3*3S=2Inceptions2ⅹ(5b)(5a)平均汇聚7*7S=1全连接层softmaxsoftmax2

平均汇聚5*5V=3卷积1*1S=1全连接层全连接层softmaxsoftmax1

平均汇聚5*5V=3卷积1*1S=1全连接层全连接层softmaxsoftmax0

4d输出4a输出6.2GoogLeNet及其应用GoogLeNet的结构两个辅助分类器一方面用于给网络增加监督信息，增加反向传播的梯度信号，从而提供了额外的正则化，减轻了梯度消失的影响；同时，通过将中间4a层和4d层的输出用作分类，并按一个较小的权重（0.3）加到最终分类结果中，起到模型融合的作用。而在实际测试的时候，这两个额外的分类器会被去掉。输入最大汇聚3*3S=2卷积7*7S=2最大汇聚3*3S=2卷积3*3S=1Inceptions2ⅹ(3b)(3a)最大汇聚3*3S=2Inceptions5ⅹ(4e)(4d)(4c)(4b)(4a)最大汇聚3*3S=2Inceptions2ⅹ(5b)(5a)平均汇聚7*7S=1全连接层softmaxsoftmax2

平均汇聚5*5V=3卷积1*1S=1全连接层全连接层softmaxsoftmax1

平均汇聚5*5V=3卷积1*1S=1全连接层全连接层softmaxsoftmax0

4d输出4a输出6.2GoogLeNet及其应用利用GoogLeNet进行TinyImageNet分类TinyImageNet又称为MicroImageNet，类似于ImageNet。200个训练类，每个类有500幅图像。包含10000幅图像；所有图像均为64*64彩色图像。dataset=torchvision.datasets.ImageFolder('./data/tiny-imagenet/tiny-imagenet-200/train',transform=transform)

train_data,val_data,test_data=torch.utils.data.random_split(dataset,[80000,10000,10000])

batch_size=32train_loader=DataLoader(train_data,batch_size=batch_size,shuffle=True,num_workers=4)val_loader=DataLoader(val_data,batch_size=batch_size,shuffle=False,num_workers=4)test_loader=DataLoader(test_data,batch_size=batch_size,shuffle=False,num_workers=4)transform=transforms.Compose([

transforms.Resize((224,224)),

transforms.ToTensor()])6.2GoogLeNet及其应用利用GoogLeNet进行TinyImageNet分类辅助分类器classAuxiliary(nn.Module):

def__init__(self,in_channels,num_classes):super(Auxiliary,self).__init__()

self.avgpool=nn.AvgPool2d(kernel_size=5,stride=3)

self.conv1x1=ConvBlock(in_channels,128,kernel_size=1)

self.fc1=nn.Linear(2048,1024)

self.fc2=nn.Linear(1024,num_classes)

self.dropout=nn.Dropout(0.7)

self.relu=nn.ReLU()defforward(self,x):x=self.avgpool(x)x=self.conv1x1(x)x=x.reshape(x.shape[0],-1)x=self.relu(self.fc1(x))x=self.dropout(x)x=self.fc2(x)

returnx6.2GoogLeNet及其应用利用GoogLeNet进行TinyImageNet分类网络结构classGoogLeNet(nn.Module):

def__init__(self,in_channels=3,use_auxiliary=True,num_classes=1000):super(Inception,self).__init__()

self.conv1=ConvBlock(in_channels,64,kernel_size=7,stride=2,padding=3)

self.conv2=ConvBlock(64,192,kernel_size=3,stride=1,padding=1)

self.maxpool=nn.MaxPool2d(kernel_size=3,stride=2,padding=1)

self.avgpool=nn.AvgPool2d(kernel_size=7,stride=1)

self.dropout=nn.Dropout(0.4)

self.linear=nn.Linear(1024,num_classes)

self.use_auxiliary=use_auxiliary

ifuse_auxiliary:

self.auxiliary4a=Auxiliary(512,num_classes)

self.auxiliary4d=Auxiliary(528,num_classes)

self.inception3a=InceptionBlock(192,64,96,128,16,32,32)

self.inception3b=InceptionBlock(256,128,128,192,32,96,64)

self.inception4a=InceptionBlock(480,192,96,208,16,48,64)

self.inception4b=InceptionBlock(512,160,112,224,24,64,64)

self.inception4c=InceptionBlock(512,128,128,256,24,64,64)

self.inception4d=InceptionBlock(512,112,144,288,32,64,64)

self.inception4e=InceptionBlock(528,256,160,320,32,128,128)

self.inception5a=InceptionBlock(832,256,160,320,32,128,128)

self.inception5b=InceptionBlock(832,384,192,384,48,128,128)

defforward(self,x):y=Nonez=None

x=self.conv1(x)x=self.maxpool(x)x=self.conv2(x)x=self.maxpool(x)

x=self.inception3a(x)x=self.inception3b(x)x=self.maxpool(x)

x=self.inception4a(x)

if

self.trainingand

self.use_auxiliary:y=self.auxiliary4a(x)

x=self.inception4b(x)x=self.inception4c(x)x=self.inception4d(x)

if

self.trainingand

self.use_auxiliary:z=self.auxiliary4d(x)

x=self.inception4e(x)x=self.maxpool(x)

x=self.inception5a(x)x=self.inception5b(x)x=self.avgpool(x)x=x.reshape(x.shape[0],-1)x=self.dropout(x)

x=self.linear(x)

returnx,y,z6.2GoogLeNet及其应用利用GoogLeNet进行TinyImageNet分类网络结构

self.inception3a=InceptionBlock(192,64,96,128,16,32,32)

self.inception3b=InceptionBlock(256,128,128,192,32,96,64)

self.inception4a=InceptionBlock(480,192,96,208,16,48,64)

self.inception4b=InceptionBlock(512,160,112,224,24,64,64)

self.inception4c=InceptionBlock(512,128,128,256,24,64,64)

self.inception4d=InceptionBlock(512,112,144,288,32,64,64)

self.inception4e=InceptionBlock(528,256,160,320,32,128,128)

self.inception5a=InceptionBlock(832,256,160,320,32,128,128)

self.inception5b=InceptionBlock(832,384,192,384,48,128,128)

defforward(self,x):y=Nonez=None

x=self.conv1(x)x=self.maxpool(x)x=self.conv2(x)x=self.maxpool(x)

x=self.inception3a(x)x=self.inception3b(x)x=self.maxpool(x)

x=self.inception4a(x)

if

self.trainingand

self.use_auxiliary:y=self.auxiliary4a(x)

x=self.inception4b(x)x=self.inception4c(x)x=self.inception4d(x)

if

self.trainingand

self.use_auxiliary:z=self.auxiliary4d(x)

x=self.inception4e(x)x=self.maxpool(x)

x=self.inception5a(x)x=self.inception5b(x)x=self.avgpool(x)x=x.reshape(x.shape[0],-1)x=self.dropout(x)

x=self.linear(x)

returnx,y,z6.2GoogLeNet及其应用利用GoogLeNet进行TinyImageNet分类网络结构defforward(self,x):y=Nonez=None

x=self.conv1(x)x=self.maxpool(x)x=self.conv2(x)x=self.maxpool(x)

x=self.inception3a(x)x=self.inception3b(x)x=self.maxpool(x)

x=self.inception4a(x)

if

self.trainingand

self.use_auxiliary:y=self.auxiliary4a(x)

x=self.inception4b(x)x=self.inception4c(x)x=self.inception4d(x)

if

self.trainingand

self.use_auxiliary:z=self.auxiliary4d(x)

x=self.inception4e(x)x=self.maxpool(x)

x=self.inception5a(x)x=self.inception5b(x)x=self.avgpool(x)x=x.reshape(x.shape[0],-1)x=self.dropout(x)

x=self.linear(x)

returnx,y,zx=self.inception4a(x)

if

self.trainingand

self.use_auxiliary:y=self.auxiliary4a(x)

x=self.inception4b(x)x=self.inception4c(x)x=self.inception4d(x)

if

self.trainingand

self.use_auxiliary:z=self.auxiliary4d(x)

x=self.inception4e(x)x=self.maxpool(x)

x=self.inception5a(x)x=self.inception5b(x)x=self.avgpool(x)x=x.reshape(x.shape[0],-1)x=self.dropout(x)

x=self.linear(x)

returnx,y,z6.2GoogLeNet及其应用利用GoogLeNet进行TinyImageNet分类训练#训练中使用辅助分类器optimizer.zero_grad()withtorch.set_grad_enabled(phase=='train'):

ifphase=='train':#使用辅助函数

ifuse_auxiliary:outputs,aux1,aux2=model(inputs)#计算损失函数值loss=criterion(outputs,labels)+0.3*criterion(aux1,labels)+0.3*criterion(aux2,labels)#不使用辅助函数

else:outputs,_,_=model(inputs)loss=criterion(outputs,labels)

_,preds=torch.max(outputs,1)loss.backward()optimizer.step()6.3ResNet及其应用何恺明等于2015年提出的一种深度卷积神经网络模型旨在解决网络深度增加带来的梯度消失问题可以包含大量的卷积层，通常在18到152之间，最多可达数千层。6.3ResNet及其应用残差块残差块是实现残差计算的功能模块，也是构成ResNet模型的基本模块。残差模块是在传统卷积神经网络模型的基础上添加了跨越多个卷积层的直连边来实现的。6.3ResNet及其应用

卷积卷积ReLUxReLUx

恒等函数

f(x)

f(x)+

x

6.3ResNet及其应用残差块残差块定义classBasicBlock(nn.Module):

def__init__(self,in_channels,out_channels,stride=1,expansion=1,downsample=None):super(BasicBlock,self).__init__()

self.expansion=expansion

self.downsample=downsample

self.conv1=nn.Conv2d(in_channels,out_channels,kernel_size=3,stride=stride,padding=1,bias=False)

self.bn1=nn.BatchNorm2d(out_channels)

self.relu=nn.ReLU(inplace=True)

self.conv2=nn.Conv2d(out_channels,out_channels*self.expansion,

kernel_size=3,padding=1,bias=False)

self.bn2=nn.BatchNorm2d(out_channels*self.expansion)6.3ResNet及其应用残差块残差块定义

defforward(self,x:Tensor):identity=xout=self.conv1(x)out=self.bn1(out)out=self.relu(out)out=self.conv2(out)out=self.bn2(out)

if

self.downsampleis

not

None:identity=self.downsample(x)out+=identityout=self.relu(out)

returnout6.3ResNet及其应用ResNet的结构以ResNet18为例，总共有18个带参数的层，从第一个卷积和汇聚层后，每两个卷积层构成一个残差块，总共8个残差块。6.3ResNet及其应用ResNet的结构定义classResNet(nn.Module):

def__init__(self,img_channelst,num_layers,

block:Type[BasicBlock],num_classes=1000):super(ResNet,self).__init__()

ifnum_layers==18:#列表layers列出基本块BasicBlock的个数layers=[2,2,2,2]

self.expansion=1

self.in_channels=64

#所有ResNets(18~152)首先包含Conv2d=>BN=>ReLU等三层，核大小为7

self.conv1=nn.Conv2d(in_channels=img_channels,out_channels=self.in_channels,kernel_size=7,

stride=2,padding=3,bias=False)

self.bn1=nn.BatchNorm2d(self.in_channels)

self.relu=nn.ReLU(inplace=True)

self.maxpool=nn.MaxPool2d(kernel_size=3,stride=2,padding=1)

self.layer1=self._make_layer(block,64,layers[0])

self.layer2=self._make_layer(block,128,layers[1],stride=2)

self.layer3=self._make_layer(block,256,layers[2],stride=2)

self.layer4=self._make_layer(block,512,layers[3],stride=2)

self.avgpool=nn.AdaptiveAvgPool2d((1,1))

self.fc=nn.Linear(512*self.expansion,num_classes)

def_make_layer(self,block:Type[BasicBlock],out_channels,blocks,stride:int=1):downsample=None

ifstride!=1:downsample=nn.Sequential(nn.Conv2d(

self.in_channels,

out_channels*self.expansion,kernel_size=1,stride=stride,bias=False

),nn.BatchNorm2d(out_channels*self.expansion),)layers=[]layers.append(block(

self.in_channels,out_channels,stride,self.expansion,downsample))

self.in_channels=out_channels*self.expansion

foriinrange(1,blocks):layers.append(block(

self.in_channels,out_channels,expansion=self.expansion))

returnnn.Sequential(*layers)

defforward(self,x:Tensor):x=self.conv1(x)x=self.bn1(x)x=self.relu(x)x=self.maxpool(x)x=self.layer1(x)x=self.layer2(x)x=self.layer3(x)x=self.layer4(x)#对所有ResNets，最后一层的空间维度应为7*7print('Dimensionsofthelastconvolutionalfeaturemap:',x.shape)x=self.avgpool(x)x=torch.flatten(x,1)x=self.fc(x)

returnx6.3ResNet及其应用ResNet的结构定义#所有ResNets(18~152)首先包含Conv2d=>BN=>ReLU等三层，核大小为7

self.conv1=nn.Conv2d(in_channels=img_channels,out_channels=self.in_channels,kernel_size=7,

stride=2,padding=3,bias=False)

self.bn1=nn.BatchNorm2d(self.in_channels)

self.relu=nn.ReLU(inplace=True)

self.maxpool=nn.MaxPool2d(kernel_size=3,stride=2,padding=1)

self.layer1=self._make_layer(block,64,layers[0])

self.layer2=self._make_layer(block,128,layers[1],stride=2)

self.layer3=self._make_layer(block,256,layers[2],stride=2)

self.layer4=self._make_layer(block,512,layers[3],stride=2)

self.avgpool=nn.AdaptiveAvgPool2d((1,1))

self.fc=nn.Linear(512*self.expansion,num_classes)

def_make_layer(self,block:Type[BasicBlock],out_channels,blocks,stride:int=1):downsample=None

ifstride!=1:downsample=nn.Sequential(nn.Conv2d(

self.in_channels,

out_channels*self.expansion,kernel_size=1,stride=stride,bias=False

),nn.BatchNorm2d(out_channels*self.expansion),)layers=[]layers.append(block(

self.in_channels,out_channels,stride,self.expansion,downsample))

self.in_channels=out_channels*self.expansion

foriinrange(1,blocks):layers.append(block(

self.in_channels,out_channels,expansion=self.expansion))

returnnn.Sequential(*layers)

defforward(self,x:Tensor):x=self.conv1(x)x=self.bn1(x)x=self.relu(x)x=self.maxpool(x)x=self.layer1(x)x=self.layer2(x)x=self.layer3(x)x=self.layer4(x)#对所有ResNets，最后一层的空间维度应为7*7print('Dimensionsofthelastconvolutionalfeaturemap:',x.shape)x=self.avgpool(x)x=torch.flatten(x,1)x=self.fc(x)

returnx6.3ResNet及其应用ResNet的结构定义self.layer1=self._make_layer(block,64,layers[0])

self.layer2=self._make_layer(block,128,layers[1],stride=2)

self.layer3=self._make_layer(block,256,layers[2],stride=2)

self.layer4=self._make_layer(block,512,layers[3],stride=2)

self.avgpool=nn.AdaptiveAvgPool2d((1,1))

self.fc=nn.Linear(512*self.expansion,num_classes)

def_make_layer(self,block:Type[BasicBlock],out_channels,blocks,stride:int=1):downsample=None

ifstride!=1:downsample=nn.Sequential(nn.Conv2d(

self.in_channels,

out_channels*self.expansion,kernel_size=1,stride=stride,bias=False

),nn.BatchNorm2d(out_channels*self.expansion),)layers=[]layers.append(block(

self.in_channels,out_channels,stride,self.expansion,downsample))

self.in_channels=out_channels*self.expansion

foriinrange(1,blocks):layers.append(block(

self.in_channels,out_channels,expansion=self.expansion))

returnnn.Sequential(*layers)

defforward(self,x:Tensor):x=self.conv1(x)x=self.bn1(x)x=self.relu(x)x=self.maxpool(x)x=self.layer1(x)x=self.layer2(x)x=self.layer3(x)x=self.layer4(x)#对所有ResNets，最后一层的空间维度应为7*7print('Dimensionsofthelastconvolutionalfeaturemap:',x.shape)x=self.avgpool(x)x=torch.flatten(x,1)x=self.fc(x)

returnx6.3ResNet及其应用ResNet的结构定义

def_make_layer(self,block:Type[BasicBlock],out_channels,blocks,stride:int=1):downsample=None

ifstride!=1:downsample=nn.Sequential(nn.Conv2d(

self.in_channels,

out_channels*self.expansion,kernel_size=1,stride=stride,bias=False

),nn.BatchNorm2d(out_channels*self.expansion),)layers=[]layers.append(block(

self.in_channels,out_channels,stride,self.expansion,downsample))

self.in_channels=out_channels*self.expansion

foriinrange(1,blocks):layers.append(block(

self.in_channels,out_channels,expansion=self.expansion))

returnnn.Sequential(*layers)

defforward(self,x:Tensor):x=self.conv1(x)x=self.bn1(x)x=self.relu(x)x=self.maxpool(x)x=self.layer1(x)x=self.layer2(x)x=self.layer3(x)x=self.layer4(x)#对所有ResNets，最后一层的空