机器学习从原理到应用Chp10-Pytorch基础

Pytorch

基础什么是PyTorch?一种基于Python语言的深度学习开发框架.特征:使用Pytorch可实现在GPU上进行N维张量的计算操作对训练网络自动进行求导Tensors—张量High-dimensional

matrices

(arrays)1-D

tensore.g.

audio2-D

tensore.g.

grayimages3-D

tensore.g.

RGB

imagesTensors

–

Shape

of

TensorsCheck

with

.shape()(5,)5(3,5)5(4,5,3)dim

0

dim

1

dim

0Note:

dim

in

PyTorch

==

axis

inNumPydim

0dim

1dim

25343Tensors

–

创建Tensors从已有数据创建(listor

numpy.ndarray)x=torch.tensor([[1,-1],[-1,1]])x=torch.from_numpy(np.array([[1,-1],[-1,1]]))Tensorof

constant

zeros

&

onesx=torch.zeros([2,2])x=torch.ones([1,2,5])shapetensor([[0.,0.],[0.,0.]])tensor([[[1.,1.,1.,1.,1.],[1.,1.,1.,1.,1.]]])tensor([[1.,-1.],[-1.,1.]])Addition

●Summationy=x.sum()Meany

=x.mean()z=x+ySubtractionz=x-yPowery

=x.pow(2)Pytorch支持一些常见的数学运算，如:Tensors

–

常见的操作Tensors

–常见的操作Transpose:

交换张量中两个指定的维度>>>x=torch.zeros([2,3])2>>>x.shapetorch.Size([2,3])>>>x=x.transpose(0,1)>>>x.shapetorch.Size([3,2])323Tensors

–常见的操作Permute:

交换张量中任意的维度>>>x=torch.zeros([2,3,

4,

5])>>>x.shapetorch.Size([2,3,4,5])>>>x=x.permute(3,1,0,2).contiguous()>>>x.shapetorch.Size([5,3,2,4])Tensors

–

Common

OperationsSqueeze:

删除张量中指定的大小为1的维度>>>x=torch.zeros([1,2,3])>>>x.shapetorch.Size([1,2,3])torch.Size([2,3])>>>x=x.squeeze(0)(dim

=

0)>>>x.shape12323Tensors

–常见的操作Unsqueeze:

扩充一个新维度>>>x=torch.zeros([2,3])>>>x.shapetorch.Size([2,3])>>>x=x.unsqueeze(1)>>>x.shapetorch.Size([2,1,3])23213(dim

=

1)Tensors

–常见的操作Cat:

将多个张量沿着指定维度进行拼接>>>x=torch.zeros([2,1,3])>>>y=torch.zeros([2,3,3])>>>z=torch.zeros([2,2,3])>>>w=torch.cat([x,y,z],dim=1)>>>w.shapetorch.Size([2,6,3])213x233y223z236wmore

operators:

/docs/stable/tensors.htmlTensors

–

数据类型Data

typedtypetensor32-bit

floating

pointtorch.floattorch.FloatTensor64-bit

integer

(signed)torch.longtorch.LongTensorseeofficial

documentation

for

more

information

ondata

types.Using

different

data

types

for

model

and

data

will

cause

errors.Tensors

–

PyTorch

v.s.NumPy相似的属性PyTorchNumPyx.shapex.shapex.dtypex.dtyperef:

/wkentaro/pytorch-for-numpy-usersseeofficial

documentation

for

more

information

ondata

types.Tensors

–

PyTorch

v.s.NumPy许多函数具有相同的名字PyTorchNumPyx.reshape/x.viewx.reshapex.squeeze()x.squeeze()x.unsqueeze(1)np.expand_dims(x,1)ref:

/wkentaro/pytorch-for-numpy-usersTensors

–

Device（GPUorCPU）创建的张量以及模型默认在CPU上进行相关计算使用

.to()可将张量以及模型移动到指定的设备上.CPUx=x.to(‘cpu’)GPUx

=x.to(‘cuda’)Tensors

–

Device

(GPU)Check

if

your

computer

has

NVIDIA

GPUtorch.cuda.is_available()Multiple

GPUs:

specify

‘cuda:0’,

‘cuda:1’,

‘cuda:2’,

...Why

useGPUs?Parallel

computing

with

more

cores

for

arithmetic

calculationsSee

What

is

a

GPU

and

do

you

need

one

in

deep

learning?Tensors

–

梯度计算>>>z=x.pow(2).sum()>>>z.backward()>>>x.gradtensor([[2.,

0.],

[-2.,

2.]])1231

>>>

x=torch.tensor([[1.,0.],[-1.,1.]],

requires_grad=True)2344See

here

to

learn

about

gradient

calculation.训练网络任务构建TrainingDefine

NeuralNetworkLoss

FunctionOptimizationAlgorithmMore

info

about

the

training

process

in

last

year's

lecture

video.训练

&

测试神经网络ValidationTestingTrainingGuide

for

training/validation/testing

can

be

found

here.重复N次(Epochs)训练

&

测试神经网络步骤-

in

PytorchStep1.torch.utils.data.Dataset&torch.utils.data.DataLoaderLoss

FunctionOptimizationAlgorithmTrainingValidationTestingLoadDataDefine

NeuralNetworkDataset

&

DataloaderDataset:

存储数据的每一个样本及其真值（ground-truth）Dataloader:

将多个样本打包成一个batchdataset=MyDataset(file)dataloader=DataLoader(dataset,batch_size,shuffle=True)Training:

TrueTesting:

FalseMore

info

about

batches

and

shuffling

here.Dataset

&

Dataloaderfrom

torch.utils.dataimport

Dataset,

DataLoaderclassMyDataset(Dataset):def

init

(self,file):self.data=...def

__getitem__(self,index):returnself.data[index]def

__len__(self):returnlen(self.data)Read

data

&

preprocessReturns

one

sampleat

atimeReturns

the

size

of

the

datasetDataset

&

Dataloaderdataset

=MyDataset(file)dataloader

=

DataLoader(dataset,

batch_size=5,

shuffle=False)Datasetmini-batchDataLoader

getitem

(0)

getitem

(1)

getitem

(2)

getitem

(3)

getitem

(4)batch_size01234训练

&

测试神经网络步骤–

in

PytorchLoss

FunctionOptimizationAlgorithmTrainingValidationTestingStep2.torch.nn.ModuleLoadDataDefine

NeuralNetworktorch.nn

–

网络层Linear

Layer

(Fully-connected

Layer)nn.Linear(in_features,out_features)Input

Tensor*x32Output

Tensor*x64nn.Linear(32,64)can

be

any

shape

(but

last

dimension

must

be

32)e.g.

(10,

32),

(10,

5,

32),

(1,

1,

3,

32),

...torch.nn

–

网络层Linear

Layer

(Fully-connected

Layer)ref:

last

year's

lecture

videotorch.nn

–

网络层Linear

Layer

(Fully-connected

Layer)x2x1x3x32y2y1y3y643264......W(64x32)yxx=b+torch.nn

–

网络参数Linear

Layer

(Fully-connected

Layer)>>>layer=torch.nn.Linear(32,64)>>>layer.weight.shapetorch.Size([64,32])>>>layer.bias.shapetorch.Size([64])W(64x32)yxx=b+torch.nn

–

非线性激活函数Sigmoid

Activationnn.Sigmoid()ReLU

Activationnn.ReLU()See

here

to

learnaboutwhy

weneedactivation

functions.torch.nn

–

自定义一个神经网络importtorch.nnasnnclassMyModel(nn.Module):def

init

(self):super(MyModel,self).

init

()=nn.Sequential(nn.Linear(10,32),nn.Sigmoid(),nn.Linear(32,1))defforward(self,x):return(x)Initialize

your

model

&

define

layersCompute

output

of

yourNNtorch.nn

–自定义一个神经网络importtorch.nnasnn

importtorch.nnasnnclassMyModel(nn.Module):def

init

(self):super(MyModel,self).

init

()=nn.Sequential(nn.Linear(10,32),nn.Sigmoid(),nn.Linear(32,1))defforward(self,x):return(x)Class

MyModel(nn.Module):def

init

(self):super(MyModel,self).

init

()self.layer1

=

nn.Linear(10,32)self.layer2=nn.Sigmoid(),self.layer3=nn.Linear(32,1)defforward(self,x):

out=self.layer1(x)out=self.layer2(out)out=self.layer3(out)returnout=训练

&

测试神经网络步骤

–

in

PytorchLoss

FunctionOptimizationAlgorithmTrainingValidationTestingStep3.torch.nn.MSELosstorch.nn.CrossEntropyLossetc.LoadDataDefine

NeuralNetworktorch.nn

–

损失函数Mean

Squared

Error

(for

regression

tasks)criterion=nn.MSELoss()Cross

Entropy

(for

classification

tasks)criterion=nn.CrossEntropyLoss()loss=criterion(model_output,ground_truth_value)训练

&

测试神经网络步骤

–

in

PytorchDefine

NeuralNetworkLoss

FunctionOptimizationAlgorithmTrainingValidationTestingStep4.torch.optimLoadDatatorch.optim包含了基于梯度下降优化网络参数以减小预测损失的算法.

(See

Adaptive

Learning

Rate

lecturevideo)E.g.

Stochastic

Gradient

Descent

(SGD)—随机梯度下降torch.optim.SGD(model.parameters(),lr,momentum=0)torch.optimoptimizer

=torch.optim.SGD(model.parameters(),lr,momentum=0)对于训练迭代中的每一个batch的数据:调用

optimizer.zero_grad()

将参数的梯度重置为0.调用

loss.backward()对预测得到的损失进行方向传播来计算参数梯度.调用optimizer.step()根据梯度值来调整模型参数.

See

official

documentation

for

more

optimization

algorithms.训练&测试神经网络步骤5

–

in

PytorchDefine

NeuralNetworkLoss

FunctionOptimizationAlgorithmTrainingValidationTestingStep5.EntireProcedureLoadDataNeural

Network

Training

Setupdataset=MyDataset(file)tr_set=DataLoader(dataset,16,shuffle=True)model=MyModel().to(device)criterion=nn.MSELoss()optimizer=torch.optim.SGD(model.parameters(),0.1)readdataviaMyDatasetputdatasetintoDataloaderconstructmodelandmovetodevice(cpu/cuda)setlossfunctionsetoptimizer循环训练过程forepochinrange(n_epochs):model.train()forx,yintr_set:optimizer.zero_grad()x,y=x.to(device),y.to(device)pred=model(x)loss=criterion(pred,y)loss.backward()optimizer.step()iteraten_epochssetmodeltotrainmodeiteratethroughthedataloadersetgradienttozeromovedatatodevice(cpu/cuda)forwardpass(computeoutput)computelosscomputegradient(backpropagation)updatemodelwithoptimizer循环验证过程model.eval()total_loss=0forx,yindv_set:x,y=x.to(device),y.to(device)withtorch.no_grad():pred=model(x)loss=criterion(pred,y)total_loss+=loss.cpu().item()*len(x)avg_loss=total_loss/len(dv_set.dataset)setmodeltoevaluationmodeiteratethroughthedataloadermovedatatodevice(cpu/cuda)disablegradientcalculationforwardpass(computeoutput)computelossaccumulatelosscomputeaveragedloss循环测试过程model.eval()preds=[]forxintt_set:x=x.to(device)withtorch.no_grad():pred=model(x)preds.append(pred.cpu())setmodeltoevaluationmodeiteratethrough