学习pytorch12 神经网络-线性层

官方文档

https://pytorch.org/docs/stable/nn.html#normalization-layers

自行了解的网络层

归一化层 Normalization

百度 弹幕 都说Normalization是归一化层【标准化层】，regulation才是正则化层

https://blog.csdn.net/weixin_40895135/article/details/130651226

循环层 Recurrent

多用于文字识别

Transformer

Alphafold

https://zhuanlan.zhihu.com/p/393699764?utm_medium=social&utm_oi=1144761973647597568

Dropout

目的： 防止过拟合

Sparse 稀疏层

https://blog.csdn.net/qq_51870267/article/details/126413057

只计算非零特征，对输入图像降维

Distence 距离函数

计算两个值的距离 衡量两个值之间的误差

Loss 损失函数

根据损失函数 可以判断模型的好坏，也可以以此去调整模型
损失函数越小，模型越好 L(f(x), Y)

线性层

code

import torch
import torchvision
from torch import nn
from torch.nn import Linear
from torch.utils.data import DataLoader

test_dataset = torchvision.datasets.CIFAR10(root='./dataset', train=False, transform=torchvision.transforms.ToTensor(),
                                            download=True)

dataloader = DataLoader(dataset=test_dataset, batch_size=64, shuffle=False, drop_last=True)

class MyLinear(nn.Module):
    def __init__(self):
        super(MyLinear, self).__init__()
        self.linear1 = Linear(196608, 10)
    def forward(self, input):
        output = self.linear1(input)
        return output

mylinear = MyLinear()


for data in dataloader:
    imgs, targs = data
    print(imgs.shape)
    input = torch.reshape(imgs, (1,1,1,-1))  # torch.Size([1, 1, 1, 196608])   # torch.Size([1, 1, 1, 10])
    # input = torch.reshape(imgs, (-1,))  # torch.Size([196608])  # torch.Size([10])
    # input = torch.flatten(imgs)  # torch.Size([196608])         # torch.Size([10])
    print(input.shape)
    output = mylinear(input)  # torch.Size([10])  # torch.Size([1, 1, 1, 10])
    print(output.shape)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

running

torch.Size([64, 3, 32, 32])
torch.Size([1, 1, 1, 196608])
torch.Size([1, 1, 1, 10])

# flatten
torch.Size([64, 3, 32, 32])
torch.Size([196608])
torch.Size([10])
1
2
3
4
5
6
7
8