• 国产化框架PaddleClas结合Swanlab进行杂草分类

    1. 项目介绍

    杂草是农业中的主要问题之一，对作物生长和产量造成严重威胁。传统的手动识别和管理方式效率低下且不够精确，因此需要借助先进的计算机视觉技术来提升农业生产的效率和质量。ResNet作为一种深度学习模型，在处理复杂的图像分类任务中表现出色，不仅可以有效解决农田中杂草多样化、形态复杂的问题，还能够推动农业智能化发展，减少对化学农药的依赖，实现农业可持续发展的目标。通过利用ResNet进行杂草分类，可以为农民提供更智能、精确的农业管理方案，促进农业生产效率的提升和农业产业的现代化进程。因此，本项目使用国产化框架PaddleClas+Swanlab+Gradio+Swanhub进行杂草分类实验。

    PaddlePaddle（飞桨）是百度开发的企业级深度学习平台，旨在支持从模型开发到部署的全流程深度学习应用。它提供了丰富的工具和库，支持多种深度学习任务，包括图像处理、自然语言处理、语音识别等。PaddlePaddle

    PaddleClas是Paddle框架中专门用于图像分类任务的工具库。它提供了一整套端到端的解决方案，包括数据处理、模型定义、训练、评估和部署，旨在帮助开发者快速构建和部署高效的图像分类模型。PaddleClas

    SwanLab是一款开源、轻量级的AI实验跟踪工具，通过提供友好的API，结合超参数跟踪、指标记录、在线协作等功能，提高ML实验跟踪和协作体验。欢迎使用SwanLab | SwanLab官方文档

    Swanhub是由极客工作室开发的一个开源模型协作分享社区。它为AI开发者提供了AI模型托管、训练记录、模型结果展示、API快速部署等功能。欢迎使用Swanhub

    Gradio是一个开源的Python库，旨在帮助数据科学家、研究人员和从事机器学习领域的开发人员快速创建和共享用于机器学习模型的用户界面。Gradio

     2. 准备部分

    2.1 环境安装

    安装以下3个库:

    1. paddle
    2. swanlab
    3. gradio

    安装命令：

    pip install paddle swanlab gradio

    2.2 下载数据集

    杂草分类数据集：DeepWeeds

    1. DeepWeeds
    2. --images
    3. ----1.jpg
    4. ----2.jpg
    5. --train.txt
    6. --val.txt
    7. --test.txt
    8. --classnames.txt

    它们各自的作用与意义：

    1.  DeepWeeds文件夹：该文件夹用于存储图片文件夹images、训练集测试集验证集文件和标签文件

    2.  images文件夹：该文件夹用于保存训练、测试、验证图片文件夹。

    3.  train.txt、val.txt、test.txt文件：该文件用于保存训练、测试、验证集图片路径与类别。

    4.  classnames文件：用于保存类别标签

    2.3 下载PaddleClas框架

    模型链接：PaddleClas模型

    解压后得到PaddleClas文件夹。

    2.4 创建文件目录

    在PaddleClas文件夹中创建app.py。

    它的作用是：运行运行Gradio Demo的脚本

    3. ResNet模型训练

    3.1 修改配置

    首先在PaddleClas文件夹中找到ppcls-->configs-->ImageNet-->Res2Net-->Res2Net50_14w_8s.yaml。

    分别修改epochs为100，类别class_num为9，训练图片路径，验证图片路径和标签文件。一共修改7处。

    1. # global configs
    2. Global:
    3.   checkpoints: null
    4.   pretrained_model: null
    5.   output_dir: ./output/
    6.   device: gpu
    7.   save_interval: 1
    8.   eval_during_train: True
    9.   eval_interval: 1
    10.   epochs: 100###########################1##############################
    11.   print_batch_step: 10
    12.   use_visualdl: False
    13.   # used for static mode and model export
    14.   image_shape: [3, 224, 224]
    15.   save_inference_dir: ./inference
    16.  
    17. # model architecture
    18. Arch:
    19.   name: Res2Net50_14w_8s
    20.   class_num: 9############################2##############################
    21.  
    22. # loss function config for traing/eval process
    23. Loss:
    24.   Train:
    25.     - CELoss:
    26.         weight: 1.0
    27.         epsilon: 0.1
    28.   Eval:
    29.     - CELoss:
    30.         weight: 1.0
    31.  
    32.  
    33. Optimizer:
    34.   name: Momentum
    35.   momentum: 0.9
    36.   lr:
    37.     name: Cosine
    38.     learning_rate: 0.1
    39.   regularizer:
    40.     name: 'L2'
    41.     coeff: 0.0001
    42.  
    43.  
    44. # data loader for train and eval
    45. DataLoader:
    46.   Train:
    47.     dataset:
    48.       name: ImageNetDataset
    49.       image_root: ./weeds/images/#################3#######################
    50.       cls_label_path: ./weeds/train.txt###########4########################
    51.       transform_ops:
    52.         - DecodeImage:
    53.             to_rgb: True
    54.             channel_first: False
    55.         - RandCropImage:
    56.             size: 224
    57.         - RandFlipImage:
    58.             flip_code: 1
    59.         - NormalizeImage:
    60.             scale: 1.0/255.0
    61.             mean: [0.485, 0.456, 0.406]
    62.             std: [0.229, 0.224, 0.225]
    63.             order: ''
    64.       batch_transform_ops:
    65.         - MixupOperator:
    66.             alpha: 0.2
    67.  
    68.     sampler:
    69.       name: DistributedBatchSampler
    70.       batch_size: 64
    71.       drop_last: False
    72.       shuffle: True
    73.     loader:
    74.       num_workers: 4
    75.       use_shared_memory: True
    76.  
    77.   Eval:
    78.     dataset: 
    79.       name: ImageNetDataset
    80.       image_root: ./DeepWeeds/images/###############5#######################
    81.       cls_label_path: ./DeepWeeds/val.txt###########6########################
    82.       transform_ops:
    83.         - DecodeImage:
    84.             to_rgb: True
    85.             channel_first: False
    86.         - ResizeImage:
    87.             resize_short: 256
    88.         - CropImage:
    89.             size: 224
    90.         - NormalizeImage:
    91.             scale: 1.0/255.0
    92.             mean: [0.485, 0.456, 0.406]
    93.             std: [0.229, 0.224, 0.225]
    94.             order: ''
    95.     sampler:
    96.       name: DistributedBatchSampler
    97.       batch_size: 64
    98.       drop_last: False
    99.       shuffle: False
    100.     loader:
    101.       num_workers: 4
    102.       use_shared_memory: True
    103.  
    104. Infer:
    105.   infer_imgs: docs/images/inference_deployment/whl_demo.jpg
    106.   batch_size: 10
    107.   transforms:
    108.     - DecodeImage:
    109.         to_rgb: True
    110.         channel_first: False
    111.     - ResizeImage:
    112.         resize_short: 256
    113.     - CropImage:
    114.         size: 224
    115.     - NormalizeImage:
    116.         scale: 1.0/255.0
    117.         mean: [0.485, 0.456, 0.406]
    118.         std: [0.229, 0.224, 0.225]
    119.         order: ''
    120.     - ToCHWImage:
    121.   PostProcess:
    122.     name: Topk
    123.     topk: 5
    124.     class_id_map_file: ./DeepWeeds/classnaems.txt###########7##################
    125.  
    126. Metric:
    127.   Train:
    128.   Eval:
    129.     - TopkAcc:
    130.         topk: [1, 5]

    3.2 使用Swanlab

    在PaddleClas文件夹中找tools-->train.py。初始化swanlab

    1. # Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
    2.  #
    3.  # Licensed under the Apache License, Version 2.0 (the "License");
    4.  # you may not use this file except in compliance with the License.
    5.  # You may obtain a copy of the License at
    6.  #
    7.  #     http://www.apache.org/licenses/LICENSE-2.0
    8.  #
    9.  # Unless required by applicable law or agreed to in writing, software
    10.  # distributed under the License is distributed on an "AS IS" BASIS,
    11.  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    12.  # See the License for the specific language governing permissions and
    13.  # limitations under the License.
    14.  
    15. from __future__ import absolute_import
    16. from __future__ import division
    17. from __future__ import print_function
    18. import os
    19. import sys
    20.  
    21.  __dir__ = os.path.dirname(os.path.abspath(__file__))
    22.  sys.path.append(os.path.abspath(os.path.join(__dir__, '../')))
    23.  
    24. from ppcls.utils import config
    25. from ppcls.engine.engine import Engine
    26. import swanlab
    27. # -*- coding: utf-8 -*-
    28.  
    29. if __name__ == "__main__":
    30.     args = config.parse_args()
    31.     config = config.get_config(
    32.         args.config, overrides=args.override, show=False)
    33.     config.profiler_options = args.profiler_options
    34.     engine = Engine(config, mode="train")
    35.     
    36.     ## 初始化swanlab
    37.     swanlab.init(
    38.         experiment_name="Swanlab_ResNet50_PaddleClas",
    39.         description="Train ResNet50 for weeds classification.",
    40.         project="Swanhub_Weeds_Classification",
    41.         config={
    42.             "model": "ResNet50",
    43.             "optim": "Adam",
    44.             "lr": 0.001,
    45.             "batch_size": 64,
    46.             "num_epochs": 100,
    47.             "num_class": 9,
    48.         }
    49.     )
    50.     engine.train()

    在PaddleClas中找ppcls-->engine-->train-->utils.py中添加以下代码：

    1. swanlab.log({"train_lr_msg": lr_msg.split(": ")[1]})  #
    2. swanlab.log({"train_CELoss": metric_msg.split(",")[0].split(': ')[1]})  ##
    3. swanlab.log({'train_loss': metric_msg.split(",")[1].split(': ')[1]})

    在PaddleClas文件夹中找ppcls-->engine-->engine.py，添加以下代码：

    swanlab.log({'best_metric': best_metric.get('metric')})

    3.3 模型训练

    在控制台输入以下命令：

    python -m paddle.distributed.launch tools/train.py -c ./ppcls/configs/ImageNet/Res2Net/Res2Net50_14w_8s.yaml

    在swanlab查看实验详细信息

    实验结果如下：

    swanlab查看实验结果

    3.4 模型推理

    将以下代码输入控制台：

    python tools/infer.py -c ./ppcls/configs/ImageNet/Res2Net/Res2Net50_14w_8s.yaml -o Infer.infer_imgs=./DeepWeeds/infer/01.jpg -o Global.pretrained_model=./output/Res2Net50_14w_8s/best_model

    4. DarkNet53模型训练

    4.1 修改配置

    首先在PaddleClas文件夹中找到ppcls-->configs-->ImageNet-->DarkNet-->DarkNet53.yaml。

    分别修改epochs为100，类别class_num为9，训练图片路径，验证图片路径和标签文件。一共修改7处。

    1. # global configs
    2. Global:
    3.   checkpoints: null
    4.   pretrained_model: null
    5.   output_dir: ./output/
    6.   device: gpu
    7.   save_interval: 1
    8.   eval_during_train: True
    9.   eval_interval: 1
    10.   epochs: 100
    11.   print_batch_step: 10
    12.   use_visualdl: False
    13.   # used for static mode and model export
    14.   image_shape: [3, 256, 256]
    15.   save_inference_dir: ./inference
    16.  
    17. # model architecture
    18. Arch:
    19.   name: DarkNet53
    20.   class_num: 9
    21.  
    22. # loss function config for traing/eval process
    23. Loss:
    24.   Train:
    25.     - CELoss:
    26.         weight: 1.0
    27.         epsilon: 0.1
    28.   Eval:
    29.     - CELoss:
    30.         weight: 1.0
    31.  
    32.  
    33. Optimizer:
    34.   name: Momentum
    35.   momentum: 0.9
    36.   lr:
    37.     name: Cosine
    38.     learning_rate: 0.1
    39.   regularizer:
    40.     name: 'L2'
    41.     coeff: 0.0001
    42.  
    43.  
    44. # data loader for train and eval
    45. DataLoader:
    46.   Train:
    47.     dataset:
    48.       name: ImageNetDataset
    49.       image_root: F:/datasets/DeepWeeds/images
    50.       cls_label_path: F:/datasets/DeepWeeds/train.txt
    51.       transform_ops:
    52.         - DecodeImage:
    53.             to_rgb: True
    54.             channel_first: False
    55.         - RandCropImage:
    56.             size: 256
    57.         - RandFlipImage:
    58.             flip_code: 1
    59.         - NormalizeImage:
    60.             scale: 1.0/255.0
    61.             mean: [0.485, 0.456, 0.406]
    62.             std: [0.229, 0.224, 0.225]
    63.             order: ''
    64.       batch_transform_ops:
    65.         - MixupOperator:
    66.             alpha: 0.2
    67.  
    68.     sampler:
    69.       name: DistributedBatchSampler
    70.       batch_size: 64
    71.       drop_last: False
    72.       shuffle: True
    73.     loader:
    74.       num_workers: 4
    75.       use_shared_memory: True
    76.  
    77.   Eval:
    78.     dataset: 
    79.       name: ImageNetDataset
    80.       image_root: F:/datasets/DeepWeeds/images
    81.       cls_label_path: F:/datasets/DeepWeeds/val.txt
    82.       transform_ops:
    83.         - DecodeImage:
    84.             to_rgb: True
    85.             channel_first: False
    86.         - ResizeImage:
    87.             resize_short: 292
    88.         - CropImage:
    89.             size: 256
    90.         - NormalizeImage:
    91.             scale: 1.0/255.0
    92.             mean: [0.485, 0.456, 0.406]
    93.             std: [0.229, 0.224, 0.225]
    94.             order: ''
    95.     sampler:
    96.       name: DistributedBatchSampler
    97.       batch_size: 64
    98.       drop_last: False
    99.       shuffle: False
    100.     loader:
    101.       num_workers: 4
    102.       use_shared_memory: True
    103.  
    104. Infer:
    105.   infer_imgs: docs/images/inference_deployment/whl_demo.jpg
    106.   batch_size: 10
    107.   transforms:
    108.     - DecodeImage:
    109.         to_rgb: True
    110.         channel_first: False
    111.     - ResizeImage:
    112.         resize_short: 292
    113.     - CropImage:
    114.         size: 256
    115.     - NormalizeImage:
    116.         scale: 1.0/255.0
    117.         mean: [0.485, 0.456, 0.406]
    118.         std: [0.229, 0.224, 0.225]
    119.         order: ''
    120.     - ToCHWImage:
    121.   PostProcess:
    122.     name: Topk
    123.     topk: 5
    124.     class_id_map_file: F:/datasets/DeepWeeds/classnames
    125.  
    126. Metric:
    127.   Train:
    128.   Eval:
    129.     - TopkAcc:
    130.         topk: [1, 5]

    4.2 使用Swanlab

    在PaddleClas文件夹中找tools-->train.py。修改初始化swanlab

    1. # Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
    2.  #
    3.  # Licensed under the Apache License, Version 2.0 (the "License");
    4.  # you may not use this file except in compliance with the License.
    5.  # You may obtain a copy of the License at
    6.  #
    7.  #     http://www.apache.org/licenses/LICENSE-2.0
    8.  #
    9.  # Unless required by applicable law or agreed to in writing, software
    10.  # distributed under the License is distributed on an "AS IS" BASIS,
    11.  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    12.  # See the License for the specific language governing permissions and
    13.  # limitations under the License.
    14.  
    15. from __future__ import absolute_import
    16. from __future__ import division
    17. from __future__ import print_function
    18. import os
    19. import sys
    20.  
    21.  __dir__ = os.path.dirname(os.path.abspath(__file__))
    22.  sys.path.append(os.path.abspath(os.path.join(__dir__, '../')))
    23.  
    24. from ppcls.utils import config
    25. from ppcls.engine.engine import Engine
    26. import swanlab
    27. # -*- coding: utf-8 -*-
    28.  
    29. if __name__ == "__main__":
    30.     args = config.parse_args()
    31.     config = config.get_config(
    32.         args.config, overrides=args.override, show=False)
    33.     config.profiler_options = args.profiler_options
    34.     engine = Engine(config, mode="train")
    35.     
    36.     ## 初始化swanlab
    37.     swanlab.init(
    38.         experiment_name="Swanlab_DrakNet53_PaddleClas",
    39.         description="Train DarkNet53 for weeds classification.",
    40.         project="Swanhub_Weeds_Classification",
    41.         config={
    42.             "model": "DarkNet53",
    43.             "optim": "Adam",
    44.             "lr": 0.001,
    45.             "batch_size": 64,
    46.             "num_epochs": 100,
    47.             "num_class": 9,
    48.         }
    49.     )
    50.     engine.train()

    4.3 模型训练

    在控制台输入以下命令：

    python -m paddle.distributed.launch tools/train.py -c ./ppcls/configs/ImageNet/DarkNet/DarknetNet53.yaml

    在swanlab查看实验详细信息

    实验结果如下：

    4.4 模型推理

    将以下代码输入控制台：

    python tools/infer.py -c ./ppcls/configs/ImageNet/DarkNet/DarkNet53.yaml -o Infer.infer_imgs=./DeepWeeds/infer/01.jpg -o Global.pretrained_model=./output/DarkNet53/best_model

    4.5 Swanlab结果展示

    从图中可以看出，ResNet50模型的效果要比DarkNet53模型好，swanlab提供了便利的对比图表功能。

    5. Gradio演示

    未完待续。。。

    6. Swanhub上传并演示demo

    未完待续。。。

  • 相关阅读:
    dp(1) - 数字三角形模型
    高通WLAN框架学习（33）-- WLAN/P2P get IOCTLs（iwpriv）命令大全
    LCD12864驱动开发
    ArrayList集合&学生管理系统
    基于SSH开发网络在线考试系统
    Spring 中的 Bean
    Jsoup解析XML文件
    C# 基类中的虚函数调用基类的虚函数执行的是派生类实现的对应函数吗
    【右击打开cmd】
    使用 Python 的基于边缘和基于区域的分割
  • 原文地址：https://blog.csdn.net/weixin_44813538/article/details/140282337