LLaMA-Factory实战推理

LLaMA-Factory官网：https://github.com/hiyouga/LLaMA-Factory

安装环境

git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory/
conda create -n py310 python=3.10
conda activate py310

按照llama-factory要求的标准格式组织数据集，保存成一个文件，比如下面这种.json文件：

[
  {
    "instruction": "user instruction (required)",
    "input": "user input (optional)",
    "output": "model response (required)",
    "system": "system prompt (optional)",
    "history": [
      ["user instruction in the first round (optional)", "model response in the first round (optional)"],
      ["user instruction in the second round (optional)", "model response in the second round (optional)"]
    ]
  }
]

大模型选择：

LLaMA、Mistral、Mixtral-MoE、Qwen、Yi、Gemmha、Baichuan、ChatGLM、Phi

推理模型

CUDA_VISIBLE_DEVICES=0 python cli_demo.py \--model_name_or_path path_to_llama_model \--adapter_name_or_path path_to_checkpoint \--template default \--finetuning_type lora

总结：

有效的微调已成为大型语言模型适应特定任务的必要条件之一。随着 Llama-Factory 的引入，这一全面的框架让训练更加高效，用户无需编写代码即可轻松为超过 100 个 LLMs 定制微调。