ContextQMD
Back to Qwen

Fine-tuning Qwen-7B-Chat on Your Own Domain-Specific Data

recipes/applications/domain_finetune/qwen_domain_finetune.ipynb

latest10.7 KB
Original Source

Fine-tuning Qwen-7B-Chat on Your Own Domain-Specific Data

This notebook uses Qwen-7B-Chat as an example to introduce how to LoRA fine-tune the Qwen model on a specific domain.

Qwen is a pretrained conversational model that supports English and Chinese. It is suitable for universal scenarios, but may lack some specialized knowledge in certain specific domain. If you would like to fine-tune it for a specific domain, or on your own private dataset, you can refer to this tutorial.

Here is an example showing the differences before and after fine-tuning.

Model Input

怎样治疗青光眼?

Before Fine-tuning

青光眼的治疗方法取决于病情严重程度。轻度青光眼可以采取药物治疗，比如激素类药物、抗生素等，以减轻炎症和改善血管紧张。

中度青光眼可能需要激光手术来降低眼压，并使用药物以保持控制。

重度青光眼则需要实施传统的眼部手术，如小梁切除术或巩膜外减压术，以防止视神经损伤。

After Fine-tuning

青光眼是一种眼睛疾病，可以引起视神经的损伤和视力下降。治疗青光眼的主要方法是药物治疗、激光治疗和手术治疗。

  1. 药物治疗：对于早期青光眼患者，通常采用抗青光眼药物治疗，通过降低眼压来预防视神经进一步损伤。目前常用的抗青光眼药物有拉坦前列素滴眼液、阿托品滴眼液等。
  2. 激光治疗：对于中晚期青光眼患者，如果药物治疗效果不佳，可能需要进行激光治疗。目前常用的激光治疗方法包括小梁切除术、YAG激光手术等。
  3. 手术治疗：对于非常严重的青光眼患者，可能需要进行手术治疗。目前常用的手术治疗方法包括前房引流术、虹膜周切术、小梁切除术、巩膜滤过术等。

需要注意的是，以上治疗方法的具体选择应根据患者的病情和医生的专业建议来进行。同时，青光眼患者在日常生活中也需要保持良好的生活习惯，避免过度疲劳、保持情绪稳定、定期检查眼睛等情况的发生。

Environment Requirements

Please refer to requirements.txt to install the required dependencies.

Run the following command line in the main directory of the Qwen repo.

bash
pip install -r requirements.txt

Preparation

Download Qwen-7B-Chat

First, download the model files. You can choose to download directly from ModelScope.

python
from modelscope.hub.snapshot_download import snapshot_download
model_dir = snapshot_download('Qwen/Qwen-7B-chat', cache_dir='.')

Download Medical Training Data

Download the data required for training; here, we provide a medical conversation dataset for training. It is sampled from MedicalGPT repo and we have converted this dataset into a format that can be used for fine-tuning.

Disclaimer: the dataset can be only used for the research purpose.

python
!wget https://atp-modelzoo-sh.oss-cn-shanghai.aliyuncs.com/release/tutorials/qwen_recipes/medical_sft.json

You can prepare your dataset in JSON format following the format below, and then modify the --data_path parameter in the training command to point to your JSON file.

These data instances can be conversations in the real world or include domain knowledge QA pairs. Besides, fine-tuning allows Qwen-chat to play like some specific roles. As Qwen-chat is a dialogue model for general scenarios, your fine-tuning can customize a chatbot to meet your requirements.

We recommend that you prepare 50~ data instances if you want to fine-tune Qwen-chat as a roleplay model.

You may prepare much more data instances if you want to infuse the domain knowledge of your field into the model.

In this tutorial, we have prepared a medical domain fine-tuning dataset consisting of 1000 data instancess as an example. You can refer to our example to fine-tune on your own domain-specific dataset.

Below is a simple example list with 1 sample:

python
[
  {
    "id": "1",
    "conversations": [
      {
        "from": "user",
        "value": "直肠腺瘤的早期症状?"
      },
      {
        "from": "assistant",
        "value": "结直肠腺瘤是起源于结直肠黏膜腺上皮的良性肿瘤，包括结肠腺瘤与直肠腺瘤，是常见的肠道良性肿瘤。因与大肠癌的发生关系密切，被认为是一种癌前病变。不同地区、不同年龄的发病率差别很大，40岁以下的发病率低，60岁以上较高，男女无明显差别。此类疾病的病因及发病机制不明确。部分患者有遗传因素存在，另外，高脂肪饮食与食物纤维不足、肠道菌群紊乱也与此病的发生有相关性。多数患者无症状。症状与腺瘤的大小、部位、病理性质等有关。有症状者可有以下表现：直肠腺瘤的临床常见症状有便血，多为无疼性便血，常呈鲜红色，量多，可引起贫血。粪便性状改变，多为粘液便，便意不尽或里急重感，有时只排出粘液，称为假性腹泻。1.腹部不适、腹部疼痛，多为大的腺瘤伴发肠套叠、肠梗阻引起。2.排便习惯改变，包括便秘、腹泻、里急后重等。3.粪便带血，最常见为间歇性便血。4.部分位于直肠的较大的、带蒂腺瘤可在排便时脱落或脱出肛。可以采取内镜下高频电凝、激光、微波凝固等方法切除，也可以选择外科手术切除，并定期随访。有恶变者根据情况选择其他治疗（如放疗、化疗、手术等）管状腺瘤切除术后复发者少见，但绒毛状腺瘤及绒毛管状腺瘤切除术后常可复发，尤其是绒毛状腺瘤，且多发的腺瘤复发率高于单发者。对于经内镜治疗或局部手术切除的结直肠腺瘤患者尤其是绒毛状腺瘤或广基的绒毛管状腺瘤患者，建议腺瘤切除后的第一年内3～6个月进行一次肠镜检查，第二年开始每年一次。"
      }
    ]
  }
]

You can also use multi-turn conversations as the training set. Here is a simple example:

python
[
  {
    "id": "2",
    "conversations": [
      {
        "from": "user",
        "value": "你好，我最近经常感觉胸口疼痛，这是怎么回事？"
      },
      {
        "from": "assistant",
        "value": "胸痛可能由多种原因引起，包括心脏问题、消化系统疾病、呼吸系统问题等。您能描述一下疼痛的性质和持续时间吗？"
      },
      {
        "from": "user",
        "value": "疼痛感觉像是挤压，大概持续了几分钟。"
      },
      {
        "from": "assistant",
        "value": "挤压感的胸痛可能与心脏问题有关，特别是如果它伴随着呼吸困难、出汗或恶心。我建议您尽快去看医生并进行适当的检查，如心电图和血液检测，以确定具体原因。"
      },
      {
        "from": "user",
        "value": "我应该去急诊室吗，还是预约我的家庭医生？"
      },
      {
        "from": "assistant",
        "value": "如果您的疼痛是突然发生的，并且还有其他症状，建议您立即去急诊室。如果疼痛不是很严重且没有其他严重症状，您可以预约家庭医生进一步评估。但请不要忽视疼痛，尤其是如果这种情况是第一次出现。"
      }
    ]
  }
]

Fine-Tune the Model

You can directly run the prepared training script to fine-tune the model.

For parameter settings, you can modify --model_name_or_path to the location of the model you want to fine-tune, and set --data_path to the location of the dataset.

You should remove the --bf16 parameter if you are using a non-Ampere architecture GPU, such as a V100.

For --model_max_length and --per_device_train_batch_size, we recommend the following configurations, ,you can refer to this document for more details:

--model_max_length--per_device_train_batch_sizeGPU Memory
512424g
1024324g
512832g
1024632g

You can use our recommended saving parameters, or you can save by epoch by just setting --save_strategy "epoch" if you prefer to save at each epoch stage. --save_total_limit means the limit on the number of saved checkpoints.

For other parameters, such as --weight_decay and --adam_beta2, we recommend using the values we provided blow.

Setting the parameters --gradient_checkpointing and --lazy_preprocess is to save GPU memory.

The parameters for the trained Lora module will be saved in the output_qwen folder.

python
!python ../../../finetune/finetune.py \
    --model_name_or_path "Qwen/Qwen-7B-Chat/"\
    --data_path  "medical_sft.json"\
    --bf16 \
    --output_dir "output_qwen" \
    --num_train_epochs 4\
    --per_device_train_batch_size 4 \
    --per_device_eval_batch_size 3 \
    --gradient_accumulation_steps 16 \
    --evaluation_strategy "no" \
    --save_strategy "epoch" \
    --save_steps 3000 \
    --save_total_limit 10 \
    --learning_rate 1e-5 \
    --weight_decay 0.1 \
    --adam_beta2 0.95 \
    --warmup_ratio 0.01 \
    --lr_scheduler_type "cosine" \
    --logging_steps 10 \
    --model_max_length 512 \
    --gradient_checkpointing \
    --lazy_preprocess \
    --use_lora

Merge Weights

The LoRA training only saves the adapter parameters. You can load the fine-tuned model and merge weights as shown below:

python
from transformers import AutoModelForCausalLM
from peft import PeftModel
import torch

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B-chat/", torch_dtype=torch.float16, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, "output_qwen/")
merged_model = model.merge_and_unload()
merged_model.save_pretrained("output_qwen_merged", max_shard_size="2048MB", safe_serialization=True)

The tokenizer files are not saved in the new directory in this step. You can copy the tokenizer files or use the following code:

python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "Qwen/Qwen-7B-chat/",
    trust_remote_code=True
)

tokenizer.save_pretrained("output_qwen_merged")

Test the Model

After merging the weights, we can test the model as follows:

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation import GenerationConfig

tokenizer = AutoTokenizer.from_pretrained("output_qwen_merged", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "output_qwen_merged",
    device_map="auto",
    trust_remote_code=True
).eval()

response, history = model.chat(tokenizer, "什么是VDAC1？", history=None)
print(response)