微调后模型推理自定义数据集方式

[katie312]

目前通过这种方式进行微调后模型推理：

CUDA_VISIBLE_DEVICES=0 swift infer --ckpt_dir /xx/ --dataset /xx/test.jsonl --load_dataset_config true

其中test.jsonl内容如下

{"query": "<image>将这张图片进行OCR提取，转换为markdown格式","response": "# ", "images": ["/xxx/xxx.jpg"]}

请问这样做是正确的吗？

通过这种方式跑推理会报错：

[INFO:swift] PeftModelForCausalLM: 564.3297M Params (0.0000M Trainable [0.0000%]), 100.6641M Buffers.
[INFO:swift] system:         You should follow the instructions carefully and explain your answers in detail.
[INFO:swift] val_dataset: Dataset({
    features: ['query', 'response', 'images'],
    num_rows: 1
})
[INFO:swift] Setting args.verbose: True
[PROMPT]<|im_start|>system
        You should follow the instructions carefully and explain your answers in detail.<|im_end|>
<|im_start|>user
<img>[151859 * 256]</img>
将这张图片进行OCR提取，转换为markdown格式<|im_end|>
<|im_start|>assistant
[OUTPUT]/opt/conda/envs/llama/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.)
  return F.conv2d(input, weight, bias, self.stride,

请问是有其他的微调后模型推理自定义数据集方式吗？