跑 qwen2-vl 的demo，报错：cudaErrorNoKernelImageForDevice

[luhui]

对应的 gpu 信息如下：

使用的是 docker，用的 paddle 镜像:ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddl:3.0.0b2-gpu-cuda12.3-cudnn9.0-trt8.6

以下是报错信息

# 加载时的提示
Hint: Your machine support AVX, but the installed paddlepaddle doesn't have avx core. Hence, no-avx core with worse performance will be imported.
If you like, you could reinstall paddlepaddle by 'python -m pip install --force-reinstall paddlepaddle-gpu[==version]' to get better performance.
W1125 07:37:50.996143   570 gpu_resources.cc:96] The GPU architecture in your current machine is Pascal, which is not compatible with Paddle installation with arch: 61 70 75 80 86 90 , it is recommended to install the corresponding wheel package according to the installation information on the official Paddle website.
W1125 07:37:50.996188   570 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 6.0, Driver API Version: 12.4, Runtime API Version: 12.3
W1125 07:37:50.999251   570 gpu_resources.cc:164] device: 0, cuDNN Version: 9.0.

# 执行推理时的报错
CUDA error 209 [../third_party/cccl/cub/cub/util_device.cuh, 83]: no kernel image is available for execution on the device
CUDA error 101 [../third_party/cccl/cub/cub/util_device.cuh, 102]: invalid device ordinal
CUDA error 209 [../third_party/cccl/cub/cub/util_device.cuh, 308]: no kernel image is available for execution on the device
CUDA error 209 [../third_party/cccl/cub/cub/util_device.cuh, 391]: no kernel image is available for execution on the device
CUDA error 209 [../third_party/cccl/cub/cub/device/dispatch/dispatch_reduce.cuh, 962]: no kernel image is available for execution on the device
CUDA error 209 [../third_party/cccl/cub/cub/util_device.cuh, 308]: no kernel image is available for execution on the device
CUDA error 209 [../third_party/cccl/cub/cub/util_device.cuh, 391]: no kernel image is available for execution on the device
CUDA error 209 [../third_party/cccl/cub/cub/device/dispatch/dispatch_reduce.cuh, 962]: no kernel image is available for execution on the device
Traceback (most recent call last):
  File "/home/PaddleMIX/paddlemix/examples/qwen2_vl/single_image_infer.py", line 65, in <module>
    generated_ids = model.generate(**inputs, max_new_tokens=128)  # already trimmed in paddle
  File "/usr/local/lib/python3.10/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.10/dist-packages/paddle/base/dygraph/base.py", line 397, in _decorate_function
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddlenlp/generation/utils.py", line 947, in generate
    return self.sample(
  File "/usr/local/lib/python3.10/dist-packages/paddlenlp/generation/utils.py", line 1189, in sample
    outputs = self(**model_inputs)
  File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1532, in __call__
    return self.forward(*inputs, **kwargs)
  File "/home/PaddleMIX/paddlemix/models/qwen2_vl/modeling_qwen2_vl.py", line 1462, in forward
    image_embeds = self.visual(pixel_values, grid_thw=image_grid_thw)
  File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1532, in __call__
    return self.forward(*inputs, **kwargs)
  File "/home/PaddleMIX/paddlemix/models/qwen2_vl/modeling_qwen2_vl.py", line 984, in forward
    hidden_states = self.patch_embed(hidden_states)
  File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1532, in __call__
    return self.forward(*inputs, **kwargs)
  File "/home/PaddleMIX/paddlemix/models/qwen2_vl/modeling_qwen2_vl.py", line 285, in forward
    hidden_states = self.proj(hidden_states.to(dtype=target_dtype)).reshape([-1, self.embed_dim])
  File "/usr/local/lib/python3.10/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.10/dist-packages/paddle/base/wrapped_decorator.py", line 40, in __impl__
    return wrapped_func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddle/base/framework.py", line 704, in __impl__
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddle/base/dygraph/tensor_patch_methods.py", line 805, in to
    return self._to(device, dtype, blocking)
  File "/usr/local/lib/python3.10/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.10/dist-packages/paddle/base/wrapped_decorator.py", line 40, in __impl__
    return wrapped_func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddle/base/framework.py", line 704, in __impl__
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddle/base/dygraph/tensor_patch_methods.py", line 664, in _to
    return transform(self, device, dtype, blocking)
  File "/usr/local/lib/python3.10/dist-packages/paddle/base/dygraph/tensor_patch_methods.py", line 628, in transform
    gpu_memory_available = core.gpu_memory_available()
OSError: (External) CUDA error(209), no kernel image is available for execution on the device. 
  [Hint: 'cudaErrorNoKernelImageForDevice'. This indicates that there is no kernel image available that is suitable for the device. This can occur when a user specifiescode generation options for a particular CUDA source file that do not include the corresponding device configuration.] (at ../paddle/phi/core/platform/device/gpu/gpu_info.cc:133)

paddle 的版本是：https://paddle-qa.bj.bcebos.com/paddle-pipeline/CompileServicing-LinuxCentos-Commit-Cuda123-WITH_PIP_CUDA_LIBRARIES_ON-WITH_AVX_OFF/59dba7a7b8b3f0f7cfa7cc4e5b4dd28a38b1f431/paddlepaddle_gpu-0.0.0-cp310-cp310-linux_x86_64.whl
使用上述版本是因为官方当前的版本对 cpu 架构不支持，import paddle 时提示Illegal instruction (core dumped) PaddlePaddle/Paddle#69003

paddlemix 的版本是 dev 分支：1ad4bfcc27efc51c373ccbfdca8b773a604d48a0

[luyao-cv]

paddle要用develop的。paddlenlp用3.0.b2。

[luhui]

paddle要用develop的。paddlenlp用3.0.b2。

develop看起来还不支持 avx和 avx2 的指令集，用以下命令安装 paddle，import paddle 报错Illegal instruction (core dumped):
python -m pip install --pre paddlepaddle-gpu -i https://www.paddlepaddle.org.cn/packages/nightly/cu123/

指令集情况

[luyao-cv]

非常感谢支持，有条件的话建议使用A卡和H卡试试。