qwen2_moe support w multipack

[winglian]

Description

resolves #1453

✅ multipack
✅ qwen2_moe 4-bit QLoRA
✅ qwen2_moe 16-bit LoRA
❓ qwen2_moe 8-bit LoRA

[winglian]

8-bit LoRA still doesn't work atm

  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 487, in backward
    grad_A = torch.matmul(grad_output, CB).view(ctx.grad_shape).to(ctx.dtype_A)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (2048x1 and 32x2048)