vision models do not currently work with MacOS

[BBC-Esq]

It's my understanding that the bitsandbytes library doesn't work with MacOS, correct me if I'm mistaken. Therefore, 4-bit and 8-bit usage of the vision models might not be possible ever on MacOS...but if that's true, it doesn't explain why float16 wouldn't work...The goal is to at least get float16 working...perhaps try float32 (no quantization), and if possible, get 8-bit and/or 4-bit working somehow on MacOS.

[cddigi]

The issue can be tracked at bitsandbytes-foundation/bitsandbytes#485.

[gramss]

Appreciate the active suppression of users running into this issue on MacOS.
Unfortunately, the error-prevention causes some errors on its own:

Traceback (most recent call last):
  File "/Users/user/gits/ChromaDB-Plugin-for-LM-Studio-3.04/src/gui.py", line 148, in on_create_button_clicked
    if platform.system() == "Darwin" and any(images_dir.iterdir()):
                                             ^^^^^^^^^^
UnboundLocalError: cannot access local variable 'images_dir' where it is not associated with a value

commenting out this guard function makes the program run further.

What I did: I never chose any images. Just one PDF.

EDIT:
Using release 3.04 - as seen in my path

[cddigi]

There is an open pull request #87 that addresses that error.

[BBC-Esq]

I'll close this issue unless/until bitsandbytes adds Apple/Metal support I I change the backend of running the vision models...