I'll beb using this git in order to keep my workflows and maybe some important things that I get asked alot...
When it comes to IP-adapters There's one important thumb rule to remember - IP-adatpers all need a clip vision models. There are only 2 Clip Vision models at the moment, In short, we can call them ViT-H and ViT-BigG.
ONLY 2 IP-adatpers use the ViT-BigG at the moment, ip-adapter_sd15_vit-G and ip-adapter_sdxl - all the rest uses ViT-H When you download the clip vision models - be carful as both files are named the same thing - personally, because of a personal mistake, I've got used to calling the ViT-H 1.5 model and the ViT-BigG the SDXL model, it's not right in anyway, but this is the way i'm used to...