Skip to content

v6.0.0 - hard fought well done

Compare
Choose a tag to compare
@BBC-Esq BBC-Esq released this 27 Jun 06:32
· 405 commits to main since this release
0461ed4

Improvements

  • Added all of these chat models to select from to chat with the vector database!
chart_chat
  • Added all of these vision models to select from when processing images.
chart_vision
  • Added ChatTTS and google TTS as text to speech backends.
  • Added hyperlinks to all model cards for vector, chat, and vision models.
  • Added the ability to restore backups if your databases.
  • Significantly improved the "Test Vision Model" tool to be able to test all vision models.
  • Revamped the User Manual
  • Added a pulldown menu to select the vector model instead of having to navigate and select a particular folder.
  • MASSIVE refactoring.
  • MASSIVE restructuring of the setup.py and requirements.txt due to the crazy increase in dependencies. "Dependency "hell" is a real thing...

CUDA no longer required (sort of):

  • Previously, you were required to install CUDA from Nvidia before the program would work...but not only that cudnn, which required a developer account (albeit, only requires an email address).
  • Now, ALL CUDA-RELATED FILES ARE PIP INSTALLED INTO THE SITE-PACKAGES FOLDER.
  • This means that you no longer have to install CUDA/CUDNN systemwide...or you can install a different version system-wide. Basically, setup.py now handles everything.
  • This was achieved by setting the paths to CUDA/CUDNN files temporarily whenever the program begins. Thus, no manual changing of paths is needed.

IMPORTANT...restructuring of model downloading procedure

  • Previously, the vector models were downloaded using a git clone command when you clicked a button. All other models (e.g. chat, vision, etc.) were automatically downloaded to the system's cache folder.
  • Now, all models (except vector models) are downloaded to the Models folder within the main src folder so you can see them. Vector models will eventually be downloaded the same way.
  • The goal is to make the program as completely portable as possible - e.g. put everything on a thumb drive and use it on your laptop without having to re-download everything. But more importantly, all paths are now relative such that even if you move your src folder on your computer the paths to the models should still work (have yet to confirm with vector models).

Bug Fixes

  • Fixed the issue where the local chat model was not being removed from memory.
  • Fixed the issue where the local chat model would exponentially increase vram when you asked more than one question.
  • A crucial script had been accidentally deleted that went unnoticed until now.
  • Fixed a huge issue creating databases due to issues between sentence-transformers, TileDB, and Langchain. Database creation should now be far more reliable!
  • Fixed numerous other bugs.

Known issues

  • Image search is NOT WORKING currently but will be fixed in an incremental release. It doesn't matter if you create an all-image database or combing images with other documents, questions regarding images supposedly entered into the database are not being returned.
  • There's an issue with Langchain specific to TileDB; specifically, the from_documents method. A temporary work around was to actually modify the sentence-transformers source code. A subsequent patch will likely use the from_texts method instead, but the database seems to be working fine.

Please create an issue with any bugs you encounter!

Credit goes to the new Claude 3.5 Sonnet for finally being able to solve the memory issue regarding loading/unloading chat models in a separate process nonetheless.