v6.0.2

Improvements

• Added all of these chat models to select from to chat with the vector database!

• Added all of these vision models to select from when processing images.

• Added ChatTTS and google TTS as text to speech backends.
• Added hyperlinks to all model cards for vector, chat, and vision models.
• Added the ability to restore backups of your databases.
• Significantly improved the "Test Vision Model" tool to be able to test all vision models.
• Revamped the User Manual
• Added a pull-down menu to select the vector model instead of having to navigate and select a particular folder.
• MASSIVE refactoring.
• MASSIVE restructuring of the setup.py and requirements.txt due to the crazy increase in dependencies. "Dependency "hell" is a real thing...

CUDA no longer required (sort of):

• Previously, you were required to install CUDA and CUDNN directly from Nvidia, which was by its very nature a system-wide installation.
• Now, ALL CUDA and CUDNN-related files are "pip installed" within the setup.py script. This allows you to either install a different version of CUDA system-wide to be used by other programs or, alternatively, not install CUDA system-wide at all.
• This was achieved by installing CUDA and CUDNN as a python dependency and temporarily adding these specific installations to the PATH whenever the program is running.

IMPORTANT...restructuring of model downloading procedure

• Previously, the vector models were downloaded using a git clone command and other types of models (i.e. chat, vision, TTS, whisper.) were automatically downloaded to the system's cache folder.
• Now, almost all of the models (except vector models) are downloaded to a specific sub-folder within the Models folder. In future releases all models will be downloaded this way.
• The goal is to make the program as portable as possible; for example, copying the "src" folder (and therefore all the models) to a thumb drive to use on a laptop without having to re-download everything. Eventually, all paths to models selected within the program will be "relative" such that wherever you move the src folder (even to a different computer) everything should work just like it did without having to download anything again or change any settings.

Bug Fixes

• Fixed the "local model" not being removed from memory even after a different chat model was is loaded.
• Fixed the "local model" exponentially increasing memory usage when asking multiple questions (best guess, it was was re-loading the model each time).
• Re-added a script that was accidentally deleted from this repository during the last release...
• Fixed a huge issue involving sentence-transformers, TileDB, and Langchain that threw a variety of un-fixable errors when trying to create a vector database. This required modifying the source code for sentence-transformers itself as a temporary fix, but everything seems to be working much better so this may become a permanent fix.
• Fixed numerous other bugs.

Known issues

• Image search is NOT WORKING but will be fixed in an incremental release.
• There's an issue with Langchain specific to TileDB; specifically, the from_documents method. A temporary work around was to actually modify the sentence-transformers source code. A subsequent patch will likely use the from_texts method instead, but the database seems to be working fine.

Please create an issue with any bugs you encounter!

Credit goes to the new Claude 3.5 Sonnet for finally being able to solve the memory issue regarding loading/unloading chat models in a separate process nonetheless.

v6.0.1 - hard fought well done

Improvements

• Added all of these chat models to select from to chat with the vector database!

• Added all of these vision models to select from when processing images.

• Added ChatTTS and google TTS as text to speech backends.
• Added hyperlinks to all model cards for vector, chat, and vision models.
• Added the ability to restore backups if your databases.
• Significantly improved the "Test Vision Model" tool to be able to test all vision models.
• Revamped the User Manual
• Added a pulldown menu to select the vector model instead of having to navigate and select a particular folder.
• MASSIVE refactoring.
• MASSIVE restructuring of the setup.py and requirements.txt due to the crazy increase in dependencies. "Dependency "hell" is a real thing...

CUDA no longer required (sort of):

• Previously, you were required to install CUDA from Nvidia before the program would work...but not only that cudnn, which required a developer account (albeit, only requires an email address).
• Now, ALL CUDA-RELATED FILES ARE PIP INSTALLED INTO THE SITE-PACKAGES FOLDER.
• This means that you no longer have to install CUDA/CUDNN systemwide...or you can install a different version system-wide. Basically, setup.py now handles everything.
• This was achieved by setting the paths to CUDA/CUDNN files temporarily whenever the program begins. Thus, no manual changing of paths is needed.

IMPORTANT...restructuring of model downloading procedure

• Previously, the vector models were downloaded using a git clone command when you clicked a button. All other models (e.g. chat, vision, etc.) were automatically downloaded to the system's cache folder.
• Now, all models (except vector models) are downloaded to the Models folder within the main src folder so you can see them. Vector models will eventually be downloaded the same way.
• The goal is to make the program as completely portable as possible - e.g. put everything on a thumb drive and use it on your laptop without having to re-download everything. But more importantly, all paths are now relative such that even if you move your src folder on your computer the paths to the models should still work (have yet to confirm with vector models).

Bug Fixes

• Fixed the issue where the local chat model was not being removed from memory.
• Fixed the issue where the local chat model would exponentially increase vram when you asked more than one question.
• A crucial script had been accidentally deleted that went unnoticed until now.
• Fixed a huge issue creating databases due to issues between sentence-transformers, TileDB, and Langchain. Database creation should now be far more reliable!
• Fixed numerous other bugs.

Known issues

• Image search is NOT WORKING currently but will be fixed in an incremental release. It doesn't matter if you create an all-image database or combing images with other documents, questions regarding images supposedly entered into the database are not being returned.
• There's an issue with Langchain specific to TileDB; specifically, the from_documents method. A temporary work around was to actually modify the sentence-transformers source code. A subsequent patch will likely use the from_texts method instead, but the database seems to be working fine.

Please create an issue with any bugs you encounter!

Credit goes to the new Claude 3.5 Sonnet for finally being able to solve the memory issue regarding loading/unloading chat models in a separate process nonetheless.

v6.0.0 - hard fought well done

Improvements

• Added all of these chat models to select from to chat with the vector database!

• Added all of these vision models to select from when processing images.

• Added ChatTTS and google TTS as text to speech backends.
• Added hyperlinks to all model cards for vector, chat, and vision models.
• Added the ability to restore backups if your databases.
• Significantly improved the "Test Vision Model" tool to be able to test all vision models.
• Revamped the User Manual
• Added a pulldown menu to select the vector model instead of having to navigate and select a particular folder.
• MASSIVE refactoring.
• MASSIVE restructuring of the setup.py and requirements.txt due to the crazy increase in dependencies. "Dependency "hell" is a real thing...

CUDA no longer required (sort of):

• Previously, you were required to install CUDA from Nvidia before the program would work...but not only that cudnn, which required a developer account (albeit, only requires an email address).
• Now, ALL CUDA-RELATED FILES ARE PIP INSTALLED INTO THE SITE-PACKAGES FOLDER.
• This means that you no longer have to install CUDA/CUDNN systemwide...or you can install a different version system-wide. Basically, setup.py now handles everything.
• This was achieved by setting the paths to CUDA/CUDNN files temporarily whenever the program begins. Thus, no manual changing of paths is needed.

IMPORTANT...restructuring of model downloading procedure

• Previously, the vector models were downloaded using a git clone command when you clicked a button. All other models (e.g. chat, vision, etc.) were automatically downloaded to the system's cache folder.
• Now, all models (except vector models) are downloaded to the Models folder within the main src folder so you can see them. Vector models will eventually be downloaded the same way.
• The goal is to make the program as completely portable as possible - e.g. put everything on a thumb drive and use it on your laptop without having to re-download everything. But more importantly, all paths are now relative such that even if you move your src folder on your computer the paths to the models should still work (have yet to confirm with vector models).

Bug Fixes

• Fixed the issue where the local chat model was not being removed from memory.
• Fixed the issue where the local chat model would exponentially increase vram when you asked more than one question.
• A crucial script had been accidentally deleted that went unnoticed until now.
• Fixed a huge issue creating databases due to issues between sentence-transformers, TileDB, and Langchain. Database creation should now be far more reliable!
• Fixed numerous other bugs.

Known issues

• Image search is NOT WORKING currently but will be fixed in an incremental release. It doesn't matter if you create an all-image database or combing images with other documents, questions regarding images supposedly entered into the database are not being returned.
• There's an issue with Langchain specific to TileDB; specifically, the from_documents method. A temporary work around was to actually modify the sentence-transformers source code. A subsequent patch will likely use the from_texts method instead, but the database seems to be working fine.

Please create an issue with any bugs you encounter!

Credit goes to the new Claude 3.5 Sonnet for finally being able to solve the memory issue regarding loading/unloading chat models in a separate process nonetheless.

v5.0.1

BREAKING CHANGES:

• This release has breaking changes. Notably, the "Embedding_Models" folder is no longer used and has been replaced with the "Models" folder.

IMPROVEMENTS:

• Added the functionality of using "Local Models" as well as LM Studio!!! CPU only installations must still use LM Studio.
• Fixed setup.py to allow for CPU only installation and the overall functionality of the program when using only CPU.
• Updated dependencies.

COMMENTS:

This may be the last release in a little while, but it's a good one! I'll continue to monitor the GitHub issues for any bugs or small changes in and address those until further notice.

NEW CHAT MODELS!

• Tested on RTX 4090.
  

V5.0 - CHAT MODELS!

BREAKING CHANGES:

• This release has breaking changes. Notably, the "Embedding_Models" folder is no longer used and has been replaced with the "Models" folder.

IMPROVEMENTS:

• Added the functionality of using "Local Models" as well as LM Studio!!! CPU only installations must still use LM Studio.
• Fixed setup.py to allow for CPU only installation and the overall functionality of the program when using only CPU.
• Updated dependencies.

COMMENTS:

This may be the last release in a little while, but it's a good one! I'll continue to monitor the GitHub issues for any bugs or small changes in and address those until further notice.

NEW CHAT MODELS!

• Tested on RTX 4090.
  

v4.4.1 - INSTRUCTOR fixed

IMPROVEMENTS:

• Normalize embeddings set to "True..." not sure how it go set to false, leading to suboptimal search results.
• Custom library for Instructor fixing all Instructor models - They're Back!
• Changed WhisperSpeech to using "base" for both s2a and t2s models.

COMMENTS:

• Feel free to experiment with different sizes of "s2a" and "t2s" models by modifying tts_module.py. Here are some benchmarks for the various model combinations.
• Benchmarks for Bark coming soon!

v4.4.0 - Dude! Upgrade your Dependencies

BREAKING CHANGES:

• Significant refactoring and some minimal script renaming as well.
• Updating certain libraries to be compatible with one another.
• Revised list of vector models so if you created a database with one that's been removed from the list be aware.

IMPROVEMENTS:

• Upgrade versions of dependencies as much as possible.
• Using custom library for Instructor models.
• Removed Jina models since they have comparable performance to existing models and, recently, require an API key to use, which is stupid.

KNOWN BUGS:

• To use any of the instructor models it requires you to download through the "Models Tab" and at runtime it'll download it a second time. This is because of the custom modified instructor script. This will be fixed in a future patch - e.g. release 4.4.1.

v4.3.0 - reliability!!!

BREAKING CHANGE

• This version is the first to use TileDB instead of ChromaDB. Therefore, any databases previously created with ChromaDB will no longer work with Version 4.3.0+ of this program.
• TileDB is much more robust, faster, will not give errors when trying to ingest massive amounts of documents, and is overall "better." ChromaDB has been used since the beginning of this repository so it's a major change. However, it was necessary due to ChromaDB not being able to handle massive amounts of documents reliably. It simply couldn't insert, for example, 200k+ vectors reliably. This was tested extensively with older versions that use duckdb + clickhouse as well as the current version that uses sqlite3. ALL HAIL TILEDB!

IMPROVEMENTS:

• WhisperSpeech was added as an alternative to Bark when doing text-to-speech. WhisperSpeech is a very promising library managed by Collabora. It rivals Bark in terms of quality, VRAM/compute requirements, and speed - even outperforms it in certain metrics. As newer models from WhisperSpeech come out this will only improve further.
• As part of the sentence-transformers library vector models were loaded with a batch size of 32. It became apparent after extensive testing that this caused slower performance and higher compute/memory requirements in EVERY SINGLE MODEL. database_interactions.py now sets the batch size based on the size of the vector model being used when creating the database with a GPU. When using a CPU, the batch size is always 2. This reduces memory requirements by over 50% in most cases with the larger models, and improves speed for GPU and CPU.
• Updated some of the dependency versions with further updates based on further testing in the near future.
• Started using the pickle library to save representations of image files and audio transcriptions within a given database. When a user doubleclicks on a image or audio file within an existing database, it will read the pickle file and route the call to where the file is located on your computer. Obviously, if you move the actual file (i.e. the image or audio file) it'll give an error.
• Multiple scripts were refactored and/or renamed.

BUG FIXES:

• Changes made after Version 4.1 resulted in "Port," "Temperature," and "Max_tokens" not being updated properly. This was fixed by reverting the specific script managing this to a prior version.
• The Models Tab was revamped to correct alignment, a longstanding bug.

v4.2.0 - general upgrade

• Eliminated multiple bugs regarding creating, saving, and searching databases.
• Faster database creation by specifying batch size based on the vector model selected, and also lower VRAM usage as a result. This was only done after extensive testing.
• Make sure that all audio and image files are displayed in the database viewer tab now and that they open the appropriate file when doubleclicked.
• Updated user guide.
• Remove certain vector models that had significantly higher compute/VRAM requirements but no greater quality than other models that require less compute/VRAM. This was only done after extensive testing.
• Other relatively minor changes.

V4.1.0 - MULTIPLE databases!!!

Finally painstakingly implemented the ability to create multiple databases.

Changed the voice recorder to use the new backend for the file transcriber, which is the amazing WhisperS2T library.

Updated multiple libraries to the most recent versions while still maintaining inter-dependency compatibility.

Updated openai library to use the new API finally.

Refactored multiple scripts, revised some conditional checks, reduced some clutter printed to the command prompt, and other improvements.