wandbot v1.3 - There be changes ahead

[morganmcg1]

wandbot v1.3.0

Note that to trigger the final initalization after running run.sh, a request has to be made to the /startup endpoint, as tiggering the heavy initialzations during app startup causes replit to timeout:

curl https://wandbot.replit.app/startup

New:

• Move to uv for package management: Installs and dependency checks cut down from minutes to seconds
• Support python 3.11 on replit
• Move to lazing loading in app.py to help with startup: Replit app deployments can't seen to handle the delay from loading the app, despite attempting async or background tasks
• Turn off web search: Currently we don't have a web search provider to use.
• Small formatting updates for weave.op
• Add dotenv in app.py for easy env var loads

Testing

• Tested on Replit, slack and discord working as expected.
• Testing by running evals

Updating the Evaluation Module

All LlamaIndex and RAGAS dependencies were removed from the evaluation code. The system prompt, model name and temperature remain the same.

Both implementations were evaluated to ensure that the new impelmentation perfectly matched the LlamaIndex version. Full evaluation details, code and results files are documented in this PR. Below are weave eval links for both implementations:

Full wandbot evals; LlamaIndex eval vs raw OpenAI

• New (raw OpenAI) eval code as of commit 840427d - 72.7%, weave eval

• wandbot v1.2 LlamaIndex eval code, 71.4% weave eval

The new code tends to evaluate slightly higher sometimes as the old LlamaIndex code wasn't fault tolerant of retries and errors and so would mark samples incorrect if there was an error/rate limit with the evaluator call itself.

[socket-security]

Report is too large to display inline.
View full report↗︎

Next steps

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of ecosystem/package-name@version specifiers. e.g. @SocketSecurity ignore npm/[email protected] or ignore all packages with @SocketSecurity ignore-all

• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/pywin32@308
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]
• @SocketSecurity ignore pypi/[email protected]

[socket-security]

New, updated, and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
pypi/[email protected] 🔁 pypi/[email protected]	None	+2831	16.2 GB	anton_chroma, jeff_chroma
pypi/[email protected] 🔁 pypi/[email protected]	Transitive: eval, shell, unsafe	+2313	8.17 GB
pypi/[email protected]	environment, filesystem, network, shell, unsafe Transitive: eval	+1760	12.5 GB	messense
pypi/[email protected] 🔁 pypi/[email protected], pypi/[email protected]	Transitive: shell, unsafe	+3119	34.1 GB	baskaryan, efriis, hwchase17, ...1 more
pypi/[email protected] 🔁 pypi/[email protected]	Transitive: network	+2066	7.84 GB	dmontagu, samuelcolvin
pypi/[email protected]	Transitive: filesystem	+89	109 MB	lebrice

🚮 Removed packages: pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected], pypi/[email protected]

View full report↗︎