Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better error handling when remote tool entry is invalid( had https:// for a remote github tool) #638

Open
sangee2004 opened this issue Jul 16, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@sangee2004
Copy link
Contributor

gptscript version - v0.0.0-dev-d3cac6e6-dirty

Steps to reproduce the problem:

  1. Execute the following script which has invalid remote tool entry - https:// for image generation tool entry
 % cat image.gpt 
Tools: https://github.com/gptscript-ai/image-generation, sys.download

Generate an image of a a squirelle playing with acorn on a tree and return only the url of the image.
Download the contents of the url to a file `myimage` with appropriate extension.
  1. It fails with context length exceeded error message since the entire html content of the tool directory gets passed as instruction.
gptscript --disable-cache image.gpt 
09:33:39 started  [main]
09:33:39 sent     [main]
         content  [1] content | Waiting for model response...
         content  [1] content | <tool call> imageGeneration -> {"defaultPromptParameter":"a squirrel playing with an acorn on a tree"}
09:33:41 started  [https://github.com/gptscript-ai/image-generation(2)] [input={"defaultPromptParameter":"a squirrel playing with an acorn on a tree"}]
09:33:41 sent     [https://github.com/gptscript-ai/image-generation(2)]
         content  [2] content | Waiting for model response...09:33:43 usage    [total=305] [prompt=280] [completion=25]
2024/07/16 09:33:43 error, status code: 400, message: This model's maximum context length is 128000 tokens. However, your messages resulted in 128642 tokens. Please reduce the length of the messages.

Expected Behavior:
There should be tool validation that would be able to catch this as an invalid tool entry and report it even before it get used for LLM calls.

@sangee2004 sangee2004 added the bug Something isn't working label Jul 16, 2024
@drpebcak
Copy link
Member

This is a tricky one because its not an invalid tool per se - its just too big.

You could serve a gptscript from a webserver that just said say hello and that would be valid syntax. Is there anything other than the length you would expect to trigger a validation error?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants