Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: change the Artifact to be folder-based #286

Merged
merged 12 commits into from
Nov 27, 2024

Conversation

yzld2002
Copy link
Member

@yzld2002 yzld2002 commented Oct 24, 2024

Currently we implement the Artifact based on the pickle.
This requires to load / save the files.

This PR changes the Artifact to be folder-based.
There is no need to load / save any more.
All the files under the cwd (current working directory) will be treated as the files inside the Artifact.

Also we want to keep the chat_with_artifacts running in the same environment with the code_interpreter. So we don't have to maintain "local" and "remote".


Local test

Code:

from vision_agent.agent import VisionAgent
from vision_agent.tools.meta_tools import Artifacts


artifacts = Artifacts("/Users/zhichao/Projects/vision-agent-chat/va-local-test")

agent = VisionAgent(verbosity=1)
resp, artifacts = agent.chat_with_artifacts(
    [
        {
            "role": "user",
            "content": "Can you write code to detect the car and visualize the detection?",
        }
    ],
    artifacts=artifacts,
    test_multi_plan=False,
)
resp.append({"role": "user", "content": "Can you describe the output image?"})
resp, artifacts = agent.chat_with_artifacts(resp, artifacts, test_multi_plan=False)

Result:
image

image

Also tested the view_media_artifact action, the result looks good now.
image

@yzld2002 yzld2002 force-pushed the zhichao-change-to-folder-for-artifact branch from be3033a to 66f91f5 Compare October 31, 2024 15:55
@yzld2002 yzld2002 force-pushed the zhichao-change-to-folder-for-artifact branch from 46e5fa1 to 19c5e28 Compare November 8, 2024 10:07
@yzld2002 yzld2002 force-pushed the zhichao-change-to-folder-for-artifact branch from 19c5e28 to c796e4e Compare November 14, 2024 19:27
@yzld2002 yzld2002 changed the title feat: [WIP] change the Artifact to be folder-based feat: change the Artifact to be folder-based Nov 25, 2024
Copy link
Member

@dillonalaird dillonalaird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! I think it mostly looks good, just one comment right now

vision_agent/agent/vision_agent.py Show resolved Hide resolved
Copy link
Member

@dillonalaird dillonalaird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one more comment

vision_agent/agent/vision_agent.py Outdated Show resolved Hide resolved
Copy link
Member

@dillonalaird dillonalaird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! One minor comment

vision_agent/agent/vision_agent.py Outdated Show resolved Hide resolved
@yzld2002 yzld2002 merged commit 6a6b3b8 into main Nov 27, 2024
8 checks passed
@yzld2002 yzld2002 deleted the zhichao-change-to-folder-for-artifact branch November 27, 2024 01:25
@yzld2002 yzld2002 restored the zhichao-change-to-folder-for-artifact branch November 27, 2024 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants