-
Notifications
You must be signed in to change notification settings - Fork 694
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Docs] Add observability section for agentops (#448)
# [Docs] Add observability section for agentops Adds a new section **Observability** under **Advanced Features** for [AgentOps](https://www.agentops.ai/). This is in accordance with this [PR](AgentOps-AI/agentops#541) and should close #445
- Loading branch information
Showing
2 changed files
with
60 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
--- | ||
id: observability | ||
description: Observability | ||
slug: /observability | ||
--- | ||
|
||
# Observability | ||
|
||
[AgentOps](https://www.agentops.ai/) is a platform that helps you to monitor and analyze the behavior and performance of your agents. | ||
TaskWeaver is integrated with AgentOps, so you can use TaskWeaver to generate code and use AgentOps to observe all the events from the planning to the code execution. | ||
|
||
## Get Started | ||
|
||
TaskWeaver integration with AgentOps is documented [here](https://www.agentops.ai/docs/taskweaver). | ||
|
||
To get started, you need an API key from AgentOps by [signing up](https://www.agentops.ai/signup) for a free account. Then you need to create a new project and generate the API key from the project settings. | ||
|
||
Once you have the API keys, the following steps will help you to get started. | ||
|
||
1. Install the `agentops`` package in your project. | ||
```bash | ||
pip install agentops | ||
``` | ||
2. Import the `agentops` package in your project. | ||
```python | ||
import agentops | ||
``` | ||
3. Initialize the `agentops` client with your API key. | ||
```python | ||
agentops.init(api_key="your_api_key") | ||
``` | ||
|
||
:::note | ||
You can also set the tags here to track your sessions in the AgentOps dashboard. By default, the "taskweaver" tag will be added to all the AgentOps sessions for TaskWeaver. | ||
::: | ||
|
||
4. Import the TaskWeaver handler `TaskWeaverEventHandler` and set it as the event handler for your TaskWeaver project. | ||
```python | ||
from agentops.providers.taskweaver import TaskWeaverEventHandler | ||
handler = TaskWeaverEventHandler() | ||
``` | ||
|
||
:::note | ||
There are two ways to set the handler: | ||
- Set the handler using the `session.event_handler.register()` method. | ||
- Set the handler in every `session.send_message` function call in the `event_handler` parameter. | ||
::: | ||
|
||
:::warning | ||
If you encounter "stuttering" in the recorded messages from the events, it is because the handler has been set with more than one instance. | ||
::: | ||
|
||
5. Now you can run your TaskWeaver project and observe the events in the AgentOps dashboard. When the client is initialized, it will automatically start to track the events and provide a link to the session in the AgentOps dashboard. | ||
|
||
:::info | ||
It is important to note that the LLM calls and the other events are tracked together in different modules of AgentOps. By default, all the LLM calls are tracked when the AgentOps client is initialized. However, the other events are only tracked when the `TaskWeaverEventHandler` handler is used in conjunction within the code. | ||
|
||
Therefore, event observability is limited to LLM calls when TaskWeaver is used in [Terminal mode](../usage/cmd.md). | ||
::: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters