Skip to content

Commit

Permalink
[Docs] Add observability section for agentops (#448)
Browse files Browse the repository at this point in the history
# [Docs] Add observability section for agentops

Adds a new section **Observability** under **Advanced Features** for
[AgentOps](https://www.agentops.ai/).

This is in accordance with this
[PR](AgentOps-AI/agentops#541) and should close
#445
  • Loading branch information
the-praxs authored Dec 23, 2024
1 parent 6d066a6 commit ead8249
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 0 deletions.
59 changes: 59 additions & 0 deletions website/docs/advanced/observability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
id: observability
description: Observability
slug: /observability
---

# Observability

[AgentOps](https://www.agentops.ai/) is a platform that helps you to monitor and analyze the behavior and performance of your agents.
TaskWeaver is integrated with AgentOps, so you can use TaskWeaver to generate code and use AgentOps to observe all the events from the planning to the code execution.

## Get Started

TaskWeaver integration with AgentOps is documented [here](https://www.agentops.ai/docs/taskweaver).

To get started, you need an API key from AgentOps by [signing up](https://www.agentops.ai/signup) for a free account. Then you need to create a new project and generate the API key from the project settings.

Once you have the API keys, the following steps will help you to get started.

1. Install the `agentops`` package in your project.
```bash
pip install agentops
```
2. Import the `agentops` package in your project.
```python
import agentops
```
3. Initialize the `agentops` client with your API key.
```python
agentops.init(api_key="your_api_key")
```

:::note
You can also set the tags here to track your sessions in the AgentOps dashboard. By default, the "taskweaver" tag will be added to all the AgentOps sessions for TaskWeaver.
:::

4. Import the TaskWeaver handler `TaskWeaverEventHandler` and set it as the event handler for your TaskWeaver project.
```python
from agentops.providers.taskweaver import TaskWeaverEventHandler
handler = TaskWeaverEventHandler()
```

:::note
There are two ways to set the handler:
- Set the handler using the `session.event_handler.register()` method.
- Set the handler in every `session.send_message` function call in the `event_handler` parameter.
:::

:::warning
If you encounter "stuttering" in the recorded messages from the events, it is because the handler has been set with more than one instance.
:::

5. Now you can run your TaskWeaver project and observe the events in the AgentOps dashboard. When the client is initialized, it will automatically start to track the events and provide a link to the session in the AgentOps dashboard.

:::info
It is important to note that the LLM calls and the other events are tracked together in different modules of AgentOps. By default, all the LLM calls are tracked when the AgentOps client is initialized. However, the other events are only tracked when the `TaskWeaverEventHandler` handler is used in conjunction within the code.

Therefore, event observability is limited to LLM calls when TaskWeaver is used in [Terminal mode](../usage/cmd.md).
:::
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ const sidebars = {
'advanced/code_verification',
'advanced/cli_only',
'advanced/telemetry',
'advanced/observability',
'advanced/plugin_only'
],
}
Expand Down

0 comments on commit ead8249

Please sign in to comment.