Skip to content

Commit

Permalink
Add info in docs
Browse files Browse the repository at this point in the history
  • Loading branch information
panregedit committed Nov 11, 2024
1 parent 961e558 commit d4677e9
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 13 deletions.
26 changes: 13 additions & 13 deletions docs/concepts/worker.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Worker
Worker is the basic unit of computation in OmAgent. It is responsible for executing tasks and generating outputs.
## 1. How to define a worker
## How to define a worker
The most basic worker can be created like this:
```python
from omagent_core.engine.worker.base import BaseWorker
Expand All @@ -14,7 +14,7 @@ class MyWorker(BaseWorker):
```
By inheriting from `BaseWorker`, you can define your own worker. The worker will be registered with the name of the class. Normally, use `@registry.register_worker()` to register the worker so that it can build from configurations. See [registry](./registry.md) for more details.

### 1.1. Parameter Handling
### 1. Parameter Handling
You can define typed parameters that are json serializable, and return in key-value format:
```python
@registry.register_worker()
Expand All @@ -26,7 +26,7 @@ class ParameterWorker(BaseWorker):
}
```

### 1.2. Integration
### 2. Integration
You can integrate workers with other libraries to extend the functionality. A most common case is to integrate with LLMs. Here is an example of how:
```python
@registry.register_worker()
Expand All @@ -43,7 +43,7 @@ class LLMWorker(BaseLLMBackend, BaseWorker)
return self.simple_infer()
```

### 1.3. Configuration Fields
### 3. Configuration Fields
You can configure worker behavior using Pydantic Fields to set default values:
```python
@registry.register_worker()
Expand All @@ -54,7 +54,7 @@ class ConfigurableWorker(BaseWorker):
```
Note: do not use ```alias``` in the field definition.

### 1.4. Async Support
### 4. Async Support
Workers can be asynchronous:
```python
@registry.register_worker()
Expand All @@ -70,9 +70,9 @@ class AsyncWorker(BaseWorker):
return {"result": "async operation completed"}
```

## 2. Configuration and build
## Configuration and build
Workers can be configured and built from YAML or JSON configuration files. You not only can set the parameters, but the recursive dependencies.
### 2.1. Worker Configuration Structure
### 1. Worker Configuration Structure
Here's the basic structure:
```yaml
name: LLMWorker
Expand All @@ -86,7 +86,7 @@ llm:
output_parser:
name: StrParser
```
### 2.2. Submodule Substitution
### 2. Submodule Substitution
You can use the ${sub|**module_name**} to substitute submodules. This is useful when you want to reuse the same submodule in different workers and also keep the configuration clean. The **module_name** should be the name of the submodule configuration file.
For example, you can define the llm_worker.yaml as follows:
```yaml
Expand All @@ -107,7 +107,7 @@ vision: true
This is equivalent to the previous LLMWorker example.
Note: Do not use ```alias``` in the field definition. Do not create Circular reference.

### 2.3. Environment Variables
### 3. Environment Variables
You can use the ${env|**env_name**, **default_value**} to substitute environment variables. This is useful when you want to set the parameters dynamically. The **env_name** should be the name of the environment variable. **default_value** is optional, and will be used when the environment variable is not set.
For example, you can define the gpt.yaml as follows:
```yaml
Expand All @@ -120,7 +120,7 @@ vision: true
```
The environment variable name is case-sensitive.

### 2.3. Default Configuration Fields
### 4. Default Configuration Fields
Workers have several default configuration fields that can be set:
- **component_stm**: The STM component for the worker. Use any registered component name. Default is the one registered with `register_stm`. Access it via `self.stm`. See [container](./container.md) and [memories](./memories.md) for more details.
- **component_ltm**: The LTM component for the worker. Use any registered component name. Default is the one registered with `register_ltm`. Access it via `self.ltm`. See [container](./container.md) and [memories](./memories.md) for more details.
Expand All @@ -130,7 +130,7 @@ Workers have several default configuration fields that can be set:
- **domain**: The domain of the workflow. Default is None.
- **concurrency**: The concurrency of the worker. Default is 5.

### 2.2. Build from Configurations
### 5. Build from Configurations
The worker instances can be built from configurations by using the ```build_from_file``` function from omagent_core.utils.build. Here's how it works:
```python
from omagent_core.utils.build import build_from_file
Expand All @@ -140,15 +140,15 @@ worker_config = build_from_file('path/to/config/directory')
```
Note: You must provide a ```workers``` directory in the configuration path which contains all configurations for the workers.

## 3. Important Notes
## Important Notes
- Always use the @registry.register_worker() decorator to register the worker
- The ```_run``` method is mandatory and contains your core logic
- Return values should be a dictionary with serializable values
- Worker behavior can be configured using Fields
- Both synchronous and asynchronous operations are supported
- The ```self.workflow_instance_id``` is automatically available in the worker context

## 4. Best Practices
## Best Practices
- Keep workers focused on a single responsibility
- Use proper type hints for better code clarity
- Implement proper error handling
Expand Down
6 changes: 6 additions & 0 deletions docs/concepts/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,12 @@ workflow >> switch_task >> {'case1': task1, 'case2': task2, 'default': task3} #
```
Note that the switch_task **MUST** output ```switch_case_value``` as indicator for branching.

You can use a workflow as a task in another workflow.
```python
sub_workflow >> task1 >> task2
workflow >> task3 >> sub_workflow >> task4
```

## Registering a Workflow
You can register a workflow by using ```register``` method.
```python
Expand Down

0 comments on commit d4677e9

Please sign in to comment.