Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor multi agents execution #376

Merged

Conversation

ruanyl
Copy link
Member

@ruanyl ruanyl commented Dec 13, 2024

Description

The refactor in the commit introduces a Pipeline to execute asynchronous operations, each operator encapsulates a specific function. The Pipeline orchestrates these operators, processing input sequentially. The pipeline architecture allows for adding, reusing or rearranging steps with minimal impact on the codebase.

The monolithic Text2Vega class also be replaced with Pipeline in this PR.

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test.
  • New functionality has user manual doc added.
  • Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Yulong Ruan <[email protected]>
*/

export abstract class Operator<T, P> {
abstract execute(v: T): Promise<P>;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need context parameter? and how about name it as Task

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about name it as Task

Sounds good, I like the name, let me update.

do we need context parameter?

Is there an example?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a real use case, but this could helpful if we have context for tasks share intermediate result during execution. For example, task 3 want to use a output of task 1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current design is each task will add extra properties to its output, and the output will become the next task's input. But I got your point which makes to me, I think we can tweaks the implementation when there is a real use case.

Copy link

codecov bot commented Dec 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.60%. Comparing base (581a7ca) to head (e56d2c3).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #376      +/-   ##
==========================================
+ Coverage   87.46%   87.60%   +0.13%     
==========================================
  Files          65       66       +1     
  Lines        1891     1912      +21     
  Branches      473      473              
==========================================
+ Hits         1654     1675      +21     
  Misses        236      236              
  Partials        1        1              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Yulong Ruan <[email protected]>
@ruanyl ruanyl merged commit fb47ce8 into opensearch-project:main Dec 27, 2024
9 checks passed
@ruanyl ruanyl added the backport 2.x Trigger the backport flow to 2.x label Dec 27, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Dec 27, 2024
* refactor(t2viz): introduced pipeline to run async operations

Signed-off-by: Yulong Ruan <[email protected]>

* add tests

Signed-off-by: Yulong Ruan <[email protected]>

* update changelog entry

Signed-off-by: Yulong Ruan <[email protected]>

* rename Operator -> Task

Signed-off-by: Yulong Ruan <[email protected]>

* cleanup console.log

Signed-off-by: Yulong Ruan <[email protected]>

---------

Signed-off-by: Yulong Ruan <[email protected]>
(cherry picked from commit fb47ce8)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

# Conflicts:
#	CHANGELOG.md
ruanyl pushed a commit that referenced this pull request Dec 27, 2024
* refactor(t2viz): introduced pipeline to run async operations

Signed-off-by: Yulong Ruan <[email protected]>

* add tests

Signed-off-by: Yulong Ruan <[email protected]>

* update changelog entry

Signed-off-by: Yulong Ruan <[email protected]>

* rename Operator -> Task

Signed-off-by: Yulong Ruan <[email protected]>

* cleanup console.log

Signed-off-by: Yulong Ruan <[email protected]>

---------

Signed-off-by: Yulong Ruan <[email protected]>
(cherry picked from commit fb47ce8)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

# Conflicts:
#	CHANGELOG.md

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Trigger the backport flow to 2.x
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants