Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor big5, eventdata, http_logs workloads #538

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

OVI3D0
Copy link
Member

@OVI3D0 OVI3D0 commented Dec 18, 2024

Description

Refactors big5, eventdata, and http_logs workloads to make use of the common_operations directory.

Issues Resolved

#489

Testing

  • New functionality includes testing

Tested by refactoring each test procedure and ensuring it runs as it did before refactoring.

Backport to Branches:

  • 6
  • 7
  • 1
  • 2
  • 3

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@OVI3D0 OVI3D0 added backport 2 Backport to the "2" branch backport 1 backport 3 Backport to the "3" branch backport 7 Backport to the "7" branch labels Dec 18, 2024
@rishabh6788
Copy link
Collaborator

rishabh6788 commented Dec 18, 2024

Do not get the intent and benefit of this change, I would prefer all the operations related to a workload to be grouped inside the workload dir. Each can be easily customized limiting to a workload. For e.g., I see that cluster health operation has been refactored to

{
    "name": "check-cluster-health",
    "operation": {
        "operation-type": "cluster-health",
        "index": "{{ index_name }}",
        "request-params": {
            "wait_for_status": "{{cluster_health | default('green')}}",
            "wait_for_no_relocating_shards": "true"
        },
        "retry-until-success": true
    }
}

This will now require user to make sure they add another workload param, why?
I believe we are reducing the flexibility and increasing the burden on the end user.

@OVI3D0
Copy link
Member Author

OVI3D0 commented Dec 19, 2024

This will now require user to make sure they add another workload param, why? I believe we are reducing the flexibility and increasing the burden on the end user.

Not necessarily, index_name is now parameterized but the index name value is provided by the workload.json file. There's no additional burden on the user unless they want to customize the index name. If you test the changes you'll notice that the index name is still provided for each workload without needing to pass in this value.

Do not get the intent and benefit of this change, I would prefer all the operations related to a workload to be grouped inside the workload dir. Each can be easily customized limiting to a workload.

The intent of these changes is to group up all of the common operations so they can be pulled from one single directory. If we want to make changes to one of these operations, we only need to do it once and the changes will be reflected across all workloads. I think if we keep these related operations inside each workloads directory then we're housing a lot of redundancy across the OSB workloads since they are the exact same operations

@IanHoang
Copy link
Collaborator

@rishabh6788 Consolidating the common basic operations from all workloads to a single shared directory should also reduce boilerplate code and allow users to focus on characteristics (operations and test procedures) that are unique to that workload. This would also simplify the experience for end users who are both experienced with and new to OSB.

Open to discussing this more in #489?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1 backport 2 Backport to the "2" branch backport 3 Backport to the "3" branch backport 7 Backport to the "7" branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants