Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/main' into feature_issue_2805
Browse files Browse the repository at this point in the history
  • Loading branch information
LantaoJin committed Jan 9, 2025
2 parents 9b0074e + 0e4acd3 commit d4ed787
Show file tree
Hide file tree
Showing 653 changed files with 25,225 additions and 7,525 deletions.
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# This should match the owning team set up in https://github.com/orgs/opensearch-project/teams
* @pjfitzgibbons @ps48 @kavithacm @derek-ho @joshuali925 @dai-chen @YANG-DB @rupal-bq @mengweieric @vamsi-amazon @swiddis @penghuo @seankao-az @MaxKsyunz @Yury-Fridlyand @anirudha @forestmvey @acarbonetto @GumpacG @ykmr1224
* @pjfitzgibbons @ps48 @kavithacm @derek-ho @joshuali925 @dai-chen @YANG-DB @rupal-bq @mengweieric @vamsi-amazon @swiddis @penghuo @seankao-az @MaxKsyunz @Yury-Fridlyand @anirudha @forestmvey @acarbonetto @GumpacG @ykmr1224 @LantaoJin @noCharger
32 changes: 32 additions & 0 deletions .github/ISSUE_TEMPLATE/request_of_comments.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
name: 📄 Request of comments
about: Create a request of comments to initiate an discussion on feature implementation / design.
title: '[RFC]'
labels: 'rfc, untriaged'
assignees: ''
---

**Problem Statement**
Describes the issue or challenge that needs to be addressed.

**Current State**
Explains the existing setup or context and its shortcomings.

**Long-Term Goals**
* Describe the ideal outcome or future state you aim to achieve.
* What are the primary objectives of this solution should fulfill in the long run?
* How will this solution ensure sustainability or scalability over time?
* Consider whether the solution addresses the root problem and aligns with the overall goals.
* How confident are we that this solution solves the problem?

**Proposal**
Summarizes the suggested solution or improvement.

**Approach**
Details the steps or strategy to implement the proposal.

**Alternative**
Alternative that can serve as workarounds to partially or temporarily solve the problem.

**Implementation Discussion**
Outlines point the discussion regarding the proposed implementation.
20 changes: 11 additions & 9 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
### Description
[Describe what this change achieves]

### Issues Resolved
[List any issues this PR will resolve]


### Related Issues
Resolves #[Issue number to be closed when this PR is merged]
<!-- List any other related issues here -->

### Check List
- [ ] New functionality includes testing.
- [ ] All tests pass, including unit test, integration test and doctest
- [ ] New functionality has been documented.
- [ ] New functionality has javadoc added
- [ ] New functionality has user manual doc added
- [ ] Commits are signed per the DCO using --signoff
- [ ] New functionality has javadoc added.
- [ ] New functionality has a user manual doc added.
- [ ] API changes companion pull request [created](https://github.com/opensearch-project/opensearch-api-specification/blob/main/DEVELOPER_GUIDE.md).
- [ ] Commits are signed per the DCO using `--signoff`.
- [ ] Public documentation issue/PR [created](https://github.com/opensearch-project/documentation-website/issues/new/choose).

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).
For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/sql/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).
25 changes: 11 additions & 14 deletions .github/workflows/integ-tests-with-security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,24 +21,21 @@ jobs:
fail-fast: false
matrix:
java: [21]

runs-on: ubuntu-latest
container:
# using the same image which is used by opensearch-build team to build the OpenSearch Distribution
# this image tag is subject to change as more dependencies and updates will arrive over time
image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
# need to switch to root so that github actions can install runner binary on container without permission issues.
options: --user root

# Allow using Node16 actions
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}

steps:
- uses: actions/checkout@v3
- name: Run start commands
run: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-command }}

- uses: actions/checkout@v4

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.java }}
Expand All @@ -50,7 +47,7 @@ jobs:
- name: Upload test reports
if: ${{ always() }}
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: test-reports-${{ matrix.os }}-${{ matrix.java }}
Expand All @@ -63,16 +60,16 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ windows-latest ]
os: [ windows-latest, macos-13 ]
java: [21]

runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.java }}
Expand All @@ -82,7 +79,7 @@ jobs:

- name: Upload test reports
if: ${{ always() }}
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: test-reports-${{ matrix.os }}-${{ matrix.java }}
Expand Down
13 changes: 8 additions & 5 deletions .github/workflows/sql-pitest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,17 @@ jobs:
# using the same image which is used by opensearch-build team to build the OpenSearch Distribution
# this image tag is subject to change as more dependencies and updates will arrive over time
image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
# need to switch to root so that github actions can install runner binary on container without permission issues.
options: --user root
options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}


steps:
- uses: actions/checkout@v3
- name: Run start commands
run: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-command }}

- uses: actions/checkout@v4

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.java }}
Expand All @@ -46,7 +49,7 @@ jobs:
- name: Upload test reports
if: always()
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: test-reports-${{ matrix.entry.java }}
path: |
Expand Down
86 changes: 63 additions & 23 deletions .github/workflows/sql-test-and-build-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,31 +34,24 @@ jobs:
# using the same image which is used by opensearch-build team to build the OpenSearch Distribution
# this image tag is subject to change as more dependencies and updates will arrive over time
image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
# need to switch to root so that github actions can install runner binary on container without permission issues.
options: --user root

# Allow using Node16 actions
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}

steps:
- uses: actions/checkout@v3
- name: Run start commands
run: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-command }}

- uses: actions/checkout@v4

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.java }}

- name: Build with Gradle
run: |
chown -R 1000:1000 `pwd`
su `id -un 1000` -c "./gradlew --continue build"
- name: Run backward compatibility tests
run: |
chown -R 1000:1000 `pwd`
su `id -un 1000` -c "./scripts/bwctest.sh"
su `id -un 1000` -c "./gradlew --continue --parallel build"
- name: Create Artifact Path
run: |
Expand All @@ -68,22 +61,22 @@ jobs:
# This step uses the codecov-action Github action: https://github.com/codecov/codecov-action
- name: Upload SQL Coverage Report
if: ${{ always() }}
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
continue-on-error: true
with:
flags: sql-engine
token: ${{ secrets.CODECOV_TOKEN }}

- name: Upload Artifacts
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: opensearch-sql-ubuntu-latest-${{ matrix.java }}
path: opensearch-sql-builds

- name: Upload test reports
if: ${{ always() }}
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: test-reports-ubuntu-latest-${{ matrix.java }}
Expand All @@ -107,19 +100,20 @@ jobs:
matrix:
entry:
- { os: windows-latest, java: 21, os_build_args: -x doctest -PbuildPlatform=windows }
- { os: macos-13, java: 21 }
runs-on: ${{ matrix.entry.os }}

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.entry.java }}

- name: Build with Gradle
run: ./gradlew --continue build ${{ matrix.entry.os_build_args }}
run: ./gradlew --continue --parallel build ${{ matrix.entry.os_build_args }}

- name: Create Artifact Path
run: |
Expand All @@ -129,22 +123,22 @@ jobs:
# This step uses the codecov-action Github action: https://github.com/codecov/codecov-action
- name: Upload SQL Coverage Report
if: ${{ always() && matrix.entry.os == 'ubuntu-latest' }}
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
continue-on-error: true
with:
flags: sql-engine
token: ${{ secrets.CODECOV_TOKEN }}

- name: Upload Artifacts
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: opensearch-sql-${{ matrix.entry.os }}-${{ matrix.entry.java }}
path: opensearch-sql-builds

- name: Upload test reports
if: ${{ always() && matrix.entry.os == 'ubuntu-latest' }}
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: test-reports-${{ matrix.entry.os }}-${{ matrix.entry.java }}
Expand All @@ -160,3 +154,49 @@ jobs:
plugin/build/reports/**
doctest/build/testclusters/docTestCluster-0/logs/*
integ-test/build/testclusters/*/logs/*
bwc-tests:
needs: Get-CI-Image-Tag
runs-on: ubuntu-latest
strategy:
matrix:
java: [21]
container:
image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}

steps:
- name: Run start commands
run: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-command }}

- uses: actions/checkout@v4

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.java }}

- name: Run backward compatibility tests
run: |
chown -R 1000:1000 `pwd`
su `id -un 1000` -c "./scripts/bwctest.sh"
- name: Upload test reports
if: ${{ always() }}
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: test-reports-ubuntu-latest-${{ matrix.java }}-bwc
path: |
sql/build/reports/**
ppl/build/reports/**
core/build/reports/**
common/build/reports/**
opensearch/build/reports/**
integ-test/build/reports/**
protocol/build/reports/**
legacy/build/reports/**
plugin/build/reports/**
doctest/build/testclusters/docTestCluster-0/logs/*
integ-test/build/testclusters/*/logs/*
12 changes: 7 additions & 5 deletions .github/workflows/sql-test-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,16 @@ jobs:
# using the same image which is used by opensearch-build team to build the OpenSearch Distribution
# this image tag is subject to change as more dependencies and updates will arrive over time
image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
# need to switch to root so that github actions can install runner binary on container without permission issues.
options: --user root
options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}

steps:
- uses: actions/checkout@v3
- name: Run start commands
run: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-command }}

- uses: actions/checkout@v4

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.java }}
Expand Down Expand Up @@ -84,7 +86,7 @@ jobs:
- name: Upload test reports
if: always()
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: test-reports-${{ matrix.entry.java }}
path: |
Expand Down
28 changes: 28 additions & 0 deletions .github/workflows/stalled.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Label Stalled PRs
on:
schedule:
- cron: '15 15 * * *' # Run every day at 15:15 UTC / 7:15 PST / 8:15 PDT
permissions:
pull-requests: write
jobs:
stale:
if: github.repository == 'opensearch-project/sql'
runs-on: ubuntu-latest
steps:
- name: GitHub App token
id: github_app_token
uses: tibdex/[email protected]
with:
app_id: ${{ secrets.APP_ID }}
private_key: ${{ secrets.APP_PRIVATE_KEY }}
installation_id: 22958780
- name: Stale PRs
uses: actions/stale@v9
with:
repo-token: ${{ steps.github_app_token.outputs.token }}
stale-pr-label: 'stalled'
stale-pr-message: 'This PR is stalled because it has been open for 30 days with no activity.'
days-before-pr-stale: 30
days-before-issue-stale: -1
days-before-pr-close: -1
days-before-issue-close: -1
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,5 @@ gen
.worktrees
http-client.env.json
/doctest/sql-cli/
/doctest/opensearch-job-scheduler/
.factorypath
Loading

0 comments on commit d4ed787

Please sign in to comment.