[Backport 2.x] Backporting commits to 2.x (#400)

* tannerjones4075- update developer guide (#385) * Updating DEVELOPER_GUIDE.md file to include docker build instructions Signed-off-by: Tanner Jones <[email protected]> * Updating DEVELOPER_GUIDE.md file to include docker build instructions Signed-off-by: Tanner Jones <[email protected]> * Updating DEVELOPER_GUIDE.md file and CHANGELOG Signed-off-by: Tanner Jones <[email protected]> * Removed link from localhost:9200 Signed-off-by: Tanner Jones <[email protected]> --------- Signed-off-by: Tanner Jones <[email protected]> Signed-off-by: Vacha Shah <[email protected]> * Bump github.com/aws/aws-sdk-go from 1.45.20 to 1.45.24 (#387) * Bump github.com/aws/aws-sdk-go from 1.45.20 to 1.45.24 Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.45.20 to 1.45.24. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.45.20...v1.45.24) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/aws-sdk-go-v2 from 1.21.0 to 1.21.1 (#388) * Bump github.com/aws/aws-sdk-go-v2 from 1.21.0 to 1.21.1 Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.21.0 to 1.21.1. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@v1.21.0...v1.21.1) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/aws-sdk-go-v2/config from 1.18.43 to 1.18.44 (#389) * Bump github.com/aws/aws-sdk-go-v2/config from 1.18.43 to 1.18.44 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.18.43 to 1.18.44. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@config/v1.18.43...config/v1.18.44) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Bump golang.org/x/net from 0.7.0 to 0.17.0 (#391) * Bump golang.org/x/net from 0.7.0 to 0.17.0 Bumps [golang.org/x/net](https://github.com/golang/net) from 0.7.0 to 0.17.0. - [Commits](golang/net@v0.7.0...v0.17.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Update DEVELOPER_GUID, Add Coverage + go ling to Makefile, Run CI tests on all branches (#392) * .github: run test workflows for all branches, test against 2.9,2.10 Signed-off-by: Jakob Hahn <[email protected]> * Makefile: use docker golangci-lint, run integration test on `.` folder, change coverage generation Signed-off-by: Jakob Hahn <[email protected]> * Update DEVELOPER_GUIDE Signed-off-by: Jakob Hahn <[email protected]> --------- Signed-off-by: Jakob Hahn <[email protected]> Signed-off-by: Vacha Shah <[email protected]> * Updated linter version to 1.54.2 (#394) * Updated linter version to 1.54.2 Signed-off-by: Tanner Jones <[email protected]> * Updated CHANGELOG.md Signed-off-by: Tanner Jones <[email protected]> * Updated CHANGELOG.md Signed-off-by: Tanner Jones <[email protected]> * Updated CHANGELOG.md - moved linter update to Dependencies Bumps Signed-off-by: Tanner Jones <[email protected]> --------- Signed-off-by: Tanner Jones <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: Vacha Shah <[email protected]> * Deprecate legacy API (#390) * deprecated legacy REST API /_template Signed-off-by: Tanner Jones <[email protected]> * deprecated legacy REST API /_template Signed-off-by: Tanner Jones <[email protected]> * removed deprecated header at the top of the doc Signed-off-by: Tanner Jones <[email protected]> * added deprecated notes to the CHANGELOG.md Signed-off-by: Tanner Jones <[email protected]> * Corrected deprecation statement Signed-off-by: Tanner Jones <[email protected]> * Update CHANGELOG.md Co-authored-by: Jakob <[email protected]> Signed-off-by: Tanner Jones <[email protected]> * Updated deprecated message on api.indices.put_template.go Signed-off-by: Tanner Jones <[email protected]> * Added deprecated message on api.indices.exists_template.go Signed-off-by: Tanner Jones <[email protected]> * Added deprecated message on api.indices.delete_template.go Signed-off-by: Tanner Jones <[email protected]> * Added deprecated message on api.indices.get_template.go Signed-off-by: Tanner Jones <[email protected]> * Updated deprecated message on api.indices.exists_template.go and added back in IndicesExistsTemplate that was delete by mistake Signed-off-by: Tanner Jones <[email protected]> --------- Signed-off-by: Tanner Jones <[email protected]> Signed-off-by: Tanner Jones <[email protected]> Co-authored-by: Jakob <[email protected]> Signed-off-by: Vacha Shah <[email protected]> * Bump github.com/aws/aws-sdk-go-v2/config from 1.18.44 to 1.19.0 (#398) * Bump github.com/aws/aws-sdk-go-v2/config from 1.18.44 to 1.19.0 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.18.44 to 1.19.0. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/v1.19.0/CHANGELOG.md) - [Commits](aws/aws-sdk-go-v2@config/v1.18.44...v1.19.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/aws-sdk-go from 1.45.24 to 1.45.26 (#397) * Bump github.com/aws/aws-sdk-go from 1.45.24 to 1.45.26 Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.45.24 to 1.45.26. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.45.24...v1.45.26) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Bump client version to 2.4.0 Signed-off-by: Vacha Shah <[email protected]> --------- Signed-off-by: Tanner Jones <[email protected]> Signed-off-by: Vacha Shah <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Jakob Hahn <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: Tanner Jones <[email protected]> Co-authored-by: Tanner Jones <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakob <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]>
opensearch-project · Oct 23, 2023 · 8f11c2d · 8f11c2d
1 parent bb74a19
commit 8f11c2d
Show file tree

Hide file tree

Showing 17 changed files with 204 additions and 125 deletions.
diff --git a/.github/workflows/links.yml b/.github/workflows/links.yml
@@ -1,11 +1,6 @@
 name: Link Checker
-on:
-  push:
-    branches:
-      - "*"
-  pull_request:
-    branches:
-      - "*"
+
+on: [push, pull_request]
 
 jobs:
   linkchecker:

diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -1,12 +1,6 @@
 name: Lint check
 
-on:
-  push:
-    branches:
-      - "*"
-  pull_request:
-    branches:
-      - "*"
+on: [push, pull_request]
 
 env:
   GITHUB_ACTIONS: true
@@ -23,7 +17,7 @@ jobs:
       - name: golangci-lint
         uses: golangci/golangci-lint-action@v3
         with:
-          version: v1.53.3
+          version: v1.54.2
           args: -v -c .golangci.yml
           only-new-issues: true
 

diff --git a/.github/workflows/test-compatibility.yml b/.github/workflows/test-compatibility.yml
@@ -3,7 +3,7 @@ name: Integration for Compatibility
 on: [push, pull_request]
 
 jobs:
-  integration-test-compatibility:
+  integ-test-compat:
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false
@@ -22,6 +22,8 @@ jobs:
           - { opensearch_version: 2.6.0 }
           - { opensearch_version: 2.7.0 }
           - { opensearch_version: 2.8.0 }
+          - { opensearch_version: 2.9.0 }
+          - { opensearch_version: 2.10.0 }
     steps:
       - uses: actions/checkout@v3
         with: { fetch-depth: 1 }

diff --git a/.github/workflows/test-integration.yml b/.github/workflows/test-integration.yml
@@ -1,12 +1,6 @@
 name: Integration
 
-on:
-  push:
-    branches:
-      - "*"
-  pull_request:
-    branches:
-      - "*"
+on: [push, pull_request]
 
 env:
   GITHUB_ACTIONS: true
@@ -30,6 +24,9 @@ jobs:
       - name: Launch OpenSearch cluster
         run: |
           make cluster.clean cluster.build cluster.start
+          for attempt in `seq 25`; do sleep 5; \
+          if curl -s localhost:9200; \
+          then echo '=====> ready'; break; fi; if [ $attempt == 25 ]; then exit 1; fi; echo '=====> waiting...'; done
       - run: make test-integ race=true
       - uses: codecov/codecov-action@v1
         with:
@@ -57,4 +54,7 @@ jobs:
       - name: Launch OpenSearch cluster
         run: |
           make cluster.clean cluster.build cluster.start
-      - run: make test-integ-secure
+          for attempt in `seq 25`; do sleep 5; \
+          if curl -s -ku admin:admin https://localhost:9200; \
+          then echo '=====> ready'; break; fi; if [ $attempt == 25 ]; then exit 1; fi; echo '=====> waiting...'; done
+      - run: make test-integ-secure
diff --git a/.github/workflows/test-unit.yml b/.github/workflows/test-unit.yml
@@ -1,12 +1,6 @@
 name: Unit
 
-on:
-  push:
-    branches:
-      - "*"
-  pull_request:
-    branches:
-      - "*"
+on: [push, pull_request]
 
 env:
   GITHUB_ACTIONS: true

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,10 +6,12 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ### Dependencies
 
-- Bumps `github.com/aws/aws-sdk-go` from 1.44.180 to 1.45.20
-- Bumps `github.com/aws/aws-sdk-go-v2` from 1.17.4 to 1.21.0
-- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.8 to 1.18.43
+- Bumps `github.com/aws/aws-sdk-go` from 1.44.180 to 1.45.26
+- Bumps `github.com/aws/aws-sdk-go-v2` from 1.17.4 to 1.21.1
+- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.8 to 1.19.0
 - Bumps `github.com/stretchr/testify` from 1.8.1 to 1.8.4
+- Bumps `golang.org/x/net` from 0.7.0 to 0.17.0
+- Bumps `github.com/golangci/golangci-lint-action` from 1.53.3 to 1.54.2
 
 ### Added
 
@@ -41,9 +43,14 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Read, close and replace the http Reponse Body ([#300](https://github.com/opensearch-project/opensearch-go/pull/300))
 - Updated and adjusted golangci-lint, solve linting complains for signer ([#352](https://github.com/opensearch-project/opensearch-go/pull/352))
 - Solve linting complains for opensearchtransport ([#353](https://github.com/opensearch-project/opensearch-go/pull/353))
+- Updated Developer guide to include docker build instructions ([#385](https://github.com/opensearch-project/opensearch-go/pull/385)
+- Test against version 2.9.0,2.10.0, run tests in all branches, change intergration tests to wait for OpenSearch to start ([#392](https://github.com/opensearch-project/opensearch-go/pull/392))
+- Makefile: use docker golangci-lint, run integration test on `.` folder, change coverage generation ([#392](https://github.com/opensearch-project/opensearch-go/pull/392)) 
 
 ### Deprecated
 
+- Deprecate legacy API /_template ([#151](https://github.com/opensearch-project/opensearch-go/issues/151))
+
 ### Removed
 
 - Removes info call before performing every request ([#219](https://github.com/opensearch-project/opensearch-go/pull/219))

diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md
@@ -7,11 +7,15 @@
       - [Windows](#windows)
     - [Unit Testing](#unit-testing)
     - [Integration Testing](#integration-testing)
+    - [Composing an OpenSearch Docker Container](#composing-an-opensearch-docker-container)
       - [Execute integration tests from your terminal](#execute-integration-tests-from-your-terminal)
-    - [Lint](#lint)
-      - [Markdown lint](#markdown-lint)
+  - [Lint](#lint)
+    - [Markdown lint](#markdown-lint)
+    - [Go lint](#go-lint)
+  - [Coverage](#coverage)
   - [Use an Editor](#use-an-editor)
     - [GoLand](#goland)
+    - [Vim](#vim)
 
 # Developer Guide
 
@@ -25,9 +29,9 @@ Fork [opensearch-project/opensearch-go](https://github.com/opensearch-project/op
 
 ### Install Prerequisites
 
-#### Go 1.11
+#### Go 1.15
 
-OpenSearch Go Client builds using [Go](https://golang.org/doc/install) 1.11 at a minimum.
+OpenSearch Go Client builds using [Go](https://golang.org/doc/install) 1.15 at a minimum.
 
 #### Docker
 
@@ -37,7 +41,7 @@ OpenSearch Go Client builds using [Go](https://golang.org/doc/install) 1.11 at a
 
 To build the project on Windows, use [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install), the compatibility layer for running Linux applications.
 
-Install ```make```
+Install `make`
 ```
 sudo apt install make
 ```
@@ -61,6 +65,18 @@ go test -v -run TestName;
 
 In order to test opensearch-go client, you need a running OpenSearch cluster. You can use Docker to accomplish this. The [Docker Compose file](.ci/opensearch/docker-compose.yml) supports the ability to run integration tests for the project in local environments. If you have not installed docker-compose, you can install it from this [link](https://docs.docker.com/compose/install/).
 
+### Composing an OpenSearch Docker Container
+
+Ensure that Docker is installed on your local machine. You can check by running `docker --version`. Next, navigate to your local opensearch-go repository. Run the following command to build and start the OpenSearch docker container. 
+
+```
+make cluster.build cluster.start
+```
+
+This command will start the OpenSearch container using the `docker-compose.yaml` configuration file. During the build process, the necessary dependencies and files will be downloaded, which may take some time depending on your internet connection and system resources.
+
+Once the container is built and running, you can open a web browser and navigate to localhost:9200 to access the OpenSearch docker container. 
+
 In order to differentiate unit tests from integration tests, Go has a built-in mechanism for allowing you to logically separate your tests with [build tags](https://pkg.go.dev/cmd/go#hdr-Build_constraints). The build tag needs to be placed as close to the top of the file as possible, and must have a blank line beneath it. Hence, create all integration tests with build tag 'integration'.
 
 #### Execute integration tests from your terminal
@@ -90,6 +106,30 @@ To check the markdown files, run the following command:
 make lint.markdown
 ```
 
+### Go lint
+
+To check all go files, run the following command:
+
+```
+make linters
+```
+
+## Coverage
+
+To get the repository test coverage, run the following command:
+
+For the results to be display in your terminal:
+
+```
+make coverage
+```
+
+For the results to be display in your browser:
+
+```
+make coverage-html
+```
+
 ## Use an Editor
 
 ### GoLand
@@ -100,3 +140,8 @@ You can import the OpenSearch project into GoLand as follows:
 2. In the subsequent dialog navigate to the ~/go/src/opensearch-go and click **Open**
 
 After you have opened your project, you need to specify the location of the Go SDK. You can either specify a local path to the SDK or download it. To set the Go SDK, navigate to **Go | GOROOT** and set accordingly.
+
+### Vim
+
+To improve your vim experience with Go, you might want to check out [fatih/vim-go](https://github.com/fatih/vim-go).
+For example it correctly formats the file and validates it on save.
diff --git a/Makefile b/Makefile
@@ -38,7 +38,7 @@ endif
 		gotestsum --format=short-verbose --junitfile=tmp/integration-report.xml -- $(testintegargs) "./opensearchtransport" "./opensearchapi" "./opensearchutil"; \
 	else \
 		echo "go test -v" $(testintegargs) "."; \
-		go test -v $(testintegargs) "./opensearchtransport" "./opensearchapi" "./opensearchutil"; \
+		go test -v $(testintegargs) "." "./opensearchtransport" "./opensearchapi" "./opensearchutil"; \
 	fi;
 
 test-integ-secure: ##Run secure integration tests
@@ -48,11 +48,24 @@ test-bench:  ## Run benchmarks
 	@printf "\033[2m→ Running benchmarks...\033[0m\n"
 	go test -run=none -bench=. -benchmem ./...
 
-test-coverage:  ## Generate test coverage report
+coverage:  ## Print test coverage report
+	@make gen-coverage
+	@go tool cover -func=$(PWD)/tmp/total.cov
+	@printf "\033[0m--------------------------------------------------------------------------------\n\033[0m"
+
+coverage-html: ## Open test coverage report in browser
+	@make gen-coverage
+	@go tool cover -html $(PWD)/tmp/total.cov
+
+gen-coverage:  ## Generate test coverage report
 	@printf "\033[2m→ Generating test coverage report...\033[0m\n"
-	@go tool cover -html=tmp/unit.cov -o tmp/coverage.html
-	@go tool cover -func=tmp/unit.cov | 'grep' -v 'opensearchapi/api\.' | sed 's/github.com\/opensearch-project\/opensearch-go\///g'
-	@printf "\033[0m--------------------------------------------------------------------------------\nopen tmp/coverage.html\n\n\033[0m"
+	@rm -rf tmp
+	@mkdir tmp
+	@mkdir tmp/unit
+	@mkdir tmp/integration
+	@go test -cover ./... -args -test.gocoverdir="$(PWD)/tmp/unit"
+	@go test -cover -tags='integration' ./... -args -test.gocoverdir="$(PWD)/tmp/integration"
+	@go tool covdata textfmt -i=$(PWD)/tmp/unit,$(PWD)/tmp/integration -o $(PWD)/tmp/total.cov
 
 ##@ Development
 lint:  ## Run lint on the package
@@ -205,7 +218,7 @@ cluster.clean: ## Remove unused Docker volumes and networks
 	docker system prune --volumes --force
 
 linters:
-	./bin/golangci-lint run ./... --timeout=5m
+	docker run -t --rm -v $$(pwd):/app -v ~/.cache/golangci-lint/v1.53.3:/root/.cache -w /app golangci/golangci-lint:v1.53.3 golangci-lint run --timeout=5m
 
 workflow: ## Run all github workflow commands here sequentially
 

diff --git a/go.mod b/go.mod
@@ -3,10 +3,10 @@ module github.com/opensearch-project/opensearch-go/v2
 go 1.15
 
 require (
-	github.com/aws/aws-sdk-go v1.45.20
-	github.com/aws/aws-sdk-go-v2 v1.21.0
-	github.com/aws/aws-sdk-go-v2/config v1.18.43
+	github.com/aws/aws-sdk-go v1.45.26
+	github.com/aws/aws-sdk-go-v2 v1.21.2
+	github.com/aws/aws-sdk-go-v2/config v1.19.0
 	github.com/kinbiko/jsonassert v1.1.1
 	github.com/stretchr/testify v1.8.4
-	golang.org/x/net v0.7.0 // indirect
+	golang.org/x/net v0.17.0 // indirect
 )