-
Notifications
You must be signed in to change notification settings - Fork 183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added support for AWS Sigv4 for UrlLib3. #547
Conversation
Signed-off-by: dblock <[email protected]>
2a50056
to
a732d44
Compare
Codecov Report
@@ Coverage Diff @@
## main #547 +/- ##
==========================================
+ Coverage 70.63% 70.66% +0.02%
==========================================
Files 83 83
Lines 7857 7877 +20
==========================================
+ Hits 5550 5566 +16
- Misses 2307 2311 +4
|
Signed-off-by: dblock <[email protected]>
Signed-off-by: dblock <[email protected]>
Signed-off-by: dblock <[email protected]>
c9f113f
to
fb9e1b9
Compare
@harshavamsi would appreciate your CR since you did support for the requests connection class. |
Signed-off-by: dblock <[email protected]>
fb9e1b9
to
f8f2b62
Compare
Signed-off-by: dblock <[email protected]>
Signed-off-by: dblock <[email protected]>
Signed-off-by: dblock <[email protected]>
b709e40
to
8d5aeb0
Compare
Signed-off-by: dblock <[email protected]>
# Modifications Copyright OpenSearch Contributors. See | ||
# GitHub history for details. | ||
# | ||
# Licensed to Elasticsearch B.V. under one or more contributor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this license here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this code was refactored out of test_connection.py
, so let's keep it.
guides/auth.md
Outdated
@@ -9,24 +9,24 @@ OpenSearch allows you to use different methods for the authentication via `conne | |||
|
|||
## IAM Authentication | |||
|
|||
Opensearch-py supports IAM-based authentication via `AWSV4SignerAuth`, which uses `RequestHttpConnection` as the transport class for communicating with OpenSearch clusters running in Amazon Managed OpenSearch and OpenSearch Serverless, and works in conjunction with [botocore](https://pypi.org/project/botocore/). | |||
Opensearch-py supports IAM-based authentication via `RequestsAWSV4SignerAuth` and `Urllib3AWSV4SignerAuth`, which use `RequestHttpConnection` and `Urllib3HttpConnection` respectively, as the transport classes for communicating with OpenSearch clusters running in Amazon Managed OpenSearch and OpenSearch Serverless, and works in conjunction with [botocore](https://pypi.org/project/botocore/). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we recommend Urllib3AWSV4SignerAuth
here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've clarified the documentation in cb76874.
self.signer = AWSV4Signer(credentials, region, service) | ||
|
||
def __call__(self, method: str, url: str, body: Any) -> Dict[str, str]: | ||
return self.signer.sign(method, url, body) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
RequestsAWSV4SignerAuth
returns prepared request while Urllib3AWSV4SignerAuth
returns the headers, should we keep them consistent?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The requests
library expect a callable that derives from requests.auth.AuthBase
and receives a request
object. It's called inside the requests library.
The urllib3 library doesn't support such an interface. It actually splits the client constructor and exposes a perform_request
method. That's where we have the method, URL, and body, and call auth. There's no "request" object here.
We could make a similar interface to AuthBase
for urllib3, but it cannot be the same interface because that one exists in the requests library. Since these signer classes aren't interchangeable at all, they don't need to derive from the same class, and I already moved the common parts into AWSV4Signer
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That makes sense, thank you!
…uestHttpConnection. Signed-off-by: dblock <[email protected]>
Signed-off-by: dblock <[email protected]>
Signed-off-by: dblock <[email protected]>
Signed-off-by: dblock <[email protected]>
@VachaShah I wrote some more tests and addressed your comments. LMK if there's anything else needed for this one |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Signed-off-by: Djcarrillo6 <[email protected]> Added a guide on making raw JSON REST requests. (opensearch-project#542) Signed-off-by: dblock <[email protected]> Added document lifecycle guide & sample code. Signed-off-by: Djcarrillo6 <[email protected]> Updated CHANGELOG Signed-off-by: Djcarrillo6 <[email protected]> Added support for AWS Sigv4 for UrlLib3. (opensearch-project#547) * WIP: Added support for AWS Sigv4 for UrlLib3. Signed-off-by: dblock <[email protected]> * Refactored common implementation. Signed-off-by: dblock <[email protected]> * Added sigv4 samples. Signed-off-by: dblock <[email protected]> * Updated CHANGELOG. Signed-off-by: dblock <[email protected]> * Add documentation. Signed-off-by: dblock <[email protected]> * Use the correct class in tests. Signed-off-by: dblock <[email protected]> * Renamed samples. Signed-off-by: dblock <[email protected]> * Split up requests and urllib3 unit tests. Signed-off-by: dblock <[email protected]> * Rename AWSV4Signer. Signed-off-by: dblock <[email protected]> * Clarified documentation of when to use Urllib3AWSV4SignerAuth vs. RequestHttpConnection. Signed-off-by: dblock <[email protected]> * Move fetch_url inside the signer class. Signed-off-by: dblock <[email protected]> * Added unit test for Urllib3AWSV4SignerAuth adding headers. Signed-off-by: dblock <[email protected]> * Added unit test for signing to include query string. Signed-off-by: dblock <[email protected]> --------- Signed-off-by: dblock <[email protected]> Remove support for Python 2.x. (opensearch-project#548) Signed-off-by: dblock <[email protected]> Fixed guide & added link in USER_GUIDE.md Signed-off-by: Djcarrillo6 <[email protected]> Added support for AWS Sigv4 for UrlLib3. (opensearch-project#547) * WIP: Added support for AWS Sigv4 for UrlLib3. Signed-off-by: dblock <[email protected]> * Refactored common implementation. Signed-off-by: dblock <[email protected]> * Added sigv4 samples. Signed-off-by: dblock <[email protected]> * Updated CHANGELOG. Signed-off-by: dblock <[email protected]> * Add documentation. Signed-off-by: dblock <[email protected]> * Use the correct class in tests. Signed-off-by: dblock <[email protected]> * Renamed samples. Signed-off-by: dblock <[email protected]> * Split up requests and urllib3 unit tests. Signed-off-by: dblock <[email protected]> * Rename AWSV4Signer. Signed-off-by: dblock <[email protected]> * Clarified documentation of when to use Urllib3AWSV4SignerAuth vs. RequestHttpConnection. Signed-off-by: dblock <[email protected]> * Move fetch_url inside the signer class. Signed-off-by: dblock <[email protected]> * Added unit test for Urllib3AWSV4SignerAuth adding headers. Signed-off-by: dblock <[email protected]> * Added unit test for signing to include query string. Signed-off-by: dblock <[email protected]> --------- Signed-off-by: dblock <[email protected]> Remove support for Python 2.x. (opensearch-project#548) Signed-off-by: dblock <[email protected]> Fixed guide & added link in USER_GUIDE.md opensearch-project#3 Signed-off-by: Djcarrillo6 <[email protected]>
* WIP: Added support for AWS Sigv4 for UrlLib3. Signed-off-by: dblock <[email protected]> * Refactored common implementation. Signed-off-by: dblock <[email protected]> * Added sigv4 samples. Signed-off-by: dblock <[email protected]> * Updated CHANGELOG. Signed-off-by: dblock <[email protected]> * Add documentation. Signed-off-by: dblock <[email protected]> * Use the correct class in tests. Signed-off-by: dblock <[email protected]> * Renamed samples. Signed-off-by: dblock <[email protected]> * Split up requests and urllib3 unit tests. Signed-off-by: dblock <[email protected]> * Rename AWSV4Signer. Signed-off-by: dblock <[email protected]> * Clarified documentation of when to use Urllib3AWSV4SignerAuth vs. RequestHttpConnection. Signed-off-by: dblock <[email protected]> * Move fetch_url inside the signer class. Signed-off-by: dblock <[email protected]> * Added unit test for Urllib3AWSV4SignerAuth adding headers. Signed-off-by: dblock <[email protected]> * Added unit test for signing to include query string. Signed-off-by: dblock <[email protected]> --------- Signed-off-by: dblock <[email protected]> Signed-off-by: roma2023 <[email protected]>
Description
AWSV4SignerAuth
in favor ofRequestsAWSV4SignerAuth
Urllib3AWSV4SignerAuth
in the documentation since that transport is faster.Testing
AOS
AOSS
Issues Resolved
Closes #546
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.