-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] [REMOTE STORE] Unable to index data if region
key not provided in node attribute
#9942
Comments
Also the settings have not been updated in https://github.com/opensearch-project/OpenSearch/blob/2.x/distribution/src/config/opensearch.yml#L90-L100 |
Hi @rishabh6788, this is not specific to remote store. To enable remote store feature, you need to provide node attributes that will help create the repository. If we create a repository to take snapshot, then also Instead of providing region in s3 repository creation, you can also set default region by adding following in the yml file:
|
Repo settings are missing, we will get them added. Thanks for reporting it. |
Thanks for the response @sachinpkale. Also since the exception If I understand you correctly, in case a user wants to use remote-store they need to either add |
As per my discussion with @sachinpkale |
@rishabh6788 @sachinpkale I still have questions on the According to the doc of S3 repository plugin about
I think I knew that |
Where would us-west-2 come from as a default? It wasn't the original AWS region (that was us-east-1). The SDK should be able to use DefaultAwsRegionProviderChain to infer the region, especially if running on EC2. |
If I understand correctly the repository registration is now handled by the OS process and not the The default region dependency was removed from |
I found that there is no default It also means that I think there are 2 defects:
|
@sachinpkale confirmed that the repository registration flow haven't been changed as part of remote store feature. The same behavior would exist as trying to register a repository for snapshot. I just realized previously the @rishabh6788 I think maybe next step is to verify whether |
In general if you just want to verify if we can register a snapshot repo without region key and then upload snapshots to it, then we can test it out. |
@sachinpkale @tlfeng I created a 2.10.0 single node cluster with no feature, just plain docrep in
I already have
I confirmed that the snapshot was uploaded successfully as well.
As you can see that If it is mandatory parameter to make remote-store work then we need to call this out and add in the documentation till we figure out why is it not working as expected with remote-store enabled. |
The issue is not with respository creation, the issue is with data upload part while using remote-store feature. |
Describe the bug
Following changes from #9105 created a 2.10.0 OS cluster with remote-store settings mentioned below:
While running benchmark against this cluster I am seeing below failures in the opensearch log:
However when added
region
key in thenode.attr.remote_store.repository.opensearch-infra-stack-test-new-setting-v4-repo.settings:
setting it is working fine and I am able to index the data without any issue.There should be a default for
region
if not provided explicitly in the node attribute.To Reproduce
Steps to reproduce the behavior:
Expected behavior
There should be a default for
region
if not provided explicitly in the node attribute and should not throw error.Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: