[BUG]

[taborzbislaw]

What is the bug?

I am sending requests to opensearch engine like:

curl -X GET 'https://mercury.seargin.com:3005/test_cv/_search' -u 'joedoe:qwerty' --insecure

which correctly returns an answer. I can also use correctly the following python code to send requests and receive answers:

import requests

#####################################
username = "joedoe"
password = "qwerty"
url = 'https://mercury.seargin.com:3005/test_cv/_search'
auth = (username, password)
response = requests.get(url, auth=auth, verify=False)

if response.status_code == 200:
    print("Request successful!")
    data = response.json() 
    print(data)
else:
    print(f"Request failed with status code: {response.status_code}")
#######################################

But when I use opensearch python client:

#####################################
host = 'https://mercury.seargin.com'
port = 3005
auth = ('joedoe', 'qwerty') 

client = OpenSearch(
    hosts = [{'host': host, 'port': port}],
    http_auth = auth,
    use_ssl = False,
    verify_certs = False,
    ssl_assert_hostname = False,
    ssl_show_warn = False,
)

query = {
  'size': 5,
  'query': {
    'match': {
    }
  }
}

response = client.search(
    body = query,
    index = 'test_cv'
)
##############################################

I get the following error:

Traceback (most recent call last):
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/connection/http_urllib3.py", line 280, in perform_request
    response = self.pool.urlopen(
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/util/retry.py", line 445, in increment
    raise reraise(type(error), error, _stacktrace)
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 1278, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 976, in send
    self.connect()
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connection.py", line 205, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPConnection object at 0x7583932fbb50>: Failed to resolve 'https://mercury.seargin.com' ([Errno -2] Name or service not known)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/media/zbislaw/dane/Projekty/Seargin_NLP/opensearch_merkury/opensearch.py", line 28, in <module>
    response = client.search(
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/client/utils.py", line 180, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/client/__init__.py", line 2359, in search
    return self.transport.perform_request(
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/transport.py", line 453, in perform_request
    raise e
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/transport.py", line 416, in perform_request
    status, headers_response, data = connection.perform_request(
  File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/connection/http_urllib3.py", line 295, in perform_request
    raise ConnectionError("N/A", str(e), e)
opensearchpy.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7583932fbb50>: Failed to resolve 'https://mercury.seargin.com' ([Errno -2] Name or service not known)) caused by: NameResolutionError(<urllib3.connection.HTTPConnection object at 0x7583932fbb50>: Failed to resolve 'https://mercury.seargin.com' ([Errno -2] Name or service not known))

Any help is appreciated

[dblock]

I believe the host needs to be just the host, without the https part, host = 'mercury.seargin.com'. Closing this since I'm pretty sure this is the issue. Feel free to comment/reopen if it's not.

You can use urlparse if you need to convert one to the other programmatically, like here.

A good place to ask questions may be the #clients channel on the public slack. There are many working samples in https://github.com/opensearch-project/opensearch-py/tree/main/samples to try out first, too.

[taborzbislaw]

Hi. Thank you for your prompt answer. I have removed "https://" from host name (thank you for link to github samples, I should have had a look at this page earlier). The corrected code is however not working. Below is the log. On the client side I am using the latest version of Linux Mint (22 Wilma) - maybe it is the problem? The installation instructions at https://opensearch.org/docs/latest/getting-started/quickstart/ do not work for this linux release too (docker-compose exits with errors at 'sudo docker-compose up -d' command with opensearch example yaml file). And here is the error log for the corrected python source:

Traceback (most recent call last):
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connectionpool.py", line 537, in _make_request
response = conn.getresponse()
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connection.py", line 466, in getresponse
httplib_response = super().getresponse()
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 1375, in getresponse
response.begin()
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/connection/http_urllib3.py", line 280, in perform_request
response = self.pool.urlopen(
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/util/retry.py", line 445, in increment
raise reraise(type(error), error, _stacktrace)
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/util/util.py", line 38, in reraise
raise value.with_traceback(tb)
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connectionpool.py", line 537, in _make_request
response = conn.getresponse()
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/urllib3/connection.py", line 466, in getresponse
httplib_response = super().getresponse()
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 1375, in getresponse
response.begin()
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/http/client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/media/zbislaw/dane/Projekty/Seargin_NLP/opensearch_merkury/opensearch.py", line 26, in
response = client.search(
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/client/utils.py", line 180, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/client/init.py", line 2359, in search
return self.transport.perform_request(
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/transport.py", line 453, in perform_request
raise e
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/transport.py", line 416, in perform_request
status, headers_response, data = connection.perform_request(
File "/home/zbislaw/anaconda3/envs/NLP/lib/python3.10/site-packages/opensearchpy/connection/http_urllib3.py", line 295, in perform_request
raise ConnectionError("N/A", str(e), e)
opensearchpy.exceptions.ConnectionError: ConnectionError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))

[dblock]

The "Remote end closed connection without response" error means the server didn't respond with anything the client expected.

Since it looks like you can do curl -u joedoe:qwerty https://mercury.seargin.com:3005successfully, can you please re-paste the exact code you ended up with?

I do notice use_ssl = False above, which is incorrect, you are using SSL.

[taborzbislaw]

Thank you very much, setting use_ssl to True solves the problem.
Best regards