Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5b280fd590>: Failed to establish a new connection

[son7je]

Hi
I am trying to connect splunk - cloud imperva waf through API communication. However, the connection is not possible due to the following error.

Why does an error occur?
Do I need a WAF certificate? Where can I find the certificate?
I need your help.

====================================================================================

2024-02-26 14:31:33,201 process_thread INFO Sleeping for 5 seconds before trying to fetch logs again...
2024-02-26 14:31:38,206 process_thread INFO Downloading logs index file...
2024-02-26 14:31:38,208 process_thread WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5b280aa510>: Failed to establish a new connection: [Errno -2] Name or service not known')': /13735_1970844/logs.indexlogs.index
2024-02-26 14:31:38,209 process_thread WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5b280aa690>: Failed to establish a new connection: [Errno -2] Name or service not known')': /13735_1970844/logs.indexlogs.index
2024-02-26 14:31:38,209 process_thread WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5b280aa1d0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /13735_1970844/logs.indexlogs.index
2024-02-26 14:31:38,210 process_thread ERROR HTTPSConnectionPool(host='logs1.incapsula.com', port=443): Max retries exceeded with url: /13735_1970844/logs.indexlogs.index (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5b280c4910>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Traceback (most recent call last):
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/opt/splunk_hv/lib/python3.7/socket.py", line 752, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
conn.connect()
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connection.py", line 308, in connect
conn = self._new_conn()
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f5b280c4910>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/splunk_hv/etc/apps/incapsula-logs-downloader-master/script/FileDownloader.py", line 42, in request_file_content
response = self.https.request('GET', url, headers=auth_header)
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/request.py", line 76, in request
method, url, fields=fields, headers=headers, **urlopen_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/request.py", line 97, in request_encode_url
return self.urlopen(method, url, **extra_kw)
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/poolmanager.py", line 336, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
**response_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
**response_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
**response_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='logs1.incapsula.com', port=443): Max retries exceeded with url: /13735_1970844/logs.indexlogs.index (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5b280c4910>: Failed to establish a new connection: [Errno -2] Name or service not known'))
2024-02-26 14:31:38,211 process_thread ERROR Failed to downloading index file and starting to download all the log files in it - name 'response' is not defined, Traceback (most recent call last):
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/opt/splunk_hv/lib/python3.7/socket.py", line 752, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
conn.connect()
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connection.py", line 308, in connect
conn = self._new_conn()
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f5b280c4910>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/splunk_hv/etc/apps/incapsula-logs-downloader-master/script/FileDownloader.py", line 42, in request_file_content
response = self.https.request('GET', url, headers=auth_header)
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/request.py", line 76, in request
method, url, fields=fields, headers=headers, **urlopen_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/request.py", line 97, in request_encode_url
return self.urlopen(method, url, **extra_kw)
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/poolmanager.py", line 336, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
**response_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
**response_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
**response_kw
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/opt/splunk_hv/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='logs1.incapsula.com', port=443): Max retries exceeded with url: /13735_1970844/logs.indexlogs.index (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5b280c4910>: Failed to establish a new connection: [Errno -2] Name or service not known'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/splunk_hv/etc/apps/incapsula-logs-downloader-master/script/FileDownloader.py", line 73, in request_file_content
raise Exception("Connection error")
Exception: Connection error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/splunk_hv/etc/apps/incapsula-logs-downloader-master/script/LogsDownloader.py", line 140, in get_index_file
self.logs_file_index.download()
File "/opt/splunk_hv/etc/apps/incapsula-logs-downloader-master/script/LogsFileIndex.py", line 35, in download
file_content = self.file_downloader.request_file_content(self.config.BASE_URL + "logs.index")
File "/opt/splunk_hv/etc/apps/incapsula-logs-downloader-master/script/FileDownloader.py", line 79, in request_file_content
response.close()
NameError: name 'response' is not defined

[joeymoore]

@son7je this error "Name or service not known" is saying that the box is unable to resolve the URL via DNS. Can you do an nslookup on the "logs1.incapsula.com" and possibly your splunk cloud endpoint URL?

[son7je]

@son7je this error "Name or service not known" is saying that the box is unable to resolve the URL via DNS. Can you do an nslookup on the "logs1.incapsula.com" and possibly your splunk cloud endpoint URL?

================================================================================
@joeymoore
Hi

Are you saying that logs1.incapsula.com can be nslookuped from within splunk?

I think splunk can't find the path to log1.incapsula.com.

However, the imperva waf manager said that the API URI was only log1.incapsula.com.

[joeymoore]

@son7je please go to my github profile (github.com/joeymoore) and email me directly. We can jump on a zoom and resolve this.

[son7je]

@joeymoore OK. I just sent you an email. Could you check your email?

[son7je]

@son7je this error "Name or service not known" is saying that the box is unable to resolve the URL via DNS. Can you do an nslookup on the "logs1.incapsula.com" and possibly your splunk cloud endpoint URL?

[AVitg]

the log say could not download /<your_id>/logs.indexlogs.index

logs.indexlogs.index does not sound right, it should read eventually "logs.index" ... dont know why the script does this to you