Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blocksat-cli standalone monitor --report crashes #94

Open
bg002h opened this issue Jan 26, 2023 · 7 comments
Open

blocksat-cli standalone monitor --report crashes #94

bg002h opened this issue Jan 26, 2023 · 7 comments

Comments

@bg002h
Copy link

bg002h commented Jan 26, 2023

blocksat-cli crashes after a day or three when reporting satellite receiver stats...seems to be triggered by network/destination connectivity issues. Probably needs some exception handling around network commands.

@bg002h
Copy link
Author

bg002h commented Jan 27, 2023

Sample CLI output after a crash:

Lock = True; Level = -55.00dBm; SNR = 14.10dB; BER = 8.94e-2023-01-26 04:00:43 Lock = True; Level = -55.00
dBm; SNR = 14.20dB; BER = 8.94e-Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1012, in validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 411, in connect
self.sock = ssl_wrap_socket(
File "/usr/lib/python3/dist-packages/urllib3/util/ssl
.py", line 449, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl(
File "/usr/lib/python3/dist-packages/urllib3/util/ssl
.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib/python3.10/ssl.py", line 1071, in _create
self.do_handshake()TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 532, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 385, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='satellite.blockstream.space', port=443): Read timed out. (read timeout=None)During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/blocksat-cli", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/blocksatcli/main.py", line 81, in main
args.func(args)
File "/usr/local/lib/python3.10/dist-packages/blocksatcli/standalone.py", line 647, in monitor
monitor.update(stats)
File "/usr/local/lib/python3.10/dist-packages/blocksatcli/monitoring.py", line 351, in update
self.reporter.send(self.get_stats())
File "/usr/local/lib/python3.10/dist-packages/blocksatcli/monitoring.py", line 138, in send
r = requests.post(self.dest_addr,
File "/usr/lib/python3/dist-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='satellite.blockstream.space', port=443): Read timed out. (read timeout=None)

@bg002h
Copy link
Author

bg002h commented Jan 27, 2023

I've never done any exception handling in python, but I'm gonna try to add these three lines to catch any uncaught errors:

except Exception as e:
print()
logger.error("Report failed: " + str(e))

to line 147 of /usr/local/lib/python3.10/dist-packages/blocksatcli/monitoring.py

and I'll see what happens...i'm hypothesizing the error is just not handled (there is handling for requests.exceptions.ConnectionError ). I'll report back in a day or three.

@bg002h
Copy link
Author

bg002h commented Jan 27, 2023

...and apparently python cares about spaces vs tabs...yeesh. Picky picky picky. Anyhow, I got it running. Should take a day or so to run into trouble naturally...maybe I can provoke it by pulling the network cable...

@bg002h
Copy link
Author

bg002h commented Jan 27, 2023

https://github.com/bg002h/satellite_AmateurBugFix/blob/master/blocksatcli/monitoring.py

I totally don't grok git, but, even with doing the whole secret service git authentication thing, I can't create a pull request. See link above for my trivial addition (lines 147-149)

@blockstreamsatellite
Copy link
Contributor

Hi @bg002h ,

We released a new CLI version today. Could you try it and see if the new version fixes your problem?

Thanks

@bg002h
Copy link
Author

bg002h commented Mar 5, 2023 via email

@bg002h
Copy link
Author

bg002h commented Mar 12, 2023

latest update to 0.4.5 doesn't exactly fix the crash...the program seems to simply hang and stop producing output after no apparent error...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants