'i/o timeout' and mi_datagram 'failed to send the response' errors #8
Comments
|
My first guess is the opensips socket file is owned by another user than the one you're starting the opensips exporter with. This in turn boils down to the 'permission denied' error you're seeing.
A simple fix is to find out what user you're running opensips with, and then running the opensips exporter with the same user. Like so:
`sudo -u opensips_user_here ./opensips_exporter`
Let me know if that helps!
…-------- Original Message --------
On 4 Dec 2019, 22:54, jockmckechnie wrote:
Good afternoon, Ruben - apologies, I'm not entirely certain if this is a fault with my set up or in opensips_exporter, but I could sure use some help.
I'm running the latest opensips_exporter (as of a pull on 2019-12-03), with OpenSIPS v2.4.6.
I have the mi_datagram socket set up with:
modparam("mi_datagram", "socket_name", "/tmp/opensips.sock")
Which garners me a socket in /tmp/, ala:
srw-rw---- 1 opensips opensips 0 Dec 3 13:22 opensips.sock
When running the exporter I'm using the following command:
sudo ./opensips_exporter -socket /tmp/opensips.sock
Which I then attempt to pull using a curl:
curl localhost:9434/metrics
The output that opensips_exporter shows is:
Started OpenSIPS exporter, listening on :9434
Error encountered while reading statistics from opensips socket: read unixgram /tmp/opensips_exporter158593694/1.sock: i/o timeout
Problems registering the *processors.loadProcessor processor (could be due to no metrics found for this processor). Error: collector has no descriptors
Problems registering the *processors.pkmemProcessor processor (could be due to no metrics found for this processor). Error: collector has no descriptors
OpenSIPS is logging receipt of the request from opensips_exporter:
:get_statistics:#012core:#012shmem:#012net:#012uri:#012tm:#012sl:#012usrloc:#012dialog:#012registrar:#012pkmem:#012load:#012tmx:#12
Builds the tree and mi_datagram_write_node reports that it's outputting the statistics to the socket (which I can provide, but is mostly several hundred lines of it doing exactly what you'd expect it to), but then logs:
ERROR:mi_datagram:mi_datagram_server: failed to send the response: Permission denied (13)
The metrics themselves include a pile of go metrics, with the following pertinent to OpenSIPS:
HELP opensips_up Whether the opensips exporter could read metrics from the Management Interface socket. (i.e. is OpenSIPS up)
TYPE opensips_up gauge
opensips_up 0
HELP opensips_usrloc_registered_users_total Total number of AOR existing in the USRLOC memory cache for all domains.
TYPE opensips_usrloc_registered_users_total counter
opensips_usrloc_registered_users_total 0
HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0
All of this says to me that opensips_exporter is not having problems talking to the UNIX domain socket, although I can't reconcile the "timeout" and "Permission denied" errors between opensips_exporter and OpenSIPS.
This all being said, using OpenSIPS' 'opensipsunix' tool to talk to the UNIX domain socket seems to work just fine:
printf ":get_statistics:\ndialog:" | sudo opensipsunix /tmp/opensips.sock
200 OK
dialog:active_dialogs:: 0
dialog:early_dialogs:: 0
(etc)
I'm thoroughly stumped. Please let me know what else I can provide, if anything, or just point out what kind of a cretin I've been and what I've missed.
Thank you for your time, and for the project in the first place!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, [view it on GitHub](#8?email_source=notifications&email_token=AAEFREIYOF6LFHAXYXZ2WKTQXARK5A5CNFSM4JVPOKS2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H6EJFLA), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AAEFREMTK45FEDTVGFPLKQTQXARK5ANCNFSM4JVPOKSQ).
|
|
How interesting... I was using sudo to ensure that opensips_exporter had the permissions required to talk to the /tmp/opensips.sock socket - but OpenSIPS was unable to communicate in reverse presumably precisely because opensips_exporter was running as root. You are correct, of course - running opensips_exporter as the same user works perfectly. Did not see that one coming. Thank you, Ruben, very much! |
|
@jockmckechnie no problem, glad you've got it running in the end. I've run into the same issue myself so I think it'd make sense to have a privilege check in the exporter that will error and exit with some useful information if this scenario pops up. I'll open an feature request issue for this. This is a long shot, but would you be willing to help out? :) |
|
I'd seen your request for help Issue :) I would very much like to contribute - this exporter is already paying dividends, and it makes me look really good to boot :) Unfortunately I don't know go... yet.Hopefully I'll be able to come back to you at some point, but I'm afraid not presently. Thank you, again, though :) |
Really glad this is helping you! 😄 To be honest, this and the maxscale_exporter were my first Go projects and how I learned the ropes. If you feel brave enough #9 is a great way to start learning the language as it should be as simple as writing a |
Good afternoon, Ruben - apologies, I'm not entirely certain if this is a fault with my set up or in opensips_exporter, but I could sure use some help.
I'm running the latest opensips_exporter (as of a pull on 2019-12-03), with OpenSIPS v2.4.6.
I have the mi_datagram socket set up with:
modparam("mi_datagram", "socket_name", "/tmp/opensips.sock")Which garners me a socket in /tmp/, ala:
srw-rw---- 1 opensips opensips 0 Dec 3 13:22 opensips.sockWhen running the exporter I'm using the following command:
sudo ./opensips_exporter -socket /tmp/opensips.sockWhich I then attempt to pull using a curl:
curl localhost:9434/metricsThe output that opensips_exporter shows is:
OpenSIPS is logging receipt of the request from opensips_exporter:
:get_statistics:#012core:#012shmem:#012net:#012uri:#012tm:#012sl:#012usrloc:#012dialog:#012registrar:#012pkmem:#012load:#012tmx:#012Builds the tree and mi_datagram_write_node reports that it's outputting the statistics to the socket (which I can provide, but is mostly several hundred lines of it doing exactly what you'd expect it to), but then logs:
ERROR:mi_datagram:mi_datagram_server: failed to send the response: Permission denied (13)The metrics themselves include a pile of go metrics, with the following pertinent to OpenSIPS:
All of this says to me that opensips_exporter is not having problems talking to the UNIX domain socket, although I can't reconcile the "timeout" and "Permission denied" errors between opensips_exporter and OpenSIPS.
This all being said, using OpenSIPS' 'opensipsunix' tool to talk to the UNIX domain socket seems to work just fine:
I'm thoroughly stumped. Please let me know what else I can provide, if anything, or just point out what kind of a cretin I've been and what I've missed.
Thank you for your time, and for the project in the first place!
(Edited for formatting to make it considerably easier to read)
The text was updated successfully, but these errors were encountered: