Emits not consistently sending data

[sirthomasii]

Hi all (and Miguel),

I have a socketio server running on digitalocean with eventlet. It emits messages to chat rooms (chat_uuid).

I can see that the users are indeed connected to the chat rooms and an emit is made every time, however about 50% of the time, the emit message is not "sending packet data". In the server log I have highlighted in pink an emit event which did not send the data.

I have already confirmed that the data (JSON) is correctly passed to the emit function every time.

socketio.emit("chat_message_sent", socket_data, json=True, to=data['chat_uuid'], callback=message_received)

I have tried adding eventlet monkeypatching to the very beginning of my server app as well as some sleep methods to fix the issue but to no avail. It's worth noting that this was not an issue when I was developing locally, it's only when I moved to a production server that this issue surfaced.

Any advice or help is very much appreciated.

[miguelgrinberg]

@sirthomasii what command did you use to start the server?

[sirthomasii]

I asked because your Python server is on port 8080. Are you asking your users to add :8080 to the URLs that you use to connect to your production server?

No I'm not

I did not imply that you are doing things in a bad way, I'm just asking questions to understand the full context.

I see..... well, I am almost sure that I am doing things in a bad way. 😙 Haha

The other question that I have is related to those emits that don't reach the clients. Have you determined if there are specific emits in your code that have this problem? Or is a given emit in your source code sometimes working and sometimes not working? My expectation is that some of the emits that you have in your code work all the time, and other emits that you have do not work all the time, but can you please confirm?

Well to be honest this is really the only emit command (per-se) coming from the server. All the other socket events are reliable and don't fail like joining the rooms, connects, disconnects, etc... However it's important to note that the emit command always is logged, but doesn't send the packet data, and is not reached by the clients.

Update: I managed to get it running on gunicorn. In the logs, the server is now sending the data in every emit command. However the clients are inconsistently receiving the commands. I feel like this is getting closer to figuring out what is going on, but I will try to isolate the problem (adding more server emits and seeing if they are consistent) and get back to you. If you have any suggestions your input is appreciated. :))

[miguelgrinberg]

I don't have any suggestions other than you making available a small example that I can use to debug your problem here. I know you think this is helping, but the fact that you now have switched to gunicorn and get different issues makes this much more harder for me to figure out. As I said, there was nothing wrong with not using gunicorn.

I asked because your Python server is on port 8080. Are you asking your users to add :8080 to the URLs that you use to connect to your production server?

No I'm not

This is also likely not relevant to the issue, but if your server is on port 8080 and your clients connect to your server without specifying a port, then you have something in between the client and the server that is redirecting traffic on the default ports 80 or 443 to your 8080.

[sirthomasii]

Update:

At this point in time I have determined that the server emits a message to the room, but only one of the connected clients receives a message. Never both, but always one of them receives it. It also is random who gets the message.

So, the server can only emit a message to one client in a room... is there something I'm missing?

making available a small example that I can use to debug your problem here.

If neccesary I can you access to my github repo, if that helps. Then I can point you to the files where the socket client/server code is.

[miguelgrinberg]

The logs that you shared disagree with this. The working emits were sent to two clients, not one. I would prefer that you create a minimal example (one python and one html/js file, ideally) that shows the problem, as I don't have time to set up and debug a large application.

[sirthomasii]

The logs that you shared disagree with this.

You're right. After some puzzlement, I removed gevent from the requirements file and the behavior returned to the original behavior as shown in the original log image.

I would prefer that you create a minimal example (one python and one html/js file, ideally)

I understand. The issue is that, as you have correctly surmised, the application is quite large. I think it would take some time to boil it down to two files.

For now I'm thinking that I will live with spotty messages and just ajax ping the server every few seconds as well. It's definitely not efficient. For now I'm aiming for a minimum viable product, and quite scarce on time, hopefully I can make a smaller version after the rest of the features are debugged and optimized thoroughly.... maybe in a couple weeks.

In the event that I make a smaller application that can reproduce this issue, and you solve it, is there a way I can compensate you?

Cheers
T