Strange Behavior with RabbitMQ and Flask-SocketIO

[njbinbin]

Describe the bug
I'm encountering a peculiar issue while using RabbitMQ in conjunction with flask-socketio. The behavior of the program varies with each run, which is unexpected.

To Reproduce
Steps to reproduce the behavior:

1. build a simple flask app like this:

from flask import Flask
from flask_socketio import SocketIO
import pika
app = Flask(__name__)
app.config['SECRET_KEY'] = 'your_secret_key'
socketio = SocketIO(app, cors_allowed_origins="*", namespace='/door')
test_var = 0
def callback(ch, method, properties, body):
    print('app: ' + str(id(app))) # here each time the id is different
    print('test_var: ' + str(id(test_var))) #  same as above
def rabbitmq_listener():
    connection = pika.BlockingConnection(pika.URLParameters('amqp://shufan:123456@localhost:5672/examination'))
    channel = connection.channel()
    channel.queue_declare(queue='door')
    channel.basic_consume(queue='door', on_message_callback=callback, auto_ack=True)
    channel.start_consuming()
if __name__ == '__main__':
    socketio.start_background_task(rabbitmq_listener)
    socketio.run(app, allow_unsafe_werkzeug=True, host='0.0.0.0', port=5008, debug=True)

and write a rabbitmq client like this to test the app:

import pika
connection = pika.BlockingConnection(pika.URLParameters( 'amqp://shufan:123456@localhost:5672/examination'))
channel = connection.channel()
channel.queue_declare(queue='door')
# send 10 messages
for i in range(0, 10):
    channel.basic_publish(exchange='', routing_key='door', body=b'Hello World!')
connection.close()

2. launch rabbitmq server, run the flask app server, and run the test sciprt

Expected behavior
I expect that when a new message is received in the RabbitMQ queue, the callback function is called, and I can use the socketio object within it to send messages to clients. However, this does not work consistently. Upon inspecting the socketio object within the callback function, I noticed that its address changes with each call (alternating between two addresses).
When I replace socketio with a global object like app or even a separately defined global variable like test_var, the id values obtained within the callback function are different each time.

Logs

Werkzeug appears to be used in a production deployment. Consider switching to a production web server instead.
 * Serving Flask app 'socket_app'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5008
 * Running on http://192.168.86.156:5008
Press CTRL+C to quit
 * Restarting with stat
Werkzeug appears to be used in a production deployment. Consider switching to a production web server instead.
 * Debugger is active!
 * Debugger PIN: 121-598-879
app: 140172787352528
test_var: 140172816548048
app: 140172787352528
test_var: 140172816548048
app: 140172787352528
test_var: 140172816548048
app: 140612182304096
test_var: 140612213424336
app: 140612182304096
test_var: 140612213424336
app: 140612182304096
test_var: 140612213424336
app: 140612182304096
test_var: 140612213424336
app: 140172787352528
test_var: 140172816548048
app: 140612182304096
test_var: 140612213424336

Each time the script is run, the callback function on the server prints different id values for app and test_var after processing the 10 messages. The behavior seems like there are two parallel universes for all global objects.
Could you provide insights into the underlying mechanisms causing this behavior?
Is it a bug of flask-socketio or pika ?

[njbinbin]

I test it on python 3.10, with packages as below:
alembic==1.11.1
APScheduler==3.10.1
async-timeout == 4.0.3
blinker==1.6.2
cachelib==0.9.0
certifi==2023.5.7
cffi==1.15.1
charset-normalizer==3.1.0
click==8.1.3
colorama==0.4.6
cryptography==40.0.2
Flask==2.3.2
Flask-APScheduler==1.12.4
Flask-Caching==2.0.2
Flask-Cors==3.0.10
Flask-Migrate==4.0.4
Flask-SocketIO==5.3.5
Flask-SQLAlchemy==3.0.3
greenlet==2.0.2
idna==3.4
importlib-metadata==6.3.0
itsdangerous==2.1.2
Jinja2==3.1.2
Mako==1.2.4
MarkupSafe==2.1.2
minio==7.1.14
pika==1.3.2
pika-stubs=0.1.3
Pillow==10.0.0
psutil==5.9.5
pycparser==2.21
PyJWT==2.7.0
PyMySQL==1.0.3
pyOpenSSL==23.1.1
python-dateutil==2.8.2
pika==1.3.2
pika-stubs==0.1.3
pytz==2023.3
redis==4.5.5
requests==2.30.0
six==1.16.0
SQLAlchemy==2.0.15
SQLAlchemy-serializer==1.4.1
typing_extensions==4.6.2
tzdata==2023.3
tzlocal==5.0.1
urllib3==2.0.2
Werkzeug==2.3.4
zipp==3.15.0

[miguelgrinberg]

You are using the Flask reloader, which runs two instances of your app. That means that you have two RabbitMQ callbacks that are active, and two of each global variable that you define.

Could you please repeat the test using Flask in production mode instead?

[njbinbin]

Thank you so much for your prompt reply! I decided to learn and use Flask after reading your blog tutorial. However, I haven't delved deep into the underlying mechanisms and logic, nor do I know about its reloader mechanism.

I'm testing in a production environment, but I still have a question. If multiple workers are started by the server in the production environment, will they also have independent global variables and SocketIO connections? Could similar issues arise?

[miguelgrinberg]

@njbinbin Yes, each process is independent of the others and will have its own set of globals.

[njbinbin]

I installed eventlet package, set app.config['FLASK_ENV'] = 'production', then start app with:

socketio.run(app, debug=False, host='0.0.0.0', port=5008)

or

socketio.run(app, debug=True, host='0.0.0.0', port=5008 , use_reloader=False)

everything is OK :)
thanks a lot!