Python install (paho-mqtt) fails because of protected python default environment

[NateEaton]

When trying to install RPI-MQTT-shutdown on a new environment, a Pi3B running the latest Raspberry OS (Bookworm), I ran into this when installing paho-mqtt under sudo:

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

While it appears that using break-system-packages might work, eventually under Debian-based platforms it may not be an option. See here for more information. As the general recommendation is to use a virtual environment, I fiddled with this until I got it working without installing via sudo and running the service under root. I'm sharing it here in case the project team are interested in adopting this approach to add support for newer OS installs.

1. Create virtual environment with venv.
2. Activate the virtual environment.
3. Use pip (without sudo) to install paho-mqtt.
4. Modify mqtt_shutdown.py to add sudo at beginning of calls to shutdown:

# handle new MQTT command function
def handleCmnd (cmnd):
  if cmnd=="reboot":
    call(['sudo', 'shutdown', '-r', 'now'], shell=False) #reboot host
  elif cmnd=="shutdown":
    call(['sudo', 'shutdown', '-h', 'now'], shell=False) #shut down host
  elif cmnd=="test":
    MyClient.publish(teleTopic, "Reply to test msg") # Publish reply to an incom
ming msg with payload "test"

5. Create a shell script to execute under the service. It should activate the virtual environment then execute mqtt_shutdown.py. Here is mine for example (I used the same venv I was using for picframe which also uses paho-mqtt):

#!/bin/bash
source /home/pi/venv_picframe/bin/activate  # activate phyton virtual env
/home/pi/venv_picframe/bin/python3 /home/pi/mqtt_shutdown.py   #start mqtt_shutdown

I got a CHDIR error starting the service so moved the code from ~/RPI-MQTT-shutdown up to ~ since it was late and I didn't have time to work out how to make the subdirectory work.
6. Modify mqtt_shutdown.service to call shell script and run as user instead of root. Here is mine for example:

[Unit]
Description=MQTT Shutdown on Pi3

[Service]
ExecStart=/home/pi/start_mqtt_shutdown.sh
Restart=always
RestartSec=10

[Install]
WantedBy=default.target

7. mkdir ~/.config/systemd/user
8. Move mqtt_shutdown.service into ~/.config/systemd/user
9. Enable service:

systemctl --user enable mqtt_shutdown

There may be other ways to solve this but hopefully sharing this here will help someone else get this great tool up and running on Bookworm without so much frustration.

[PaoloElMarinelo]

Hello just install paho-mqtt with
sudo apt-get install python3-paho-mqtt