How to Separate Zigbee2MQTT from Home Assistant in Proxmox

Recently I wrote a tutorial covering the Easiest way to install Home Assistant on Proxmox by using automated helper scripts, which streamlines the deployment process significantly. Since I shared that article, many of you contacted me asking for a guide of how to separate Zigbee2MQTT from HA in Proxmox.

This article is a continuation of the first one, and I’m going to show you how to do exactly that: Separate Zigbee2MQTT from Home Assistant entirely, so It can run as separate container independent from Home Assistant.

Why separate Zigbee2MQTT from Home Assistant?

For the uninitiated, you might wonder why would you want to separate Zigbee2MQTT from Home Assistant? Running it like an add-on gives you full control and access to it’s features right from Home Assistant.

Well, a separate Zigbee2MQTT instance on your local network has a few advantages, with the most important being your Zigbee network will no longer be affected by Home Assistant restarts, updates, braking changes and other downtime you server might experience.

Zigbee2MQTT will run completely independently and still push it’s data through MQTT in Home Assistant.

Prerequisites

Before starting with this tutorial, a few conditions need to be met to be able to follow along. This article is specifically dedicated to a Proxmox VE, since that’s the most popular virtualization platform among Home Assistant users. If you have different hardware or VM, the guide can be easily adapter if needed.

• Running Proxmox Virtual Environment
• Running Home Assistant Container in Proxmox
• All devices on the same local network

My Proxmox is running on a cheap Mini PC, but you can deploy yours on any hardware you want. It should be powerful enough to make the experience better, faster and more stable than a Raspberry Pi.

Methods of Separating Zigbee2MQTT from Home Assistant

There are two actual ways to separate Zigbee2MQTT from Home Assistant in Proxmox:

In this guide, I will cover both so you can decide which one is for you. Method A is a partial separation, as the MQTT Broker is still installed in Home Assistant as an add-on. Method B is a complete separation, as both are deployed as separate containers along side HA and connected with the MQTT Integration.

Installing MQTT Broker (Mosquitto)

Before we can fully setup and deploy a Zigbee2MQTT Container in Proxmox, we are going to setup the MQTT broker. Like I mentioned above, we can do this in two ways:

Method A: Install MQTT Broker as an Add-on in Home Assistant

1. Open up your Home Assistant instance
2. Navigate to Settings > Add-ons > Add-on Store and search for MQTT



3. Select Mosquitto broker and click Install
4. Turn on the Start on boot and Watchdog toggles
5. Navigate to the Configuration tab and create a new user with any credentials you want to
6. Optionally, at the bottom of the screen you can change the access port

7. Your MQTT Broker access IP address is: homeassistant_ip:port (e.g. 192.168.0.XXX:1883)
8. Your MQTT Broker login credentials are set in the add-on (e.g. username: z2m-proxmox, password: z2m-proxmox)
8. Done

Method B: Install MQTT Broker as a Proxmox Container

1. Open up your Proxmox VE and click Shell
2. Run the following script to install MQTT Mosquitto

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/mqtt.sh)"
Code language: YAML (yaml)

3. The script will install MQTT with default settings: 512MiB RAM – 2GB Storage – 1vCPU
4. You can customize these by selecting Advanced settings during installation
5. Once Mosquitto is installed, you will get a Completed Successfully message

6. Mosquitto comes with a built in password creating tool
7. Click the MQTT container on the left and click Console
8. Run the following command

#Replace <user> with your desired MQTT username
sudo mosquitto_passwd -c /etc/mosquitto/passwd <user>
Code language: YAML (yaml)

9. Input a password twice to make sure it matches
10. Next, we need to create configuration file for Mosquitto pointing to the password file we have just created.
11. Run the following command in the console

sudo nano /etc/mosquitto/conf.d/default.conf
Code language: YAML (yaml)

12. This will create an empty file, paste the following configuration in it:

allow_anonymous false
persistence true
password_file /etc/mosquitto/passwd
listener 1883
Code language: YAML (yaml)

13. Save the file with CTRL + O and hit Enter. You will get a wrote XX lines confirmation message.
14. Exit with CTRL + X
15. Restart Mosquitto by running the following command

sudo systemctl restart mosquitto
Code language: YAML (yaml)

15. Done, your MQTT server is up and running on Proxmox
16. Your MQTT credentials are the username and password you created during installation
17. You MQTT access is Mosquitto’s IP Address:1883

Installing Zigbee2MQTT Container in Proxmox

Assuming you have everything ready, Home Assistant Operating System deployed as a container in Proxmox and your MQTT Broker running on your local network, we can deploy Zigbee2MQTT easily by running tteck‘s helper scripts. This step is the same for both methods:

1. Open your Proxmox VE
2. Login with your username and password
3. Click your node on the left and open Shell
4. Run the following command:

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/zigbee2mqtt.sh)"
Code language: YAML (yaml)

5. The script will start to install Zigbee2MQTT
6. Choose advanced settings to setup dedicated RAM, Storage, CPU cores and set Privileged mode. This will automatically perform USB Passthrough and necessary permission for the container automatically. Otherwise choose Default Settings: 1GB RAM – 4GB Storage – 2vCPU and set the USB passthrough yourself later
7. You will get a prompt mid-install to switch to Edge (Dev) branch, choose Y or N

8. Once the process is finished, you will see the Zigbee2MQTT Container under your node on the left
9. A confirmation message Completed Successfully will appear in the shell

10. Done

Before we can start Zigbee2MQTT, we need to edit it’s configuration.yaml file and input the MQTT credentials and passthrough your Zigbee dongle to the container.

Zigbee Dongle USB Passthrough

If your Zigbee dongle was plugged in while you installed Zigbee2MQTT and the LXC was created privileged, the script will automatically setup USB Passthrough to the container. To verify and find out the serial-id of your dongle, open the Proxmox shell and input the following command:

ls -l /dev/serial/by-id
Code language: YAML (yaml)

1. You will get an output similar to this, depending on your adapter type:



2. Copy the serial-id just before the -> symbol
(eg. usb-ITEAD_SONOFF_Zigbee_3.0_USB_Dongle_Plus_V2_20220813094657-if00)
3. Save it in a file, we are going to need it for the configuration.yaml file

For some people, the adapter is not identified when listing by serial id. As an alternative, you can pass the USB port directly, but you need to keep in mind that with this method the dongle will always have to stay in the same USB port. Instead of copying the adapter-id, copy the port at the end, in this case tty/ACM0:

/dev/ttyACM0
Code language: YAML (yaml)

Add the port in Zigbee2MQTT’s configuration.yaml file, see the example bellow.

Editing Zigbee2MQTT Configuration

Before Zigbee2MQTT can be started, we need to edit the main configuration.yaml file. Open the Zigbee2MQTT Container, click Console and input the following command:

nano /opt/zigbee2mqtt/configuration.yaml
Code language: YAML (yaml)

1. In the opened file, paste the following configuration and edit it to match your MQTT Credentials and Adapter:

frontend:
  port: 8080 #Change Frontend UI Access Port Default: 8080
homeassistant: true
permit_join: false
mqtt:
  base_topic: zigbee2mqtt
  server: 'mqtt://192.168.0.50:1883'
  user: z2m-proxmox #MQTT Username
  password: z2m-proxmox #MQTT Password
  keepalive: 60
  reject_unauthorized: true
  version: 4
serial:
  port: /dev/serial/by-id/usb-ITEAD_SONOFF_Zigbee_3.0_USB_Dongle_Plus_V2_20220813094657-if00
  #port: /dev/ttyACM0 #Uncomment for USB Port Method
  #adapter: deconz (Uncomment For Conbee II)
  #adapter: ezsp (Uncomment For Sonoff ZBDongle-E)
advanced:
  pan_id: GENERATE
  network_key: GENERATE
  channel: 20 #Choose Zigbee channel
Code language: YAML (yaml)

2. Click CTRL + O, Enter you should get a wrote XX lines confirmation prompt. Exit the editor with CTRL + X.
3. Start Zigbee2MQTT with:

cd /opt/zigbee2mqtt && npm start
Code language: YAML (yaml)

4. Zigbee2MQTT should start automatically and you will get the confirmation in the log:



5. Done

Connecting Zigbee2MQTT to Home Assistant

Once everything is up and running, all that remains is to connect Zigbee2MQTT to Home Assistant via the Mosquitto MQTT Broker:

• If you’ve used Method A: Partial separation, Zigbee2MQTT should already be connected to your MQTT Broker in Home Assistant (Auto-discovered)
• If you’ve used Method B: Complete separation, we will use the MQTT Integration to connect Z2M to Home Assistant

1. In Home Assistant, navigate to Settings > Devices & Services > Add Integration
2. Search for MQTT
3. Add the first MQTT Integration and input your broker credentials

4. If everything is correct, you will get a confirmation message
5. Done

Accessing Zigbee2MQTT in your Web Browser

Since you don’t have the Zigbee2MQTT Add-on installed in Home Assistant directly, you can access it’s UI by navigating to the IP_address:Port directly in your web browser.

You can now add devices like you normally would, and Zigbee2MQTT will push them to your MQTT Broker (Mosquitto) which will in turn add them to Home Assistant!

For example, if I pair the Tuya IR Blaster it will automatically show up in Home Assistant:

Troubleshooting and Best Practices

Before you get to troubleshooting installation errors, open up your router and set static IP addresses to everything (Proxmox, Home Assistant, Mosquitto MQTT, Zigbee2MQTT). A static IP will make sure the deployed containers are always accessible at the same local IP and will not be shifted around by DHCP, resulting in connectivity errors.

Errors are possible while going through the deployment process of Zigbee2MQTT and Mosquitto Broker on Proxmox. Different hardware may present different obstacles. Here are a few common errors and how to solve them:

• If you get no such file or directory errors, you will need to create the Zigbee2MQTT directory manually and set the correct user permissions. Run the following commands in the Proxmox shell:

# Create a directory for zigbee2mqtt and set your user as owner of it
sudo mkdir /opt/zigbee2mqtt

#Change USER with Your Proxmox User
sudo chown -R ${USER}: /opt/zigbee2mqtt
Code language: YAML (yaml)

• Verify you are using the correct port for the Zigbee adapter
  • Always use serial-id instead of ttyACM0
  • Run the following command in the Proxmox shell to get the correct serial-id

ls -l /dev/serial/by-id
Code language: YAML (yaml)

• See the complete list for Zigbee2MQTT failing to start here
  • https://www.zigbee2mqtt.io/guide/installation/20_zigbee2mqtt-fails-to-start.html

Summary

Hopefully, this guide helped you separate your Zigbee network from Home Assistant in Proxmox. If you were successful, you are now running Zigbee2MQTT as a separate LXC container, which means independent uptime and backups from Home Assistant.

For those you are not using Proxmox, or have different hardware available the general process is the same and can be adapted to your setup. For example, let’s say you have two Raspberry Pi’s and want to use one for your Zigbee2MQTT network, Method A: Partial Separation is applicable to you:

• 1st Raspberry Pi:
  • Install and setup Home Assistant
  • Install and setup Mosquitto MQTT as an Add-on within Home Assistant
• 2nd Raspberry Pi:
  • Install and setup Zigbee2MQTT directly on the Pi
  • Connect it to your MQTT Server running on the first Pi

There are several other ways this guide can be adapted to fit your hardware, let me know in the comments if you need some help or advice for your particular setup.