Data not flowing to Rpi - Error: Failed to connect to broker

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Data not flowing to Rpi - Error: Failed to connect to broker

#1 Post by Maarten71 » 17 Oct 2021, 16:45

I have an ESP Easy with temperature sensors connect to an RPI running Mosquitto. The setup has been working flawless till March (end heating season) and is now longer working (October, start of the new season). Might be due to an update of the Mosquitto software but I cannot find hints.
Information
ESP Easy
Build: 120
Core Version: 2_3_0

Log on ESP Easy

Code: Select all

408531740: MQTT : Connection lost
408531759: MQTT : Failed to connect to broker
408531762: MQTT : Failed to connect to broker
408531845: DS : : Temperature: 19.94 (28-ff-.....
408531846: DS : : Temperature: 19.94 (28-ff-.....
408531847: DS : : Temperature: 19.94 (28-ff-.....
etc
Log on the RPi
pi@IOT-server:/var/log $ mosquitto -v

Code: Select all

1634480173: mosquitto version 2.0.12 starting
1634480173: Using default config.
1634480173: Starting in local only mode. Connections will only be possible from clients running on this machine.
1634480173: Create a configuration file which defines a listener to allow remote access.
1634480173: For more details see https://mosquitto.org/documentation/authentication-methods/
1634480173: Opening ipv4 listen socket on port 1883.
1634480173: Opening ipv6 listen socket on port 1883.
1634480173: mosquitto version 2.0.12 running
1634480176: New connection from 127.0.0.1:50424 on port 1883.
1634480176: New client connected from 127.0.0.1:50424 as mqtt_4c533bf4.c02174 (p1, c1, k60).
1634480176: No will message specified.
1634480176: Sending CONNACK to mqtt_4c533bf4.c02174 (0, 0)
1634480176: Received SUBSCRIBE from mqtt_4c533bf4.c02174
1634480176:     /Sensors-12/# (QoS 0)
1634480176: mqtt_4c533bf4.c02174 0 /Sensors-12/#
1634480176: Sending SUBACK to mqtt_4c533bf4.c02174
1634480236: Received PINGREQ from mqtt_4c533bf4.c02174
1634480236: Sending PINGRESP to mqtt_4c533bf4.c02174
1634480296: Received PINGREQ from mqtt_4c533bf4.c02174
So it looks like there is a connection but the temp data is not flowing.
I replace the ESP power supply and not result. Increased the wifi signal and no result.

Any suggestion would be welcome

User avatar
Ath
Normal user
Posts: 3386
Joined: 10 Jun 2018, 12:06
Location: NL

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#2 Post by Ath » 17 Oct 2021, 17:27

Your ESPEasy version is very old, updating that to a more current release could help. Latest release can be found here: https://github.com/letscontrolit/ESPEasy/releases
Has your mosquitto server possibly changed ip-address or port? Or have you changed settings so it now requires a username/password? As the ESP can't connect to it. You may need to update your Controller settings on the ESPEasy side.

Edit:

Please read the mosquitto log, it is quite clear what's happening there, you will have to update the configuration there...
/Ton (PayPal.me)

Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#3 Post by Maarten71 » 17 Oct 2021, 17:47

Thanks for suggestion, will perform upgrade to mega-20211005.

For my learning, can you please give some context to
Please read the mosquitto log, it is quite clear what's happening there,

User avatar
Ath
Normal user
Posts: 3386
Joined: 10 Jun 2018, 12:06
Location: NL

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#4 Post by Ath » 17 Oct 2021, 18:39

Maarten71 wrote: 17 Oct 2021, 16:45 Log on the RPi
pi@IOT-server:/var/log $ mosquitto -v

Code: Select all

1634480173: mosquitto version 2.0.12 starting
1634480173: Using default config.
1634480173: Starting in local only mode. Connections will only be possible from clients running on this machine.
1634480173: Create a configuration file which defines a listener to allow remote access.
1634480173: For more details see https://mosquitto.org/documentation/authentication-methods/
That is the part I was pointing at, check out the documentation at that link, it will tell you why your mosquitto installation is in local-only mode, and how to make it accessible (again) for other users in your network, like the ESP.
/Ton (PayPal.me)

Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#5 Post by Maarten71 » 18 Oct 2021, 08:10

Thanks, totally missed the link. So indeed the change to Mosquitto 2.x is the root cause. I will update the firmware and, if I read correctly, I also have to make changes on the Mosquitto side.
Keep you posted.

TD-er
Core team member
Posts: 8607
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#6 Post by TD-er » 18 Oct 2021, 09:42

Well, you better do it step-by-step.
So first update the Mosquitto settings to allow connections from other than localhost.
Then update ESPEasy.

Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#7 Post by Maarten71 » 20 Oct 2021, 22:02

Thanks for the tip. Started to work in the MQTT side, but so far no luck.

To my understanding I have to add listener to the /etc/mosquitto/mosquitto.conf file. As per the doc started to create my own file in conf.d directory . Created a file listener.conf and mosquitto.conf but no success.
Next, added the following options to /etc/mosquitto/mosquitto.conf
Tried 3 options

listener 1883
or
listener 1883 0.0.0.0
or
listener 1883 192.168.x.yy

Still have the same error.

/edit also tried to add allow_anonymous statement

Code: Select all

# listener
listener 1883
allow_anonymous true
or

Code: Select all

# listener
listener 1883 192.168.x.yy
allow_anonymous true

Code: Select all

1634759415: mosquitto version 2.0.12 starting
1634759415: Using default config.
1634759415: Starting in local only mode. Connections will only be possible from clients running on this machine.
1634759415: Create a configuration file which defines a listener to allow remote access.
1634759415: For more details see https://mosquitto.org/documentation/authentication-methods/
1634759415: Opening ipv4 listen socket on port 1883.
1634759415: Opening ipv6 listen socket on port 1883.
1634759415: mosquitto version 2.0.12 running
1634759417: New connection from 127.0.0.1:48104 on port 1883.
1634759417: New client connected from 127.0.0.1:48104 as mqtt_c76f625f.ff5ae (p1, c1, k60).

User avatar
Ath
Normal user
Posts: 3386
Joined: 10 Jun 2018, 12:06
Location: NL

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#8 Post by Ath » 20 Oct 2021, 22:23

To allow 'old fashion' unrestricted access, like the old mosquitto did, you should have this in the listener configuration:

Code: Select all

listener 1883 0.0.0.0
allow_anonymous true
And restart mosquitto, of course ;)
/Ton (PayPal.me)

Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#9 Post by Maarten71 » 20 Oct 2021, 22:42

Probably me :? . Doesn't work. Made the change and rebooted the RPI.
file: mosquitto.conf

Code: Select all

pi@IOT-server:/etc/mosquitto $ cat mosquitto.conf
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.gz

pid_file /run/mosquitto/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

# listener
listener 1883 0.0.0.0
allow_anonymous true

After the reboot

Code: Select all

pi@IOT-server:~ $ sudo service mosquitto stop
pi@IOT-server:~ $ mosquitto -v
1634762321: mosquitto version 2.0.12 starting
1634762321: Using default config.
1634762321: Starting in local only mode. Connections will only be possible from clients running on this machine.
1634762321: Create a configuration file which defines a listener to allow remote access.
1634762321: For more details see https://mosquitto.org/documentation/authentication-methods/
1634762321: Opening ipv4 listen socket on port 1883.
1634762321: Opening ipv6 listen socket on port 1883.
1634762321: mosquitto version 2.0.12 running
1634762330: New connection from 127.0.0.1:58230 on port 1883.
1634762330: New client connected from 127.0.0.1:58230 as mqtt_d05ebe37.72e24 (p1, c1, k60).
1634762330: No will message specified.
1634762330: Sending CONNACK to mqtt_d05ebe37.72e24 (0, 0)
1634762330: Received SUBSCRIBE from mqtt_d05ebe37.72e24
1634762330:     /Sensors-12/# (QoS 0)
1634762330: mqtt_d05ebe37.72e24 0 /Sensors-12/#
1634762330: Sending SUBACK to mqtt_d05ebe37.72e24
^C1634762333: mosquitto version 2.0.12 terminating
/Edit
Or should I create a separate listener.conf file?

TD-er
Core team member
Posts: 8607
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#10 Post by TD-er » 20 Oct 2021, 23:56

Make sure there is no listener definition in another file in the conf.d folder.
The configuration does load all files in that folder, just as if they were all concatenated into a single file.
However I don't know what will happen is the listener definition is added multiple times for the same port.

I just looked in the config on my Domoticz pi and there it doesn't mention the IP, only the port numbers.

In my config there are 2 listener entries, but for different ports (to also support TLS over MQTT)

Code: Select all

listener 1883
listener 8883
As you can see, no IP, and it is reachable on my network, so not just bound to localhost.

User avatar
ThomasB
Normal user
Posts: 1064
Joined: 17 Jun 2018, 20:41
Location: USA

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#11 Post by ThomasB » 21 Oct 2021, 01:18

So indeed the change to Mosquitto 2.x is the root cause.
If you're running OpenHab2 and the system was previously running Mosquitto 1.x, and you've switched to 2.x, then be sure that the 1.x binding has been removed.

If you've recently performed the 2.x upgrade then maybe this OpenHab community discussion will help resolve your problem:
https://community.openhab.org/t/solved- ... -v2/106798

FWIW, I'm still using the 1.x binding. Works fine, haven't found a reason to switch (which is a lot of work).

- Thomas

User avatar
Ath
Normal user
Posts: 3386
Joined: 10 Jun 2018, 12:06
Location: NL

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#12 Post by Ath » 21 Oct 2021, 09:16

That listener definition (I posted above) should go in the listeners.conf file in /etc/mosquitto/conf.d folder, not in mosquitto.conf, I guess. (I copied that from my own, working, configuration, the only change I did to get it working again after the upgrade to 2.x :))
/Ton (PayPal.me)

Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#13 Post by Maarten71 » 21 Oct 2021, 19:45

Ok, I'm making progress.
Added the listeners.conf file to conf.d and MQTT is recognising the connection. The log is confirming the connection and the ESP Easy log changed from failed to connect to ConnectFailure 50xxx, where xxx is a decreasing number. Next step is to upgrade the ESP.

TD-er
Core team member
Posts: 8607
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#14 Post by TD-er » 21 Oct 2021, 20:25

Why is the connection refused?
The MQTT code in ESPEasy hasn't changed a lot in the last 2 years. (at least the MQTT client library)

Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#15 Post by Maarten71 » 21 Oct 2021, 21:14

I was advised to do an ESP Easy upgrade
Your ESPEasy version is very old, updating that to a more current release could help. Latest release can be found here: https://github.com/letscontrolit/ESPEasy/releases
Will check if I can find more info in why the connection is not working.

Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#16 Post by Maarten71 » 21 Oct 2021, 21:32

Correction! data is flowing. Apparently the result of the command Mosquitto -v has changed.
Now I understand the ESP message
772066430 : WD : Uptime 12712 ConnectFailures 50173
This looks to be a status message. Rebooted the ESP and ConnectFailures is 0
Thanks for the support. Next step is to debug my node-red software. ;)

Maarten71
Normal user
Posts: 14
Joined: 19 Sep 2019, 16:09

Re: Data not flowing to Rpi - Error: Failed to connect to broker

#17 Post by Maarten71 » 21 Oct 2021, 22:31

My final post to share my learnings
1) the name of the 'listener' file in conf.d seems to be irrelevant
2) I had to add the port to listen to and configure authentication
2) In my case there was no need to upgrade ESP Easy
3) Mosquitto v2 is pushing the device name and Value name. In v1 only the value name was provided. I had to change my Node-Red script.

My file

Code: Select all

 Starting with the release of Mosquitto version 2.0.0 the default config
# will only bind to localhost as a move to a more secure default posture.
# If you want to be able to access the broker from other machines you will need to explicitly edit
# the config files to either add a new listener that binds to the external IP address (or 0.0.0.0)
# or add a bind entry for the default listener.
# listener 1883 ip_address_of_the_machine(192.168.1.1 etc.)
# listener 1883
# You also need to configure unauthenticated access, use the allow_anonymous option:
# allow_anonymous true
#
listener 1883 0.0.0.0
allow_anonymous true
Thanks for all your support and let me know if I missed something.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests