Out of Sync state due to MQTT QOS 0

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
Alexs
New user
Posts: 3
Joined: 10 Jan 2017, 18:46

Out of Sync state due to MQTT QOS 0

#1 Post by Alexs » 13 Feb 2017, 14:51

Currently ESPEasy uses the pubsub client, which only supports publishing messages with QOS 0. This can lead to messages getting lost and system states getting out of sync.
MQTT QOS are defined as:
0: The broker/client will deliver the message once, with no confirmation.
1: The broker/client will deliver the message at least once, with confirmation required.
2: The broker/client will deliver the message exactly once by using a four step handshake.

I'm running node-red on a raspberry pi (rpi) as the main contrtol unit and espeasy on remote esp2866 devices to control things like heaters. Communication between everything is done solely by MQTT.
I've encountered some recent situations where the esp2866 heater control gets out of sync with the known state on the RPI due to MQTT messages getting lost. ie.
  1. RPI sends a MQTT command to the ESPEasy controlled heater heater/gpio/13 to turn the switch off.
  2. espeasy heater turns off switch and then publishes the switch state on mqtt:heater/power/state.
  3. Unfortunately this message never reaches the mqtt broker, so the RPI (and all other subscribing clients) think that the heater is still on.
  4. Because the RPI thinks the heater is on, it continues to try to turn the heater off by publishing mqtt commands to heater/gpio/13, but because the switch is already off, espeasy doesn't publish any state change on the mqtt topic heater/power/state.
A couple solutions could be:
  1. Support a more robust QOS for MQTT. There is an async mqtt client that supports QOS 0, 1 & 2 that could work https://github.com/marvinroger/async-mqtt-client. It seems to support both persistent session and non-persistent sessions. Will likely increase the memory footprint.
  2. Return state of ESPEasy switch on receipt of every command. ie. when ESPEasy receives a command to turn a switch off, it will publish the current state of the switch even if it doesn't change the state of the switch. In the above example, while the first publish to mqtt:heater/power/state on step 2 may have failed, a subsequent command in step 4 would result in a publish of the current state. I believe another library https://github.com/arendst/Sonoff-Tasmota does this effectively where it publishes the current state response for all command control actions received.
In addition to this, I'm thinking to expand the switch plugin so that its state can be controlled via publishing a command direct to the switch plugin topic, instead of a gpio topic. ie,

Code: Select all

heater/power/control 1
where 'heater' is the espeasy device and 'power' is the swtich device. 
instead of

Code: Select all

 heater/gpio/13 1

CCOSTAN
New user
Posts: 1
Joined: 01 May 2017, 23:21

Re: Out of Sync state due to MQTT QOS 0

#2 Post by CCOSTAN » 01 May 2017, 23:30

has there been an update to this? I'm also experiencing this out of sync phenomena. QOS 1 would most likely solve it.

User avatar
lucaberta
Normal user
Posts: 59
Joined: 09 May 2016, 11:26
Location: Lausanne, Switzerland
Contact:

Re: Out of Sync state due to MQTT QOS 0

#3 Post by lucaberta » 03 May 2017, 09:23

This is definitely something that should be addressed to have ESPEasy become a reliable solution for devices in production environment, where more stringent QoS settings are needed.

We would likewise need to support SSL/TLS, in my view, and neither the pubsub library nor the Async MQTT client support SSL/TLS.

This library does support all QoS levels and SSL/TLS too, on the other hand:

https://github.com/tuanpmt/esp_mqtt
http://tuanpm.net/native-mqtt-client-li ... r-esp8266/

I also very much like the idea of the original poster to add more customized topics to control directly states to plugins, rather than relying on controlling GPIO pins directly.

This would also mean that we would finally have plugins for OUTPUT devices such as LEDs and relays, something which has always bothered me. Yes, we can control directly GPIO pins without configuring anything, but sometimes it's very difficult to remember what is connected to what pin and GPIO line from the web interface, and having a plugin which reminds us of the hardware configuration would be very helpful.

Looking forward to amazing new developments for ESPEasy!

Luca

psy0rz
Normal user
Posts: 232
Joined: 02 Feb 2017, 12:12

Re: Out of Sync state due to MQTT QOS 0

#4 Post by psy0rz » 04 May 2017, 00:28

Yeah we need to fix a lot of MQTT stuff still. I dont know if we have enough memory for SSL/TLS.

But making MQTT more robust and more complete is certainly something we will do in the near future.
Please support ESPEasy development via Patreon or buy us a coffee.

User avatar
lucaberta
Normal user
Posts: 59
Joined: 09 May 2016, 11:26
Location: Lausanne, Switzerland
Contact:

Re: Out of Sync state due to MQTT QOS 0

#5 Post by lucaberta » 04 May 2017, 02:14

psy0rz wrote: 04 May 2017, 00:28 Yeah we need to fix a lot of MQTT stuff still. I dont know if we have enough memory for SSL/TLS.
you raise a very good point Edwin. Memory.

I have a couple of boards with ESP8285 on them, and I could not upload successfully ESPEasy Mega Normal, as the PlatformIO uploader was telling me that the size of the sketch was too large for the flash memory (8285 come with on-chip 1M of flash).

In end I had to remove all the _P0xx plugin files from the source directory, add only the ones I am interested in, and finally recompile. The upload then worked and I now have the latest Github code running on the 8285.

Still, having a more modular way to build the image, skipping the modules we don't need and only including the important ones for our application would be a great thing to have, in my view.

This will also need to be well documented, and only available for the real hackers with 1M flash on their boards.

I appreciate that the issue with SSL/TLS is probably more RAM related than flash, but I am convinced that having unnecessary code should also help on the RAM side.

Many thanks for all you do!

MvG, Luca

psy0rz
Normal user
Posts: 232
Joined: 02 Feb 2017, 12:12

Re: Out of Sync state due to MQTT QOS 0

#6 Post by psy0rz » 04 May 2017, 08:37

someone has been working on a java program that allows the end user to enable and disable plugins. i didnt have time to look at it so far.
Please support ESPEasy development via Patreon or buy us a coffee.

User avatar
lucaberta
Normal user
Posts: 59
Joined: 09 May 2016, 11:26
Location: Lausanne, Switzerland
Contact:

Re: Out of Sync state due to MQTT QOS 0

#7 Post by lucaberta » 04 May 2017, 10:15

psy0rz wrote: 04 May 2017, 08:37 someone has been working on a java program that allows the end user to enable and disable plugins. i didnt have time to look at it so far.
given that PlatformIO is now the tool to use to build ESPEasy, how about configuring things in a more granular way, not down to individual module, but doing something akin to [Normal], [Dev], [Test] like we have today.

That might be enough.

But then again, it depends on how many people are affected by something like this.

The discussion is open.

Thanks,

Luca

psy0rz
Normal user
Posts: 232
Joined: 02 Feb 2017, 12:12

Re: Out of Sync state due to MQTT QOS 0

#8 Post by psy0rz » 04 May 2017, 10:25

i could just make a config file in which you can enable/disable plugins?
Please support ESPEasy development via Patreon or buy us a coffee.

User avatar
lucaberta
Normal user
Posts: 59
Joined: 09 May 2016, 11:26
Location: Lausanne, Switzerland
Contact:

Re: Out of Sync state due to MQTT QOS 0

#9 Post by lucaberta » 04 May 2017, 12:18

psy0rz wrote: 04 May 2017, 10:25 i could just make a config file in which you can enable/disable plugins?
that would definitely help.

The thing that reminds of something similar is the Linux kernel configuration procedure, with all the choices you get in terms of, say, building certain drivers directly into the kernel, versus building them as loadable modules, versus skipping them altogether.

We don't need anything so articulate, but you get the idea.

Thanks,

Luca

harry56
New user
Posts: 4
Joined: 26 May 2017, 06:41

Re: Out of Sync state due to MQTT QOS 0

#10 Post by harry56 » 26 May 2017, 09:09

but from simply reading around I've found the answers to many of my own questions. Good luck!


บาคาร่า
Last edited by harry56 on 27 Mar 2018, 15:46, edited 1 time in total.

tuxmartin
Normal user
Posts: 34
Joined: 10 Sep 2017, 13:07

Re: Out of Sync state due to MQTT QOS 0

#11 Post by tuxmartin » 04 Dec 2017, 22:38

Hi,
is possible to recompile espeasy with mqtt qos 1 or 2?

I found at espeasy mqtt lib (pubsubclient):
It can only publish QoS 0 messages. It can subscribe at QoS 0 or QoS 1.
https://github.com/letscontrolit/ESPEas ... /README.md

and https://github.com/letscontrolit/ESPEas ... ent.h#L136

Post Reply

Who is online

Users browsing this forum: No registered users and 35 guests