MQTT Import - stability problems ...?

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
dirk_k
Normal user
Posts: 19
Joined: 19 Aug 2017, 22:41

MQTT Import - stability problems ...?

#1 Post by dirk_k » 04 Jun 2018, 17:39

Hi all,
i use SonOff S20 to control my dehumidifier.
FHEM send every minute a MQTT message to restart the Count-Down-Timer.
Works fine after ESP restart.
But after some time (1-2h) the MQTT import stops reading MQTT messages. The counter expires and the dehumidifier stops working.
Manually send MQTT messages are ignored too.
ESP- Easy send his data (uptime, load, relay-state,...) with MQTT to my server without problems ... but don't receive messages.
Version is "mega-20180524".
Logfile shows problems too.
The "EVENT: MQTT-IN#State" is missing, the "WD : Uptime X ConnectFailures X FreeMem XXX" messages are no longer there ... take a look to the logfile
S20-Fehler_2.zip
(8.73 KiB) Downloaded 257 times

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: MQTT Import - stability problems ...?

#2 Post by TD-er » 05 Jun 2018, 13:43

This is the last I see in the log:

Code: Select all

2391086: IMPT : [MQTT-IN#State] : 600.00
2391087: EVENT: MQTT-IN#State=600.00
2391158: ACT  : gpio,12,1
2391160: SW   : GPIO 12 Set to 1
2391163: ACT  : timerSet,1,1
2391173: Command: timerset
2391183: ACT  : timerSet,2,600*60
2391193: Command: timerset
2391195: ACT  : Publish fhem/sensors/sonoff_S20_01/in/Relay,1
2391205: Command: publish
We really have to look into this MQTTimport, since its current implementation really is a hack.

waspie
Normal user
Posts: 127
Joined: 09 Feb 2017, 19:35

Re: MQTT Import - stability problems ...?

#3 Post by waspie » 08 Jun 2018, 14:18

TD-er wrote: 05 Jun 2018, 13:43 This is the last I see in the log:

Code: Select all

2391086: IMPT : [MQTT-IN#State] : 600.00
2391087: EVENT: MQTT-IN#State=600.00
2391158: ACT  : gpio,12,1
2391160: SW   : GPIO 12 Set to 1
2391163: ACT  : timerSet,1,1
2391173: Command: timerset
2391183: ACT  : timerSet,2,600*60
2391193: Command: timerset
2391195: ACT  : Publish fhem/sensors/sonoff_S20_01/in/Relay,1
2391205: Command: publish
We really have to look into this MQTTimport, since its current implementation really is a hack.
I depend heavily on MQTTImport as to *me* it *seems* far more reliable than http events especially when getting commanded to do several things one after another. While I'm not seeing the same problem, I do want to emphasize that this plugin is very important so thank you for getting it right (when you do!).


I'm running 5/24 source on a few ESPs at this time and most have MQTTImport running. A few of them I have a "watchdog" to be sure they're online and working. I have openhab "ping" them every 5 minutes via MQTT and the ESP answers back with an MQTT publish and it works very reliably.

dirk_k
Normal user
Posts: 19
Joined: 19 Aug 2017, 22:41

Re: MQTT Import - stability problems ...?

#4 Post by dirk_k » 14 Jun 2018, 10:48

Hi waspi,
which firmware do you use.
With "older" fw (last year) i don't see such problems too, but i flash the current because the "wlan-setting-writes kill the flash" problem.

i configure another ESP (NodeMCU), point to the same MQTT topic for import and get the same result.
Booth devices stops working nearly the same time.

waspie
Normal user
Posts: 127
Joined: 09 Feb 2017, 19:35

Re: MQTT Import - stability problems ...?

#5 Post by waspie » 14 Jun 2018, 14:40

dirk_k wrote: 14 Jun 2018, 10:48 Hi waspi,
which firmware do you use.
With "older" fw (last year) i don't see such problems too, but i flash the current because the "wlan-setting-writes kill the flash" problem.

i configure another ESP (NodeMCU), point to the same MQTT topic for import and get the same result.
Booth devices stops working nearly the same time.
i've been using many versions this year, generally current.

I also compile my own because I like to remove unused plugins. I don't alter any sketches (generally).

I have some units running with 4 MQTTImports and no problems...

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: MQTT Import - stability problems ...?

#6 Post by TD-er » 14 Jun 2018, 19:18

waspie wrote: 14 Jun 2018, 14:40 [..]
I also compile my own because I like to remove unused plugins. I don't alter any sketches (generally).
A bit off-topic; If you have some good "minimal" set of plugins, could you share that list?
I was thinking about adding some "minimal" version to the pre-built list and I guess a generic "fits most" would be nice.
That could be a default for the 1M builds, since these cannot be flashed anymore via 2-step OTA.

waspie
Normal user
Posts: 127
Joined: 09 Feb 2017, 19:35

Re: MQTT Import - stability problems ...?

#7 Post by waspie » 14 Jun 2018, 21:37

TD-er wrote: 14 Jun 2018, 19:18
waspie wrote: 14 Jun 2018, 14:40 [..]
I also compile my own because I like to remove unused plugins. I don't alter any sketches (generally).
A bit off-topic; If you have some good "minimal" set of plugins, could you share that list?
I was thinking about adding some "minimal" version to the pre-built list and I guess a generic "fits most" would be nice.
That could be a default for the 1M builds, since these cannot be flashed anymore via 2-step OTA.
Off the top of my head:

for me, what covers most of my usage is: switch (obviously), dallas (probably add DHT?), mqttimport, sysinfo, dummy, mcp23017 (probably pca too for people that use them instead), sr04, and TSL2561
Slightly more? maybe oled and neopixel stuff

I dump all the controllers but Openhab (probably should have Domo). I also get rid of the 2 notifications.

waspie
Normal user
Posts: 127
Joined: 09 Feb 2017, 19:35

Re: MQTT Import - stability problems ...?

#8 Post by waspie » 14 Jun 2018, 21:44

dirk_k wrote: 14 Jun 2018, 10:48 Hi waspi,
which firmware do you use.
With "older" fw (last year) i don't see such problems too, but i flash the current because the "wlan-setting-writes kill the flash" problem.

i configure another ESP (NodeMCU), point to the same MQTT topic for import and get the same result.
Booth devices stops working nearly the same time.


here's one with 12 devices. 2 mqttimports. 6 plugins, 9 days uptime
Load 11% (LC=19970)
Free Mem 21384 (18264 - parseCommandString)
and really this doesn't even show all of it. I have two 16 channel MCPs on it. They drive 3 L293Ds which open and close 6 different vents via MQTTimports. Tons of rules based on MCP switch inputs and SR04 distance measurements.
garage1.PNG
garage1.PNG (74.6 KiB) Viewed 8128 times
I have another with 4 or 5 mqttimports and a lot of dummies that runs a nextion with 5 pages, all 4 rule pages almost completely full and it runs fine. It's not up currently as im waiting for my 3d printer to get all the pieces together.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: MQTT Import - stability problems ...?

#9 Post by TD-er » 15 Jun 2018, 20:25

@Waspie
That's an impressive setup and glad to read it is running stable :)

dirk_k
Normal user
Posts: 19
Joined: 19 Aug 2017, 22:41

Re: MQTT Import - stability problems ...?

#10 Post by dirk_k » 02 Jul 2018, 11:49

I use only one MQTT Import with only one value.
But because a non optimal software configuration i send the value 20 times per minute (10 times within 2-3 seconds).
After some hours the ESP-Easy don't get new/changed MQTT values. I open the webpage and check this within device-view.
An external MQTT Tool see the new values at the MQTT server.
I build a second ESP (NodeMCU-Board - Release mega-20180625), point to the same MQTT Server&topic and get the same problems with stopping MQTT receive.
Both devices stopping nearly at the same time.
After rebooting one device, this one get the new values while the other (not rebooted) don't get the values...

waspie
Normal user
Posts: 127
Joined: 09 Feb 2017, 19:35

Re: MQTT Import - stability problems ...?

#11 Post by waspie » 03 Jul 2018, 04:09

dirk_k wrote: 02 Jul 2018, 11:49 I use only one MQTT Import with only one value.
But because a non optimal software configuration i send the value 20 times per minute (10 times within 2-3 seconds).
After some hours the ESP-Easy don't get new/changed MQTT values. I open the webpage and check this within device-view.
An external MQTT Tool see the new values at the MQTT server.
I build a second ESP (NodeMCU-Board - Release mega-20180625), point to the same MQTT Server&topic and get the same problems with stopping MQTT receive.
Both devices stopping nearly at the same time.
After rebooting one device, this one get the new values while the other (not rebooted) don't get the values...
Im not very aggressive with speed. About as close i get to doing that is if a unit reboots i send all the mqtt values at once at boot time (which can be 4-12 or so messages all at once).

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: MQTT Import - stability problems ...?

#12 Post by TD-er » 04 Jul 2018, 21:43

What is the MQTT message delay setting?
I suggest setting that to 100 msec instead of 1000.
N.B. with the next build, that will be the new default setting and I will continue to run tests to reduce the effects of that setting, since it has a lot of impact on the entire node's operation.

dirk_k
Normal user
Posts: 19
Joined: 19 Aug 2017, 22:41

Re: MQTT Import - stability problems ...?

#13 Post by dirk_k » 04 Jul 2018, 22:17

currently my Message Interval is 1000msec
but this is for outgoing/sending messages only ... i think?

i correct the settings within my home control (FHEM) and send the control message one time once (instead 10 times)
now it works for days without problems.
i will report new problems as they occur.

Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot] and 37 guests