Issues with Dallas 18B20 in version 20210114

[jgrad]

Hi,

I am running version ESP_Easy_mega_20210114_normal_ESP8285_1M.bin on SONOFF 4CH and I identified following issues with Dallas sensors 18B20 after upgrade from version 20201227:
* When I connect sensors (I use 2 sensors, both connected to GPIO-2) load (info page) goes up significantly and I am experiencing crashes of device after about 15-20minutes since reboot - such crashes were not present running 20201227
* device tasks have setting that they send measured values to controller every 10seconds - when device start entering in problems some readings are not send regularly (eg 2-4 readings are not sent, then interval is respected, then again happens that some readings are not sent)
* If I disconnect sensors temperature values on device tasks do not go to 125 (setting I use under Error State Value) but last measured temperature stays
* For each sensor I use separate device task (measuring one temperature per task, I don’t use multivalue possibility introduced end of last year)
* If I downgrade from 20210114 to 20201227 problems dissapear.

Since I see in release notes of 20210114 that some code changes were done related to Dallas please check if problems can arise from that change?

Do somebody else identified similar problems?

BR J.

[TD-er]

Can you test the latest code on the mega branch? (soon a new nightly build will be made)

https://www.dropbox.com/s/07vv1cspmtq4s ... 6.zip?dl=0
https://www.dropbox.com/s/mkekzbsli4h85 ... 6.zip?dl=0

[jgrad]

thanks @TD-er. I downloaded linked versions - I report my findings.

[TD-er]

* If I disconnect sensors temperature values on device tasks do not go to 125 (setting I use under Error State Value) but last measured temperature stays

I think this may actually be a bug introduced by my big update of the Dallas plugin.
So that's probably still an issue with the test build I made last night for you.

* device tasks have setting that they send measured values to controller every 10seconds - when device start entering in problems some readings are not send regularly (eg 2-4 readings are not sent, then interval is respected, then again happens that some readings are not sent)

I think this may somehow be related to your first issue regarding stability.
It sounds like your WiFi connection may somehow be having issues, leading to crashes.
I really hope the test build I made may fix this as it does contain a HUGE update in how WiFi is handled.

[jgrad]

I am in middle of testing but:
* one part of problems is related to wifi - if ESP is connected to SSID which is used on 2 different access points (and ESP "see" signal from both) then ESP is constantly crashing. I now switched to another SSID which is served just from one AP.
* regarding testing Dallas sensor - problem with disconnected sensors which does not reflect in temperature values to go to 125 (Error State Value) is still present.

[TD-er]

Hmm that's strange to hear you see this "hopping" on the latest build as that's one of the things I tried to address here.
It does now always try to connect to a specific BSSID, which is unique.

If it does hop between APs, then I guess you have some strange setting on the AP to disconnect this node on purpose.
Some APs use some threshold of RSSI to force a node to reconnect to "the other AP".
For example TP-link Omada series does have such a setting (and I have tested it with them thoroughly)

It is also possible an AP may disconnect a node if it sees a number of CRC errors in packets received.

So if you're seeing such hopping with the latest build I linked, then I really would like to know the AP used and want to know more about its used settings.

[jgrad]

I am trying to isolate problem - I run also quite a lot of code within rules.

Same rules does not have problems running on last year versions, but first I identified problems on 20210114.

[jgrad]

seems that problem is not related to wifi since I get same behaviour if ESP is connected to SSID served from 2APs or SSID served only from one AP.

WIFI signal is good and I dont see any disconnects under /sysinfo

[TD-er]

Could you try to get some logs from serial while it is running normally and also crashing?

Maybe also try to grab periodical the amount of free memory. To see if we can find some correlation between crash and free memory

[jgrad]

if I dont change anything (same configuration and same rule set as working on 20201227) then I get following parameters from operation on 20210217 (consumption of RAM through time, load is all the time 100%, ESP is crasshing within 4-6min from start)

Dallas_connected_sensors_full_rules.jpg (177.34 KiB) Viewed 9269 times

I am currently removing parts of code from rules in order to isolate problematic parts which cause such behaviour in RAM consumption.

[TD-er]

Do you call events from rules?
If so, please check if you can change them into "asyncevent" (thus it will be queued, not executed immediately)

[jgrad]

I call events from rules but only on some rare occasions (once per day, on button click). I replaced events with asyncevents commands.

Is it possible that new features in Y2021 versions are consuming too much ESP resources that code in rules are not processed with same performance - LOAD is all the time on 100%. It fall below 100% only if I disable timer events.

within rules I have
* 3 clock events (1 triggered at midnight, 2 triggered every 10min)
* 1 looptimer running every 1000ms (must be accurate) - code within it mainly comparing some dummy device values, setting dummy device values and switching on/off GPIO pins
* 1 looptimer running every 10s (this not need to be accurate) - code within it reads 2 Dallas sensors, compare temperatures/dummy device values, set dummy device values

BR J.

[TD-er]

Some simple optimization tips:
Place the most frequent occuring events at the top of the rules file.
Try to figure out if you have other events happening too, which may not be needed, nor handled in your rules and place empty rules blocks for them in the rules file, still ordered with most frequently occuring one on top.

What reading interval is set for the Dallas sensor tasks?

[jgrad]

As per recommendation here (viewtopic.php?f=6&t=8265&p=49858#p49858) I have moved event handlers for timers at the beginning of ruleset1 and boot procedure in ruleset4.
Dallas sensors 18b20 within 2 separate device tasks are read with 60s interval.

In device tasks I have additional 6 dummy devices, each with 4 values which have interval 60s (for sending data to MQTT controller - Home Assistant (openHAB) MQTT, Mosquito), I use this dummy devices as variables within rules which values shall be prevented if reboot happens.

since I use dummy devices just for internal/local use within rules I tried to set interval to 0s (in order not to fire events) but it seems it is not possible - now I set them to much higher value.

I still work on isolating problem - I already changed some code (replaced elseif-s with normal if) and it seems performance improved little bit.

[jgrad]

I think I managed to isolate problem - on all 6 Dummy devices (each containing 4 values) I had interval set to 60seconds and this generated so much events that ESP was crashing. When I configured interval to higher number (currently 7200s) consumption of RAM become stable, LOAD is also below 100%. I see this pattern for more than 12hours so it seems that problem is isolated.

I see that that on Dummy devices I cannot set Interval to 0 in order to disable events - is this intentionally?

[TD-er]

Well I don't think a dummy device should use any interval, so I don't know why it can't be set to 0.
Will have a look at it.

[jgrad]

@TD-er, can you explain also difference btw Internal variables (let..) and Dummy Device values (Task valueSet) in regards to consumption of system resources while using them within rules - only difference I know from documentation and forum posts is that DummyDevice values "survive" soft reboot, while internal variables not.

[TD-er]

Dummy variables:
- As it is part of a task, task values are stored in RTC and restored at boot (no power loss)
- Type is "float", thus 22 bit of resolution for storing an int without loosing decimals
- Dummy tasks can be connected to upto 3 controllers, which will receive the dummy values when called "taskrun" on that task.
- Set values via taskvalueset
- Retreive values via [taskname#varname] syntax

Variables:
- Content lost after reboot.
- Since recent update no longer a limit on number of variables
- Since recent update variables are stored in a 'double' so upto 52 bit of resolution.
- Set values via "let" command (also supports all available calculations)
- Retreive values via [var#1] or [int#1] or %v1% (for variable #1)

[jgrad]

@TD-er, thanks for explanation. What about performance regards to usage of CPU when manipulating with DummyVariables vs InternalVariables within rules?

[TD-er]

Handling double precision is of course more CPU intensive, but I doubt it is really an issue unless you're running really intensive computations.
But I guess the rules parsing does have a much bigger impact.
N.B. during rules parsing I do also use double precision to perform the calculations and formatting of values.
This does allow for more intuitive number formatting and is slightly more forgiving if a user may check for "==" on floating point values.

One other aspect that may differ in performance is that dummy variables also need an extra store to RTC when changed.
So I guess it all cancels out.

Dummy variables don't really need extra memory, as the floats are already allocated for max. nr of tasks.
Variables will take up roughly 16 byte of memory per variable as soon as they are set via the "let" command.

[jgrad]

I share results from testing over last weekend - pictures below shows how different ESPeasy versions consume RAM and CPU. In all 3 cases ESP (Sonoff 4CH pro V3) had same config and same set of rules.
Application of my project is controlling 2 roller shutters (controlling two 24V motors for opening/closing side "windows") on greenhouse based on temperature inside greenhouse.

ESPeasy_20201227_vs_20210114_performance

ESPeasy_20201227_vs_20210114_performance.jpg (252.82 KiB) Viewed 9125 times

ESPeasy_20210217_performance

ESPeasy_20210217_performance.jpg (274.88 KiB) Viewed 9125 times

TD-er, I hope it will help you by improving excellent SW for ESP chips.

BRJ.