Send reset info and log via POST or MQTT

[Misiu]

I'm using ESPEasy on Wemos D1 which has two original DS18B20 sensors connected (with 4k7 resistor).
I'm sending the results to Home Assistant via MQTT.
I noticed that sometimes (very often lately) I get multiple offline messages to LWT, this results in bad readings:

(each value below 0 is when the device is disconnected)

I have a couple of devices in my network, but only this one has two sensors (not sure if this changes anything).
I'm aware that things can go wrong sometimes, but I'd like to know what is the reason. This will help me to eliminate it.

Is there a way to send log info and reset reason (which is probably a part of log) via POST or via MQTT?
I don't have access to the device all the time (it is mounted in my parent's house), but if it could send the log to my server then it will be easier for me to find the reason.

Can I send WiFi and connection to MQTT server details? I'd like to debug this and remove the problem once for all.

[TD-er]

Does it have a long wire ?
Maybe it is picking up some noise as it is acting like an antenna.
Such behavior could be prevented by adding a pull-up resistor.

The faulty values do seem to have a very specific value.
What is the set value in the plugin to send when a NaN value is read from the sensor?

[Misiu]

Hi @TD-er,

the issue is more complicated. Let me start from the beginning.
I have an electrical box in the wall that is between living room and kitchen. In that box I have Wemos D1 and two sensors going to each room, the max cable length is 30 cm.
I'm using pull-up resistor (4k7)

I've created another screenshot of the graph:


The blue line is the value from my MQTT sensor and the green is from the template sensor. I'm using a template sensor to calibrate value in the room (I need to subtract 1.9 to have the correct value). Because the sensor is unavailable for some time the value is 0 and after subtraction, it is -1.9. This is specific to Home Assistant, but after you do that for the first time it is more intuitive

The problem is that the wemos D1 is losing connection with MQTT server. I was monitoring LWT topic and I saw it is sending offline and offline messages. The problem is I don't know why this behavior is happening.

This is how the sensor is configured in Home Assistant

Code: Select all

  - platform: mqtt
    state_topic: "Temp1/Sypialnia/Temperature"
    name: "sypialnia_temperature"
    unit_of_measurement: "°C"
    expire_after: 180
    availability_topic: "Temp1/status"
    payload_available: "Online"
    payload_not_available: "Offline"

so when LWT topic gets `Offline` message the whole sensor is offline and I get those spikes (bad readings).
I think there is something wrong with the wemos or my power source or wifi or config. The power supply is very good. I'm using Wean Well 5V 3A DIN Rail power supply and using CAT5 wires.
I'd like to monitor wifi and MQTT parameters on wemos and report them at specific intervals to my server.
I also want to send log info and reset reason.
This way I'll be able to check what is going on and what was the reset reason.

Sadly I can't go there to check the electrical connections, but I can use Team Viewer to connect to my parent's laptop and reconfigure the sensors. The offline messages aren't happening regularly so it will be hard to wait a couple of hours for the problem to occur, but if the crash report and logs can be sent to my server I won't lose them and will be able to investigate.

I'll try to log in today to my parent's network and will make screenshots and get version info and configs.

I hope this is a more clear image of the problem I have

[Wiki]

I do have similar issues with two of my Wemos, one equipped with DS18B20, the other one with DHT22.
Workaround:
For me the temperature readings are not time critical and I am assuming that the indoor temperatures won't fall below zero (and if so, I would have a totally different problem). So I don't send the data out of the tasks of the devices but using rules and publish the data only if the values > 0.

[TD-er]

About the flaky MQTT connection.
Do you also have some information about the WiFi stability itself?

If possible, you could also try to run the "beta" build from the set of files, as that one is using a different pre-compiled SDK image which sometimes (no idea why) may help in unstable WiFi connections on just those units that have unstable WiFi.

You could also try rotating the ESP 90 degree, so its antenna is pointing in another direction (hopefully vertical).
Quite often the presence of metal near the WiFi antenna may also cause issues.

If the unit is used in an area where there are lots and lots of WiFi access points, then you must make sure the access point doesn't hop to other channels.
It may also be useful to set the wifi to B/G only (better not switch that from a remote location if the router/access point only allows N-only) and enable to send Gratuitous ARP packets.

[Misiu]

@TD-er

thank you for the tips
I didn't know that rotating the antenna could cause such issues.

I have two other sensors in rooms near the problematic sensor and one sensor outside, they work stable, only this single unit is causing troubles (only this one has two DS18B20, rest has an only single sensor connected).
I have two routers in the house, both have assigned different channels, but maybe the wemos d1 (the one with the problem) is connecting from one router to another. They both have wifi with the same name (bridge config, single dhcp, used to extend the range of wifi).

Where I must enable "send Gratuitous ARP packets"? This must be done on ESP or on the router?

About wifi stability: can I send such info from ESP via MQTT? This way I'll be able to collect logs.
Are there events like wifi disconnect/reconnect in ESPEasy? I know that I can connect USB cable to ESP and collect logs via serial, but there is much irregularity in disconnects and sometimes everything is stable for a day or two.

If I'll be able to send logs and wifi/mqtt events to an external log server then it will be easier to diagnose such problems.

[TD-er]

You can send all logs to a syslog server.
All Linux distributions have one available, but I guess you can also have one for other OS'es as it is a really ancient, but effective way to collect logs in a central place.

You also can get wifi#connected and wifi#disconnected events. See documentation: https://espeasy.readthedocs.io/en/lates ... ore-events
N.B. those are also present for MQTT connection state changes.

Gratuitous ARP can be set in the Tools => Advanced settings (almost at the bottom of the web page)

About the orientation of the antenna.
Most of the time, the antenna of an access point has a vertical orientation, which means it will have the strongest signal and sensitivity in the circle around it (on the same floor).
Right above or below the access point, you will have almost no reception.
The same applies also for the devices. The antenna of the ESP also has a rather flat radiation pattern. So rotating it 90 degree may make quite a difference if that will move the access point into this radiation pattern.


"Elevation pattern" is the X/Z (or Y/Z) cut-out of the 3D pattern.
"Azimuth pattern" is the X/Y plane of the 3D pattern.
For a normal dipole antenna (straight wire) you must imagine the antenna being positioned at the Z-axis, on the XY-plane at (0, 0)
The inside of that donut is where you have the best signal and the colors on the 3D shape give an indication on the signal strength.

[Misiu]

@TD-er,

thank you for the answer.
I don't see WiFi#Disconnected event. Is it missing in the docs? There are events for MQTT, but no WiFi#Disconnected - https://espeasy.readthedocs.io/en/lates ... ore-events
I was going to ask how to send RSSI and other info, but I see they are available as variables, so I'll be able to create something from them

Any examples of how to send logs to a controller? I can create something in .NET or PHP that will allow POST or GET requests.
I'm aware syslog requires just a couple of settings, but I'd like to send logs to a generic HTTP controller.

Ideally, a rule that fires after boot, waits for WiFi to connect and send the log to HTTP, especially restart/reboot reason.

[TD-er]

Reboot/restart reason is one thing that's still missing as a variable.
If that's added, then I guess you can do all that (except pushing all logs)
You can already fetch the reset reason in the JSON output at <ip>/json

There can be quite a lot of log entries, so sending them all via a HTTP call will quite likely cause reboots.
But if you look at the web log.
That's a Javascript doing a fetch on the node and receiving JSON formatted log entries.
At the end of the JSON, is a TTL value, signalling when you need to retry the next call. This TTL is dynamic and based on how much the buffer is filled and how long since the last call you made fetching it.

Maybe that's more practical to fetch the logs?

[Misiu]

@TD-er thanks for the reply.
Reboot/reset reason as a variable would be helpful.
The problem is that I don't want to expose the devices to the outside world (which was mentioned many times in the forum as a bad idea).
I'd like the device (node) to push logs to my server, not in the opposite way.

I think my use case is valid: I have devices in my parent house, there is no server that can fetch logs from the devices. I have an external server that can support GET and POST operations. I'd like to use that external server as my log server to gather info about reboots/restarts and any potential problems.
https://sentry.io/ is a good example of every that could be used. It supports up to 5k error messages per month.
Having a way to send logs and errors to an external server would be awesome maybe a plugin? WDYT?

[ThomasB]

Reboot/restart reason is one thing that's still missing as a variable.

+1. %reset_reason% would be a useful system var to add.

- Thomas

[TD-er]

[...]
Having a way to send logs and errors to an external server would be awesome maybe a plugin? WDYT?

We already have the syslog server as an option to send logs to.

[Misiu]

We already have the syslog server as an option to send logs to.

I'm aware of that, but the syslog server must/should be placed in the local network.
I must search for something that I can run online.

Any chance to add a reset/reboot reason variable? This would be super useful.
Then we can create a rule that would fire on WiFi connection that would check if reset reason isn't null, if not it would be sent to a controller (MQTT or HTTP)

[TD-er]

[...]

Any chance to add a reset/reboot reason variable? This would be super useful.
Then we can create a rule that would fire on WiFi connection that would check if reset reason isn't null, if not it would be sent to a controller (MQTT or HTTP)

Sure. I guess there should be 2 versions of it.
One representing the string so it can be used in writing to displays, logentries, etc.
But that may be a bit tricky as we currently have nothing similar, so I don't know how much work it will be.
And one representing the int value of the reboot reason as it is stored in the RTC.

Maybe also separate events to make it readable, like "reboot#hwwatchdog" so you can actually read the rules using them as a human