MQTT settings

[Dick60]

MQTT implementation. I am using Domoticz as my controler. Most of my devices are ESP32 or 8266 all with the latest firmware of ESPEASY. I am totaly a beginner with MQTT (yesterday 1st time installed). I Want to make a Lux value also avilable on another ESP8266 unit, call it ESP1. The data is produced on ESP2. On both units I installed the MQTT controler and activated. On unit 2 i alo activated the MQTT controler to send the values to , in my case, IDX 167. On ESP1 i created a new device called "Generic - MQTT Import" and now it stops. I have red the documentation but it is not clear. What are the settings to receive the IDX167 value. No filters are requered, nothing special, only the value. Can you help me with that?

[TD-er]

Maybe Ton can explain this a lot better as he is the one who extended the possibilities of the MQTT import plugin.

But I can already try to explain the basics to you so it can already sink in a bit till Ton is back from work

MQTT is all about publishing messages to a topic and subscribing to topics.
If you publish a message to a topic, the MQTT broker will make sure anyone subscribed to that topic will receive the message.

By default you need to be connected to the broker to receive the messages sent to topic(s) you subscribed to.
But there is an exception to this, when messages are sent with the "retain" flag set, then the last message sent to that topic will be delivered as soon as someone connects and subscribes to this topic.
Typically this is done with "LWT" messages like "node bladiebla is disconnected". (not literally but you get the idea)

When you look at the controller settings, you can see there is a publish string and a subscribe string.
Domoticz has made a rather poor choice years ago to only have 2 topics (in and out) where all nodes publish to and subscribe to.
So when you send task data from a task to a Domoticz MQTT controller, your message ends up in all the other messages of other nodes.
It is also formatted as JSON, so you need to parse all messages to look for some IDX value.

If you only need to send messages between nodes via MQTT which don't need to be processed by Domoticz, you can perhaps better send them via the rules by using the "publish" command to a specific topic.
Then in the MQTT import you can specify those topics to subscribe to. (you can use "#" as wildcard to subscribe to several topics, like "espeasy/#")

Well I think you can let this sink in until Ton will come up with a more to-the-point reply

[Ath]

Well, the MQTT explanation by TD-er is about as clear as it gets, so there isn't much that I can add there

When using Domoticz there is one big gotcha: It's not an MQTT server...
MQTT is a server oriented solution, having a central server to store & forward the messages. Most used MQTT server combined with Domoticz is Mosquitto, and the installation and configuration process is documented in the Domoticz Wiki

Then there's the 'problem' you are trying to solve: Getting a sensor value from one ESPEasy unit to another.
TL;DR;
I'd not use MQTT to transport that from one ESP to the next ESP, that's what we have the ESPEasy P2P protocol for.

Using MQTT for that feels like taking the Intercity train to the next street, yes it's fast, but getting to the station (read: setting stuff up) takes way longer than taking the bicycle to that next street

Next to sending out any data via P2P to another node, you can still send the same data to Domoticz (or any other controller-supported server, just not 2 MQTT servers) to be able to collect the data and make nice graphs from that.
And an ESPEasy MQTT controller expects a complete MQTT server on the other end, not a relatively simple client like the MQTT Import plugin.

For using the MQTT Import plugin, you (also) need an MQTT server (and a working/enabled MQTT Controller on the ESPEasy side), so the plugin can subscribe to the topics available there. And, as TD-er already explained, Domoticz uses only 2 topics to handle all incoming and outgoing messages, that's why I added the filtering, mapping and JSON parsing features to that plugin.
The filtering is needed to reduce the number of messages being processed when your sensor network increases, and you don't need (most of) the data sent out via the topic. Usually it works best filtering on the Domoticz IDX value, as that's unique for each object (sensor, switch, group, etc.) in Domoticz.
The mapping is needed to transform the yes/no/ok/on/off/up/down values sent by some Zigbee and ZWave devices like (dimmer)switches into something numeric to be handled by ESPEasy.
The JSON parsing is needed to pick out the correct values from (often) semi-colon separated messages sent by Domoticz. Parsing those in ESPEasy rules would be 'challenging', to say the least

To get some good info/knowledge on MQTT in general, there is this site: https://www.hivemq.com/mqtt/

[Dick60]

Thanks for the reply. I know, I use the p2p protocol but o a regular base I loose the p2p connection between the nodes. I was a good reason to make a simple start with the MQTT server on my raspberry pi and solve the problem of the lost connection, but again see it as a start. I will continue in finding a working solution using the MQTT server even it is not the best solution for now but perhaps I can use it in the future.

[Ath]

Well, using MQTT to transfer data from one ESPEasy unit to another is quite a 'heavy' solution, that can be solved much simpler, that's why I suggested the P2P route.
Losing the P2P connection is something to dive deeper into, as that's not supposed to happen, though it's using UDP, a non-guaranteed delivery method from one station to another. I've seen short hickups in this communication, but all in all it's quite reliable over here, and it will recover quickly.

When going for the MQTT route, you could follow these steps:
- Setup Domoticz with MQTT
- Create a virtual sensor to receive the Lux value you want to send/store
- Set up the Domoticz MQTT Controller on the node having the sensor (ESP2)
- Configure the Lux Device to send out the value(s) to the idx for the virtual sensor
- Configure the Domoticz MQTT Controller on the receiving ESPEasy side (ESP1)
- Configure the MQTT Import plugin on ESP1, for the "domoticz/in" topic, filtering on the idx used on ESP2
NB: Using domoticz/in will pick up the value when it is sent to Domoticz! You can also use the 'domoticz/out' topic, but that's probably a much higher traffic topic
- Add a Rule on ESP1 to handle the received value by assigning it to a dummy task, or use it directly, f.e. on a display

[Dick60]

Thanks for the explanation of the MQTT. Tomorrow I give it a try. For me it is good to learn more about this protocol. About the p2p it works good in most of the cases but I have 2 nodes that give me problems. Sometimes it takes a day before the connection is working again. I already replaced them by other 8266 but still the same problem. I ordered 2 esp8266 where I can connect an antenna. Hope that solves the problem. I test the MQTT solution tomorrow and again only for testing.