Victron to ESP using SerialProxy plugin

[giorgino]

Scheme-it-export-Kim-ESP32-2024-11-16-08-25.png

I'm new on ESP32.
ESP32 has more hardware UART. The Victron plugin will run on multiple victron devices, on the same ESP32 board? Or will flood the CPU?

Ath or TD-er will know how many serial devices the serial engine of ESPEasy can cope with but FWIW my setup works just fine with 2 Victron devices.
(Shunt and MPPT)

the difference between your system and mine is that you have a shunt instead of my BMV712.
anyway, I hope I will solve the issue separing the gnd, using optocoupler based system


And now (as usual) I need help
What do you think about this schematic ?

[giorgino]

optocoupler scheme:

OPTOISOLATORI3.jpg (70.63 KiB) Viewed 2107 times

and now I'm a little in trouble after reading this:
https://www.tarthorst.net/victron-ve-direct/

perhaps in "ESP32 side" It's better to connect the 3,3v instead of 5V, can someone check the value of the resistors?

[TD-er]

I am not sure about the LED.
I think the signal will be inverted if it would light up at all.

I would expect the LED to be mounted between TX and GND (thus red wire to TX and black to GND)
Can you try with a LED and the calculated resistor to see if it even starts blinking when connected between TX and GND?

[TD-er]

Ah now I see you have the transistor part of the optocoupler which will pull it down when the LED lights up, so I guess you may have had it wired correctly.
Still I think testing with a LED is important to see if it even lights up.

[TD-er]

Oh and the transistor part on the ESP side should be pulled up to 3V3, not 5V.

[bonti69]

I buyed small and cheap ADUM1201 isolators and convertor
https://www.analog.com/en/products/adum ... rt-details

ADuM1201-fbl.gif (10.3 KiB) Viewed 1995 times

The ADuM1201 is a dual-channel, digital isolator with 1/1 channel directionality based on the Analog Devices, Inc., iCoupler® technology. Combining high speed CMOS and monolithic transformer technologies, these isolation components provide outstanding performance characteristics superior to alternatives, such as optocouplers.

[TD-er]

That's even better.
The reason I did not mention it specifically is because those are not always easy to find and/or sometimes way too expensive.

However I really do like these icoupler chips.
They are available in many configurations, like for SPI, I2C, UART, etc.
And you can get them for quite high speeds to.
They also clean up the signal if needed.

Just only one thing to keep in mind... these can be damaged by ESD, so when adding some connector to them, make sure the GND is connecting to the device first.

[giorgino]

https://images.app.goo.gl/aMMhFZtcdqDftUrR9
https://www.amazon.it/dp/B07YKQ36H9?psc ... 0S690C29AG

in 2 days on my desk !

[giorgino]

so this will be my schematic solution. What do you think?

vedirect to esp32.jpg (280.94 KiB) Viewed 1928 times

[TD-er]

I don't see any immediate issue.
Only thing I don't understand is the mentioning of 2V on the Victron.

[giorgino]

ops sorry, mistake

[bonti69]

ops sorry, mistake

What software do you use for drawing schematics ?

[giorgino]

easyEDA, freeware, online, for me the best

It can solve the circuit drawing the PCB too

[bonti69]

Thank's giorgino, I saw you use NodeRed, it's complicated to read the values from ESPEasy?
A little bit offtopic, hope TD-Er and Ath will have some time to take a look...
Few days ago I have installed Home Assistant on a minipc, with ESPHome add-on.
Good news HA has native integration for Tasmota and Shelly, Tado etc. Sonoff also works, but not as good on some devices...
Searched google but no solution yet to collect sensor/variables data from an ESPEasy device...
I am frustrated because Tasmota devices are fully compatible, once connected with mqtt running on HA, both the switches/relays and sensors.
I have no skills on mqtt, just managed to connect ESPEasy on mosquito running on HA. Nothing more.
Is there any way to link ESPEasy to HA environement? If Tasmota can, I think also ESPEasy can...
Funny thing: I have two ESPHome devices connected to Victron mppt, and they publish data on a remote mqtt broker. Well , HAos intercepts exactly those data sent to another mqtt, maybe via some API method.

[TD-er]

What you're looking for is not yet implemented in ESPEasy and it's called "MQTT Auto Discovery"
Ton is looking into it as it really makes things a lot easier to add things to HA.

I know there are users who managed to publish the right topics to the MQTT broker to make HA work with ESPEasy, but it is quite a hassle as you may need to send some messages via rules and do have to manually setup quite a lot to get it working.

[bonti69]

What you're looking for is not yet implemented in ESPEasy and it's called "MQTT Auto Discovery"
Ton is looking into it as it really makes things a lot easier to add things to HA.

I know there are users who managed to publish the right topics to the MQTT broker to make HA work with ESPEasy, but it is quite a hassle as you may need to send some messages via rules and do have to manually setup quite a lot to get it working.

Good luck to Ath...
Currently ESPEasy allow only one mqtt controller, only one at a time. I wonder if could be possible to send data to 2 brokers, one remote, hosted on internet and one locally, on same lan. This feature especially for data storage, graphs, plots...

Well well, Battle of giants : https://www.home-assistant.io/integrati ... th-espeasy

[TD-er]

That's going to take quite a lot of code changes as there is currently only one MQTT client instance.
Even the used library (pubsubclient) probably needs some changes to be made.

[bonti69]

According with the instructions https://www.home-assistant.io/integrati ... th-espeasy
I have managed to read in HA the sensor from ESPEasy device.
Have to manually create/edit the yaml configuration, then create a card in dashboard

Code: Select all

mqtt:
  sensor:
    - name: "Humidity"
      unit_of_measurement: '%'
      device_class: humidity
      state_topic: "home/ESPname/HTU/Humidity"
      
    - name: "Temperature"
      unit_of_measurement: '°C'
      device_class: temperature
      state_topic: "home/ESPname/HTU/Temperature"

Screenshot 2024-12-05 at 19-23-02 Overview – Home Assistant.png (11.85 KiB) Viewed 1377 times

Screenshot 2024-12-05 at 19-24-27 X19.png (98.82 KiB) Viewed 1377 times

Screenshot 2024-12-05 at 19-25-56 X19.png (46.63 KiB) Viewed 1377 times

Sorry, the htu sensor just crashed during my tests or I am messing something up, so ignore the -47°C

Screenshot 2024-12-05 at 19-27-45 Overview – Home Assistant.png (58.03 KiB) Viewed 1377 times

[bonti69]

I noticed that, if the device has two mqtt controllers defined, the device connects to both of them, but publish sensor data only to one. Meantime , accept commands from the disabled mqtt. Interesting... maybe some rule to enable /disable mqtt1 and mqtt2 alternatively , in order to send data to both controllers, for example to local broker every minute and to remote broker every 5 minutes.

Screenshot 2024-12-05 at 19-36-17 X19.png (63.31 KiB) Viewed 1368 times

Is there a command to enable/ disable the controllers, which can be used in rules?

[Affinite]

@bonti69
do you live in Siberia ?
-47 Degrees ?

[bonti69]

@bonti69
do you live in Siberia ?
-47 Degrees ?

senzor failure, temperature measurement is down, humidity works
or maybe plugin viewtopic.php?t=9312
ESPEasy bug, flashed an older firmware and works again...

Screenshot 2024-12-05 at 21-54-57 X19.png (79.23 KiB) Viewed 1343 times

[Ath]

Is there a command to enable/ disable the controllers, which can be used in rules?

There are ControllerEnable and ControllerDisable commands, that take a numeric parameter 1..3 for the involved controller.

MQTT commands (from rules) are handled by the first enabled MQTT controller found.

[TD-er]

Also the same "first enabled MQTT controller" rule should apply to making the connection.
If it does actually try to connect to both, then that's a bug and I think it will probably result in a quite unstable system if it actually tries to connect to both.

[Ath]

I noticed that, if the device has two mqtt controllers defined, the device connects to both of them, but publish sensor data only to one. Meantime , accept commands from the disabled mqtt. Interesting... maybe some rule to enable /disable mqtt1 and mqtt2 alternatively ,

AFAICS, the MQTT Controllers don't unsubscribe from the "Controller Subscribe" topic the when the controller is disabled (been digging in the MQTT code a lot recently ), so they may still receive and process 'stuff' from the server, that is then handled by the active controller...

in order to send data to both controllers, for example to local broker every minute and to remote broker every 5 minutes.

A better scenario would probably be that when you enable each controller you also TaskRun all the tasks that send data to that controller, and either disable the Interval (if the plugin allows that) or set it to a very high value, so it won't be triggered too often

[TD-er]

Frequent disable/enable of controllers will probably have unsdesired side-effects.
When disabling a controller, the data in the controller queue will be attempted to flush.
However the minimum send interval is going to be obeyed so it may take some time.
If flushing fails, then the disable call will be completed and this also erases the queue.
Frequent (re)connecting to a MQTT broker can cause significant stalls, which does also occur when the MQTT controller cannot connect to the broker (or connection is dropped almost immediately) and this does significantly delay just about everything running on ESPEasy.

[Ath]

I noticed that, if the device has two mqtt controllers defined, the device connects to both of them, but publish sensor data only to one.

OTOH, what you seem to ask is how to relay data from one MQTT broker to another.
Here's a guy that wrote some documentation on that subject: http://www.steves-internet-guide.com/mo ... iguration/
(and you probably already have Mosquitto running on your HA server )

We've not yet seen a (working) configuration of sending to multiple MQTT servers, as at least the Arduino-based MQTT Client library (PubSubclient) doesn't support that, and if it did, then having multiple instances would take quite some memory (rather large buffers are reserved) so it might not run smoothly.

[TD-er]

Also as far as I can see, even Home Assistant doesn't allow for >1 MQTT broker to be configured.
In HA: Settings->Devices and integrations -> MQTT
This only allows for 1 item, which looks to me like it cannot have >1 MQTT configuration

[bonti69]

Well, my knowledge on mqtt is limited, but I wonder why can be enabled to send a certain task to BOTH controllers, as long you can't enable both of them...

Screenshot 2024-12-05 at 23-28-20 X19.png (17.42 KiB) Viewed 1300 times

Also Home Assistant cannot import data from another mqtt server, have to do some mqtt bridge, configuration is beyond my understanding.

[bonti69]

Also as far as I can see, even Home Assistant doesn't allow for >1 MQTT broker to be configured.
In HA: Settings->Devices and integrations -> MQTT
This only allows for 1 item, which looks to me like it cannot have >1 MQTT configuration

You are right

[TD-er]

There are lots of other non-MQTT controllers, like HTTP, or store to flash.
So that makes it a valid use case to have multiple controllers available to a task.

From a task perspective it should not matter whether the configured controller is disabled or enabled.
You simply configure a task to send to controller X and for some specific controllers you may get an extra IDX field to send along with the taskvalues. (IDX is used for Domoticz for example and only a few other controllers)

As is already mentioned, a controller can be enabled or disabled from rules and it would be quite tedious if task configurations should be altered when another controller is enabled/disabled.

[bonti69]

Right again. Two mqtt controllers are not allowed simultaneously

[TD-er]

Please note that this is not just having 2 of the same MQTT controllers, but just any combination of MQTT controllers as they do share a lot of common code. (only the platform specifics like message formatting differs)
For any combination of MQTT controllers, only the first enabled MQTT controller will be considered when you use commands like publish or use the MQTT import plugin.
A MQTT client needs to make a connection to a broker and since there is (at most) only a single MQTT client instance running in ESPEasy, you simply cannot have multiple MQTT controllers connect to different MQTT brokers at the same time.

For ESP8266 it simply would not work as we don't have enough free resources available (mainly memory)
But also for ESP32-xx quite a lot needs to be changed to allow to have multiple MQTT controllers active at the same time:

- Look at max. nr of open connections
- Make sure no global or static variables/objects are being used by a MQTT controller
- MQTT scheduling code needs to be changed
- MQTT related commands (e.g. 'publish') need to be adapted to include a parameter to what controller should be used
- MQTT pubSubClient library needs to be adapted to have multiple instances running on the same unit
- Special case for when 2 controllers are configured to connect to the same MQTT broker.
- Received commands via MQTT must all be looked at to see where any reply/ack must be sent to (extend command source/origin handling code)

And probably quite a few more changes are needed.
But like I said, there currently is no other similar firmware (ESPHome/Tasmota/ESPruna/etc) which supports connecting to multiple MQTT brokers, not even Home Assistant itself.

[bonti69]

deleted

[Affinite]

In other news the Victron plugin is still working well in the field.
I think this thread is way "off-piste" now.

[bonti69]

sorry, I started a new thread