forwarding Rx/TX of UART via MQTT

[Eisbahn]

Hi experts,

I'm having an Microchip/Atmel ATmega 8-bit uController doing some calculation and making usage of PWM on its I/Os. Major control is running at the moment via UART, and I could change that easily (e.g. every telegram has a fixed length of 10chars, get a CRLF at the end...). Final solution should be to have a chance to control it via MQTT. I used an NodeMCU with latest ESPeasy/Lets control it on it and enabled the Ser2Net function and for output the Openhab MQTT feature. Therefore some questions:
- What baud rates are supported by ESP and how accurate are they (as the ESP doen's seem to have a "clock" oscillator like 14.7456 MHz)?
- At the moment I could see the output on the Ser2Net port with 115200baud, but nothing on MQTT: no topic, no subscription... Is this feature even possible?
- If yes: is there soem kind of How-To?

Thanks, Jürgen

P.S. Alternative solution would be an old RPi with paho lib and small Phyton script

[TD-er]

Please have a look at the SerialProxy plugin and also the discussion here: viewtopic.php?f=6&t=7960&p=47301&hilit= ... oxy#p47301

[Eisbahn]

Hello TD-er,

tried the SerialProxy: this would be a very great tool, but it seems to work only for very slow and rare communication. I need something around 600Bytes/sec as I first need to make a base64 conversion (as the plugin is limited to ASCII strings) bloating up the data amount. Tried it with some bursts of 100Bytes every 100msec at 115200baud and also some 10Bytes continiously at 38400baud. However: after few seconds the ESP seems to have issue with its WiFi connection and transmission is interrupted. Could it be an issue of a nightly/self build version? Afterwards I tried it with 10Bytes every 100msec at both speeds and that worked much longer/perfect.
As even the ATMega uC could handle that data easily: I disabled under tools the serial output (for logging) and selected the HW Serial0 pins in the SerialProxy and hoped those are handled via Interrupt to not block all CPU ressources, but it seems ESP is doing everything via SW polling and therefore failing... Is this true/limitation of ESPeasy concept?

Best, Jürgen

P.S. as it is working in genral, three sugestions:
1) maybe you could also have some option to make an update/transmission either on
a) a fixed length of chars (e.g. you always need to receive 10 Bytes on Rx Pin before forwarding them via MQTT or need to get first 10 chars via MQTT before sending them out on Tx)
b) after getting a CRLF
c) combination of a)+b)
2) allow the user to make settings for parity, stop bits...
3) give some warning regarding usage of higher baud rates and

[TD-er]

Well I've tested the Serial Proxy plugin for a while forwarding strings upto 500+ bytes per message. (at a rate of around 1 per second)
However this data still needs to be sent to the controller and as it has a queue mechanism per controller (see controller settings) it might be the memory fills up and/or gets fragmented.

Make sure to lower the controller's minimum time between messages to keep up with the flood of messages.
Also the serial port only has a receive buffer of 128 bytes, so at baud rates of over 38400 bps you may need to call it with at least 50/sec calls (PLUGIN_FIFTY_PER_SECOND)

I think you really should run this from a HW serial port, as SW serial does cause way too many interrupts to be handled and thus slowing things down quite a lot.
SW serial for the GPS for example (which only runs at 9600 bps) already has a significant effect on the ESP's performance compared to HW serial.

There is also a rather extensive filtering in that plugin, but I guess processing at high message rates may be too much for the ESP.

I'm all open for suggestions for the SerialProxy plugin.
It was initially meant for a project, but that one is now processed differently and it appears there is quite some demand for such a generic plugin.

[Eisbahn]

Tried it several times, but never got it working for longer time. Also if I try to use the webinterface, I get quite often a white page with just a text telling me memory is low if the UART is in use. Therefore I now use an old RPi2 which I found in my "garbage box".
Netherverless: perfect work from your side and mainly astonishing and quick support!

Have a nice weekend, Jürgen

[TD-er]

Ah running out of memory is indeed a clue to why it might crash.
What controller settings do you use?
It may also help to create a specific build with only the needed plugins and controllers included as having more included in the build (even without using) still consumes RAM.

[Eisbahn]

I made a custom build via Platform.io by instructions here for my NodeMCU board and took only the SerialProxy additional inside: https://www.letscontrolit.com/wiki/inde ... platformio
During runtime only the SerialProxy and OpenHab MQTT are active, no filtering is active in SerialProxy settings and also no other devices are running.
However: it runs fine now on a RPI, even if that is much oversized

[Ath]

it runs fine now on a RPI

Just curious: Are you running RPIEasy, or plain Raspbian/Raspberry OS with some custom scripting?

[Eisbahn]

I'm running Raspbian on an older RPi2 and modified a Pyhton script using paho lib.

it runs fine now on a RPI

Just curious: Are you running RPIEasy, or plain Raspbian/Raspberry OS with some custom scripting?