Best plugin to start from for 2-way serial communication

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Best plugin to start from for 2-way serial communication

#1 Post by fluppie » 06 Jun 2019, 00:36

I want to develop a serial plugin for ESP Easy that sends and receives commands over software serial.
Which plugin is the best to start from? Using 2-way communication, send and receive.

For example the command to fetch the serial number of the connected device:

Code: Select all

byte stx[] = {0x02};
byte etx[] = {0x03};
byte ack[] = {0x06};
byte cmd1[] = {0x30, 0x34, 0x36, 0x34}; //0464 // Serial Number
byte cmd2[] = {0x30, 0x38, 0x36, 0x38}; //0868 // Unknown
byte cmd3[] = {0x30, 0x39, 0x36, 0x39}; //0969 // Operating time

        mySerial.write(stx[0]);
        mySerial.write(cmd1[0]);
        mySerial.write(cmd1[1]);
        mySerial.write(cmd1[2]);
        mySerial.write(cmd1[3]);
        mySerial.write(etx[0]);
Then to parse incoming data:

Code: Select all

        
//============serial num=============//
void serial_num()
{
  data_array[10] = 0; //terminate original string early
  Serial.print("Serial number: ");
  Serial.println(&data_array[2]);

}

void operating_time()
{
  data_array[10] = 0; //terminate original string early
  runningtime = strtoul(&data_array[2], NULL, 16);
  Serial.println(runningtime); // in quarter seconds

  Serial.print("Running Time: ");
  float runningtimef = runningtime;
  Serial.println(runningtimef / 4 / 60 / 60); // divide decimal number / 4 / 60 / 60 to get running time in hours
}

fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Re: Best plugin to start from for 2-way serial communication

#2 Post by fluppie » 03 Aug 2019, 10:57

Anyone who can tell me which serial plugin is using the best for send / receive within the plugin?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Best plugin to start from for 2-way serial communication

#3 Post by TD-er » 03 Aug 2019, 13:12

I am already working on it.
See my progress here:
https://github.com/TD-er/ESPEasy/tree/f ... rial_proxy

Current status:
It can read a string from serial and send it (as a string) to a controller.

Still to do:
- user define init string (to initiate device connected)
- Add some timeout settings (when is a sentence finished, or when to send init string again)
- Add pin to allow to reset connected device
- Option to interpret received data as numerical.
- Receive strings/values from controller. (that's going to be a lot of work)

fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Re: Best plugin to start from for 2-way serial communication

#4 Post by fluppie » 05 Aug 2019, 11:09

OK cool, I tried the custom and dev build. Or should I self-compile to get this one in?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Best plugin to start from for 2-way serial communication

#5 Post by TD-er » 05 Aug 2019, 20:47

Self compile.
I have not yet included in any build, since it is really a work in progress.
But since it also needed some changes in the rest of ESPeasy I didn't want to wait to long to merge it or else I may trouble myself in merging future pull requests.
So that's why I already merged it, but it will change in the next week.
A work-in-progress so to speak.

fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Re: Best plugin to start from for 2-way serial communication

#6 Post by fluppie » 07 Aug 2019, 14:28

Do I need to define/undefine something? Since it's not appearing in the normal, testing and dev build when self-compiling.

fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Re: Best plugin to start from for 2-way serial communication

#7 Post by fluppie » 06 Sep 2019, 11:34

Any update on this? Is it included in the testing builds or do we need to uncomment something to use it?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Best plugin to start from for 2-way serial communication

#8 Post by TD-er » 06 Sep 2019, 11:50

The SerialProxy plugin is merged into mega branch and should be included in the "custom" build (if that one is included in the nightly's)

fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Re: Best plugin to start from for 2-way serial communication

#9 Post by fluppie » 06 Sep 2019, 20:48

ESP_Easy_mega-20190903_custom_ESP8266_4M1M

Should be this one then :)?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Best plugin to start from for 2-way serial communication

#10 Post by TD-er » 07 Sep 2019, 00:28

Yep, please let me know if you like the general idea of it or need some changes to make it really useful.
It is currently only sending 1 string you can specify in the plugin and sending any line of text to the controller you specify.
N.B. not all controllers support sending strings, but at least the MQTT controllers do.

hipek189
New user
Posts: 1
Joined: 22 Nov 2018, 19:00

Re: Best plugin to start from for 2-way serial communication

#11 Post by hipek189 » 15 Oct 2019, 17:30

Hey great thing would be good if two sides could communicate with the mqtt controller

fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Re: Best plugin to start from for 2-way serial communication

#12 Post by fluppie » 17 Mar 2020, 23:48

I flashed an ESP32 with the latest firmware and saw that the Serial Proxy is included.
What values do init 1 and init 2 expect?

I have a sensor that's putting out a value 8-9 times a second on the serial using serial.print and serial.println
How can I get these values to MQTT or another controller, also 8-9 times a second, let's say an update everytime an updated value is sent.

Or should I do this with this technique: viewtopic.php?f=2&t=6443

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Best plugin to start from for 2-way serial communication

#13 Post by TD-er » 18 Mar 2020, 00:06

I'm updating that plugin to make it more usable.
Initially the init strings were meant to be sent at the call of PLUGIN_READ.
So you can call it using the taskrun command.

Some devices need some string to trigger some action (e.g. some sensors)

But in the new update there will be a command to trigger sending some data.
This makes it more flexible.

By the way, that plugin is highly experimental and was not my intention to have it included in the nightly builds for ESP32.

fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Re: Best plugin to start from for 2-way serial communication

#14 Post by fluppie » 18 Mar 2020, 00:14

Aha ok. I did compile myself using VS Code and used "custom_ESP32"
But it should be working? Do you have an example screenshot of a working configuration? I wanted to use SoftSerial on an Arduino Uno/Nano/Mini at 115200 baud to Serial1 or 2 of the ESP32.
Would this 8 to 9 Hz datarate work? Or will the plugin loose/skip some readings?

To test, I just made a empty counter sketch that sends:
1
2
3
4
5
6
and so on.

Is the plugin able to read that? And also what if values are decimal and/or negative?
Example:
45.8118
-46.4829
-43.5288
44.7851

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Best plugin to start from for 2-way serial communication

#15 Post by TD-er » 18 Mar 2020, 00:39

The plugin is the first to handle string type values.
So what it does is it will receive a string up-to a line end character and hand it over to the active controller(s).
For now I have not yet added the option to select a controller, as it is still highly experimental.

But it has been used in a test setup to capture strings up-to 600 bytes and send it via MQTT.
The max. receive rate depends on the length of the strings, amount of free memory, etc.

I've found that using software serial on ESP8266 (doesn't work on ESP32) at a high line rate may cause SW watchdog reboots.
For example processing GPS data, which outputs roughly 11 lines a second (at 9600 baud) may cause issues, but it will work just fine on HW serial.
Make sure the active controller settings can also handle the traffic so don't have too high values for minimum message interval. (e.g. 200 msec interval will limit the number of messages to 5 per second)
The serial port is polled in the 50/sec call, so it has proven to be able to keep up 600 byte strings at 38400 baud.


Also be aware that strings will take more resources, so limit the queue depth.

Like I said, the output is not a numerical value, but a string value.
This can be used in rules and when used as an argument for commands, the rules interpreter will try to convert it to a numerical value.
Just refer to the event parameter to access this string.


In the updated version you will be able to do some basic (Lua syntax) regexp for filtering the messages.


I am also working on an I2C to UART integration for ESPEasy, which does add a near unlimited number of UART ports to both ESP8266 as well as ESP32 and it will be integrated in the ESPEasySerial library so it does allow all plugins using UART to use that I2C to UART chip.
I have it working here on my desk and I hope to receive the assembled PCBs tomorrow so I can test with a number of ports simultaneously. :)

If it is working and stable enough, I will include it in the mega branch.

User avatar
ThomasB
Normal user
Posts: 1064
Joined: 17 Jun 2018, 20:41
Location: USA

Re: Best plugin to start from for 2-way serial communication

#16 Post by ThomasB » 18 Mar 2020, 02:17

I am also working on an I2C to UART integration for ESPEasy, which does add a near unlimited number of UART ports to both ESP8266 as well as ESP32 and it will be integrated in the ESPEasySerial library so it does allow all plugins using UART to use that I2C to UART chip.
Nice. My crystal ball says that a SC16IS75X may be in our future.
I've found that using software serial on ESP8266 (doesn't work on ESP32) at a high line rate may cause SW watchdog reboots.
And I've found that ESPEasy's SoftSerial is unreliable when receiving data. RxD data corruption occurs due to competing interrupts, which results in serial bit-decoding errors caused by latency. So providing a way to add more hardware serial ports is the way to go.

- Thomas

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Best plugin to start from for 2-way serial communication

#17 Post by TD-er » 18 Mar 2020, 10:27

ThomasB wrote: 18 Mar 2020, 02:17 [...]
Nice. My crystal ball says that a SC16IS75X may be in our future.
Your crystal ball may be correct & Pull Request :)
ThomasB wrote: 18 Mar 2020, 02:17 And I've found that ESPEasy's SoftSerial is unreliable when receiving data. RxD data corruption occurs due to competing interrupts, which results in serial bit-decoding errors caused by latency. So providing a way to add more hardware serial ports is the way to go.
Yep that's true.
For small messages which have a checksum, like interacting with a CO2 sensor, it is working fine.
I have the Serial Proxy also running on a node here using SW serial (capturing GPS data) and it is crashing quite often due to SW watchdog reboots.

Recently the SWserial library has had a number of fixes and changes, so I may try again to use that one as we are using a patched version of that library which is roughly 2 years old by now.
That one was more stable than the "official" library, when I tried using the "official" one about a year ago.

On the PCB's I hope to receive today, I can use GPS on Serial0 swapped and connected to that NXP chip the RN2483 LoRa and SenseAir S8 or MH-Z19 CO2 sensor + a slave board with 3 of those NXP chips for an additional 6 UART ports.


Edit:
Make sure to use the SC16IS752, as the -750 is not supported (yet).

fluppie
Normal user
Posts: 89
Joined: 09 Oct 2015, 13:23

Re: Best plugin to start from for 2-way serial communication

#18 Post by fluppie » 18 Mar 2020, 23:28

NIce, I have it working. Limited my string 12 chars, reduced baud from 115200 (which I think is to fast for softwareserial on an Arduino Nano with Atmega328) to 9600 baud.
If I'm right, the potential of 9600 baud = 9600 bits per second = 960 bytes per second (1 start bit+8 bits+1 stop bit). Right?

Sentences (pass/fail): 2054/1
Length Last Sentence: 12

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Best plugin to start from for 2-way serial communication

#19 Post by TD-er » 19 Mar 2020, 00:25

9600 baud is indeed quite a bit of data.
12 byte messages can be sent over 50 per second.
But ESPEasy will not be able to handle 50 messages/sec :)

The pass/fail is determined if the received chars have an ASCII code < 32 or > 127
CR/LF is still accepted. (marks end of a message)

Post Reply

Who is online

Users browsing this forum: No registered users and 47 guests