Page 4 of 12

Re: Nextion display plugin

Posted: 21 May 2018, 10:56
by BertB
waspie wrote: 21 May 2018, 01:26 I just got my first nextion and i'm already making some progress. I have to say, it's a bit confusing... someone should make a nice noob summary at some point (pointing at me too, don't worry :P )

This will be great with openhab / mqtt. I don't plan to use a slider yet, but i may at least test so I can help answer arnold's question.

I think I know the answer to this already but I'll ask anyway.

Is there any way to remotely update the Nextion? I'd love to get this pinned to a wall and halfway usable but i also need a way to update it. I know i'll be tinkering with the interface until i get it just right.
I didn't see much in the way of updating it remotely. I hate the thought of actually having to finalize some sort of design before getting it on the wall. that could take months :O

awesome plugin!


also, all the spec sheets i've looked at from the current ones show +3.3v max on the serial line. maybe older ones used 5v?
There is some kind of a summary halfway this topic. I agree, it deserves a proper one, but I simply have no time.
Remote update would be awsome, but I honestly have no clue how to do it.
I have my displays direcly connected to WeMos boards without any problem. If you interprete the description of the interface, you see it gives an output of max 3.2 volts, but accepts between 3.3 and 5 volt as a logic high at its input.

Re: Nextion display plugin

Posted: 21 May 2018, 14:18
by BertB
I started to create a WIKI in the devices list, but it is in its very preprepre stadium.
https://www.letscontrolit.com/wiki/inde ... IONDisplay

Re: Nextion display plugin

Posted: 23 May 2018, 02:23
by waspie
BertB wrote: 21 May 2018, 14:18 I started to create a WIKI in the devices list, but it is in its very preprepre stadium.
https://www.letscontrolit.com/wiki/inde ... IONDisplay
Nice! I may go in there and at least fix grammatical issues.

on another note, for you or hopefully anyone else that can answer...

How can I get imported text into the nextion?

Code: Select all

on foyerNextion#temp do
nextion,page0.t0.txt="%[foyerNextion#temp]%"
endon
doesn't work. neither does

Code: Select all

on foyerNextion#temp do
nextion,page0.t0.txt="[foyerNextion#temp]"
endon
or

Code: Select all

on foyerNextion#temp do
nextion,page0.t0.txt="%foyerNextion#temp%"
endon
I just get some version of this in the log:

Code: Select all

2009401 : IMPT : [foyerNextion#temp] : 69.13
2009402 : EVENT: foyerNextion#temp=69.13
2009450 : ACT  : nextion,page0.t0.txt="%%"
page0.t0.txt="%%"

***EDIT**** it's %eventvalue% duh. trying to remember all the stuff on the rules tutorial page is hard :P got it!

Re: Nextion display plugin

Posted: 23 May 2018, 11:11
by BertB
waspie wrote: 23 May 2018, 02:23
Nice! I may go in there and at least fix grammatical issues.
Great! :D

Re: Nextion display plugin

Posted: 29 May 2018, 02:32
by waspie
hamster wrote: 21 Mar 2018, 23:21 I have trying for days now to get my nextion working as thermostat control

I'm using the following on for the logic on the nextion for a up / increment button

Code: Select all

va0.val=va0.val+2
if(va0.val>9)
{
  va1.val=va1.val+1
  va0.val=va0.val-10
}
if(va1.val>25)
{
  va1.val=va3.val
  va0.val=va4.val
}
cov va0.val,t9.txt,1
cov va1.val,t10.txt,2
t4.txt=t10.txt+"."+t9.txt
for the increment down button

Code: Select all

va0.val=va0.val-2
if(va0.val<1)
{
  va1.val=va1.val-1
  va0.val=va0.val+9
}
if(va1.val<5)
{
  va1.val=va5.val
  va0.val=va6.val
}
cov va0.val,t9.txt,1
cov va1.val,t10.txt,2
t4.txt=t10.txt+"."+t9.txt
this then increments t4 as a decimal xx.x
I then use on touch press event on t4.txt

Code: Select all

print "set-temp"
print t4.txt
t4.pco=18242
on the touch release of t4.txt

Code: Select all

t4.pco=65535
This then allows me to use the up + down buttons to create a setpoint where the font is red, I then press the t3.txt this then changes the font white, and sends the selected value to the esp
on the log is see :

Code: Select all

send command: set-temp19.11
Does any one know how get this

Code: Select all

Nextion : send command: set-temp19.1116
to my mqtt broker
switches work fine by using the following rules

Code: Select all

on nextion2#code=3.00 do
sendtohttp 192.168.0.48,8080,/json.htm?param=switchlight&type=command&idx=91&switchcmd=Toggle
endon
but trying to send a value doesn't seem to be possible?
ever make any progress on this? I think I'm looking to do something similar. I may end up doing the counting in ESP rules but maybe better to do in Nextion? not sure the best route to go

and rather than pressing something to "enter" the number why not a timer that waits 5-10? seconds after last press to send the number? not sure if that's an option?

what about in the rules of ESP do something like

on ESPNXT#code=5 (say 5 is up and 6 is down)
if ( dummval#TMP <= 79 )
dummyval#TMP = dummyval#TMP+1
else
dummyval = 65
endif
timerSet,1,10
endon

on ESPNXT#code=6
if ( dummvalTMP >= 66 )
dummyval#TMP = dummyval#TMP+1
else
dummyval = 80
endif
timerSet,1,10
endon

on Rules#Timer=1 do
Publish /%sysname/dummyval#TMP
endon

this is all untested and off the top of my head. I'm not a rules expert so take this as a general IDEA of what you need to do. Would this not be better then? The rules portion of the ESP handles all the incrementing.
When no button has been pressed then after 10 seconds it publishes the value to MQTT. I think that's how it will work? TD-er?

then with an MQTT import you can set the display to show the setpoint AND update the dummyval so it all stays in sync.

Re: Nextion display plugin

Posted: 09 Jun 2018, 20:39
by waspie
is there a way to make a button with a cropped picture change based on a variable without refreshing the page?

say page0.b0 has picc 1 (on) and 2 (off)
and there's a global variable va0 used to storing that state for when pages are changed.

trying to keep espeasy, the button, and openhab (mqtt) involves some rules. many buttons means many rules and running out of rule space

I could cut down a lot of i could just set the variable to the nextion and make the nextion update the picture based on the variable instead of having to set the variable and the picture.

so say on some mqtt#import do
nextion,page0.va0.val=0

then the nextion sees that va0.val changed and changes the b0 picc = 2

is this possible (without sending a page refresh) ?

Re: Nextion display plugin

Posted: 10 Jun 2018, 10:29
by TD-er
If you need some extra commands to make the rules easier, just ask.

What I need then is:
- Command-name
- Parameters + description
- Description of the call to the Nextion, based on such parameters.

We can add commands to the Nextion plugin, which can be called from rules.

If the discussion about what commands are needed and how to format them is closing to something useful, we can add an issue on Github about it.

Re: Nextion display plugin

Posted: 10 Jun 2018, 14:11
by waspie
Ok, let me think about that. A lot of the stuff is pretty repetitive but I don't know how else to do it. I'll get back to you on Monday when I'm at work and have time to tinker

Re: Nextion display plugin

Posted: 14 Jun 2018, 21:55
by waspie
TD-er wrote: 10 Jun 2018, 10:29 If you need some extra commands to make the rules easier, just ask.

What I need then is:
- Command-name
- Parameters + description
- Description of the call to the Nextion, based on such parameters.

We can add commands to the Nextion plugin, which can be called from rules.

If the discussion about what commands are needed and how to format them is closing to something useful, we can add an issue on Github about it.
all right, so I haven't had a lot of time to dedicate to this yet. I've been busy at work and when i get home i'm tending to try and wrap up several of my HA things that are in progress. Replacing bedroom switches with nextions that control the light via relay and my vents via openhab. car presence. etc etc.

So one way i found to update the nextion display is to create a hotspot on the page, typically named m0 (this was suggested in their forums by one of the two main guys i think). in the m0 on the nextion IDE you put all your commands for that page similar to the page pre-init stuff.
then you "click" the m0 like:
nextion m0,click (IIRC, i'd have to look at my rules on one of the new units i've set up).

So a function like "Check this box to "click" m0 when a variable is sent to the nextion" hah no idea how you'd do that... that's why you get paid the big bucks :O

A rule looks like

on MQTT#import=1 do
nextion,page0.va1.val=1
TaskValueSet,2,1,1
nextion m0,click
endon

I do this stuff to keep the nextion,esp, and openhab all in sync.

I know you can do "nextion,page0" and it refreshes the page but for whatever reason "clicking" the m0 doesn't redraw the page so it looks much smoother.
Try both, you'll see what I mean.

so i suppose maybe the command could just be "nxtupdate" ? and then the parameter would be the hotspot name. m0, m1, m2 etc. that makes sense i think...right?

on whatever do
nextion, stuff
nxtupdate
endon

Re: Nextion display plugin

Posted: 14 Jun 2018, 23:51
by TD-er
waspie wrote: 14 Jun 2018, 21:55 [...]
So a function like "Check this box to "click" m0 when a variable is sent to the nextion" hah no idea how you'd do that... that's why you get paid the big bucks :O
[...]
Just to be clear, I don't get paid anything for ESPeasy.
It only costs me money to let the postmen (and -women) visit my front door very frequently with bags filled with all kind of stuff from Ali or Banggood. :)

I really have to free some time to get my Nextion display connected so I could play with it a bit.
Then I am sure I would get annoyed enough by missing features and add them. Until then you'll have to formulate your annoyances for me to convert into C++ code. :lol:

Nextion display plugin, random RxD data corruption

Posted: 17 Jun 2018, 22:17
by ThomasB
The ESP Easy's Nextion Plugin is ideal for my project. Thank you for creating it!

Issue:
My project is nearly done but I have one nagging issue to resolve. The problem is that the ESP is randomly corrupting the data packet message received from the Nextion display. The good news is that only RxD data is affected; The TxD data sent by Easy ESP to the display is always perfect.

Project Description:
I'm using a NX3324T028 display with Easy ESP installed on a LoLin NodeMCU V3. Easy ESP receives button press commands from the Nextion display and then communicates them via MQTT to my OpenHabian home automation system. It's a simple touch screen app for my wife to let her know when the washer and/or dryer machine have finished. When the laundry is ready (as soon as the machine stops) she gets a voice alert courtesy of the Amazon Echo Dots in our home.

RxD Corruption Description:
The issue is that sometimes the |s data packet payload that is received by the ESP Easy contains random characters. The corruption randomly occurs to either the "idx" or "value" data.

Here are some examples of the corruption.

ESP Log entry of good RxD from Nextion's print "|s,i98,sOff":

Code: Select all

  
  626580 : Nextion : send command: |s,i98,sOff
  98.00
  630852 : Nextion : send command: |s,i98,sOff
  98.00
ESP Log entry of corrupted RxD from Nextion's print "|s,i98,sOff":

Code: Select all

  628723 : Nextion : send command: |s,i98,sO¦–Ô98.00
  629787 : Nextion : send command: |s,i98–®e™98.00
  635212 : Nextion : send command: |s,i98,sO³–¨ø98.00
  637356 : Nextion : send command: |s,iN±sOff…Ã0.00


Troubleshooting Info:

* I've flashed the NodeMCU with a self-compiled build of "20102 - Mega" (PLUGIN_BUILD_TESTING) and I also tried the pre-compiled ESP_Easy_mega-20180615_dev_ESP8266_4096.bin. Both experience the problem.
* For development/troubleshooting I'm using the Nextion editor's Debug->User MCU Input emulation function rather than a real display.
* The serial baud rate is the default 9600. The 3.3V Serial Logic levels were verified by o-scope.
* All ESP Easy devices are disabled except the Nextion plug in.
* I have ensured that all communication between the Nextion and Easy ESP is only one |s data packet at a time with a minimum of 1 second interval between transmissions.
* I have tried using the |s and the equivalent |u packet formats, as follows:
Nextion data packet example using |s format

Code: Select all

  print "|s,i10,sOff"   // CMD: Turn Dryer Buzzer off
  printh 0a                // eol, Process Packet
Nextion data packet example using |u format

Code: Select all

  print "|u,i10,n,s0"   // CMD: Turn Dryer Buzzer off
  printh 0a                // eol, Process packet.
I've looked through the forum and cannot find any other reports of the issue. I would be grateful to receive some advice on solving it. No problem if more info is needed, just ask. :)

- Thomas

Edit: Flash with "20102 - Mega" (PLUGIN_BUILD_TESTING), same problem.

Re: Nextion display plugin

Posted: 18 Jun 2018, 04:52
by ThomasB
{Continued from previous post.}

Nextion Screenshots:

Re: Nextion display plugin, random RxD data corruption

Posted: 19 Jun 2018, 21:58
by ThomasB
After some debugging I found the explanation to my ESP's corrupted Rx data. The Nextion<->ESP communication uses the ESPeasySoftwareSerial plugin which is an emulated UART. This function will sometimes produce corrupted serial data due to the ESP's background interrupts. The issue is random and only affects Rx data (ESP's SoftSerial Tx data is fine).

Reliable Rx byte decoding is based on a nominal interrupt latency time; Unfortunately the actual latency will vary a lot due to other interrupts that are sometimes being processed while a new Rx byte is received. That is to say, if the ESP is currently processing an interrupt, and a RXD start bit arrives, the ESPeasySoftwareSerial::rxRead() data byte is corrupted due to late entry into the RxRead function (because bit sampling timing is skewed).

Typical latency into the RxRead function is ~10uS and fortunately this is what occurs most of the time. But sometimes the latency is much greater and breaks Rx byte decoding.

Below is a o-scope snapshot captured during debugging. It shows the RxRead function's timing while decoding a "|" character that had too much interrupt latency.
Too much latency on this data byte cause it to be a &quot;corrupted&quot; byte.
Too much latency on this data byte cause it to be a "corrupted" byte.
esp_latency_bad1.jpg (119.84 KiB) Viewed 139762 times
The top red trace is the actual incoming serial signal and the bottom yellow trace is driven by a ESP pin that was toggled at each Rx bit's sampling. Because of the interrupt latency on the Rx data byte the start of sampling occurred too late. So the data's 0x7c ("|") value was shifted and incorrectly decoded as a 0x7e. There's no way to know which byte might be corrupted since it's just a roll of the dice.

I suspect that there's not much that can be done to prevent the randomly corrupted Rx data. So as a workaround I will have the Nexion send each ESP message multiple times to help mask the problem (sloppy big hammer approach). Too bad the ESP8266 doesn't have two hardware serial ports!

- Thomas

Re: Nextion display plugin

Posted: 20 Jun 2018, 22:16
by TD-er
That's a very plausible explanation. (scope images always help improve credibility ;) )
Is the entire data packet skewed, or only the first byte mangled?
And does the Nextion provide some kind of checksum to check before using the data?

If the first byte is mangled and it is always the same byte, or at least a small subset, we could reconstruct the data using the checksum.

Re: Nextion display plugin

Posted: 21 Jun 2018, 04:33
by ThomasB
Is the entire data packet skewed, or only the first byte mangled?
The ESP can corrupt any softserial character in the received string. That is to say, you could have zero, one, or more corrupted characters in the message sentence string. Due to the background interrupts (they're the culprit) the chance of getting hit with bad Rx data is best described as a roll of the dice.
And does the Nextion provide some kind of checksum to check before using the data?
Currently no checksum is provided.

Rather than changing the ESP code I decided to apply a brute force workaround. It is based on my observations of the ESP and OpenHab log files.

The corruption has four symptoms / conditions:
  • 1. The |s or |u message strings can be partially corrupted and sometimes still provide the correct idx and value data.
  • 2. If the |s or |u message string is too garbled it will be ignored (no idx or value data). This is essentially a missing message.
  • 3. If the |s or |u preamble is good, but all numerical chars in the idx or value are corrupted, then they are set to zero.
  • 4. If one of the numerical chars in the idx and/or value are corrupted (non-numeric) then their value will be missing a digit. For example, 25 would be changed to 2 or 5 depending on the corrupted digit position.
So this information lead to a reliable solution that I tested today; 100% success. My app only sends a few button codes which keeps things simple. Here's what I do.
  • A. Send each message three times, with 100mS pacing delay between them. This cured the random missing message problem. Here's an example snippet of a Nextion Button Touch Press Event:

    Code: Select all

        print "|s,i12,sOff"  // CMD: Start Dryer Buzzer without voice ack
        printh 0a
        delay=100
        print "|s,i12,sOff"  // CMD: Start Dryer Buzzer without voice ack
        printh 0a
        delay=100
        print "|s,i12,sOff"  // CMD: Start Dryer Buzzer without voice ack
        printh 0a
    
  • B. During Rx I only care about the idx data; The value data is ignored. In the Nextion message I pad the value parameter with the "sOff" boolean.
  • C. Only use idx data that is between 10 and 99. This provides up to 90 button press states (but I only use 8 states). This allows filtering out the corrupted idx values that have a missing digit or were falsely set to zero.
In the ESP's NEXTION Device config page the Send to Controller checkbox is unchecked. The Value Name1 is idx, Value Name2 is value.

In the ESP rules I have code for each button press that forwards the "legal" key-presses to my OpenHab home controller. A key-press action rule looks like this:

Code: Select all

on NEXTION#idx=12 do  // Start Dryer Buzzer without voice ack.
Publish /%sysname%/NEXTION/idx,[NEXTION#idx]  // MQTT the button action to OpenHab
endon
Any idx data that does not have a rule is considered to be dead zombie data (corrupted). No action will happen with these zombie values because they don't have a action rule, so they are essentially filtered out.

- Thomas

Re: Nextion display plugin

Posted: 21 Jun 2018, 18:48
by waspie
mine does the same so i just never used the value portion. if this were to actually work i may be able to cut down on the size of my rules files :O

Re: Nextion display plugin

Posted: 21 Jun 2018, 18:50
by waspie
TD-er wrote: 14 Jun 2018, 23:51
Just to be clear, I don't get paid anything for ESPeasy.
[/quote]

i know english isn't your first language so i hope you know i was joking!

Re: Nextion display plugin

Posted: 24 Jun 2018, 22:38
by r_255
You could try ESP Pins 13 RDX2 and 15 TDX2, as these can be used for serial2 communication if i am right, thats how its declared in this mqtt firmware wich never missed a hit.

As i described earlier i had the same issues but then this firmware communicates fine but isnt espeasy, so using gpio13 and 15 might do the trick for the communication issues you guys experience.
https://github.com/maragelis/NextionMqtt

Re: Nextion display plugin

Posted: 25 Jun 2018, 00:38
by ThomasB
Thanks for the link to the NextionMqtt code. It also uses Soft Serial. But its single purpose (Nextion->MQTT) avoids the Serial Rx corruption because it does not have any overhead interrupts to process.

Simply changing ESP Easy to use Pins 13 RDX2 and 15 TDX2 won't solve the problem since it would continue to use soft serial. I suppose someone could rewrite ESP to boot on Serial1, then swap the Serial1/Serial2 pins to the Serial2 pair and use hardware UART instead of Soft Serial. This would mean the loss of the serial command processor and logging.

My workaround has been working well. But if I run into more problems with ESP Easy / Nextion then I will consider using the NextionMqtt sketch and add more code to handle the other things that I need it to do. That will be a cruel circumstance because the convenience of ESP Easy's plug-ins and rule processor are hard to beat!

- Thomas

Re: Nextion display plugin

Posted: 25 Jun 2018, 12:33
by BertB
For simple switching actions, I only use the Send Component ID in the Touch Release Event tab of a switch and let the Rules or Domoticz do the toggling.

In fact, I have a panel with a number of switches that only send the ID. In Rules I have a line like:

Code: Select all

on E16Nextion#code=774 do
    SendToHTTP 192.168.xxx.yyy,8080,/json.htm?type=command&param=switchlight&idx=387&switchcmd=Toggle
endon
774 is the ID of a switch on page 3 (3 x 256 + 7=774)
I send this to a toggle switch in Domoticz, that in turn changes the color of the switch in Nextion to green, when active and grey when off.

Code: Select all

On Action:http://192.168.xxx.yyy/control?cmd=NEXTION,page3.b5.bco=1024
Off Action:http://192.168.xxx.yyy/control?cmd=NEXTION,page3.b5.bco=48631
This works quite well, but it consumes lines in Rules.

Re: Nextion display plugin

Posted: 25 Jun 2018, 12:40
by r_255
Thanks for clearifying,

Well i guess i most cases loosing your serial doesnt hurt anything. Debug and commands can be send thru the webinterface.

I agree on that its a pitty we cant have it working as intended within espeasy, as 90% of my esps run on it and it was my startpoint.
It would be sweet to simply make a rule and set the dim screen level based on the esp witty its lux levels.

I got my nextion running on a esp witty and changed the gpio's to the hardware serial as you can detach the usb to serial part leaving you a hardware rx and tx exposed. I did not try that with espeasy, as the rule engine in espeasy did not have enough space for my remote controller.

In the end i did decide to use node red as man in the middle, as my setup is kind of dynamic and i dont want to change my interface that much, and if i have to its a mather of changing a few options in node red. instead of using the nextion software, upload it thru sd etc. This way works for me and keeps things dynamic.

So basicly you can make buttons with labels and a id nummer, fill the labels dynamic from node red and do a action based on a button id catch.

Re: Nextion display plugin

Posted: 26 Jun 2018, 20:01
by BertB
@ThomasB
I have an ESP8266 (WeMos) with an OLED, a switch, an MH-Z19CO2 device (serial interface) and a PMSx003 DUST sensor, also serial.
Both serial devices use SoftSerial and hardly ever miss data.

The Ser2Net device uses the hardware Rx and Tx. So, it shouldn't be too difficult to embed that option in the Nextion plugin.

I noticed that you have a switch in your device list. Is that to wake-up the OLED? You do not need to declare it seperately
Furthermore, I see two OLED displays. Dou you use both?

Re: Nextion display plugin

Posted: 26 Jun 2018, 20:42
by ThomasB
@BertB

* Just curious, is your Wemos using 160MHz CPU clock or the default 80MHz?

* Thanks for pointing out that ser2net is using hardware UART. Too bad the existing Nextion Plugin does not support the UART as an optional implementation. If I get desperate I may take on the challenge, but so far my workaround is fine.

* The OLED devices are disabled. The project started with an OLED, but their small text is a problem for my wife's eyes. So OLED is abandoned and has been replaced with Nextion.

* Presently there are two "switch" devices. One is assigned to a HV optoisolator for sensing the Washing machine is running, the other goes to vibration detector on the dryer. But I'm changing the vibration detector to MPU6050 Accelerometer (higher sensitivity to detect the dryer spin). Just waiting for the sensor to arrive from the China supplier.

- Thomas

Re: Nextion display plugin

Posted: 26 Jun 2018, 21:02
by BertB
@ThomasB
My WeMos runs on 80 MHz.

Perhaps TD-er can modify the plugin to use the hardware USART. I would give it a try myself, but for some reason, I cannot get useful compilations.

Re: Nextion display plugin

Posted: 26 Jun 2018, 21:29
by ThomasB
Using the hardware UART would be the holy grail. :)

Re: Nextion display plugin

Posted: 26 Jun 2018, 23:39
by TD-er
ThomasB wrote: 26 Jun 2018, 21:29 Using the hardware UART would be the holy grail. :)
And it isn't that hard to do.
Only problem is that you also have to make sure no other things make use of these pins.
So you have to disable serial logging and be able to disconnect the sensor when flashing when using uart0.
I know there are also pins designated to uart1, but I don't know if that's a hardware serial or not.

Re: Nextion display plugin

Posted: 26 Jun 2018, 23:57
by BertB
TD-er wrote: 26 Jun 2018, 23:39
ThomasB wrote: 26 Jun 2018, 21:29 Using the hardware UART would be the holy grail. :)
And it isn't that hard to do.
Only problem is that you also have to make sure no other things make use of these pins.
So you have to disable serial logging and be able to disconnect the sensor when flashing when using uart0.
I know there are also pins designated to uart1, but I don't know if that's a hardware serial or not.
From the ESP8266 datasheet:
Data transfers to/from UART interfaces can be implemented via hardware. The data
transmission speed via UART interfaces reaches 115200 x 40 (4.5
Mbps).
UART0 can be used for communication. It supports fluid control. Since UART1 features
only data transmit signal (Tx), it is usually used for printing log.

Re: Nextion display plugin

Posted: 27 Jun 2018, 00:14
by ThomasB
I agree, using hardware serial means disabling the debug and serial command processor. Then turn on the UART's Rx interrupt and point it to a Rx data buffer handler routine that Nextion can access. Seems easy, but "easy" coding is sometimes full of twists and turns.

There's only one full hardware UART in the ESP8266. But there's two sets of Rx & Tx pins. I understand that a pin swap register determines which pin pairs are routed to the UART. This pin swapping feature would be convenient for wiring the NEXTION display to the UART if someone enhances the Nextion plugin.

Edit1: "Since UART1 features only data transmit signal (Tx), it is usually used for printing log."
Too bad they skimped on the silicon. We need a Rx UART to fix this problem.

Edit2: Here's some useful information about UART2's Tx, along with the pin swapping features:
http://smallbits.marshall-tribe.net/blo ... quiet-uart

Not to be a broken record, but I have fixed my data corruption problem using the workaround explained earlier. But I would definitely switch to the hardware UART solution if someone does it without breaking the other functions provided by ESPEasy.

- Thomas

Re: Nextion display plugin

Posted: 27 Jun 2018, 10:10
by BertB
As far as I know, debug info stays available via the web interface.

Re: Nextion display plugin

Posted: 27 Jun 2018, 16:11
by BertB

Re: Nextion display plugin

Posted: 27 Jun 2018, 18:32
by ThomasB
The ESP8266 UART features certainly have a lot of flexibility. But it's frustrating to find that the second UART is crippled (Tx only). Sure there are workarounds to this shortcoming, but two full UARTS would have been a more desirable implementation.

- Thomas

Re: Nextion display plugin

Posted: 27 Jun 2018, 20:14
by BertB
Well, after doing some diging ... , a simple Serial.swap(); is enough to swap Rx and Tx to DPIO13 and GPIO15 respectively.
Serial

Serial object works much the same way as on a regular Arduino. Apart from hardware FIFO (128 bytes for TX and RX) HardwareSerial has additional 256-byte TX and RX buffers. Both transmit and receive is interrupt-driven. Write and read functions only block the sketch execution when the respective FIFO/buffers are full/empty.

Serial uses UART0, which is mapped to pins GPIO1 (TX) and GPIO3 (RX). Serial may be remapped to GPIO15 (TX) and GPIO13 (RX) by calling Serial.swap() after Serial.begin. Calling swap again maps UART0 back to GPIO1 and GPIO3.

Serial1 uses UART1, TX pin is GPIO2. UART1 can not be used to receive data because normally it's RX pin is occupied for flash chip connection. To use Serial1, call Serial1.begin(baudrate).

If Serial1 is not used and Serial is not swapped - TX for UART0 can be mapped to GPIO2 instead by calling Serial.set_tx(2) after Serial.begin or directly with Serial.begin(baud, config, mode, 2).

By default the diagnostic output from WiFi libraries is disabled when you call Serial.begin. To enable debug output again, call Serial.setDebugOutput(true). To redirect debug output to Serial1 instead, call Serial1.setDebugOutput(true).

You also need to use Serial.setDebugOutput(true) to enable output from printf() function.

Both Serial and Serial1 objects support 5, 6, 7, 8 data bits, odd (O), even (E), and no (N) parity, and 1 or 2 stop bits. To set the desired mode, call Serial.begin(baudrate, SERIAL_8N1), Serial.begin(baudrate, SERIAL_6E2), etc.
Source: http://esp8266.github.io/Arduino/versio ... rence.html

And this is what you see via the normal serial port during boot:
⸮U47865 :


INIT : Booting version: (custom) (ESP82xx Core 2_4_1, NONOS SDK 2.2.1(cfd48f3), LWIP: 2.0.3)
47865 : INIT : Warm boot #24 - Restart Reason: External System
47870 : FS : Mounting...
47895 : FS : Mount successful, used 76053 bytes of 957314
47909 : CRC : No program memory checksum found. Check output of crc2.py
47940 : CRC : SecuritySettings CRC ...OK
And this can still be seen on the web log:
948350: EVENT: E8DHT#Temperature=24.60
948356: EVENT: E8DHT#Humidity=52.90
948382: Domoticz: Sensortype: 2 idx: 312 values: 24.60;52.90;1
948446: WD : Uptime 15 ConnectFailures 0 FreeMem 16216
978446: WD : Uptime 16 ConnectFailures 0 FreeMem 16240
985324: EVENT: Clock#Time=Wed,20:26
1008446: WD : Uptime 16 ConnectFailures 0 FreeMem 15936
1008500: DHT : Temperature: 24.60
1008500: DHT : Humidity: 52.60
1008502: EVENT: E8DHT#Temperature=24.60
1008508: EVENT: E8DHT#Humidity=52.60
1008536: Domoticz: Sensortype: 2 idx: 312 values: 24.60;52.60;1
1019950: SaveToFile: config.dat index: 0 datasize: 1220
1020012: FILE : Saved config.dat
1038446: WD : Uptime 17 ConnectFailures 0 FreeMem 15864
1044476: EVENT: Clock#Time=Wed,20:27
1044482: EVENT: Clock#Time=Wed,20:27 Processing time:6 milliSeconds

Re: Nextion display plugin

Posted: 27 Jun 2018, 20:49
by ThomasB
Hopefully you continue with your efforts and successfully get Nextion communication moved over to the hardware UART.

- Thomas

Re: Nextion display plugin

Posted: 28 Jun 2018, 14:51
by BertB
I spent a number of hours on experimenting, reading and searching but.
There is an issue with Serial.swap() as it redirects TX to GPIO15.
At bootup of the ESP8266, GPIO has to be low and the Nextion pulls it high. :?

Re: Nextion display plugin

Posted: 28 Jun 2018, 16:39
by ThomasB
I wish I could help with the UART code, but at this point all I can do is cheer you on.
- Thomas

Re: Nextion display plugin

Posted: 28 Jun 2018, 23:27
by grovkillen
You could always try to lower the "message delay" found under Advanced settings (MQTT). Default is set to 1000mSec but I tend to always lower this to "100". But setting it to "0" will also work, could you please just try to set it to zero (0)? We are experimenting with this value to see what it actually does to the system and we suspect it to be making more harm then good (as in we need to fix whatever it is for in a better way).

Re: Nextion display plugin

Posted: 28 Jun 2018, 23:56
by ThomasB
@grovkillen, I'm not sure if you are directing your comments to me. But to offer some feedback, I've already experimented with the MQTT interval. Currently MQTT it is 75mS. I had tried 0mS and other values, none helped solve my problem.

That was before I discovered that the problem was due to the interrupt latency. The issue became clear when I got out the o-scope and analyzed the RxD bit stream.

To summarize, the data corruption isn't with MQTT communication. Instead, it is due to the limitations of SoftSerial and its interrupt latency sensitive "bit-bang" data reads. I have posted a scope waveform that shows an example of too much latency and its impact on the affected serial data byte.

- Thomas

Re: Nextion display plugin

Posted: 29 Jun 2018, 00:00
by grovkillen
Thanks Thomas, yes I was talking to you :)

And thanks for the feedback on the test of values, just to inform you and whoever reads this, the message delay is used by more things than just MQTT. We are experimenting with it right now to see where it impacts the performance.

Re: Nextion display plugin

Posted: 29 Jun 2018, 00:01
by TD-er
We (very) recently discovered this Message Delay setting to be stalling the normal loop execution.
This also means the "50/sec" and "10/sec" loop calls may not be services as often als the names suggest.

So it was just a suggestion to look into this.
If it is indeed interrupt related (or better, lack of interrupt), then it is indeed something that cannot be fixed by adjusting this delay.
Glad you already did test that.

Re: Nextion display plugin

Posted: 29 Jun 2018, 00:14
by ThomasB
If the MQTT routines employ any service interrupts (or disable global interrupts) that take too much time then it would definitely impact software serial.

What I have observed is that typical latency into the SoftSerial read function is about 10uS, which is great. At 9600baud, I expect it would still work reliably with as much as 80uS. But randomly something is blocking the interrupt for a very long time (>135uS).

- Thomas

Re: Nextion display plugin

Posted: 29 Jun 2018, 00:20
by TD-er
I just found out (as in this evening) that the Message Delay setting is causing the sendData function to stall for about this time.
That function is called from the once-per-second loop and the message delay is being used even when no MQTT is being used.

So I will definitely change how this affects performance.

Also the wifi has to be served every now and then and that may take some time.
You cannot call the yield function from within the interrupt handler in the Software Serial routine. But you could do a call to yield when parsing the data.
If it is handled more often, the time needed for handling the wifi connection is often less per call.

Re: Nextion display plugin

Posted: 29 Jun 2018, 00:45
by ThomasB
I just found out (as in this evening) that the Message Delay setting is causing the sendData function to stall for about this time.
That is interesting to hear.
You cannot call the yield function from within the interrupt handler in the Software Serial routine. But you could do a call to yield when parsing the data.
Once inside the SoftSerial Rx interrupt handler any other interrupts shouldn't cause a problem because they would be disabled at that point. That is to say, as long as the initial entry latency is acceptable there should be no other threats to RxD parsing the data character.

- Thomas

Re: Nextion display plugin

Posted: 29 Jun 2018, 15:03
by BertB
@ThomasB

I would like to do some experimenting myself.
So far I can see, the below is the rx interrupt service routine. Correct?

Code: Select all

void ICACHE_RAM_ATTR ESPeasySoftwareSerial::rxRead() {
   // Advance the starting point for the samples but compensate for the
   // initial delay which occurs before the interrupt is delivered
   unsigned long wait = m_bitTime + m_bitTime/3 - 500;
   unsigned long start = ESP.getCycleCount();
   uint8_t rec = 0;
   for (uint8_t i = 0; i < 8; i++) {
     WAIT;
     rec >>= 1;
     if (digitalRead(m_rxPin))
       rec |= 0x80;
   }
   if (m_invert) rec = ~rec;
   // Stop bit
   WAIT;
   // Store the received value in the buffer unless we have an overflow
   uint16_t next = (m_inPos+1) % m_buffSize;
   if (next != m_inPos) {
      m_buffer[m_inPos] = rec;
      m_inPos = next;
   }
   // Must clear this bit in the interrupt register,
   // it gets set even when interrupts are disabled
   GPIO_REG_WRITE(GPIO_STATUS_W1TC_ADDRESS, 1 << m_rxPin);
}
Where did you put your toggle bit and where do you see all intterupts are blocked, once in the rx interrupt routing?

Re: Nextion display plugin

Posted: 29 Jun 2018, 16:26
by ThomasB
Here's the debug code:

Code: Select all

void ICACHE_RAM_ATTR ESPeasySoftwareSerial::rxRead() {
   // Advance the starting point for the samples but compensate for the
   // initial delay which occurs before the interrupt is delivered

   pinMode(5, OUTPUT); // DEBUG
   digitalWrite(5, HIGH);  // DEBUG
   
   unsigned long wait  = m_bitTime + m_bitTime/3 - 500;
//   unsigned long wait  = m_bitTime + m_bitTime/3 - 3000;  // Test
   unsigned long start = ESP.getCycleCount();
   uint8_t rec = 0;
   
   
   for (uint8_t i = 0; i < 8; i++) {
     WAIT;
     digitalWrite(5, LOW);  // DEBUG
     rec >>= 1;
     if (digitalRead(m_rxPin))
       rec |= 0x80;
       delayMicroseconds(1);   // DEBUG
       digitalWrite(5, HIGH);  // DEBUG
   }
   if (m_invert) rec = ~rec;
   // Stop bit
   WAIT;
   digitalWrite(5, LOW);  // DEBUG
   
   // Store the received value in the buffer unless we have an overflow
   uint16_t next = (m_inPos+1) % m_buffSize;
   if (next != m_inPos) {
      m_buffer[m_inPos] = rec;
      m_inPos = next;
   }
   // Must clear this bit in the interrupt register,
   // it gets set even when interrupts are disabled
   GPIO_REG_WRITE(GPIO_STATUS_W1TC_ADDRESS, 1 << m_rxPin);
}
You'll need either a 2-channel digital o-scope or logic analyzer. Sync on RxD and compare RxD to the toggled DEBUG bit. There will be a DEBUG bit event at entry (Latency measurement) into the function and at each bit reading (Data Alignment measurement).

Using the bit toggle you can mentally reconstruct the incoming data byte and determine if it is corrupt. You have to study each character of an entire string sentence message. And be patient, a corruption event is random when it affects a character in the 0Ah terminated string.

- Thomas


- Thomas

Re: Nextion display plugin

Posted: 29 Jun 2018, 17:02
by waspie
so there's no "canned" way to do the workaround?
ThomasB wrote: 29 Jun 2018, 16:26 Here's the debug code:

Code: Select all

void ICACHE_RAM_ATTR ESPeasySoftwareSerial::rxRead() {
   // Advance the starting point for the samples but compensate for the
   // initial delay which occurs before the interrupt is delivered

   pinMode(5, OUTPUT); // DEBUG
   digitalWrite(5, HIGH);  // DEBUG
   
   unsigned long wait  = m_bitTime + m_bitTime/3 - 500;
//   unsigned long wait  = m_bitTime + m_bitTime/3 - 3000;  // Test
   unsigned long start = ESP.getCycleCount();
   uint8_t rec = 0;
   
   
   for (uint8_t i = 0; i < 8; i++) {
     WAIT;
     digitalWrite(5, LOW);  // DEBUG
     rec >>= 1;
     if (digitalRead(m_rxPin))
       rec |= 0x80;
       delayMicroseconds(1);   // DEBUG
       digitalWrite(5, HIGH);  // DEBUG
   }
   if (m_invert) rec = ~rec;
   // Stop bit
   WAIT;
   digitalWrite(5, LOW);  // DEBUG
   
   // Store the received value in the buffer unless we have an overflow
   uint16_t next = (m_inPos+1) % m_buffSize;
   if (next != m_inPos) {
      m_buffer[m_inPos] = rec;
      m_inPos = next;
   }
   // Must clear this bit in the interrupt register,
   // it gets set even when interrupts are disabled
   GPIO_REG_WRITE(GPIO_STATUS_W1TC_ADDRESS, 1 << m_rxPin);
}
You'll need either a 2-channel digital o-scope or logic analyzer. Sync on RxD and compare RxD to the toggled DEBUG bit. There will be a DEBUG bit event at entry (Latency measurement) into the function and at each bit reading (Data Alignment measurement).

Using the bit toggle you can mentally reconstruct the incoming data byte and determine if it is corrupt. You have to study each character of an entire string sentence message. And be patient, a corruption event is random when it affects a character in the 0Ah terminated string.

- Thomas


- Thomas

Re: Nextion display plugin

Posted: 29 Jun 2018, 17:13
by ThomasB
My workaround strategy was posted earlier. It involved sending each Nextion->ESP message string (|s format) multiple times and filtering the data in the ESP Rules by having MQTT only send valid idx codes.

- Thomas

Re: Nextion display plugin

Posted: 29 Jun 2018, 18:43
by BertB
Below, a recording with a logic analyser, connected to a WeMos with a PMS5003 Dust detector attached.
It sends bursts of 32 bytes of data.
As you can see, somstimes a complete byte is not at all detected.
missing frame.PNG
missing frame.PNG (35.67 KiB) Viewed 146362 times

Re: Nextion display plugin

Posted: 29 Jun 2018, 19:09
by ThomasB
Good stuff. My interpretation is that the second RxD character byte (ref CH0) has excessive latency on entry into the SoftSerial function. The initial debug bit (ref CH1, about +4ms marker time) demonstrates that the parsing sampling is delayed (because it appears too late in the incoming RxD's bit stream).

This causes the entire RxD char's bit order to be shifted after parsing. It will be a Nextion->ESP data character byte that is corrupted.

Edit: Also, to make it easier to measure the entry latency for each received character you should include the digitalWrite(5, LOW) statement after the stop bit's WAIT. See my posted code above.

- Thomas

Re: Nextion display plugin

Posted: 29 Jun 2018, 23:01
by BertB
Well, the way I see it, ESP and serialsoft do not work reliably together, no matter how often we measure. Is am going to focus on good old hardware.

Re: Nextion display plugin

Posted: 29 Jun 2018, 23:10
by ThomasB
My workaround coding has been a good band-aid for my Nextion project. It's been running for several days without any issues. But I agree with you and would not recommend the Nextion plugin due to the SoftSerial issue.

- Thomas