ESPeasy & PlatformIO debugging question (other POVs welcome!)

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
falcongoat
New user
Posts: 4
Joined: 10 Jun 2020, 10:59

ESPeasy & PlatformIO debugging question (other POVs welcome!)

#1 Post by falcongoat » 10 Jun 2020, 13:29

Hi everyone,

this is going to be a somewhat lengthy post, because I try to give you all the information neccessary while describing the project it is embedded in.
What I thought to be a simple project got a bit out of hand now :) So I am looking for anyone interested in helping me wrapping my head around developing for ESPEasy.

Also, if I am missing some obvious solution that doesn't involve hacking away at the code that I didn't think of, I would love to hear from you!

My main problem related to the topic is as follows:
Currently I try to debug my fork of P141 & P142 to make them directly adressable or react to TaskValueSetAndRun in a predictable way.
I tried to locate every resource available for developers, but I could only find bits and pieces across different websites and GitHub repositories. Some questions that are still open for me: What functions can be passed to a plugin? What is the structure of common objects like event?
So i decided to debug my way through the code and look at the variable stacks to see what I could use for my logic.

What works:
  • Breakpoint works (only 1 hardware breakpoint possible)
  • Watch expression works (only 1 hardware watchpoint possible)
  • Temporary hardware assisted breakpoints working
What doesn't work:
  • Step over nearly always causes an exeption
  • Variable values are <optimized away> quite often
  • Variable values are diplayed as <anonymous union> containing some blob (?) data
  • Often the variable value is <error: cannot access memory at address 0x...>
Image

What I think the problem is:
I cannot compile with compiler optimization turned off (-O0). I have to use -O1 or -Og, leading to the results you see in the screenshot.
When I try to compile with -O0, I get a compiler error like this:

Code: Select all

-DDEBC:\Users\xxx\.platformio\packages\framework-arduinoespressif8266@src-a516163d55c87c23296626d8ca889c7e\tools\sdk\libc\xtensa-lx106-elf\include/sys/pgmspace.h:107:1: error: a15 cannot be used in asm here
UG -DBUILD_DEBUG -DDEBUG_ESP_PORT=Serial "-I.pio\libdeps\deb }
ug_ ^
ESP8*** [.pio\build\debug_ESP8285_1M\libfef\Adafruit GFX Library_ID13\Adafruit_GFX.cpp.o] Error 1
I found a reddit post describing this problem. The solution was to downgrade in the board manager from esp8266 v2.7.1 to 2.5.2. It seemed to have helped others, but didn't work for me.
Since then I tried a variety of board, package and tree version combinations, but to no avail, so I suspect Adafruit GFX to be the culprit.
Here are my compiler flags, maybe I haved missed something:

Code: Select all

    -c 
-fno-rtti 
-std=c++11 
-mtarget-align 
-mlongcalls 
-mtext-section-literals 
-falign-functions=4 
-U__STRICT_ANSI__ 
-ffunction-sections 
-fdata-sections -fno-exceptions 
-Wall 
-Og 
-fvar-tracking 
-fvar-tracking-assignments 
-g3 
-ggdb3 
-DUSE_CUSTOM_H 
-DPLATFORMIO=40304 
-DESP8266 
-DARDUINO_ARCH_ESP8266 
-DARDUINO_ESP8266_WEMOS_D1MINILITE 
-DBUILD_GIT=\"\" 
-DVTABLES_IN_FLASH 
-DPIO_FRAMEWORK_ARDUINO_LWIP2_HIGHER_BANDWIDTH_LOW_FLASH 
-DPUYA_SUPPORT=1 
-DCORE_POST_2_5_0 
-DBEARSSL_SSL_BASIC 
-DCORE_POST_2_6_0 
-DPIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK22x_190703 
-DSIZE_1M -DMQTT_MAX_PACKET_SIZE=1024 
-DHTTPCLIENT_1_1_COMPATIBLE=0 
-DESP8285 -DF_CPU=80000000L 
-D__ets__ 
-DICACHE_FLASH 
-DARDUINO=10805 
-DARDUINO_BOARD=\"PLATFORMIO_D1_MINI_LITE\" 
-DFLASHMODE_DOUT 
-DLWIP_OPEN_SRC 
-DNONOSDK22x_190703=1 
-DTCP_MSS=1460 
-DLWIP_FEATURES=0 
-DLWIP_IPV6=0 
-D__PLATFORMIO_BUILD_DEBUG__ 
-DDEBUG 
-DBUILD_DEBUG 
-DDEBUG_ESP_PORT=Serial
Why am I doing this to me? (aka The Project) (:

I want to build a dimmer for 2 led strip panels, controlling 2 sides of a light box. I want to use a home automation platform for this and plan to do more projects after I finished this. Right now I'm using OpenHAB, but the decision is not final. I want to do this with one D1 mini lite. There my trouble began, because I still don't know how to address these devices (in ESPEasy terms) directly. So here's what I did:

Steps of the project:
  1. build a controller for 2 LED strips to be controlled by OpenHAB (works, directly issuing PWM,<GPIO>,<n> commands control led brightness)
  2. learn that I can't use the same plugin for 2 devices, because the first one to receive incoming commands executes and there is no way right now to address a device directly
  3. learn that 2 different plugins don't help either, because they use the same commands
  4. Dug into the Controller Code to learn that you can make it queue a PLUGIN_WRITE if the last segment of your MQTT topic is a number (it ignores the payload and takes the last part of the topic as value, the remainder is issued as command. Same problem as #2)
  5. learn that Rules triggered on MQTTimport don't work either, because TaskValueSetAndRun triggers PLUGIN_DEVICE_READ instead of PLUGIN_DEVICE_WRITE (learned that later). So the values get updated, but not the device/plugin/task.
My Setup:
  • Windows 10
  • VSCode 1.45.1 (recent version)
  • PlatformIO (Core 4.3.4, Home 3.2.2)
  • D1 mini lite
  • ESPEasy mega-20200608
I really think I missed a lot of information you guys need, but I am happy to answer any question.

Hoping to understand ESPeasy better :)

Peace!

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESPeasy & PlatformIO debugging question (other POVs welcome!)

#2 Post by TD-er » 10 Jun 2020, 14:06

Hello and welcome.

I think you're trying to approach the problem from a rather difficult perspective.
Sure there are hardware debugging options, but those hardly work (as you've seen)
So what I do is the old fashioned type of debugging, which is outputting strings to the log.

Forget the debugging you may have known from other platforms.
You cannot pause the code on an ESP node reliable when hitting a break point and also variable inspection + proper stack trace is not available.


TaskValueSetAndRun is actually a rather simple command, which does only 2 things:
- Set a value in the UserVar array at the right position.
- Call PLUGIN_READ from the task.


I guess it would be best if you post here a snippet (or the whole code) of what you run into and then I can help you out.
I think it is best to only describe the things you need right now, or else you would be overwhelmed with knowledge you cannot organize right now since you don't know the context where to put it.
Apart from the fact it would take me a lot of time as I don't know where the lack of knowledge is for you :)

falcongoat
New user
Posts: 4
Joined: 10 Jun 2020, 10:59

Re: ESPeasy & PlatformIO debugging question (other POVs welcome!)

#3 Post by falcongoat » 10 Jun 2020, 15:49

Thanks for your quick feedback!

Knowing me I almost probably chose the most convoluted way to solve my problem :)
When I want to know how stuff works I like to tear it apart, look under the hood while it's running and poke it with a stick...I even read the docs most of the time :D

But fun aside, is there any way I can make a plugin check if it is the target of a command? Right now I see no solution for this.

When the MQTT Controller receives a topic ending in <nested/topic/here>/cmd it just uses the payload and makes a PluginCall(PLUGIN_WRITE, &TempEvent, cmd) with cmd=payload.

When I use the "number workaround" (<nested/topic/here>/Val/96 with payload 64) it builds a TempEventObject with both Values (Par1=96, Par2=64) and cmd=Val. Only the first parameter is used then for the cmd. Since the rest of the topic is not passed on I have no way to let the plugin check by name if it is the intended target.
Are there any methods to realize this?
And can you point me to some documentation regarding the structure of the event object?

Regarding PLUGIN_READ: Since this is made to read incoming sensor data I was hesitant to put the same functionality as in PLUGIN_WRITE there. But that would enable me to force update the plugin.

On the topic of debugging: I would like to get back to that when I could solve the problem at hand, but I'm quit interested in knowing what the difficulties are.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESPeasy & PlatformIO debugging question (other POVs welcome!)

#4 Post by TD-er » 10 Jun 2020, 19:50

About the event struct: https://github.com/letscontrolit/ESPEas ... ntStruct.h

The basic event flow in ESPEasy is quit simple:

A plugin describes how to interface with hardware.

A task is an instance of a plugin, with some settings.
So multiple tasks can use the same plugin. (important!)

A task will output values (up-to 4)

A call to PLUGIN_READ:
- will return true when there are new values.
- Will generate an event per new value, to be used in rules.
- Calls connected controllers on new values.

A controller receives new data from a task and hands them over in a pre-formatted form to some place else.
For example:
- Send to Domoticz/OpenHAB/Thingspeak
- Send to generic HTTP server
- Store on the flash
- Send to TTN via LoRaWAN


Commands can be given from a lot of origins, but mainly from within the rules.
There is a number of internal commands, which are not specific to tasks or controllers. See: https://espeasy.readthedocs.io/en/lates ... mmand.html

On top of that, there are commands specific to plugins (N.B. see the difference between plugin and a task).
When a command is not an internal command, it will be tried to send to a task. For this the PLUGIN_WRITE call is made.
The first task that accepts the command will execute it.
Some (not all) commands also have a parameter to indicate for what task it is meant to be used.
For example taskRun.

What taskRun does, is it will (re)schedule a call to PLUGIN_READ to be executed as soon as possible.

Some plugins (e.g. BME280) do try to read the sensor in a number of very short calls (to make sure reading the sensor is not blocking for other tasks to run).
These short calls can be made from PLUGIN_FIFTY_PER_SECOND, or similar periodical calls.
As soon as the BME280 (as an example, there are others too) has completed to read the sensor, it will reschedule a call to PLUGIN_READ.
See: https://github.com/letscontrolit/ESPEas ... #L287-L296

falcongoat
New user
Posts: 4
Joined: 10 Jun 2020, 10:59

Re: ESPeasy & PlatformIO debugging question (other POVs welcome!)

#5 Post by falcongoat » 11 Jun 2020, 00:19

Thanks, that helped clear up some conceptual questions for me!
And thanks again for pointing me to EventStruct.h, could have thought of that myself :oops:
The structure isn't too complicated...BTW, what is the function of TaskIndex? Since OriginTaskIndex holds the caller's index, TaskIndex holds that of the callee (target)?
The first task that accepts the command will execute it.
That was the beginning of my search for a way to adress tasks explicitly via MQTT. Raw commands are consumed by the first possible task.
Then I found TaskValueSetAndRun which almost did what I wanted. Now I understand that it updates the values for the tasks via PLUGIN_READ, but the task is responsible for handling any incoming command calls (the plugin object, of which the task is an instance, right?). Thats why the values on the Devices Tab updated without triggering any further action.

So there is nothing in the layout of the call architecture that strongly advises against putting commands in PLUGIN_READ? Then that would be the place where I start implementing a lot of the functions as in PLUGIN_WRITE.

The periodical calls you mentioned were also used for color fades in P142, one of the two LED strip plugins I want to cook into one.

I am going to read some more code now :)

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESPeasy & PlatformIO debugging question (other POVs welcome!)

#6 Post by TD-er » 11 Jun 2020, 09:51

what is the function of TaskIndex? Since OriginTaskIndex holds the caller's index, TaskIndex holds that of the callee (target)?
TaskIndex is used mainly in events.
It is the task index (0...11) to which the event is addressed, or the task index of sender of the event.

Well, sending and receiving of an event is maybe a bit too much credit here, as we loop over all tasks or controllers (or notifications, but that's hardly used) for a specific call (e.g. PLUGIN_READ) and hand over the event.
Whoever needs to do something with it, will read from the event or change values in it.
The event is more of a structured container handed over to whom may use it.
So there is nothing in the layout of the call architecture that strongly advises against putting commands in PLUGIN_READ? Then that would be the place where I start implementing a lot of the functions as in PLUGIN_WRITE.
Well I advice you to see how the commands flow through the code.
Then you'll see commands are processed by calling PLUGIN_WRITE.
So if you try to handle commands in PLUGIN_READ, they will not be processed.

You have to realize that the actions in a single call (PLUGIN_xxx) are self contained.
Meaning you take actions like reading a sensor value, set the values in an array or perform actions like in the PLUGIN_TEN_PER_SECOND call.
Sometimes you need to keep track of things, like keeping track of a state while reading a sensor.
Some plugins declare global variables for this, but it has the drawback that you can't have multiple instances of the same plugin without them influencing each other.


One example of a plugin to keep track of data for 2 instances, see: https://github.com/letscontrolit/ESPEas ... 0.ino#L198

For a more generic approach, see how the structure for plugin data is handled here: https://github.com/letscontrolit/ESPEas ... PS.ino#L68
Search for PluginTaskData_base in the code.

falcongoat
New user
Posts: 4
Joined: 10 Jun 2020, 10:59

Re: ESPeasy & PlatformIO debugging question (other POVs welcome!)

#7 Post by falcongoat » 11 Jun 2020, 14:25

Thank you for taking the time to give all these insights to me. I had a hard time unpacking ESPEasyDevelopment wiki, but it all begins to fall into place now. Especially sifting through the src/ made a lot clearer.

But I also realized that I have to brush up on some Arduino concepts again, because I could have answered some of my questions myself by taking a step back and looking at the whole picture. Instead i was in microscope mode.

After reading the developer page again I realized that I'm asking for something that is explicitely stated not to be in the focus of development. Sorry for that.

One question regarding the example in the wiki for PLUGIN_WRITE:
The function getTaskIndexByName() is nowhere to be found in the source and using it leads to a compler error. Is it deprecated/removed?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESPeasy & PlatformIO debugging question (other POVs welcome!)

#8 Post by TD-er » 11 Jun 2020, 19:59

I guess that part of the wiki may be a bit outdated.

The task name is defined here: https://github.com/letscontrolit/ESPEas ... ruct.h#L34
Keep in mind that the ExtraTaskSettings must be loaded from the settings file every time you try to access it.
So on each access you must double check the TaskIndex is the intended one.

We don't have the getTaskIndexByName function anymore, as it is renamed to findTaskIndexByName

This returns the first task index matching that name, or an invalid task index if not found.
Remember it is not guaranteed the task name is unique.
There is also a function to get the task name based on the task index.
See getTaskDeviceName

We do have a check for it to report a warning when you try to save settings with a name that already exists.
See: https://github.com/letscontrolit/ESPEas ... #L948-L980

The reason we like to warn about it is because otherwise referring to [taskname#varname] in the rules may result ambiguous results.

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 20 guests