ESP-32 plans for ESPEasy (?)

Moderators: grovkillen, Stuntteam, TD-er

Message
Author
Martinus

ESP-32 plans for ESPEasy (?)

#1 Post by Martinus » 10 Feb 2018, 20:57

Just upgraded my production ESPEasy units to Arduino Core 2.4.0 with the latest github source and all still seems to work without issues. So now i must be protected against the KRACK issue...

To round things up, i also tried to upgrade the dev ESP32 modules to the latest source, but they failed to compile. Tested some older releases and found out that the Release mega-20180112 is actually the last one that still works on ESP32 (Wemos ESP32 Lite). Looks like MD5 and timezone stuff has broken compatibility with ESP-32.

Could be nice to have something like travis checks also on the Arduino ESP32 platform.

But are we planning future support for ESPEasy on ESP32? I think it could still going to work in 2.1.0 but maybe the uPyEasy development makes all obsolete?

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#2 Post by TD-er » 10 Feb 2018, 21:55

Hmm, I was not even aware the Mega branch could run on an ESP32.
The timezone stuff is not something that may break support for other platforms.
Not sure about the MD5, since that may patch some parts of the code I guess.

Maybe have a grab on ESP32 defines in the code? Some are in the webserver part, where also a lot has changed for the 2.4 Arduino code

Edit:
Just looked at the commits since:
preparations for lwip2 (#707)
The include of ESPTimeTypes.h could be positioned a bit earlier in ESPeasy.ino. Some users mentioned compile issues when setting the define for VCC.

Martinus

Re: ESP-32 plans for ESPEasy (?)

#3 Post by Martinus » 11 Feb 2018, 12:43

When i move the time struct definitions to ESPEasy.ino and exclude the MD5.h, things are running again on ESP32 with the latest github release dated 20180209
Might be an idea to remove the ESPEasyTimeTypes.h and put it's contents into the ESPEasy.ino file.

Why do we have *.h files in the project anyway? I don't think it's common practice within Arduino is it? Shouldn't things like "Dialog_Plain_12_font.h" be moved to the actual library to where it belongs? Same for OLED***.h files?

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#4 Post by TD-er » 11 Feb 2018, 13:51

Problem with Arduino .ino files is you cannot use structs as a parameter or return value when these structs are not forward declared.
You can 'prototype' or forward declare them in the main project file, or in a .h file which you then have to include ofcourse.

I would like to have things separated in files with a filename describing the functionality or some other type of grouping.
Then it is predictable what is stored where, so people can find already existing functions instead of writing them again (which happened a lot in ESPeasy).
That's also the reason I put the forward declarations for the time related functions into a .h file. Just to be able to find things again.

The reason I put the font inside the src dir is because it is a font used by ESPeasy, not included by the OLED library. But I agree, it may be better in there, since it is only to be used by that library.

If you prefer not to have .h files, I guess the ESPeasy.ino file should get the role of forward declarations and the 'member variables' for ESPeasy.
However there should be a bit more structure in the core of ESPeasy, since there is a lot of code duplication which will lead to strange bugs.

My plan was also to add support for writing all stored structs to JSON and read them again from JSON.
That will make it possible to add more of a REST interface which will make the project more scalable to use. And also upgrading settings to a newer version and easier sharing of settings. Perhaps even a central 'backup' of settings and deploy of settings to multiple nodes.
Also the web interface could then be separated from the project. This will allow a shared interface for uPyEasy and ESPeasy.
Using a single (e.g. REST-like) interface for interaction will also lead to fewer bugs (single path of interface) and perhaps a smaller codebase.
And generating the web page in C++ like how it is done now, uses a lot of heap resources and is very hard to maintain.

But I guess having all the structs in ESPeasy will add too much clutter, when also adding import/export serializers.
So my plan was to split those into separate files, which will require forward declarations either in ESPEasy.ino or separate .h files.

That's about the ideas I have for ESPeasy. (in short: remove bugs, add options + support for devices and make it future proof)
What do you think about them?
And if there is an option to support the ESP32, I think we should add that. I simply wasn't aware the ESP32 was already possible to use for ESPeasy.
Maybe we even should add some support to allow to build smaller versions to allow the 512k ESP's to be able to use ESPeasy again. (just only use the things you'll need)

Martinus

Re: ESP-32 plans for ESPEasy (?)

#5 Post by Martinus » 11 Feb 2018, 15:13

TD-er wrote:
11 Feb 2018, 13:51
Problem with Arduino .ino files is you cannot use structs as a parameter or return value when these structs are not forward declared.
You can 'prototype' or forward declare them in the main project file, or in a .h file which you then have to include ofcourse.

If you prefer not to have .h files, I guess the ESPeasy.ino file should get the role of forward declarations and the 'member variables' for ESPeasy.
However there should be a bit more structure in the core of ESPeasy, since there is a lot of code duplication which will lead to strange bugs.
During testing, i first decided to skip the "ESPEasyTimeTypes.h", moved the structs to ESPEasy.ino and deleted the forward declarations and it compiled without issues on Arduino. So it seems possible to do without a header file. But anyway, i figured out the main cause for all the trouble: The header file was not included in the ESP32 branch because it was #ifdef for ESP8266 only. So at the end, i only moved the include outside the #ifdef condition.
Along with a few small other changes, the result is committed back to github so it's friendly towards ESP32 again...
TD-er wrote:
11 Feb 2018, 13:51
My plan was also to add support for writing all stored structs to JSON and read them again from JSON.
That will make it possible to add more of a REST interface which will make the project more scalable to use. And also upgrading settings to a newer version and easier sharing of settings. Perhaps even a central 'backup' of settings and deploy of settings to multiple nodes.
Also the web interface could then be separated from the project. This will allow a shared interface for uPyEasy and ESPeasy.
Using a single (e.g. REST-like) interface for interaction will also lead to fewer bugs (single path of interface) and perhaps a smaller codebase.
And generating the web page in C++ like how it is done now, uses a lot of heap resources and is very hard to maintain.
Sounds like a large impact...
Does "separate the web interface from the project" mean that the ESP will no longer have it's build-in webinterface? It that's the case, we would disappoint users that use the ESP as a standalone unit (including myself)
TD-er wrote:
11 Feb 2018, 13:51
That's about the ideas I have for ESPeasy. (in short: remove bugs, add options + support for devices and make it future proof)
What do you think about them?
"the remove bugs" sounds really great if more people could focus on that. We haven't had a new stable version for a looooong time...
TD-er wrote:
11 Feb 2018, 13:51
And if there is an option to support the ESP32, I think we should add that. I simply wasn't aware the ESP32 was already possible to use for ESPeasy.
Maybe we even should add some support to allow to build smaller versions to allow the 512k ESP's to be able to use ESPeasy again. (just only use the things you'll need)
About ESP32:
- The code runs on ESP32 in one RTOS thread. Not perfect, but it runs with only minor changes to the code.
- Not all modules compile without errors, there's still work to do on the IR and serial modules.
- UDP works but makes the system reboot quite often. Looks like a concurrency issue on RTOS. I have it turned of now.

About 512k:
- I have tried to keep everything compatible with the classis 512k modules for a long time, but as the SDK and Core keeps growing and SPIFFS was really needed, i decided to drop support for 512k and went into the "Mega" edition. Be prepared for a huge amount of conditional code to even try...
- Even a bare setup with one controller and the switch plugin compiles to ~450k. Adding 64k Spiffs and it will not fit the module.

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#6 Post by TD-er » 11 Feb 2018, 15:43

I saw your ESP32 commit to the Mega branch and to keep files a bit in sync, I was just testing a commit for the v2.0 branch to avoid future merge conflicts :)
Does "separate the web interface from the project" mean that the ESP will no longer have it's build-in webinterface? It that's the case, we would disappoint users that use the ESP as a standalone unit (including myself)
Nope that's absolutely not what I intend to do.
It is really nice to have ESPeasy as a stand-alone unit.
But I think doing it all in C++ is not easy to maintain.

In general, my idea is to somehow separate the data and the layout of the page.
That is not yet started so all options are still open.
One of the suggestions is to have the page layout (say the static parts) in SPIFFS and let the browser collect all information from the ESP.
That would allow for a lot faster interaction, less overhead in the ESP and even auto update of new measured values in the browser.

So rest assured, the main idea is to make development/maintenance easier and add features. Not remove them :)

User avatar
vader
Normal user
Posts: 241
Joined: 21 Mar 2017, 17:35

Re: ESP-32 plans for ESPEasy (?)

#7 Post by vader » 11 Feb 2018, 17:09

@Martinus: There is a bug in the current Git sources of core 2.4.0. Enter e.g. 'if %systime%' in the ESPeasy rules field and save it. It should be cut to 'if' !! It's a parser problem of the web server....

Martinus

Re: ESP-32 plans for ESPEasy (?)

#8 Post by Martinus » 11 Feb 2018, 19:12

vader wrote:
11 Feb 2018, 17:09
@Martinus: There is a bug in the current Git sources of core 2.4.0. Enter e.g. 'if %systime%' in the ESPeasy rules field and save it. It should be cut to 'if' !! It's a parser problem of the web server....
So i'm lucky not using %systime%... (this is about ESP8266 core right?)
Back to ESP32:
- Updated Arduino ESP32 core to github commit from 2017-12-19 and the UDP issues seems to be fixed.
- Do not use the later commits that use a newer SDK as it contains a bug in SPIFFS lib, making the ESP crashloop forever.
- Currently two ESP32 units are running latest ESPEasy source with Arduino ESP32 core 20171219. Both talking to Domoticz using HTTP protocol.
- Let's see how long they can run without a reboot...

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#9 Post by TD-er » 11 Feb 2018, 20:01

Experienced any issues with the webinterface causing chunked errors (mainly in Chrome)?

Martinus

Re: ESP-32 plans for ESPEasy (?)

#10 Post by Martinus » 11 Feb 2018, 20:13

Web gui seems fine, using chrome.

User avatar
budman1758
Normal user
Posts: 234
Joined: 15 Apr 2017, 05:13
Location: Riverside CA USA

Re: ESP-32 plans for ESPEasy (?)

#11 Post by budman1758 » 11 Feb 2018, 23:37

Martinus wrote:
11 Feb 2018, 19:12
Back to ESP32:
- Updated Arduino ESP32 core to github commit from 2017-12-19 and the UDP issues seems to be fixed.
- Do not use the later commits that use a newer SDK as it contains a bug in SPIFFS lib, making the ESP crashloop forever.
- Currently two ESP32 units are running latest ESPEasy source with Arduino ESP32 core 20171219. Both talking to Domoticz using HTTP protocol.
- Let's see how long they can run without a reboot...
Can you make a bin file available for downloading? I have an ESP32 running the version from this link.
http://www.letscontrolit.com/downloads/ ... R20000.zip
Would like to update it and see how it does also.
"The glass is twice as big as it needs to be".

Martinus

Re: ESP-32 plans for ESPEasy (?)

#12 Post by Martinus » 12 Feb 2018, 08:34


psy0rz
Normal user
Posts: 232
Joined: 02 Feb 2017, 12:12

Re: ESP-32 plans for ESPEasy (?)

#13 Post by psy0rz » 20 Feb 2018, 02:23

Martinus,

Would it be an idea if i configured our build system to also automaticly create ESP-32 nightly's? Is it just a matter of changing the board type?

Also did you get my slack invite, for https://espeasy.slack.com .

Edwin
Please support ESPEasy development via Patreon or buy us a coffee.

User avatar
budman1758
Normal user
Posts: 234
Joined: 15 Apr 2017, 05:13
Location: Riverside CA USA

Re: ESP-32 plans for ESPEasy (?)

#14 Post by budman1758 » 20 Feb 2018, 02:35

psy0rz wrote:
20 Feb 2018, 02:23
Would it be an idea if i configured our build system to also automaticly create ESP-32 nightly's? Is it just a matter of changing the board type?

+1
"The glass is twice as big as it needs to be".

Martinus

Re: ESP-32 plans for ESPEasy (?)

#15 Post by Martinus » 20 Feb 2018, 17:34

psy0rz wrote:
20 Feb 2018, 02:23
Would it be an idea if i configured our build system to also automaticly create ESP-32 nightly's? Is it just a matter of changing the board type?
That would be great, but I don't think it's ready for automated builds because it needs some patching before you can compile it without errors. Current approach to get it compiled is documented here:
https://www.letscontrolit.com/wiki/inde ... ng_Started

psy0rz
Normal user
Posts: 232
Joined: 02 Feb 2017, 12:12

Re: ESP-32 plans for ESPEasy (?)

#16 Post by psy0rz » 22 Feb 2018, 19:19

Ah i didnt know there was some handywork involved. :)

Still i might script those changes so that we at least have an automated build to test up testing and development.

We also have an ugly PUYA workaround/patch in the system currently. :)
Please support ESPEasy development via Patreon or buy us a coffee.

Martinus

Re: ESP-32 plans for ESPEasy (?)

#17 Post by Martinus » 01 Mar 2018, 15:04

The Hardware Serial plugins P020 and P044 are now working in ESPEasy on ESP32. Compiled using the github source from 2018-03-01.
A new binary is available for download on the Wiki.

The amount of free RAM is overwhelming:
ESPEasy32.png
ESPEasy32.png (38.77 KiB) Viewed 6313 times

AndrewJ
Normal user
Posts: 229
Joined: 14 Feb 2017, 12:38

Re: ESP-32 plans for ESPEasy (?)

#18 Post by AndrewJ » 01 Mar 2018, 19:27

Martinus,
Thanks for this, good work, I'll definitely give it a try! :)
AndrewJ

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#19 Post by TD-er » 01 Mar 2018, 21:55

At first glance I was like "hmm that's worse than on the ESP8266's"
And then I started counting digits ;)

By the way, how much effort does it take to build it for the ESP32?
When can we integrate it in the daily builds of ESPeasy? (so I can see when I break something again for the ESP32 ;) )

VoBo
Normal user
Posts: 3
Joined: 02 Mar 2018, 17:26

Re: ESP-32 plans for ESPEasy (?)

#20 Post by VoBo » 02 Mar 2018, 18:06

Hi everybody

I´m playing around with R20100 on my ESP32 board.
Installing was easy and everything what I need, is already working :)
"One" of the advantages of the ESP32 ist the increased amount of GPIO Ports; but the amount of devices in R20100 ist only 12.
Is it possible to increase also the amounts of devices ? I think that 24 or 32 should be enough. As far as I know, there are only 12 device because of the limits at an 8266.
The ESP32 should have enough ressources for more devices ?
Is it possible to change this ? And, if yes, can somebody compile a version with more devices ? (for me, that is too deep in arduino)

greetings from germany
Volker

Martinus

Re: ESP-32 plans for ESPEasy (?)

#21 Post by Martinus » 04 Mar 2018, 17:49

VoBo wrote:
02 Mar 2018, 18:06
Hi everybody

I´m playing around with R20100 on my ESP32 board.
Installing was easy and everything what I need, is already working :)
"One" of the advantages of the ESP32 ist the increased amount of GPIO Ports; but the amount of devices in R20100 ist only 12.
Is it possible to increase also the amounts of devices ? I think that 24 or 32 should be enough. As far as I know, there are only 12 device because of the limits at an 8266.
The ESP32 should have enough ressources for more devices ?
Is it possible to change this ? And, if yes, can somebody compile a version with more devices ? (for me, that is too deep in arduino)

greetings from germany
Volker
It's done. Beware that you will experience a full reset because this version has a different memory layout for the config.dat file.

Now we have a theoretical max of 32 tasks to be configured. It doesn't imply that performance will be sufficient to actually run 32 tasks as this has never been tried before. I can imagine that when you run a large number of tasks that use the 10PERSECOND polling scheme, the system could slow down to a hog...

But hey, this route is highly experimental anyway... :mrgreen:

User avatar
soif
Normal user
Posts: 39
Joined: 20 Feb 2018, 19:11
Location: Solar System
Contact:

Re: ESP-32 plans for ESPEasy (?)

#22 Post by soif » 05 Mar 2018, 09:28

Happy to see that the ESP32 dev is strongly on its way : Thanks for that 8-)
My plan was also to add support for writing all stored structs to JSON and read them again from JSON.
That will make it possible to add more of a REST interface which will make the project more scalable to use. And also upgrading settings to a newer version and easier sharing of settings
That would be really GREAT !

BTW would you please consider to split the device configuration + hardware config from the whole config file:
viewtopic.php?f=6&t=4965

Perhaps even a central 'backup' of settings and deploy of settings to multiple nodes.
My EspBuddy (see signature) already do a whole backup in one command :mrgreen: , but splitting files as explained in my linked post above, would pave the way for great centralized features, ie modify global settings on all nodes at once, or duplicate a node into another in a matter of seconds

Also the web interface could then be separated from the project. Using a single (e.g. REST-like) interface for interaction will also lead to fewer bugs (single path of interface) and perhaps a smaller codebase.
And generating the web page in C++ like how it is done now, uses a lot of heap resources and is very hard to maintain.
+1000, we might then integrate bootstrap, to get beautiful, mobile friendly web UI :geek:
Soif
-----
Want to update ALL your ESP babies OverTheAir, or backup their settings in ONE simple command, + many other cool features.... Adopt EspBuddy

VoBo
Normal user
Posts: 3
Joined: 02 Mar 2018, 17:26

Re: ESP-32 plans for ESPEasy (?)

#23 Post by VoBo » 05 Mar 2018, 23:00

Hi,
Martinus wrote:
04 Mar 2018, 17:49
It's done. Beware that you will experience a full reset because this version has a different memory layout for the config.dat file.

Now we have a theoretical max of 32 tasks to be configured. It doesn't imply that performance will be sufficient to actually run 32 tasks as this has never been tried before. I can imagine that when you run a large number of tasks that use the 10PERSECOND polling scheme, the system could slow down to a hog...

But hey, this route is highly experimental anyway... :mrgreen:
thank you very much :)
But the 32 Tasks seem to be a too much :mrgreen:
I´ve connected 8 Input switches and 8 Relais, which are switched by rules (like "sonoff´s")
If I press the first switch, the relay switches immediately (because it´s the first one in the "rules")
if I press switch Nr.8, which is the last one in the rules, it takes nearly 2 seconds until the relay ist activated :mrgreen: :mrgreen:
... also the "wroom" get´s quiet hot

greetings,
Volker

Martinus

Re: ESP-32 plans for ESPEasy (?)

#24 Post by Martinus » 12 Mar 2018, 20:21

VoBo wrote:
05 Mar 2018, 23:00
Hi,
Martinus wrote:
04 Mar 2018, 17:49
It's done. Beware that you will experience a full reset because this version has a different memory layout for the config.dat file.

Now we have a theoretical max of 32 tasks to be configured. It doesn't imply that performance will be sufficient to actually run 32 tasks as this has never been tried before. I can imagine that when you run a large number of tasks that use the 10PERSECOND polling scheme, the system could slow down to a hog...

But hey, this route is highly experimental anyway... :mrgreen:
thank you very much :)
But the 32 Tasks seem to be a too much :mrgreen:
I´ve connected 8 Input switches and 8 Relais, which are switched by rules (like "sonoff´s")
If I press the first switch, the relay switches immediately (because it´s the first one in the "rules")
if I press switch Nr.8, which is the last one in the rules, it takes nearly 2 seconds until the relay ist activated :mrgreen: :mrgreen:
... also the "wroom" get´s quiet hot

greetings,
Volker
Could you check the processing time on the rules? To see if this matched the 2 seconds delay.

Martinus

Re: ESP-32 plans for ESPEasy (?)

#25 Post by Martinus » 12 Mar 2018, 20:23

ESP32 binary has been updated to the latest github dev version (2018-03-12). Using the latest libraries.
https://www.letscontrolit.com/wiki/index.php/ESPEasy32
C010 is fixed.
core_version.h no longer needed for ESP32.

VoBo
Normal user
Posts: 3
Joined: 02 Mar 2018, 17:26

Re: ESP-32 plans for ESPEasy (?)

#26 Post by VoBo » 17 Mar 2018, 21:22

Martinus wrote:
12 Mar 2018, 20:21
Could you check the processing time on the rules? To see if this matched the 2 seconds delay.
how can I do that ?

Martinus

Re: ESP-32 plans for ESPEasy (?)

#27 Post by Martinus » 30 Mar 2018, 16:38

ESP32 binary has been updated to the latest github dev version (2018-03-30)
https://www.letscontrolit.com/wiki/index.php/ESPEasy32

User avatar
budman1758
Normal user
Posts: 234
Joined: 15 Apr 2017, 05:13
Location: Riverside CA USA

Re: ESP-32 plans for ESPEasy (?)

#28 Post by budman1758 » 30 Mar 2018, 20:33

Martinus wrote:
30 Mar 2018, 16:38
ESP32 binary has been updated to the latest github dev version (2018-03-30)
https://www.letscontrolit.com/wiki/index.php/ESPEasy32
Thanks for keeping this updated. I really hope to see this continue to be developed. I think ESPEasy is a fantastic firmware for the ESP32.
I am working on a fairly complex project for it and so far the only real bug is the OLED issue reported here.
https://github.com/letscontrolit/ESPEasy/issues/1029
"The glass is twice as big as it needs to be".

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#29 Post by TD-er » 31 Mar 2018, 01:53

It started with some very strange build error after a very normal and sound commit of Martinus.
It took me almost all evening and now it is fixed... and there is a build entry for ESP32 firmware in the PlatformIO.ini file.

Now there are 2 kinds of developers; those who start building and compiling and those who read the commit first :)

Martinus

Re: ESP-32 plans for ESPEasy (?)

#30 Post by Martinus » 31 Mar 2018, 14:49

TD-er wrote:
31 Mar 2018, 01:53
It started with some very strange build error after a very normal and sound commit of Martinus.
It took me almost all evening and now it is fixed... and there is a build entry for ESP32 firmware in the PlatformIO.ini file.

Now there are 2 kinds of developers; those who start building and compiling and those who read the commit first :)
There are also "developers" with poor skills and that only use Arduino IDE :mrgreen:
Maybe i should really consider to retire from the project entirely.

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#31 Post by TD-er » 31 Mar 2018, 15:30

Martinus wrote:
31 Mar 2018, 14:49
TD-er wrote:
31 Mar 2018, 01:53
It started with some very strange build error after a very normal and sound commit of Martinus.
It took me almost all evening and now it is fixed... and there is a build entry for ESP32 firmware in the PlatformIO.ini file.

Now there are 2 kinds of developers; those who start building and compiling and those who read the commit first :)
There are also "developers" with poor skills and that only use Arduino IDE :mrgreen:
Maybe i should really consider to retire from the project entirely.
Please don't stop working on the project.
My remark was just a wink at the comment I placed near the PlatformIO entry.

Code: Select all

### ESP32 pre-alpha test build ###########################################################
# Very limited build for ESP32, to start testing regular building for ESP32.             #
# Will probably not work, not tested and guaranteed to take a few hours time of some     #
# still trying to build the version without reading this warning.                        #
##########################################################################################
[env:esp32dev]
And it is just to make sure I also see the build errors I may cause when changing stuff in ESP8266-land.
It is just that we have had a few 'build' issues last few weeks where I was not able to see what's going on and was thus unable to help fixing stuff in ESP32 versions.
Let's hope I did not change stuff to cause build errors again in Arduino IDE using ESP32, because I did not test that yet.
At the moment we have about 4 build environments:
- PlatformIO with ESP8266 and ESP32
- Arduino IDE with ESP8266 and ESP32.

And as long as we don't test all of them regularly, there is a guarantee there will be new broken builds on one of them.

So my attempt to incorporate the ESP32 build on PlatformIO is just to make sure less build errors will occur.

I was hoping you could help with the SPIFF settings needed for the ESP32.

Martinus

Re: ESP-32 plans for ESPEasy (?)

#32 Post by Martinus » 31 Mar 2018, 15:46

TD-er wrote:
31 Mar 2018, 15:30
I was hoping you could help with the SPIFF settings needed for the ESP32.
We don't select this anymore within the Arduino IDE. The ESP32 seems to use some sort of partition table that seems to default to:

Code: Select all

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x140000,
app1,     app,  ota_1,   0x150000,0x140000,
eeprom,   data, 0x99,    0x290000,0x1000,
spiffs,   data, spiffs,  0x291000,0x16F000,
file: \hardware\espressif\esp32\tools\partitions\defaults.csv

Martinus

Re: ESP-32 plans for ESPEasy (?)

#33 Post by Martinus » 31 Mar 2018, 15:57

budman1758 wrote:
30 Mar 2018, 20:33
Thanks for keeping this updated. I really hope to see this continue to be developed. I think ESPEasy is a fantastic firmware for the ESP32.
I am working on a fairly complex project for it and so far the only real bug is the OLED issue reported here.
https://github.com/letscontrolit/ESPEasy/issues/1029
I think we need another can of devs to get the ESP32 development on the road, because there's a lot of work to be done here.
I merely tried to get the code through the compiler and even that can be a struggle sometimes. (the Arduino core developer teams on ESP8266 and ESP32 are not really in 'sync' at the moment)

I can't spend as much time as back in the 'old days' and my testlab has also been reduced:
Testlabs.png
Testlabs.png (295.07 KiB) Viewed 8082 times
Just enough to verify some core features of ESPEasy32.

LisaM
Normal user
Posts: 513
Joined: 11 Apr 2017, 18:29

Re: ESP-32 plans for ESPEasy (?)

#34 Post by LisaM » 31 Mar 2018, 17:33

TD-er wrote:
31 Mar 2018, 15:30
At the moment we have about 4 build environments:
- PlatformIO with ESP8266 and ESP32
- Arduino IDE with ESP8266 and ESP32.
Four? Pfff, i only have one... :lol:
Even the one is to much, i hope to make it zero. Using standard micropython firmware on a ESP32-WROVER with 4MB PsRam, i can just ftp my source code over and run that. With 3MB of flash drive there's plenty of room left, so there's no need for optimization anymore. To me it would be ideal to be just an app on micropython and let the other micropython groups deal with building the firmware, that would also expand the number of people working on uPyEasy significantly (by outsourcing a lot of firmware build&burn issues).

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#35 Post by TD-er » 31 Mar 2018, 19:26

LisaM wrote:
31 Mar 2018, 17:33
TD-er wrote:
31 Mar 2018, 15:30
At the moment we have about 4 build environments:
- PlatformIO with ESP8266 and ESP32
- Arduino IDE with ESP8266 and ESP32.
Four? Pfff, i only have one... :lol:
Even the one is to much, i hope to make it zero. Using standard micropython firmware on a ESP32-WROVER with 4MB PsRam, i can just ftp my source code over and run that. With 3MB of flash drive there's plenty of room left, so there's no need for optimization anymore. To me it would be ideal to be just an app on micropython and let the other micropython groups deal with building the firmware, that would also expand the number of people working on uPyEasy significantly (by outsourcing a lot of firmware build&burn issues).
I agree the Python way of doing things is the way to the future.
But until then, it is nice to see what can be done with really limited resources.
And I don't know how well you can assign tasks in Python to the 2nd core?
I think that's a great new feature, to allow to run some tasks really realtime and all other tasks on the other core.
Quite often you see feature requests here that need some realtime aspect, which the current ESPeasy can't do.

LisaM
Normal user
Posts: 513
Joined: 11 Apr 2017, 18:29

Re: ESP-32 plans for ESPEasy (?)

#36 Post by LisaM » 31 Mar 2018, 21:10

TD-er wrote:
31 Mar 2018, 19:26
LisaM wrote:
31 Mar 2018, 17:33
TD-er wrote:
31 Mar 2018, 15:30
At the moment we have about 4 build environments:
- PlatformIO with ESP8266 and ESP32
- Arduino IDE with ESP8266 and ESP32.
Four? Pfff, i only have one... :lol:
Even the one is to much, i hope to make it zero. Using standard micropython firmware on a ESP32-WROVER with 4MB PsRam, i can just ftp my source code over and run that. With 3MB of flash drive there's plenty of room left, so there's no need for optimization anymore. To me it would be ideal to be just an app on micropython and let the other micropython groups deal with building the firmware, that would also expand the number of people working on uPyEasy significantly (by outsourcing a lot of firmware build&burn issues).
I agree the Python way of doing things is the way to the future.
But until then, it is nice to see what can be done with really limited resources.
And I don't know how well you can assign tasks in Python to the 2nd core?
I think that's a great new feature, to allow to run some tasks really realtime and all other tasks on the other core.
Quite often you see feature requests here that need some realtime aspect, which the current ESPeasy can't do.
MicroPython task is running on the ESP32 App core, everything else on the first core. The current uPyEasy for ESP32 firmware is however using a single core approach only (so far).
Realtime requires good hardware interrupts and that's problem with the ESP32 (and ESP8266). The ESP's have software interrupts, while the STM32's have hardware interrupts. The result is that the GPIO latency with STM32's is only 1/10 of the ESP's. The ESP's will never be a good realtime platform, for that you'll need to go the STM32's (like anybody else using realtime).

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#37 Post by TD-er » 01 Apr 2018, 00:44

OK, that's good to know about the hardware ints, or actually the lack of.

About the ESPeasy + ESP32 combo.
I just tested an ESP32 module with the PlatformIO build I made yesterday and it just works :)
So expect some more debugging on that platform too. (Already found the first bug...)

Martinus

Re: ESP-32 plans for ESPEasy (?)

#38 Post by Martinus » 01 Apr 2018, 13:40

TD-er wrote:
31 Mar 2018, 19:26
I agree the Python way of doing things is the way to the future.
But until then, it is nice to see what can be done with really limited resources.
And I don't know how well you can assign tasks in Python to the 2nd core?
I think that's a great new feature, to allow to run some tasks really realtime and all other tasks on the other core.
Quite often you see feature requests here that need some realtime aspect, which the current ESPeasy can't do.
We actually have two different discussions here:

1) Latency on ISR handling
==========================
AFAIK, the ESP8266 and ESP32 do have hardware interrupts, but for proper handling and low latency, it is required that the response handling fits entirely in iRAM. I guess that this requirement prohibits higher level languages like microPython from handling this properly.
As explained at the end of this topic
https://forum.micropython.org/viewtopic ... 4&start=30
But using lean and mean C-language ISR handlers within ESPEasy could still do some nice jobs.

2) Realtime operating system
=========================
I think this is really the issue that Gijs is talking about. ESPEasy could really benefit from having an RTOS that will give the impression of a real multitasking system. As the SDK behind the ESP32 Arduino core is based on freeRTOS, i think that it would be possible to schedule more RTOS tasks within ESPEasy running on the ESP32.

Right now, the main.cpp only schedules a single task to run the entire user code like this:

Code: Select all

xTaskCreatePinnedToCore(loopTask, "loopTask", 8192, NULL, 1, NULL, ARDUINO_RUNNING_CORE);
Maybe it's possible to schedule more tasks for specific tasks/plugins?

Martinus

Re: ESP-32 plans for ESPEasy (?)

#39 Post by Martinus » 01 Apr 2018, 13:43

TD-er wrote:
01 Apr 2018, 00:44
OK, that's good to know about the hardware ints, or actually the lack of.

About the ESPeasy + ESP32 combo.
I just tested an ESP32 module with the PlatformIO build I made yesterday and it just works :)
So expect some more debugging on that platform too. (Already found the first bug...)
Good news! Does that mean that we will have nightly builds also for ESP32?

chrille
Normal user
Posts: 88
Joined: 26 Aug 2015, 15:11
Location: Horsens, Denmark

Re: ESP-32 plans for ESPEasy (?)

#40 Post by chrille » 01 Apr 2018, 20:11

TD-er wrote:
01 Apr 2018, 00:44
About the ESPeasy + ESP32 combo.
I just tested an ESP32 module with the PlatformIO build I made yesterday and it just works :)
So expect some more debugging on that platform too. (Already found the first bug...)
Did anyone succeed running on other ESP32 boards than the LoLin32 lite? I have two different ESP32 boards (the basic Lolin ESP32 and the Goouuu ESP32). I have no success with any of the two boards. I tried both the version from https://www.letscontrolit.com/wiki/index.php/ESPEasy32 and building the latest version from github with platform-io

Code: Select all

INIT : Booting version: (custom)
INIT : Cold Boot
FS   : Mounting...
CRC  : No program memory checksum found. Check output of crc2.py
CRC  : Settings CRC           ...OK
CRC  : SecuritySettings CRC   ...OK
INIT : Free RAM:168836
INIT : I2C
INIT : SPI not enabled
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0010,len:4
load:0x3fff0014,len:812
load:0x40078000,len:0
load:0x40078000,len:10164
entry 0x400789f8
U
After the U it just boots again

It seems that the first boot where the initial config is written is OK, but I haven't managed to start a terminal program fast enough after esptool finishes, to see the initial boot message

- Jan
Jan Chrillesen, Denmark

chrille
Normal user
Posts: 88
Joined: 26 Aug 2015, 15:11
Location: Horsens, Denmark

Re: ESP-32 plans for ESPEasy (?)

#41 Post by chrille » 01 Apr 2018, 22:55

chrille wrote:
01 Apr 2018, 20:11
Did anyone succeed running on other ESP32 boards than the LoLin32 lite? I have two different ESP32 boards (the basic Lolin ESP32 and the Goouuu ESP32). I have no success with any of the two boards. I tried both the version from https://www.letscontrolit.com/wiki/index.php/ESPEasy32 and building the latest version from github with platform-io
Me bad! It turned out to be a bad USB cable. As soon as the wifi stack was initialized the module rebooted. I observed the exact same behavior with micropython on the same module, which got me on track
After replacing the USB cable it just works!

- Jan
Jan Chrillesen, Denmark

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#42 Post by TD-er » 01 Apr 2018, 23:03

Martinus wrote:
01 Apr 2018, 13:43
TD-er wrote:
01 Apr 2018, 00:44
OK, that's good to know about the hardware ints, or actually the lack of.

About the ESPeasy + ESP32 combo.
I just tested an ESP32 module with the PlatformIO build I made yesterday and it just works :)
So expect some more debugging on that platform too. (Already found the first bug...)
Good news! Does that mean that we will have nightly builds also for ESP32?
For the nightly builds, we probably need a change made by Edwin.
But is really easy now to build an ESP32 version.
There are some strange things yet with the WiFi and the email notification.
I've been testing a bit this morning, but I've not had much time today.
I did find this discussion which may or may not be related.
Not sure yet how to upgrade the ESP32 core library.

LisaM
Normal user
Posts: 513
Joined: 11 Apr 2017, 18:29

Re: ESP-32 plans for ESPEasy (?)

#43 Post by LisaM » 01 Apr 2018, 23:27

TD-er wrote:
01 Apr 2018, 00:44
OK, that's good to know about the hardware ints, or actually the lack of.

About the ESPeasy + ESP32 combo.
I just tested an ESP32 module with the PlatformIO build I made yesterday and it just works :)
So expect some more debugging on that platform too. (Already found the first bug...)
From the ESP32 board:
I have done a measurement and delay from external trigger to application-provided ISR handler is around 2us (at 240MHz clock), which is around 500 cycles.
https://www.esp32.com/viewtopic.php?t=422
So it takes the ESP32 2us just to start the interrupt handler...
A technical explanation what interrupt latency is: https://community.arm.com/processors/b/ ... processors
And here is somebody complaining it takes a very long time for a STM32F4 ISR to react to an external interrupt: 200ns :shock:
Complaining because it should be possible to have a STM32F4 react within 20ns... :lol:

I've heard that the ESP32 is so slow because the ESP32 has a hidden software interrupt handler inside the ESP32, micro code, which explains why it takes so long to even start the ISR...

Conclusion: the ESP32 is a nice SOC, but not for realtime purposes... (don't use it in your nuclear power plant! ;) )

LisaM
Normal user
Posts: 513
Joined: 11 Apr 2017, 18:29

Re: ESP-32 plans for ESPEasy (?)

#44 Post by LisaM » 02 Apr 2018, 00:43

Martinus wrote:
01 Apr 2018, 13:40
I guess that this requirement prohibits higher level languages like microPython from handling this properly.
As explained at the end of this topic
https://forum.micropython.org/viewtopic ... 4&start=30
The reported maximum latency is 380us, that leaves 19.620us for the micropython ISR to complete the ISR (considering the max of 50 measurements per second).
Considering that only 5 plugins are using the max of PLUGIN_FIFTY_PER_SECOND, i'm not to worried. ;)

LisaM
Normal user
Posts: 513
Joined: 11 Apr 2017, 18:29

Re: ESP-32 plans for ESPEasy (?)

#45 Post by LisaM » 03 Apr 2018, 18:52

If your still worried that micropython is to slow, here's a discussion about the speed: https://forum.micropython.org/viewtopic.php?f=2&t=1349
To be short, the speed at toggling pin's:
Viper: 18,618,506 toggles/s, 9.31 MHz, 18 Clock Cycles/iter
ASM: 27,929,060 toggles/s, 14 MHz, 12 Clock Cycles/iter
Where Viper is micropython in Viper mode (very limited set of commands) and ASM is micropython embedded assembly. There's also Native mode which is faster, but more limited, then normal micropython mode. These numbers are all for an STM32F405. So far i've spent practically zero time on optimizing the code.
In case you really need speed (which i think is rare), they are possibilities.

waspie
Normal user
Posts: 111
Joined: 09 Feb 2017, 19:35

Re: ESP-32 plans for ESPEasy (?)

#46 Post by waspie » 05 Apr 2018, 13:54

do i see in github that the esp32 build is now running both cores?

i don't know how to read this stuff that well but i got the impression from some commits

happytm
Normal user
Posts: 66
Joined: 15 Aug 2016, 17:53

Re: ESP-32 plans for ESPEasy (?)

#47 Post by happytm » 13 Apr 2018, 21:54

Dear Martinus,

Does it compile with following plugins ? :

_P059_Encoder
_P062_MPR121_KeyPad
_P017_PN532

I know you mentioned a while ago IRTX and IRRX still do not compile but have you made any progress after that?

I tried and failed to compile so asking for your help.

Thanks

hamster
Normal user
Posts: 61
Joined: 27 Sep 2015, 21:01
Location: UK

Re: ESP-32 plans for ESPEasy (?)

#48 Post by hamster » 14 Apr 2018, 19:12

Latest esp32 version http://www.letscontrolit.com/downloads/ ... R20100.zipseems very good and quite stable.
I was wondering how to compile a version myself as I want to add a sensor [Nextion display] from the playground
I tried and managed to create one in PlatformIO but the result only had a few devices available and not the nextion.
Any assistance much appreciated.

TD-er
Core team member
Posts: 1615
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP-32 plans for ESPEasy (?)

#49 Post by TD-er » 14 Apr 2018, 20:34

hamster wrote:
14 Apr 2018, 19:12
Latest esp32 version http://www.letscontrolit.com/downloads/ ... R20100.zipseems very good and quite stable.
I was wondering how to compile a version myself as I want to add a sensor [Nextion display] from the playground
I tried and managed to create one in PlatformIO but the result only had a few devices available and not the nextion.
Any assistance much appreciated.
The Nextion plugin is on its way, so have a little patience. :)

User avatar
grovkillen
Core team member
Posts: 3262
Joined: 19 Jan 2017, 12:56
Location: Hudiksvall, Sweden
Contact:

Re: ESP-32 plans for ESPEasy (?)

#50 Post by grovkillen » 15 Apr 2018, 00:10

TD-er wrote:
14 Apr 2018, 20:34
The Nextion plugin is on its way, so have a little patience. :)
:mrgreen:
ESP Easy Flasher [flash tool and wifi setup at flash time]
ESP Easy Webdumper [easy screendumping of your units]
ESP Easy Netscan [find units]
Official shop: https://firstbyte.shop/
Sponsor ESP Easy, we need you :idea: :idea: :idea:

Post Reply

Who is online

Users browsing this forum: No registered users and 15 guests