Please help with 1MB module

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
tim
Normal user
Posts: 52
Joined: 05 Dec 2015, 11:24

Please help with 1MB module

#1 Post by tim » 08 Feb 2020, 13:44

Today i download the ESPEasy 20200204 release where the static IP crash is fixed.
Testing on a Wemos D1 Mini and this version works ok again.

Now i want to test this version on an LSC smartplug that has an ESP8285 module.
It now runs ESPCoreRules but i want to see how it will work with ESPEasy and compare

I was reading the old wiki and readthedocs website but did not find how to create a small image that can be uploaded on the 8285 module with OTA.
The size is standard more then 800 kB

Using Arduino IDE 1.8.5 and this setting:

#define PLUGIN_BUILD_MINIMAL_OTA

the size will be 629188 bytes and is still too big.

I feel stupid because this must be easy i guessed. :mrgreen: :mrgreen: :mrgreen: :mrgreen: :mrgreen:
Please help with instructions what setting i need to make

TD-er
Core team member
Posts: 2099
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Please help with 1MB module

#2 Post by TD-er » 08 Feb 2020, 22:24

You can download the nightly build and see if there is an 1M OTA version included based on the more recent core library.
If it isn't then the builds we make ourselves are also too big for the 1M 2-step OTA and thus it isn't that easy to get it small enough :)

FanOfHue
Normal user
Posts: 91
Joined: 06 Oct 2018, 10:08

Re: Please help with 1MB module

#3 Post by FanOfHue » 09 Feb 2020, 16:56

TD-er wrote:
08 Feb 2020, 22:24
If it isn't then the builds we make ourselves are also too big for the 1M 2-step OTA and thus it isn't that easy to get it small enough :)
Same issue here on one of my test ESP01 modules. Instead of trying to get a smaller image, i think we could also try to gain some more space on ESP01 modules and ESP8285 based hardware.

I demand OTA so i decided to tweak ESPEasy source a bit and use 64kB SPIFFS instead of default 128kB:

Code: Select all

#define PLUGIN_BUILD_MINIMAL_OTA
#define TASKS_MAX                        8
#define DAT_OFFSET_TASKS                 4096
#define DAT_OFFSET_CONTROLLER            (DAT_OFFSET_TASKS + (DAT_TASKS_DISTANCE * TASKS_MAX))
#define DAT_OFFSET_CUSTOM_CONTROLLER     (DAT_OFFSET_CONTROLLER + (DAT_CONTROLLER_SIZE * CONTROLLER_MAX))
#define CONFIG_FILE_SIZE                 28672
and change this check:
if (SpiffsSectors() < 32)
into:
if (SpiffsSectors() < 16)

This way i can easily switch between ESPEasy and ESPCoreRules by just using OTA.
(ESPCoreRules and ESPEasy can be switched without reformatting or reconfiguring)

ESPCoreRules can still do one-step OTA, but for ESPEasy a two-step OTA is needed. But with 64kB SPIFFS, the max size is a lot larger:
1M_uploader.png
1M_uploader.png (12.45 KiB) Viewed 376 times
And then it seems to fit with a lot smaller config file:
ESPEasy64kSPIFFS.png
ESPEasy64kSPIFFS.png (24.22 KiB) Viewed 376 times
This has been verified to work on release 20200204 in my lab, sending data on HTTP to Domoticz, but likely unsupported.

But maybe ESPEasy developers could think of a similar solution that has official support?

TD-er
Core team member
Posts: 2099
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Please help with 1MB module

#4 Post by TD-er » 09 Feb 2020, 17:52

I tried the same once, with 64k SPIFFS, but that was causing lots of issues.

FanOfHue
Normal user
Posts: 91
Joined: 06 Oct 2018, 10:08

Re: Please help with 1MB module

#5 Post by FanOfHue » 09 Feb 2020, 18:29

TD-er wrote:
09 Feb 2020, 17:52
I tried the same once, with 64k SPIFFS, but that was causing lots of issues.
I started out with 32768 and also had serious issues. Net free space caused other dat files to truncate and the config.dat did not store any data. But all issues were solved with 28672 size.
Test unit is running fine. even added the P253 plugin and it's now part of the UDP message bus network.

I think the config.dat size can even be reduced by setting nr of controllers to one or two. Or less tasks.
Just calculating, 4k basic + 8k for 4 tasks and 2*2*1k for one controller: 16kB (right?)

Could also skip notification.dat (who uses this?)

Also, recompiled the uploader using old Arduino core 2.0.0. and it offers even more upload space. Sketch is only 251k:
1M_uploader_core200.png
1M_uploader_core200.png (12.57 KiB) Viewed 362 times

TD-er
Core team member
Posts: 2099
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Please help with 1MB module

#6 Post by TD-er » 09 Feb 2020, 18:48

Run "meminfodetail" as command on the ESP and when you then open the sysinfo page, you can see the layout of the settings file graphically represented.
Changing the number of tasks will render the settings incompatible with other nodes.
Saving settings on a small SPIFFS will later lead to issues as the file system needs a number of free (unallocated) blocks to operate.
With a 64k SPIFFS you are already at the limit of the number of free blocks and thus any fragmentation of the file will keep pages occupied so you don't have 2 blocks (of 16 or 32 pages) where all pages are either free or marked as not used.

FanOfHue
Normal user
Posts: 91
Joined: 06 Oct 2018, 10:08

Re: Please help with 1MB module

#7 Post by FanOfHue » 09 Feb 2020, 21:42

So we're living on the edge here :mrgreen:
Some changes to save another 10kB:

Code: Select all

#define TASKS_MAX                        4
#define CONTROLLER_MAX                   3
#define DAT_OFFSET_TASKS                 4096
#define DAT_TASKS_DISTANCE               2048
#define DAT_CONTROLLER_SIZE              1024
#define DAT_OFFSET_CONTROLLER            (DAT_OFFSET_TASKS + (DAT_TASKS_DISTANCE * TASKS_MAX))
#define DAT_OFFSET_CUSTOM_CONTROLLER     (DAT_OFFSET_CONTROLLER + (DAT_CONTROLLER_SIZE * CONTROLLER_MAX))
#define CONFIG_FILE_SIZE                 18432
Even tried to set it to one task and one controller, but the "static_assertions" build into the source code won't let me change some parameters. Are there some (hidden) dependencies to disallow setting those to 1 ??

It also seems that the 1MB minimum OTA does not have the notification feature, but the notification.dat file is still present and it can't be deleted from the webgui. Could save another 4kB.

SPIFFS now has 16kB free. I don't expect many updates on SPIFFS filesystem. This firmware will typically run on a simple switch like many currently available Wifi Smart-Plugs with ESP8285 modules. They can only do one thing (on/off) and i think they only need a one-time config to setup anyway. Much more important for me is the ability to change firmware through OTA. (for the LSC smart plug, it is not so easy to get access to serial!!)

TD-er
Core team member
Posts: 2099
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Please help with 1MB module

#8 Post by TD-er » 10 Feb 2020, 11:27

You can also have a look at the webserver code.
We have a lot of defines to strip out pages, which may save quite a bit in code size.

tim
Normal user
Posts: 52
Joined: 05 Dec 2015, 11:24

Re: Please help with 1MB module

#9 Post by tim » 10 Feb 2020, 17:15

Thanks for discussion, but this all looks not really easy and standard solution for me
Also no other users comments on this, so must be the only one here on this forum
Tried TuyaConvert and you have Tasmota running so easy
So not waiting anymore, i moved over to Tasmota with better support for the Tuya modules

Post Reply

Who is online

Users browsing this forum: No registered users and 6 guests