Mega (V2.0.0) OTA firmware updates on 1MB modules

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
Martinus

Mega (V2.0.0) OTA firmware updates on 1MB modules

#1 Post by Martinus » 03 Jun 2017, 14:59

If you have uploaded the newer stock Mega ESP Easy firmware to 1MB modules like the ESP-01 or the Sonoff, you will no longer be able to use OTA because the firmware is a lot larger (533k) and there's not enough free space to hold a temp copy of a newer upload. You can use a workaround by uploading a smaller image and then upload a normal size image after that.

We have provided a very small sketch to use this workaround:
https://www.letscontrolit.com/wiki/inde ... g_firmware

Martinus

grumpy
Normal user
Posts: 5
Joined: 16 Mar 2018, 12:59

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#2 Post by grumpy » 16 Apr 2018, 16:41

Where can I find this small sketch?

User avatar
grovkillen
Core team member
Posts: 3614
Joined: 19 Jan 2017, 12:56
Location: Hudiksvall, Sweden
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#3 Post by grovkillen » 16 Apr 2018, 17:28

ESP Easy Flasher [flash tool and wifi setup at flash time]
ESP Easy Webdumper [easy screendumping of your units]
ESP Easy Netscan [find units]
Official shop: https://firstbyte.shop/
Sponsor ESP Easy, we need you :idea: :idea: :idea:

RichEO
Normal user
Posts: 3
Joined: 27 Jul 2018, 21:21

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#4 Post by RichEO » 02 Aug 2018, 02:03

I was having major slowdown when I had an MQTT controller enabled. So I decided to update to a newer version.

I uploaded the small sketch, the upload said it completed successfully but the device dropped off my network.

I have power cycled the device (a sonoff SV) and it is not showing up on my router.

Has anyone experienced this? It is inside my gate motor so in a few days I will open it up and reflash it, but maybe before then anyone has any ideas what is wrong?

kenkoknz
Normal user
Posts: 64
Joined: 03 Jul 2018, 23:46
Location: New Zealand

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#5 Post by kenkoknz » 03 Aug 2018, 03:59

Hi @grovkillen,

I flashed a sonoff(1m) with normal 1044 bin (716k) and a esp01 (puya) with dev puya 1024bin(785k) just using the normal flashESP8266, and it all seems to work normally. I can access both OTA to cnfig etc. Can you clarify if the special sketch is needed as both the files are over 533k and seems to be working fine with normal flasher?
Thanks,
Ken

User avatar
grovkillen
Core team member
Posts: 3614
Joined: 19 Jan 2017, 12:56
Location: Hudiksvall, Sweden
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#6 Post by grovkillen » 03 Aug 2018, 06:31

Current releases are too big for OTA (2 step procedure for 1Mb units). We need to rethink how to support those units. They need to be physically attached using cable and serial programming to be updated. Sorry for any inconveniences but hopefully we can think of something.
ESP Easy Flasher [flash tool and wifi setup at flash time]
ESP Easy Webdumper [easy screendumping of your units]
ESP Easy Netscan [find units]
Official shop: https://firstbyte.shop/
Sponsor ESP Easy, we need you :idea: :idea: :idea:

kenkoknz
Normal user
Posts: 64
Joined: 03 Jul 2018, 23:46
Location: New Zealand

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#7 Post by kenkoknz » 03 Aug 2018, 08:32

Hi,
This is really just a minor inconvenience, there are possible lots more urgent things on your plate to get a stable release 2. Connecting a cable for firmware upgrade is not too much of a hassle, I think we are just getting a bit lazy...

Keep up the good work,

Cheers,
Ken

sheppy
Normal user
Posts: 49
Joined: 28 Jun 2016, 05:53

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#8 Post by sheppy » 20 Aug 2018, 02:04

I hope a working OTA solution can be found, I have 38 ESP's, many are in difficult to access locations, and I have just discovered a possible DHT Humidity bug in the January Sketch I used

ewaldharmsen
Normal user
Posts: 55
Joined: 16 Feb 2017, 09:23

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#9 Post by ewaldharmsen » 26 Aug 2018, 20:47

I also look forward to be able to update OTA my devices!

ewaldharmsen
Normal user
Posts: 55
Joined: 16 Feb 2017, 09:23

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#10 Post by ewaldharmsen » 31 Aug 2018, 16:27

In the mean time: home can I make a custom build with only the plug-ins included I need?

User avatar
grovkillen
Core team member
Posts: 3614
Joined: 19 Jan 2017, 12:56
Location: Hudiksvall, Sweden
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#11 Post by grovkillen » 31 Aug 2018, 18:44

I think we're beyond the point of OTA even with no plugins in the compilation. I might be wrong though.
ESP Easy Flasher [flash tool and wifi setup at flash time]
ESP Easy Webdumper [easy screendumping of your units]
ESP Easy Netscan [find units]
Official shop: https://firstbyte.shop/
Sponsor ESP Easy, we need you :idea: :idea: :idea:

TD-er
Core team member
Posts: 3337
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#12 Post by TD-er » 31 Aug 2018, 23:51

Please have a look at the file define_plugin_sets.h and PlatformIO.ini.
Those two make it clear how the different builds are made together.
It should be possible to perform OTA on 1 M modules, when using 2-step flash. But some changes have to be made.

- filesize of the config,dat file has to be reduced to something like 36k (rest is not used)
- Flash layout may have to be changed to 1M.64k (current is 1M.128k)

At least the last step will render existing installations invalid.

User avatar
Methuselah
Normal user
Posts: 45
Joined: 04 Feb 2018, 01:39
Location: Boston, MA

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#13 Post by Methuselah » 01 Sep 2018, 12:48

grovkillen wrote: 03 Aug 2018, 06:31 Current releases are too big for OTA (2 step procedure for 1Mb units). We need to rethink how to support those units. They need to be physically attached using cable and serial programming to be updated. Sorry for any inconveniences but hopefully we can think of something.
What would be helpful is a configure script, a-la linux's "configure" which automatically enables base functionality (based on the type of device the user specifies) and then prompts the user to add in optional modules for those features s/he is going to use. For example I do not use the MQTT functionality whatsoever and could easily eliminate that from a build.

Unfortunately, the size of this project is only going to grow, as more and more functionality is added in... eventually it will get to the point where the builds will be too big for the 1MB modules, and then a whole bunch of people will be orphaned.
N00b to the world of ESPEasy, but I've been programming since the days of punch cards and paper tape :)

User avatar
grovkillen
Core team member
Posts: 3614
Joined: 19 Jan 2017, 12:56
Location: Hudiksvall, Sweden
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#14 Post by grovkillen » 01 Sep 2018, 14:55

Methuselah wrote: 01 Sep 2018, 12:48
grovkillen wrote: 03 Aug 2018, 06:31 Current releases are too big for OTA (2 step procedure for 1Mb units). We need to rethink how to support those units. They need to be physically attached using cable and serial programming to be updated. Sorry for any inconveniences but hopefully we can think of something.
What would be helpful is a configure script, a-la linux's "configure" which automatically enables base functionality (based on the type of device the user specifies) and then prompts the user to add in optional modules for those features s/he is going to use. For example I do not use the MQTT functionality whatsoever and could easily eliminate that from a build.

Unfortunately, the size of this project is only going to grow, as more and more functionality is added in... eventually it will get to the point where the builds will be too big for the 1MB modules, and then a whole bunch of people will be orphaned.
We have much in plan for the upcoming autumn and next year will be great. 1MB users are not going to be left in the dust.
ESP Easy Flasher [flash tool and wifi setup at flash time]
ESP Easy Webdumper [easy screendumping of your units]
ESP Easy Netscan [find units]
Official shop: https://firstbyte.shop/
Sponsor ESP Easy, we need you :idea: :idea: :idea:

Djoe
Normal user
Posts: 21
Joined: 27 Aug 2018, 09:04

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#15 Post by Djoe » 15 Sep 2018, 08:00

I have try to update a sonoff basic (1m) like this :

- I have build my own firmware with a size of 580kb approximately
- I have uploaded it in the sonoff
- Now I can upload ESPEasyUploaderMega_1024 through OTA, the first step
- The sonoff reboot and I select the same custom firmware (580k)

But after the reboot the new firmware not start... The wiki say that we can upload max 604k, so I think my firmware is okay ?

Is someone have already use ESPEasyUploaderMega_1024 successfully ?

Thank you and have a good day!

neoseb
Normal user
Posts: 3
Joined: 01 Oct 2018, 19:36

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#16 Post by neoseb » 02 Oct 2018, 11:12

Hi all

Very interested by this topic and by knowing how to build my custom lighter firmware for my Sonoff so that I will be able to update them as they will not be easily accessible in the coming weeks (they'll be used for my covers!)

Is there a tutorial somewhere in the web? Didn't fond anything yet.

Thank you for your help.

Djoe
Normal user
Posts: 21
Joined: 27 Aug 2018, 09:04

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#17 Post by Djoe » 02 Oct 2018, 11:55

neoseb wrote: 02 Oct 2018, 11:12 Is there a tutorial somewhere in the web? Didn't fond anything yet..
You can have a look to :

https://www.letscontrolit.com/wiki/inde ... platformio

riker1
Normal user
Posts: 333
Joined: 26 Dec 2017, 18:02

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#18 Post by riker1 » 17 Jan 2020, 09:05

grovkillen wrote: 03 Aug 2018, 06:31 Current releases are too big for OTA (2 step procedure for 1Mb units). We need to rethink how to support those units. They need to be physically attached using cable and serial programming to be updated. Sorry for any inconveniences but hopefully we can think of something.
Hi is this still valid?

So i do not understand why there is a firmware with
ESPEasy_2step_UploaderMega_1024.bin
which can not be used?

Did I miss anything?

when I call firmwareupdate via gui.

Code: Select all

URI: /update
Method: GET
Arguments: 0
So finally on 1 MB devices it is not working? right?

would be very usefull

thanks a lot T

User avatar
grovkillen
Core team member
Posts: 3614
Joined: 19 Jan 2017, 12:56
Location: Hudiksvall, Sweden
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#19 Post by grovkillen » 17 Jan 2020, 10:13

Well I think it is working work OTA now but since I'm doing 100% development of the new GUI as of the last 4-5 month I haven't been testing any new firmware stuff. Perhaps @TD-er can shed some light?
ESP Easy Flasher [flash tool and wifi setup at flash time]
ESP Easy Webdumper [easy screendumping of your units]
ESP Easy Netscan [find units]
Official shop: https://firstbyte.shop/
Sponsor ESP Easy, we need you :idea: :idea: :idea:

TD-er
Core team member
Posts: 3337
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#20 Post by TD-er » 17 Jan 2020, 15:48

As some of you may know, I've also been way too busy the last month in moving to a new house (last build was > 1 month ago :( )
In the mean time there has been a new addition to the core lib which allows for flashing gzip'ed images via OTA, which I really want to add.
It would be a really great addition to extend support for 1M modules.

It isn't there yet, but I will try to add it in a few weeks time.
Then I can also determine the true gain in size of the compressed binaries as it is really hard to predict the amount of compression we can expect.

In short the current situation:

We have in the nightly builds a set of so called "minimal_OTA" versions.
These are meant to be flashed to 1M modules in order to be able to perform OTA updates.

The 1M flash is split in 2 parts:
- roughly 128k SPIFFS filesystem
- 1M - 128k = 873k for active sketch + OTA image

The smallest 2-step OTA image we have is roughly 273k, which leaves 600k for the "active sketch". (N.B. all numbers out of my head)
So if the sketch is =< 600k, you still have room for a small "2-step OTA image" which can be used for only uploading the new image.

My hope is that the compression can shrink this 2-step image, so we have more room for the active sketch.

lumajo
Normal user
Posts: 9
Joined: 03 Feb 2018, 13:19

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#21 Post by lumajo » 12 Jul 2020, 18:00

I have ugraded ESP01 to 2M Flash. (they where sent wrongly to my, but I still gave it a try) However, it also says I am not able to make a OTA upgrade. Why? There must be enough room. The flash size is correctly recognized.
Any Idea? However, it works with the 1M workaround.

TD-er
Core team member
Posts: 3337
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#22 Post by TD-er » 12 Jul 2020, 18:06

The flash chip is partitioned in a partition for the sketch + OTA and the SPIFFS file system
A build for 1M flash is only using the first 1M of the flash chip, regardless its size.
But if you have 2M flash, it makes sense to use a 2M build (there are some 2M builds in the nightly builds, but not a lot)
With a 2M build, the SPIFFS is roughly 500k, which leaves 1500 k for the sketch + OTA, so you could use roughly 750 kB builds, or a few 100 k larger builds if you flash .bin.gz files. (only supported since core 2.7.0)

There are also ESP8285 chips with 2M, although I have never seen them myself.

N.B. if you change builds between 1M build and 2M build, the file system will be erased so you will loose all stored settings.

lumajo
Normal user
Posts: 9
Joined: 03 Feb 2018, 13:19

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#23 Post by lumajo » 12 Jul 2020, 21:59

Thanks a Lot, i will give it a try.

So I used ESP_Easy_mega_20200703_normal_WROOM02_2M256 to flash the ESP01, which works quite well.
It now says OTA yes, and it works...
However I do not know the differences to the normal one (w/Vcc). The time server ntp does not work here. So the time always stays on 1970-1-1

So maybe there is a bug here?

TD-er
Core team member
Posts: 3337
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Mega (V2.0.0) OTA firmware updates on 1MB modules

#24 Post by TD-er » 13 Jul 2020, 15:24

If you don't enter a host for the NTP server, then it will use a random one from pool.ntp.org.
Only requirement then is that it has access to a gateway and DNS server. So please check those are correct.
In the sysinfo page both are reported.

VCC build is about being able to measure its own voltage on the analog input pin.
If you need to read an analog value from some sensor, you don't need the VCC feature.

Post Reply

Who is online

Users browsing this forum: No registered users and 18 guests