ESP Easy development

[Martinus]

Hello to you all!

Almost a year has passed since we introduced the first ESP Easy release with support for an easy extendable device plugin mechanism. That would open doors for easy contribution by plugin developers. And about one month later, we introduced the same mechanism for controller plugins. Starting out with 12 device plugins and 6 controller plugins.

Since then a lot has happened. We've gained 21 additional device plugins and 3 controller plugins, while the plugin playground holds another 16 additional plugins and two more controller plugins. And there may well be some plugins hidden in forum posts...

It started out as a Domoticz focussed community but we've also seen more activity on other platforms like OpenHAB, ThingSpeak, Jeedom, FHEM and Pimatic. I guess the Nodo Telnet has become obsolete and i'm not sure if anything happened on PiDome (This seemed to be a nice candidate for my own home solution but it seems to have never left the alpha stage...)
OpenHAB MQTT could also be renamed "Generic MQTT" and likely there are non OpenHAB users using this protocol selection.
Would be nice to know who's using the generic HTTP and for what kind of platform...

Yesterday we released the most recent stable build (R120). A big thanks to everyone who has contributed in any way to help us getting to this point with the ESP Easy project!

We're still looking forward to welcome more developers to broaden the feature set of this project. But at the same time, we're experiencing challenges in terms of program size. For all users that are still on the classic ESP modules with only 512kB flash, they will not be able to run the full set of plugins once the playground versions are included.

I think it's important to keep the current master repository lean enough to run on all ESP modules. We have to think about other strategies, but up until we have those in place, we need some guidelines on development. The draft edition can be found on our wiki site: http://www.esp8266.nu/index.php/ESPEasy ... Guidelines

@namirda: We have to figure out how to proceed best with your work as it seems quite popular (but also quite large in code size...). It would be nice to have this in the standard branch somehow.

Because this should be a community driven initiative, feel free to comment, discuss, launch ideas, build new plugins, ...

And let's not forget: Have fun!

[andy]

Martinus

It's been remarkable watching this firmware project grow over the last year. I've found it invaluable for our product development as well as personal projects around the property.

Can't wait to see what the future brings!!

[JR01]

Martinus, thank you for your leadership, and drive to make this a success. I really enjoy playing with ESPEasy, and olating with this community, have learnt so much!

You have not mentioned people using NodeRed as controller, which we all use very successfully with OpenHab mqtt, would support renaming that to mqtt generic.

Ps: I have one esp01, 1MB, would not mind sacrifing it to the esp12's to partake in the rest of the journey!

Looking forward to the next phase of the journey!

[tozett]

Martinus, thank you for your leadership,

.. OpenHab mqtt, would support renaming that to mqtt generic.

Ps: I have one esp01, 1MB, would not mind sacrifing it to the esp12's to partake in the rest of the journey!

Looking forward to the next phase of the journey!

fully acknowledge!

(most people dont use the forum or the wiki for help, but maybe it despite that habit useful to link your new development pages in the wiki?

[JR01]

It would be an honour, let me know what to do

[bertrik]

Hi Martinus,
I've worked on an embedded project with similar challenges, rockbox, which is an alternative firmware for mp3-players. Some very old mp3 players are very limited in resources (flash and RAM) compared to newer mp3 players. For us, it is a quite hard problem to keep supporting old limited hardware alongside new fancier hardware.

Some things we did to keep things in check a bit:
1) Some very restricted players just didn't have certain features enabled at all. Each type of mp3 player basically has it's own set of defines for what features to include. This can become a bit ugly ("ifdef-hell") to allow features to be enabled/disabled at compile-time.
2) We created a system that allows plugins to be loaded at run-time. This requires some low-level trickery, to get everything compiled right for dynamic loading, and we had to create a common plugin API that can fit all plugins, (This doesn't really help with ESP Easy because there's not something like a disk we can read from).
3) At first we tried to fit everything simply by static allocation of memory, but in the end we did finally implement a kind of dynamic memory manager for some memory-hungry things.
4) We have a script ("bloat-o-meter.py" if I remember correctly) to parse the map linker output file and give insight in the RAM use of various parts of the code, so we can see what parts of the code are the biggest resource hogs. This script also helps to show memory increase vs new commits.

I think 4) is something we can try in the short term, I'll try to have a look.

Personally, I got the impression that there is a bit of code duplication between the various controllers, perhaps I can help to get that reduced too.

[tozett]

in the development guidlines are some short headings for ramsize/sketzsize.
on version r121 i found on the frontpage some information

Code: Select all

Flash Size:	4096 kB
Sketch Size/Free:	401 kB / 620 kB
Free Mem:	27728

i dont see (as a noob), why sketch (free+used) is altogether only 1000kb from the 4000kb Flash-size.
maybe helpful to elaborate a littel bit on the guidelines?

----noob-discovered somethin, update on this :
realy easy: 4000 flash minus 3000 SPIFF (filesystem) = 1000 for sketch...

[bertrik]

Actually, the bloat-o-meter script appears to work without any changes: https://paste.sigio.nl/pkh0octto
After checking out and compiling a specific revision in the Arduino IDE, just copy the resulting .elf file from /tmp/buildXXXX (to R121.elf for example)
Then do this for another revision and you can run the bloat-o-meter script with both .elf files to see which parts got bigger and which parts got smaller.

If you just run nm on a single file, for example:
nm R121.elf --size-sort --demangle -l
you get a list of all parts, their type and their size, with a reference to the source code.

It seems that the ArduinoJson library contributes quite a bit to the size, but this is useful functionality too ...
Oh, and BTW I guess I wrong earlier about not having a file system, we can have the SPIFFS as file system.

[tozett]

Oh, and BTW I guess I wrong earlier about not having a file system, we can have the SPIFFS as file system.

and there is a tool to upload to SPIFFS..
https://github.com/esp8266/arduino-esp8266fs-plugin

(i did not test, only a founding..)