ESP Easy development

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
Martinus

ESP Easy development

#1 Post by Martinus » 11 Aug 2016, 18:47

Hello to you all!

Almost a year has passed since we introduced the first ESP Easy release with support for an easy extendable device plugin mechanism. That would open doors for easy contribution by plugin developers. And about one month later, we introduced the same mechanism for controller plugins. Starting out with 12 device plugins and 6 controller plugins.

Since then a lot has happened. We've gained 21 additional device plugins and 3 controller plugins, while the plugin playground holds another 16 additional plugins and two more controller plugins. And there may well be some plugins hidden in forum posts...

It started out as a Domoticz focussed community but we've also seen more activity on other platforms like OpenHAB, ThingSpeak, Jeedom, FHEM and Pimatic. I guess the Nodo Telnet has become obsolete and i'm not sure if anything happened on PiDome (This seemed to be a nice candidate for my own home solution but it seems to have never left the alpha stage...)
OpenHAB MQTT could also be renamed "Generic MQTT" and likely there are non OpenHAB users using this protocol selection.
Would be nice to know who's using the generic HTTP and for what kind of platform...

Yesterday we released the most recent stable build (R120). A big thanks to everyone who has contributed in any way to help us getting to this point with the ESP Easy project!

We're still looking forward to welcome more developers to broaden the feature set of this project. But at the same time, we're experiencing challenges in terms of program size. For all users that are still on the classic ESP modules with only 512kB flash, they will not be able to run the full set of plugins once the playground versions are included. :(

I think it's important to keep the current master repository lean enough to run on all ESP modules. We have to think about other strategies, but up until we have those in place, we need some guidelines on development. The draft edition can be found on our wiki site: http://www.esp8266.nu/index.php/ESPEasy ... Guidelines

@namirda: We have to figure out how to proceed best with your work as it seems quite popular (but also quite large in code size...). It would be nice to have this in the standard branch somehow.

Because this should be a community driven initiative, feel free to comment, discuss, launch ideas, build new plugins, ...

And let's not forget: Have fun!

andy
Normal user
Posts: 65
Joined: 26 Jan 2016, 23:07

Re: ESP Easy development

#2 Post by andy » 11 Aug 2016, 23:27

Martinus

It's been remarkable watching this firmware project grow over the last year. I've found it invaluable for our product development as well as personal projects around the property.

Can't wait to see what the future brings!! :shock: :o :D

JR01
Normal user
Posts: 260
Joined: 14 Feb 2016, 21:04
Location: South Africa

Re: ESP Easy development

#3 Post by JR01 » 12 Aug 2016, 07:44

Martinus, thank you for your leadership, and drive to make this a success. I really enjoy playing with ESPEasy, and olating with this community, have learnt so much!

You have not mentioned people using NodeRed as controller, which we all use very successfully with OpenHab mqtt, would support renaming that to mqtt generic.

Ps: I have one esp01, 1MB, would not mind sacrifing it to the esp12's to partake in the rest of the journey!

Looking forward to the next phase of the journey!
-----------
IOTPLAY. Tinkerer, my projects are @ http://GitHub.com/IoTPlay, and blog https://iotplay.org. Using RPi, Node-Red, ESP8266 to prove Industry 4.0 concepts.

tozett
Normal user
Posts: 734
Joined: 22 Dec 2015, 15:46
Location: Germany

Re: ESP Easy development

#4 Post by tozett » 14 Aug 2016, 12:53

JR01 wrote:Martinus, thank you for your leadership,
:!: :D
JR01 wrote:.. OpenHab mqtt, would support renaming that to mqtt generic.
:!: :D
JR01 wrote:Ps: I have one esp01, 1MB, would not mind sacrifing it to the esp12's to partake in the rest of the journey!
:!: :D
JR01 wrote:Looking forward to the next phase of the journey!
:!: :D

fully acknowledge!

(most people dont use the forum or the wiki for help, but maybe it despite that habit useful to link your new development pages in the wiki?

JR01
Normal user
Posts: 260
Joined: 14 Feb 2016, 21:04
Location: South Africa

Re: ESP Easy development

#5 Post by JR01 » 14 Aug 2016, 19:46

It would be an honour, let me know what to do
-----------
IOTPLAY. Tinkerer, my projects are @ http://GitHub.com/IoTPlay, and blog https://iotplay.org. Using RPi, Node-Red, ESP8266 to prove Industry 4.0 concepts.

bertrik
Normal user
Posts: 12
Joined: 27 Jul 2016, 19:11

Re: ESP Easy development

#6 Post by bertrik » 14 Aug 2016, 21:43

Hi Martinus,
I've worked on an embedded project with similar challenges, rockbox, which is an alternative firmware for mp3-players. Some very old mp3 players are very limited in resources (flash and RAM) compared to newer mp3 players. For us, it is a quite hard problem to keep supporting old limited hardware alongside new fancier hardware.

Some things we did to keep things in check a bit:
1) Some very restricted players just didn't have certain features enabled at all. Each type of mp3 player basically has it's own set of defines for what features to include. This can become a bit ugly ("ifdef-hell") to allow features to be enabled/disabled at compile-time.
2) We created a system that allows plugins to be loaded at run-time. This requires some low-level trickery, to get everything compiled right for dynamic loading, and we had to create a common plugin API that can fit all plugins, (This doesn't really help with ESP Easy because there's not something like a disk we can read from).
3) At first we tried to fit everything simply by static allocation of memory, but in the end we did finally implement a kind of dynamic memory manager for some memory-hungry things.
4) We have a script ("bloat-o-meter.py" if I remember correctly) to parse the map linker output file and give insight in the RAM use of various parts of the code, so we can see what parts of the code are the biggest resource hogs. This script also helps to show memory increase vs new commits.

I think 4) is something we can try in the short term, I'll try to have a look.

Personally, I got the impression that there is a bit of code duplication between the various controllers, perhaps I can help to get that reduced too.

tozett
Normal user
Posts: 734
Joined: 22 Dec 2015, 15:46
Location: Germany

Re: ESP Easy development

#7 Post by tozett » 15 Aug 2016, 14:32

in the development guidlines are some short headings for ramsize/sketzsize.
on version r121 i found on the frontpage some information

Code: Select all

Flash Size:	4096 kB
Sketch Size/Free:	401 kB / 620 kB
Free Mem:	27728
i dont see (as a noob), why sketch (free+used) is altogether only 1000kb from the 4000kb Flash-size.
maybe helpful to elaborate a littel bit on the guidelines?

----noob-discovered somethin, update on this :oops: :
realy easy: 4000 flash minus 3000 SPIFF (filesystem) = 1000 for sketch...
Attachments
spiff.png
spiff.png (34.06 KiB) Viewed 8963 times
Last edited by tozett on 17 Aug 2016, 09:43, edited 1 time in total.

bertrik
Normal user
Posts: 12
Joined: 27 Jul 2016, 19:11

Re: ESP Easy development

#8 Post by bertrik » 15 Aug 2016, 21:57

Actually, the bloat-o-meter script appears to work without any changes: https://paste.sigio.nl/pkh0octto
After checking out and compiling a specific revision in the Arduino IDE, just copy the resulting .elf file from /tmp/buildXXXX (to R121.elf for example)
Then do this for another revision and you can run the bloat-o-meter script with both .elf files to see which parts got bigger and which parts got smaller.

If you just run nm on a single file, for example:
nm R121.elf --size-sort --demangle -l
you get a list of all parts, their type and their size, with a reference to the source code.

It seems that the ArduinoJson library contributes quite a bit to the size, but this is useful functionality too ...
Oh, and BTW I guess I wrong earlier about not having a file system, we can have the SPIFFS as file system.

tozett
Normal user
Posts: 734
Joined: 22 Dec 2015, 15:46
Location: Germany

Re: ESP Easy development

#9 Post by tozett » 21 Aug 2016, 19:03

bertrik wrote: Oh, and BTW I guess I wrong earlier about not having a file system, we can have the SPIFFS as file system.
and there is a tool to upload to SPIFFS..
https://github.com/esp8266/arduino-esp8266fs-plugin

(i did not test, only a founding..)

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 19 guests