Size Matters: Dynamic Linking, Flexible builds etc.

[roman]

Hi!

I have read through the history of the project and the development guidelines and it seems that (as probably with any microcontroller project) sketch size strongly limits the possibilities to include ever more hardware support for sensors, loggers, etc. and software extensions like extended web forms etc.

Since then I have been asking myself if there would be any means to use dynamic linking or something alike where the user could either compose an image to install (e.g. on a website that compiles the image on demand) or even download and link modules at runtime.

Beeing a software developer but someone who has never touched on anything below C-code I have a rather basic understanding of what is going on "under the hood". So I have tried to understand what could be necessary to make something like this work and tried to find if there would be examples but I did not find anything alike. As far as I understand it would need for dynamic linking - also of the arduino base libraries - and this seems hardly achievable with the means provided at the moment.

Has anyone with more experience in low level programming ever thought about this?

cheers
roman

[Shardan]

Hello Roman,

well, i'm not a programmer at all, but the problem ist undenieable.

A dynamic linking via download feels a bit overpowered from my point of view.
Such a model has even more security aspects as any IoT has by itself.
As an ESP is placed usualy in a sensor or actuator that is programmed once and should then just run,
this seems to be not really necessary.

What fits this type of MCU more from my point of view would be a simple "clickbox".
- All basic modules (networking, webserver) are included by Default, they are needed.
- A list of modules where i can collect my Firmware Setup just by (un-)checking boxes.

At least for me it won't matter if i do this on a Webpage and get my firmware downloaded
or if that is done on a local PC with atom/platformio or another IDE.

A very nice addon to that might be a ressource listing.
With every plugin a value is stored how much Flash / iRAM it takes and adding these
values up with checking the boxes.
This way a webpage or a local compiler script can say "Not possible with a 1024KB-flash" or "using too much iRAM".

Anyways: At the moment the dev's are very busy with getting V2.0.0 into
a stable state and i think this has priority one.


Regards
Sardan

[AndrewJ]

I agree with @roman, this is a problem.
@Shardan, I like your click-box idea.
Quite recently I have seen a table/list of plugin memory sizes, but I'm having trouble finding it again now!!
Even nicer would be if the memory sizes were presented alongside the click-boxes in some sort of interface before uploading firmware. I am also not a programmer, so I have no idea about how to achieve this, I'm afraid.
I quite agree that the priority at present is getting v.2.0.0 ready.

[Shardan]

Hello,

a list of used ressources can be found on github.

Well, i'd prefer if the amount of used ressources gets calculated automatically.

Try to imagine:
You check a box "1024" for the memory model, a 1MB in this case (Sonoff Basic for example).
Then you check some modules and at one certain point the webpage or the IDE script
warns "Out of Memory Limits" or something like that.

The danger of a "checkbox" is that some people will do a "I want it all and i want it now".
And then they want support for the so bad firmware that never works...

Well, this was the original goal of martinus as far as i remember right: To get all modules in one firmware.
Seeing the flood of new feature requests i doubt if that still is a real option.

On another note i see that things like Arduino and Platformio is a bit much for some non-tech people.
So it's a bit of a dilemma

[roman]

Thanks for the replies. I have been rethinking this and I think its a matter of audiences.

For non-programmers something like a "clickable" platfform would be ideal. And I think there is quite some potential for the project to be very accessible. E.g. the rules engine is another step aimng into this direction.

I have been talking with a friend who ist more into low-level programming (I am more familiar with python and scala than c/c++) and he had some fun thinking through the idea of dynamic linking. But I think this would be somehting not very closely connected to the project itself as if one would achieve such a platform it would be a very generic solution that would for multiple such frameworks.

What I have not been considering is the security implications. Still I would say that there is two sides to updates of a box over the internet. One is the risk associated with it, something that can be reduced by the use of cryptopgrahy and the other is the benefit of allowing for updates of critical bugs. In total ths is something pretty important we will be worrying about for the next decades...