ESPEasyDevelopment

You may hear some construction noise in the background...

Welcome to the developers documentation page!

To aid in contributions to the project, it may come in handy to have some background information. Still very limited, but we have to start somewhere (and we did not start from the beginning...)

General concept

ESP Easy is build upon the Arduino ESP8266 core (check out on https://github.com/esp8266). This code package provides the Arduino way of working on an ESP8266 module with many ported libraries.

ESP Easy consists of a framework that provides basic functionality and a collection of controller and device plugins. Controller plugins (_Cxxxx files) provide communication to Home Automation platform like Domoticz, OpenHAB, etc. Device plugins (_Pxxx files) provide communication to a wide collection of sensor devices and some actuators. Because ESP Easy was designed as a "wireless multi-sensor" project, the actuator part is limited.

Remember that ESP Easy is primarily a sensor device. It handles measurements and stores output in numerical floating point variables that can be send over the network.

Data structures

We have a lot of data structures in ESP Easy. The most important ones:

Settings - Holds the majority of system configurations
SecuritySettings - Holds things like Wifi SSID, passwords
ExtraTaskSettings - Holds the current actual copy of additional task settings into RAM

Important: Most of the data within ESP Easy is always accessible with exception to ExtraTaskSettings. Because most structures use global RAM they put a heave load on resources. Tasks have a lot of string data and we can't afford to keep this data in RAM for all 12 tasks! So there's only one copy available in ExtraTaskSettings. Before working on some task, the configuration needs to be loaded from flash. The framework handles this before a call is made to a device plugin.

Device plugins

In many cases, it is quite easy to add support for a specific device. The framework periodically calls each device plugin with a different type of function type. The minimum set:

PLUGIN_DEVICE_ADD - Describes device characteristics to the framework
PLUGIN_GET_DEVICENAME - Returns the device name to the framework
PLUGIN_GET_DEVICEVALUENAMES - Returns the default value names to the framework
PLUGIN_READ - This is called when the framework reads a device (sensor)

Because ESP Easy is basically a sensor device, everything is hooked upon a large floating point variable array (UserVar[]). It provides a maximum of 4 variables for each task. A device can be loaded more than once, so each 'instance' has it's own set of variables. The event->BaseVarIndex variable points to the first available variable so you don't have to calculate this yourself.

Two other useful calls, often used in switch devices that use a fast polling technique to read data:

PLUGIN_TEN_PER_SECOND - This is called 10 times per second for speedy response in polling scenario's
PLUGIN_ONCE_A_SECOND - This is called once per second for not so speedy stuff

The easiest way to create a new plugin is just by copying one that looks a bit similar to the new device. Renumber all XXX numbered references to a new unique id (check the plugin playground) to avoid conflicts with the original.

Remember that you can only store floating point numbers as output from your device. Also note that the floating point precision is limited. For most application this will not lead to issues but storing a large 32 bit number like RFID tags will get rounded. As a workaround, you can store this into two floating point variables as 16 bit parts. Selecting SENSOR_TYPE_LONG will handle this within the framework. But it supports only one long value!

Plugin configuration

Basic configuration is already provided by the framework, like names, GPIO selection, port selection, etc. So maybe you don't have to worry about this at all. You just have to read the required variables if you need them:

Settings.TaskDevicePin1[event->TaskIndex] - The first GPIO pin selected within the task
Settings.TaskDevicePin2[event->TaskIndex] - The second GPIO pin selected within the task
Settings.TaskDevicePort[event->TaskIndex] - The port in case the device has multiple in/out pins

If you need some custom device configuration, the framework provides 8 variables to store these custom settings. They are 16 bit so the value can range between -32767 and +32768. They can be used in this way:

Settings.TaskDevicePluginConfig[event->TaskIndex][x] - Custom setting

Where x can be 0 to 7. If a 16 bit var does not fit your purpose, there are also 4 float and long variables available:

Settings.TaskDevicePluginConfigFloat[event->TaskIndex][x] - Custom setting
Settings.TaskDevicePluginConfigLong[event->TaskIndex][x] - Custom setting

In case you have a custom config, of course the framework does not know this custom stuff. So you need to expand the web gui with additional fields. The framework has two calls to handle this:

PLUGIN_WEBFORM_LOAD - code to be executed when the web form loads, typically a HTML input box
PLUGIN_WEBFORM_SAVE - code to be executed when the form is submitted, typically read the field and store into the settings variable

Controller plugins

The framework calls the selected controller during init phase and each time data from a sensor must be send. The minimum set:

CPLUGIN_PROTOCOL_ADD - Describes controller characteristics to the framework
CPLUGIN_GET_DEVICENAME - Returns the controller name to the framework
CPLUGIN_PROTOCOL_SEND - This is were the magic happens, sending data to your Home Automation platform.

Data transfer between devices and controllers is handled though the UserVar[] array and a special data structure "event". This data structure holds additional information needed to process the sensor data.

event->TaskIndex - The index to the task that made the request
event->BaseVarIndex - points to the actual position in the uservar array for this send request
event->idx - holds the IDX value if needed
event->sensorType - provides information on what type of sensor made the request
event->String1 - in case of MQTT, this holds the topic
event->String2 - in case of MQTT, this holds the payload

And you have access to other task related information like:

ExtraTaskSettings.TaskDeviceName - the active task name that made the request
ExtraTaskSettings.TaskDeviceValueNames[x] - An array of value names for this task
ExtraTaskSettings.TaskDeviceValueDecimals[x] - Corresponding decimal rounding requested

Configuration storage

The project needs to store it's configuration and this will be done in the on board flash memory chip. In the beginning we have used the emulated EEPROM but we have abandoned that because of it's limitations and permanent RAM usage. We moved over to SPIFFS but that also introduced a rather large decrease of available RAM. So we moved over to reading/writing directly to flash memory. For convenience, we use the same area that is normally reserved for SPIFFS, so we don't have to worry about locating a suitable area. This also means that we can't use SPIFFS and compiling without reservation for SPIFFS wont work with ESP Easy.

With a minimum of 64k SPIFFS reservation and the available emulated EEPROM area, we have a total of 17 sectors available, 4k each.

This is our current layout:

Sector 0	Global settings struct
Sector 1-3	Task settings (1kB per task, first 512 bytes for system, 512 remaining can be used for custom task config)
Sector 4-6	Reserved for future expansion for Task settings
Sector 7	Custom controller config
Sector 8	Security settings
Sector 9	CSS config
Sector 10	Rules
Sector 11-16	future use

When saving the configuration, sectors 0-7 (32k) are saved to file. This is because sector 8 contains security data. Because CSS and rules are plain text, they can be saved in other ways. Sector 0-7 contains binary data.

The framework handles most of the work so you don't have to worry about this yourself. But the layout can explain why things are limited and that's something you need to be aware of if you start to use custom data structures for configuration.

A controller plugin can have custom configuration up to 4096 bytes. A device plugin can only use 512 bytes for it's custom configuration.

In most cases you don't need a custom config, but the framework provides these system calls to easy things if you need it:

To load/save data structures from/into flash when using a device plugin:

 LoadCustomTaskSettings(event->TaskIndex, (byte*)&customConfig, sizeof(customConfig));
 SaveCustomTaskSettings(event->TaskIndex, (byte*)&customConfig, sizeof(customConfig));

To load/save data structures from/into flash when using a controller plugin:

 LoadCustomControllerSettings((byte*)&customConfig, sizeof(customConfig));
 SaveCustomControllerSettings((byte*)&customConfig, sizeof(customConfig));

In these samples, the 'customConfig' is a data structure that can hold a collection of variables.