RFC: How to custom-format display Task data in the Devices page

[Ath]

A request that started here

This is intended as a Request for Comment/discussion piece. Please reply if you think you have something useful/relevant to add.

Goal: Present Task data more userfriendly on the Devices page

The challenge:
- Some devices hold Values data that is either not very comprehensible or even incomplete when displayed as a mere numerical value
Examples:
- P008 RFID, P016 IR, P017 PN352 and P040 ID12 -> Store their data in 2 variables (to avoid losing accuracy), but only the lower 16 bits are displayed
- Dummy Devices can be used to store all kinds of numerical data -> Storing %syssec_d% gives a number like 12345, but it is quite hard to comprehend that it is actually 03:25:45 in HH:MM:SS format
- Some numerical data would be more useful when translated to an enumerator-like string (0-off, 1-ON) etc. (Some formatting options are already available in the [taskname#varname#transformation#justification] processing)
- Temperatures, barometric pressures, etc. would be helped when showing a unit before or after the value

To avoid overly complex code, as mentioned in the thread above as seen in the Domoticz plugins/controllers for receiving/sending values, there should probably be multiple, area-specific, solutions to these multiple cases.

A few possible solutions/directions I've been thinking of:

1) Plugins that store their result in multiple variables (P008, P017, P040)
-> A plugin-event that is called while displaying the Devices page, and the possibly dashboard.esp page?, that returns a presentation-version (string) for all possible values (1..4) of the plugin (event->String1..4?)
: Quite easy to implement, as it is controlled by plugin authors

2) Dummy devices, or more generic, numerical data as transformed data
-> Some kind of formatting formula to transform the numeric data to the required output
: This could require some settings space to store these formulas, and that can turn out to be somewhat challenging
: Could be handled by rules/events, but currently a) returning a value from rules is 'difficult' and b) string support in ESPEasy is limited

[TD-er]

Please have a look at those plugins where "PLUGIN_WEBFORM_SHOW_VALUES" is implemented.

I think that does cover all the issues you're seeing?

On top of that I guess we may need to have a look at the "UserVar" data as it is now used.
Every piece of code can access it, which may lead to a plethora of implementations to handle special cases.
Maybe we should make it just an array of bytes which will be read by default as 4 floats, or in some other form based on the current sensor Vtype.
For example Sensor_VType::LONG is now used in a very inefficient way to store its value in 2 floats to not loose any resolution.

But in the same 16 bytes per task, you can store whatever you like.
- 4x 32 bit uint/int
- 4x float
- 8x 16 bit int with some standard factor (see for inspiration the packed raw data format I added for the LoRaWAN controller)
- etc.

So I guess that's what you meant by extending the Sensor_VType enum?

If that's the reason to do it, then I am quite fond of the idea, as long as we take a very good look at the Domoticz controllers as they really rely on this type.
So maybe we need to add a very specific field in the DeviceStruct to add a "Domoticz override sensor VType" only to be set for a very specific plugin?

[Ath]

Please have a look at those plugins where "PLUGIN_WEBFORM_SHOW_VALUES" is implemented.

That should take care of item 1) indeed, but the documentation on this part is quite sparse, to say the least, that's probably why I didn't understand how this is to be used.

I think that does cover all the issues you're seeing?

No, as there is no way to format the data in a Dummy Device, you would need some kind of typing or user-controlled formatting, as it is very likely different for every instance used.

On top of that I guess we may need to have a look at the "UserVar" data as it is now used.
Every piece of code can access it, which may lead to a plethora of implementations to handle special cases.
Maybe we should make it just an array of bytes which will be read by default as 4 floats, or in some other form based on the current sensor Vtype.
For example Sensor_VType::LONG is now used in a very inefficient way to store its value in 2 floats to not loose any resolution.

But in the same 16 bytes per task, you can store whatever you like.
- 4x 32 bit uint/int
- 4x float
- 8x 16 bit int with some standard factor (see for inspiration the packed raw data format I added for the LoRaWAN controller)
- etc.

Yes, something in this direction is the way to go, though 16 bytes isn't much space, but it'll have to suffice for now I think.

So I guess that's what you meant by extending the Sensor_VType enum?

If that's the reason to do it, then I am quite fond of the idea, as long as we take a very good look at the Domoticz controllers as they really rely on this type.
So maybe we need to add a very specific field in the DeviceStruct to add a "Domoticz override sensor VType" only to be set for a very specific plugin?

I wasn't trying to extend that enum just yet, more like trying to get custom formatting on task values.

Hadn't yet found the special handling of Sensor_VType::LONG before (but now I have), so the RFID tags are actually shown correctly, great! (Have no RFID reader available to validate...)