Page 1 of 1

Plugin timing/architecture questions [TE AmbiMate MS4 multisensor)

Posted: 04 Sep 2019, 04:16
by bkhobby
Hey all - I'm working again on my ESPEasy plugin for the TE AmbiMate MS4 multisensor (temp/humidity/light/audio/co2/voc and motion), which is a great, commercial quality sensor with good bang/buck. I'm using the sensor with my Kube home automation sensor platform ... r-platform, with ESPEasy as the main firmware. The sensor uses I2C for communications, and thought I thought I had it mostly figured out (submitted a PR to ESPEasyPlayground in February - ... d/pull/127), there was definitely room for improvement, mostly as far as the ability to send all 8 channels of data where ESPEasy only allows 4, and avoiding the use of "delayBackground" calls in the I2C comms.

So, here I am, I've spent some more time looking at the code and rearranging things, but I think I'm stuck and need some help from the experts. I've solved the issue of 8 channels into 4 (by adding webforms with selectable sensor inputs and allowing multiple instances of the plugin to send all 8 channels), but the one main issue I still haven't solved is the fact that the PIR motion sensor requires different handling than the rest of the sensors. So, here's what the sensor requires for I2C comms for handling the different sensors:

PIR Motion (constant polling)
1. Send an I2C write to set register to output PIR motion event
2. Wait 100 ms
3. Read I2C register and check for motion event

OTHER SENSORS (on Plugin Timer period)
1. Send I2C write for AmbiMate to populate values of all sensors on next cycle
2. Wait 100 ms
3. Send I2C write command to read all sensors
4. Read 15 bytes with sensor values and process.

So, due to the different timing/read requirements for the sensors, I decided to restructure my plugin like so:

1. In "PLUGIN_TEN_PER_SEC" I only run step 1 of the PIR Motion sequence, then schedule a timer to event->TaskIndex (setting a flag to let PLUGIN_READ know there is a PIR event to read)
2. In "PLUGIN_READ", I run an if statement with the motion read flag
a. If the motion read flag is set, I run step 3 of the PIR Motion sequence (reading Motion I2C register from the sensor, and setting the appropriate UserVar for the motion sensor)
b. If the motion read flag is NOT set, I run steps 1-4 of the Other Sensors sequence. (I have not worked out a way to avoid the 100ms delay in this sequence, though I could probably reschedule the read with another flag, if it's too troublesome)

Now, I've come across a few issues with this, especially as I try to combine it with the openHAB MQTT controller:

1. The motion events don't come through the MQTT controller (with logs, I'm able to see the event all the way through setting it in UserVar)
2. The other sensors appear to get weird/random values, even though I think I'm setting the UserVars correclty based on my Webform config
3. The MQTT controller is constantly updating (sending all the way up to the nimimum send rate, and flooding the MQTT broker) <- I'm pretty sure this is because PLUGIN_READ is now being called 10 times per second.

So, what am I doing wrong? I've spent a full day looking at this code, and probably am missing some simple things, but I don't nearly have the full level of understanding of how the plugin architecture works to be able to figure these special cases out.

Rather than post the code, I'll link to the file on my GitHub ... e_MS4_.ino. I'd appreciate any suggestions/help with this, and I'll be publishing the full design (including my sensor) once completed.


-BK Hobby

Re: Plugin timing/architecture questions [TE AmbiMate MS4 multisensor)

Posted: 04 Sep 2019, 04:54
by ThomasB
1. In "PLUGIN_TEN_PER_SEC" I only run step 1 of the PIR Motion sequence, then schedule a timer to event->TaskIndex (setting a flag to let PLUGIN_READ know there is a PIR event to read)
I didn't have time to evaluate all your code. But I saw something that looks unusual to me. After deleting all the comment statements in PLUGIN_TEN_PER_SEC, it boils down to this:

Code: Select all

    case PLUGIN_TEN_PER_SECOND: //For motion event processing
        motionRead = true;
        schedule_task_device_timer(event->TaskIndex, millis() + 80);
        success = true;  // SEE NOTE BELOW
Since the user values are not available yet, I think the success var should be false, not true. That is to say, set it true in the PLUGIN_READ section when the new values are available.

- Thomas

Re: Plugin timing/architecture questions [TE AmbiMate MS4 multisensor)

Posted: 04 Sep 2019, 09:14
by TD-er
I have not yet read your (updated) code, to give some blank replies first on what to look for.

Plugin_read does evaluate the returned success.
A success from the read means: There are new values. So data is sent to the controller and an event is triggered for the updated value to be used in rules.

This is used in the GPS plugin to only send out new values when N meters have passed since the last updated position.
The UserVar values are updated on every new position update though, so in the rules you can have access to the latest position.

If you have separate "sensors" to read from the same module, you may also consider to have some kind of global defined object (or shared among all instances) to handle the reading and have 2 or more plugins to access those.
The Eastron plugin does something like this.
It has a global defined object, which can be accessed from multiple instances of the plugin to collect more than 4 values.
We also plan to do something similar for the Dallas sensors, where there is a single handler object to synchronize all calls to all sensors and have multiple plugins query that handler (not yet implemented, but similar issues)

About 2 weeks ago I also started on having a new option to get out more than the 4 values ESPeasy now supports.
This is being used for the LoRaWAN/TTN controller.
It is a new kind of format to have a very tightly packed binary stream of data containing all possible sensor values in a known order and representation.
The Sysinfo plugin and the GPS plugin are the first to support this.
LoRaWAN/TTN does have very limited bandwidth (or allowed air time) so every byte counts. On the receiving end, the data is then sent to a server running a decoder to interpret the stream and unpack it.
Not sure if this approach is useful on your project, but I wanted to let you know the options.

About the 10/sec and read calls.
I think you should keep track of some kind of state machine (see BME280 plugin) and just process that state machine.
Only when you have a new value to report, schedule the read call.
This state machine then should keep track of:
- Current state ;) (I2C write sent / waiting / Read register type X) As alternative you can keep track of not the current state, but just the "next thing to do"
- Timestamp of the last action (not to flood the sensor, but also to know if a sample could be ready)

Re: Plugin timing/architecture questions [TE AmbiMate MS4 multisensor)

Posted: 06 Sep 2019, 03:41
by bkhobby
Thank you both very much! I have a much better understanding of the architecture based on your posts, and even cleaning my code up a bit has made the plugin more stable. :D

@TD-er, the state machine you suggested makes perfect sense to deal with the AmbiMate's varied read times. I've stubbed it out in my code, but would love to run this by you to see if my pseudo-code/design will work.

Code: Select all

//State machine states
        //0 = START (reset stateTimer to millis(), go to WAITING)
        //1 = WAITING (if external plugin timer elapses, PLUGIN_READ will send to POLL ALL SENSORS;
        //             else if stateTimer elapses to 500ms, send to POLL PIR)
        //2 = POLL PIR (send status write, start 100 ms timer, go to READ PIR)
        //3 = READ PIR (process and set UserVar; send task_timer to PLUGIN_READ; PLUGIN_READ will send to WAITING)
        //4 = POLL ALL SENSORS (send status write, start 100 ms timer, go to READ ALL SENSORS)
        //5 = READ ALL SENSORS (process and set UserVar; send task_timer to PLUGIN_READ; PLUGIN_READ will send to WAITING)
Basically, I will run this in my 10/sec, polling the PIR twice per second and fitting in the reads for the remaining sensor word in between (based on the external plugin timer, up to the maximum rate of 1/sec).

In my PLUGIN_READ, I will check the current state, and:
If WAITING, I'll set success=false (no controller send), and set the state to POLL ALL SENSORS <-- Assuming PLUGIN_READ got called by the external plugin timer
If READ PIR or READ ALL SENSORS, I'll set success=true (controller send), and set the state to WAITING

I figure this will give me plenty of control over reads/writes, without having to use delays, but I'd appreciate any comments if something doesn't make sense.


Thank you for letting me know about the LoraWAN/TTN method of sending >4 values! I'll definitely use that as well, after I get the basic reads done.

P.S. How do I explicitly call one plugin task from another? E.g. If I want to call PLUGIN_GET_DEVICEVALUENAMES from PLUGIN_WEBFORM_SAVE, to repopulate the sensor value text boxes after selecting my sensors, can I do that explicitly? I could just populate the values in WEBFORM_SAVE, but it seems to me GET_DEVICEVALUENAMES is specifically built for that purpose.


Thanks again and I look forward to submitting my plugin once completed!


Re: Plugin timing/architecture questions [TE AmbiMate MS4 multisensor)

Posted: 06 Sep 2019, 11:02
by TD-er
I think you're making it a bit more complex than needed.

You have 2 entry points in the plugin:

You can re-schedule the PLUGIN_READ as you may have seen in the BME280 plugin.

So all handling of the state machine should be done from the PLUGIN_TEN_PER_SECOND call and the PLUGIN_READ should only check if there is a new value to report and just base the state of success on that.

So the state machine should share at least 1 value with the READ call, that is whethere there is a new value.
This can be as simple as a boolean, which is only set to true by the state machine and set to false by the READ call.

Whether you make this state machine a separate class with a get function (that does reset the boolean flag when read) or do it all in the 10/sec call, is up to you.
Personally I would go for a separate object, which you then can share among multiple instances of the plugin.

This object should then have a function which you should call from the 10/sec call to "feed" the state machine. Something like a clock or whatever analogy you want to make.
It would be nice if this clock then reports if the state for the sensor you're interested in has a new value, so the 10/sec call can schedule a new read.

If you want to share the object among multiple plugins, you should let the function know what value you're interested in.
If you only want to know if all sensors are read, then you don't have to give it a parameter.

So something like this:

Code: Select all

class p090_mystatemachine {

p090_mystatemachine(<parameters needed>) {}

bool loop() {
  <handle your state>
  if (state == STATE_allread) { newValue = true; }
  return newValue;

// Only called from PLUGIN_READ.
// Will also start a new run if state was idle.
bool read_and_clear_newValue() {
  if (newValue) {
    newValue = false;
	return true;
  if (state == STATE_IDLE) {
    // Your code for starting a new reading
  return false;

bool newValue = false;
unsigned long timer = 0;
byte state = STATE_uninitialized;  // Some defines you have to make, or use an enum