pulsecounter acting strange after latest changes
Moderators: grovkillen, Stuntteam, TD-er
pulsecounter acting strange after latest changes
with build 20200410 i had a completely stable solution for at least 3 weeks
no extra or missing pulses.
now with build 20200426 i am experiencing strange things
i changed my rules script so i could use the setpulsecountertotal command instead of a dummy -device
first day i experienced that the pulsecounter total was missing 1 pulse
no rules involved because this happens directly in the plugin
to be save i restored the old rules which use count=1 as a trigger and increment a dummy device total
in this setup the dummy device total had an extra pulse.
strange thing in this situation is that the pulsecounter total was right
no reboots happened at that time so i suspect the changes made regarding restoring RTC values did not create this, nor did using the 4th var as a value.
at the moment i am going back to 20200410 to see if i still have a stable solution again
@TD-er,
can you see if changes between 20200410 and 20200426 has something to do with this
no extra or missing pulses.
now with build 20200426 i am experiencing strange things
i changed my rules script so i could use the setpulsecountertotal command instead of a dummy -device
first day i experienced that the pulsecounter total was missing 1 pulse
no rules involved because this happens directly in the plugin
to be save i restored the old rules which use count=1 as a trigger and increment a dummy device total
in this setup the dummy device total had an extra pulse.
strange thing in this situation is that the pulsecounter total was right
no reboots happened at that time so i suspect the changes made regarding restoring RTC values did not create this, nor did using the 4th var as a value.
at the moment i am going back to 20200410 to see if i still have a stable solution again
@TD-er,
can you see if changes between 20200410 and 20200426 has something to do with this
Re: pulsecounter acting strange after latest changes
OK, can you itemize what parts you think may have different behavior?
Reading these changes in story-like format are a bit hard to interpret and itemizing may help to categorize what things have been checked and ruled out and what not.
Just by reading your post (for a few times), it looks like:
- the plugin itself does record an extra pulse
- You made sure it was never a count>1 between intervals. (or at least not in the run with the old rules)
Do you still use the setpulsecountertotal command?
If so, it could be you set it to some value and almost immediately there is a new pulse registered.
Any offset should then be visible right after setting the counter total.
Reading these changes in story-like format are a bit hard to interpret and itemizing may help to categorize what things have been checked and ruled out and what not.
Just by reading your post (for a few times), it looks like:
- the plugin itself does record an extra pulse
- You made sure it was never a count>1 between intervals. (or at least not in the run with the old rules)
Do you still use the setpulsecountertotal command?
If so, it could be you set it to some value and almost immediately there is a new pulse registered.
Any offset should then be visible right after setting the counter total.
Re: pulsecounter acting strange after latest changes
i added the complete setup here with the 3 scenariosTD-er wrote: ↑29 Apr 2020, 11:15 OK, can you itemize what parts you think may have different behavior?
Reading these changes in story-like format are a bit hard to interpret and itemizing may help to categorize what things have been checked and ruled out and what not.
Just by reading your post (for a few times), it looks like:
- the plugin itself does record an extra pulse
CORRECT: it is adding an extra pulse in case of the 20200426 firmware and with the old rules it is sometimes missing a pulse
i check on Count=1 but this does not change the total because the plugin does that
- You made sure it was never a count>1 between intervals. (or at least not in the run with the old rules)
Do you still use the setpulsecountertotal command?
Only on a cold boot, i retrieve the value from domoticz and set it to the pulsecounter so this does not account for extra/missing pulses
If so, it could be you set it to some value and almost immediately there is a new pulse registered.
Any offset should then be visible right after setting the counter total.
in a nutshell
20200410 build using count=1 as trigger to increment dummy counter is stable
20200426 using build in pulsecounter total is adding an extra pulse
20200426 build using count=1 as trigger to increment dummy counter is missing a pulse
- Attachments
-
- esp-devices-rules.rar
- (153.73 KiB) Downloaded 235 times
Re: pulsecounter acting strange after latest changes
This does look like it is still possible the pulse counter sometimes has "count=2".
Can you extend the rules slightly to store the counts where count > 1? You can use the %eventvalue% for that.
Re: pulsecounter acting strange after latest changes
do you mean like a test to see if there are pulses with values higher then 1
can you explain how to do this with %eventvalue% because i do not understand
can you explain how to do this with %eventvalue% because i do not understand
Re: pulsecounter acting strange after latest changes
Yep, that's what I mean.
Just taken from your Word document:
Not going to make it all, but just as illustration:
Just based on your description, it is incorrect behavior if the count > 1, but if you ignore it you will loose counts.
So maybe a simple fix in your rules can be to count any count > 1 as just a single count.
Just taken from your Word document:
Code: Select all
On Watermeter#Count do // When Pulse is detected
if [Watermeter#Count] = 1 // if pulse is received
TaskValueSet 3,4,[Water#CounterTotal]+1 // increase Water#CounterTotal with 1 liter
SendToHTTP 192.168.1.50,8084,/json.htm?type=command¶m=udevice&idx=344&nvalue=0&svalue=[Water#CounterTotal]
TaskValueSet 3,1,60000/[Watermeter#Time] // set Water#Flow
SendToHTTP 192.168.1.50,8084,/json.htm?type=command¶m=udevice&idx=338&nvalue=0&svalue=[Water#Flow]
endif
if [Water#Flow] > [Water#MaxFlow]
TaskValueSet 3,3,[Water#Flow] // copy Water#Flow to Water#MaxFlow
EndOn
Code: Select all
On Watermeter#Count do // When Pulse is detected
if [Watermeter#Count] = 1 // if pulse is received
// Normal operations, do your stuff.
else
// Not normal so it is either 0 or > 1
// Just assuming you have a dummy task and task variable called [debug#total] at task position 12, variable position 1
TaskValueSet 12,1,[debug#total]+[Watermeter#Count]
endif
EndOn
So maybe a simple fix in your rules can be to count any count > 1 as just a single count.
Code: Select all
if [Watermeter#Count] >= 1 // if pulse is received
Re: pulsecounter acting strange after latest changes
ok, that i can do(the %eventvalue% was not clear)
that would proof why pulses are missing but that will not proof where the extra pulse came from because that one is added directly to the total of the pulsecounter.
in this thread we tried to solve a problem regarding the RTC values which turned out to be a different problem becaue Bulthaup was using a formula
https://www.letscontrolit.com/forum/vie ... f=6&t=6926
i remember that you changed a sequence just to be sure this was not a problem.
i think it was this issue
https://github.com/letscontrolit/ESPEasy/pull/2996
could this have caused it or is this change already in build 20200410
that would proof why pulses are missing but that will not proof where the extra pulse came from because that one is added directly to the total of the pulsecounter.
in this thread we tried to solve a problem regarding the RTC values which turned out to be a different problem becaue Bulthaup was using a formula
https://www.letscontrolit.com/forum/vie ... f=6&t=6926
i remember that you changed a sequence just to be sure this was not a problem.
i think it was this issue
https://github.com/letscontrolit/ESPEasy/pull/2996
could this have caused it or is this change already in build 20200410
Re: pulsecounter acting strange after latest changes
About the %eventvalue%
Just have a look at this part of your own rules:
The event starts with: "Watermeter#Count"
But the actual event is: "Watermeter#Count=1" (or =2..., you get the idea)
So the event has a value, which exists after the "=" sign.
In the rules you can refer to this value using %eventvalue%
So just like referring to [Watermeter#Count], which could have changed, you can also use %eventvalue% in the rules.
The content of %eventvalue% is the same as it was at the moment the event was fired.
Since the pulse counter acts on interrupts, it is possible the actual value of [Watermeter#Count] has changed by the time you refer to it from rules.
That pull request you refer to was already merged in build 20200410.
Just have a look at this part of your own rules:
Code: Select all
On Watermeter#Count do
But the actual event is: "Watermeter#Count=1" (or =2..., you get the idea)
So the event has a value, which exists after the "=" sign.
In the rules you can refer to this value using %eventvalue%
So just like referring to [Watermeter#Count], which could have changed, you can also use %eventvalue% in the rules.
The content of %eventvalue% is the same as it was at the moment the event was fired.
Since the pulse counter acts on interrupts, it is possible the actual value of [Watermeter#Count] has changed by the time you refer to it from rules.
That pull request you refer to was already merged in build 20200410.
Re: pulsecounter acting strange after latest changes
ah,
finally clear to me
i let it run for a couple of days to see if it stays stable with current build 20200410(it did for 3 weeks)
after that i only update the firmware and let it run again for a couple of days.
finally i will change the rules to see if that affects things and will start the debugging
finally clear to me
i let it run for a couple of days to see if it stays stable with current build 20200410(it did for 3 weeks)
after that i only update the firmware and let it run again for a couple of days.
finally i will change the rules to see if that affects things and will start the debugging
Re: pulsecounter acting strange after latest changes
well i did a lot of testing but sofar
after a week instantly 4 extra pulses
after another week of testing 1 missing pulse
i tested if the pulsecounter was triggered on values higher then 1 but this is not the case
i also did set the pulsecounter to the total amount and that value always matched to the one on the dummy device
so it looks like the extra pulse is cominf from the pulsecounter itself
because it ran completely stable for 3 weeks eith the previous build i am going to downgrade to version 20200410 or even 20200328
i will let it run again for another 3 weeks
if in those 3 weeks i do not get any extra or missing pulses then we might need to revert some changes we made in the p003 pulsecounter
not the setpulsecountertotal but the things before that.
will get back after 3 weeks
#update with 20200328 2 missing pulses so going back to 20200328
after a week instantly 4 extra pulses
after another week of testing 1 missing pulse
i tested if the pulsecounter was triggered on values higher then 1 but this is not the case
i also did set the pulsecounter to the total amount and that value always matched to the one on the dummy device
so it looks like the extra pulse is cominf from the pulsecounter itself
because it ran completely stable for 3 weeks eith the previous build i am going to downgrade to version 20200410 or even 20200328
i will let it run again for another 3 weeks
if in those 3 weeks i do not get any extra or missing pulses then we might need to revert some changes we made in the p003 pulsecounter
not the setpulsecountertotal but the things before that.
will get back after 3 weeks
#update with 20200328 2 missing pulses so going back to 20200328
Re: pulsecounter acting strange after latest changes
ok.
it seems that after running stable for so long i tried to looosten up the settings
i apparently changed the wifi no sleep
after setting wifi to no sleep again the system seems stable also with the 20200426 build.
it seems that this one setting can cause extra pulses
could it be that some way when the wifi is sleeping and a pulse is counted the system comes back to life and again registers a pulse?
it has nothing to do with pulses missing because the total is calculated in the esp instead of sending over 1 pulse at a time
it seems that after running stable for so long i tried to looosten up the settings
i apparently changed the wifi no sleep
after setting wifi to no sleep again the system seems stable also with the 20200426 build.
it seems that this one setting can cause extra pulses
could it be that some way when the wifi is sleeping and a pulse is counted the system comes back to life and again registers a pulse?
it has nothing to do with pulses missing because the total is calculated in the esp instead of sending over 1 pulse at a time
Re: pulsecounter acting strange after latest changes
If sleeping wifi was responsible for less pulses, then it would make sense.
I can't explain what you're experiencing.
I can't explain what you're experiencing.
Re: pulsecounter acting strange after latest changes
@TD-er
i let this one run for a very long time but i still get some extra pulses some time.
to make things easy i use the direct numbers from the pulsecounter itself
eg. the Watermeter#total is the exact watermeternumber which i set with SetPulseCounterTotal
so if here is an extra pulse we know for sure it is coming from the pulsecounter itself(do to waterpressure jitter or a bug in the debouncetime functionality)
now rather than trying to find the cause i want to correct it.
i will try to measure the Watermeter#Time and if i record a low value(below 2000) i know it is caused by a faulty reading
now my question is
what is the best way to intercept this: i do not know if all the variables of the pulsecounter(Count, Total, Time do update at the same time)
On Watermeter#Count do (i use a 1 second interval on the pulsecounter so it probes every second)
If Watermeter#count=1
or
On Watermeter#Time do
if Watermeter#Time <2000
and should i test the Watermeter# variables or is it better to use %eventvalue%
if %eventvalue% <2000
if i can figur out how to intercept faulty reading i can then autocorrect is by substracting a pulse from the total
i let this one run for a very long time but i still get some extra pulses some time.
to make things easy i use the direct numbers from the pulsecounter itself
eg. the Watermeter#total is the exact watermeternumber which i set with SetPulseCounterTotal
so if here is an extra pulse we know for sure it is coming from the pulsecounter itself(do to waterpressure jitter or a bug in the debouncetime functionality)
now rather than trying to find the cause i want to correct it.
i will try to measure the Watermeter#Time and if i record a low value(below 2000) i know it is caused by a faulty reading
now my question is
what is the best way to intercept this: i do not know if all the variables of the pulsecounter(Count, Total, Time do update at the same time)
On Watermeter#Count do (i use a 1 second interval on the pulsecounter so it probes every second)
If Watermeter#count=1
or
On Watermeter#Time do
if Watermeter#Time <2000
and should i test the Watermeter# variables or is it better to use %eventvalue%
if %eventvalue% <2000
if i can figur out how to intercept faulty reading i can then autocorrect is by substracting a pulse from the total
Re: pulsecounter acting strange after latest changes
You should always use the %eventvalue% notation in handling events.
The reason is very simple;
Events may be queued and processed one at a time.
So if you handle an event, the event values were stored at the time the event was created and added to the queue, but the task values they reflect may have been changed at the moment you handle the event.
This is an issue if two of the same events end up in the event queue before they are handled.
When an event from the queue is being handled, only one event is processed in a loop.
Also the event queue is only handled if the ESP doesn't have some urgent scheduled things to do, like the 50/sec call etc.
Just take a look at the timing stats page to see if there are pieces of blocking code taking a long time to execute. Some may take several seconds, which make it very plausible the event queue may contain the same event twice (with different event values).
If you then look at the task values instead of the event values, you may see the values reflecting the later event in the queue. Especially for counters this may result in extra registered counts.
The reason is very simple;
Events may be queued and processed one at a time.
So if you handle an event, the event values were stored at the time the event was created and added to the queue, but the task values they reflect may have been changed at the moment you handle the event.
This is an issue if two of the same events end up in the event queue before they are handled.
When an event from the queue is being handled, only one event is processed in a loop.
Also the event queue is only handled if the ESP doesn't have some urgent scheduled things to do, like the 50/sec call etc.
Just take a look at the timing stats page to see if there are pieces of blocking code taking a long time to execute. Some may take several seconds, which make it very plausible the event queue may contain the same event twice (with different event values).
If you then look at the task values instead of the event values, you may see the values reflecting the later event in the queue. Especially for counters this may result in extra registered counts.
Re: pulsecounter acting strange after latest changes
looked at the timimgstats but do not know what to look for. i attached it
regarding the queue
i only had 1 On --Do command which was active on a 1 second base. i now added a second one to record possible low timings and store it in Maxflow for the time being
if i record a reading lower than 2000 and find out i have an extra pulse i know how to solve it.
regarding the queue
i only had 1 On --Do command which was active on a 1 second base. i now added a second one to record possible low timings and store it in Maxflow for the time being
if i record a reading lower than 2000 and find out i have an extra pulse i know how to solve it.
Code: Select all
On System#Boot do // When the ESP boots, do
TaskValueSet 3,2,0 // TaskValueSet TASKnr,VARnr,Value, Reset the Flow counter to 0
EndOn
On MQTTimport#Connected do
if [Watermeter#Total] = 0
SendToHTTP 192.168.1.50,8084,json.htm?type=command¶m=switchlight&idx=334&switchcmd=On //initiate mqtt broadcast
endif
Endon
On DomoticzWatermeter#CounterTotal do
if [Watermeter#Total] = 0
SetPulseCounterTotal,[DomoticzWatermeter#CounterTotal],1
endif
Endon
On Clock#Time=All,00:00 do
TaskValueSet 3,1,[Watermeter#Total]
EndOn
On Watermeter#Count do // When Pulse is detected
if [Watermeter#Count] = 1 // if pulse is received
Publish domoticz/in,'{"idx":344,"nvalue":0,"svalue":"[Watermeter#Total]"}'
TaskValueSet 3,2,60000/[Watermeter#Time] // set Water#Flow
Publish domoticz/in,'{"idx":338,"nvalue":0,"svalue":"[DomoticzWatermeter#Flow]"}'
TaskValueSet 3,4,[Watermeter#Total]-[DomoticzWatermeter#CounterTotal]
endif
Endon
On Watermeter#Time do
if %eventvalue% < [DomoticzWatermeter#MaxFlow]
TaskValueSet 3,3,%eventvalue% // copy time to maxflow
endif
EndOn
Re: pulsecounter acting strange after latest changes
When looking at your rules, I notice you have 2 entries for the "Watermeter"
When handling the Watermeter#Count event, you also look at the [Watermeter#Time] value.
However, the order of values of the pulse plugin is:
- Count
- Total
- Time
That's also the order in which the events are sent.
So if you process the #Count event, the #Time event is already placed in the queue.
It probably doesn't matter a lot, as long as those events are processed before the new events of that plugin are added to the queue.
If a new event is added to the event queue before the others are processed, you're in trouble.
So what you may want to do, is store the event value of #Count in a variable and use that one in the handling of #Time. (as that's the last one in a row)
This way you don't have to check the task values but can rely on the event values.
When handling the Watermeter#Count event, you also look at the [Watermeter#Time] value.
However, the order of values of the pulse plugin is:
- Count
- Total
- Time
That's also the order in which the events are sent.
So if you process the #Count event, the #Time event is already placed in the queue.
It probably doesn't matter a lot, as long as those events are processed before the new events of that plugin are added to the queue.
If a new event is added to the event queue before the others are processed, you're in trouble.
So what you may want to do, is store the event value of #Count in a variable and use that one in the handling of #Time. (as that's the last one in a row)
This way you don't have to check the task values but can rely on the event values.
Re: pulsecounter acting strange after latest changes
Based on this discussion I added a new feature and will now make a test build for you.
See the last commit on this PR: https://github.com/letscontrolit/ESPEasy/pull/3439
See the last commit on this PR: https://github.com/letscontrolit/ESPEasy/pull/3439
Re: pulsecounter acting strange after latest changes
See for test build:
https://github.com/letscontrolit/ESPEas ... -758278897
This allows you to set the checkbox to combine all values of a task in a single event.
https://github.com/letscontrolit/ESPEas ... -758278897
This allows you to set the checkbox to combine all values of a task in a single event.
Re: pulsecounter acting strange after latest changes
thanks.
the second on Watermeter#Time was only added to see if i can spot an anommaly (very low time)
normally i only have 1 on Watermeter event and with that one i still get extra pulses.
regarding your extra functionality.
1 event only has one %eventvalue%, this is the reason i masde the second event(to capture the %eventvalue% which represends the Time)
is it possible that when an event is launched we get all the different eventvalues so we can check on those
EG
On Watermeter#Count do
%eventvalue1% (represents Count)
%eventvalue2% (represents Total)
%eventvalue3% (represents Time)
UPDATE:
looking at your code this is exactly what you mean, very nice.
will test the new build.
FYI:
the Time represents the amount of liters
i checked and in my situation the max amount of liters per minute is 28
if i use 30 liters this means 2000ms (60000/2000)
so if i spot a time lower than 2000ms this means it is an anomally which could be a sign of an exptra pulse
i suspect that an extra pulse time will be much lower than 2000ms but below 2000ms is save.
the second on Watermeter#Time was only added to see if i can spot an anommaly (very low time)
normally i only have 1 on Watermeter event and with that one i still get extra pulses.
regarding your extra functionality.
1 event only has one %eventvalue%, this is the reason i masde the second event(to capture the %eventvalue% which represends the Time)
is it possible that when an event is launched we get all the different eventvalues so we can check on those
EG
On Watermeter#Count do
%eventvalue1% (represents Count)
%eventvalue2% (represents Total)
%eventvalue3% (represents Time)
UPDATE:
looking at your code this is exactly what you mean, very nice.
will test the new build.
FYI:
the Time represents the amount of liters
i checked and in my situation the max amount of liters per minute is 28
if i use 30 liters this means 2000ms (60000/2000)
so if i spot a time lower than 2000ms this means it is an anomally which could be a sign of an exptra pulse
i suspect that an extra pulse time will be much lower than 2000ms but below 2000ms is save.
Re: pulsecounter acting strange after latest changes
UPDATE:
eveything seems to be working after all
important to mention is that if we use the values combined then we MUST USE #All
aparently the #Count, #Total and #Time do not work to trigger events
So in my code i needed to change Watermeter#Count into Watermeter#All
eveything seems to be working after all
important to mention is that if we use the values combined then we MUST USE #All
aparently the #Count, #Total and #Time do not work to trigger events
So in my code i needed to change Watermeter#Count into Watermeter#All
Code: Select all
On Watermeter#Count do // When Pulse is detected
if [Watermeter#Count] = 1 // if pulse is received
Publish domoticz/in,'{"idx":344,"nvalue":0,"svalue":"[Watermeter#Total]"}'
TaskValueSet 3,2,60000/[Watermeter#Time] // set Water#Flow
Publish domoticz/in,'{"idx":338,"nvalue":0,"svalue":"[DomoticzWatermeter#Flow]"}'
TaskValueSet 3,4,[Watermeter#Total]-[DomoticzWatermeter#CounterTotal]
endif
if %eventvalue3% < [DomoticzWatermeter#MinTime]
TaskValueSet 3,3,%eventvalue3% // copy time to MinTime
endif
EndOn
Re: pulsecounter acting strange after latest changes
Yep, indeed you need to change the event to use "#All" in the rules block handling the event.
I'm glad we had this discussion, as it gave me an idea for a very simple optimization which also makes the rules a lot less complex in the end.
Especially on plugins like the pulse counter which can be updated very irregularly.
I guess further optimizations on the pulse counter are mainly on the accuracy of the pulse counting and thus may probably only be possible on ESP32 as that one has proper interrupts on GPIO pins and can provide a lot more insight in the actual pulse received.
I'm glad we had this discussion, as it gave me an idea for a very simple optimization which also makes the rules a lot less complex in the end.
Especially on plugins like the pulse counter which can be updated very irregularly.
I guess further optimizations on the pulse counter are mainly on the accuracy of the pulse counting and thus may probably only be possible on ESP32 as that one has proper interrupts on GPIO pins and can provide a lot more insight in the actual pulse received.
Who is online
Users browsing this forum: No registered users and 42 guests