tips to create a really stable pulsecounter for watermeter

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
GravityRZ
Normal user
Posts: 206
Joined: 23 Dec 2019, 21:24

tips to create a really stable pulsecounter for watermeter

#1 Post by GravityRZ » 13 Apr 2020, 15:01

Hello,

i use the M12 LJ12A3-4-Z Bx 5 V Inductive Proximity Sensor to count pulses on my watermeter.
This has been done before and with good results.
The things I (and many others) experience is that this is not a 100% reliable solution. sometimes you get multiple extra pulses and sometimes pulses are missing.
It took me a lot of testing where these extra/missing pulses came from.

there are a lot of factors which can cause this behaviour
sensor jittering,
bug in p003 plugin
ESPEasy software
wifi transmission
protocol used(HTTP, MQTT)
data send through controller or through rules
timing problems in rules

after 4 weeks of testing and testing i think i have a solution which seems 100% reliable.
it is 1 week runnig and absolutely no missing or extra pulses.

to help others i will describe my setup and name the points i find important. Maybe you are one of the lucky ones who have a good setup without these ;points but still it would not hurt

Always use a 10K pullup resistor on the ESP input

use an optocoupler, i see a lot of people without electronics knowledge who either do not use a pull up resistor or do use one but connect it to the 5V
the inputs of the ESP are 3.3v and are 5v tolerant, this means in can accept 5v levels but in doing so you might blow up the port. to be save stay on 3.3v logic on the ESP input side
of course you could always use a resister divider but please note this is NOT a logic shifter. true it acts as one and will probably work but it is not a clean solution
proximity-aansluiting.JPG
proximity-aansluiting.JPG (122.83 KiB) Viewed 13897 times

when using the P003 pulsecounter, use a debounce time to avoid jitter, i use 2000ms(if set to high you might miss a pulse, if set to low you might get extra pulses

set the p003 pulsecounter to a 1 second interval. this means whenever it receives a pulse it will flush the data to the variable every second
since my maximum water consumption is 25 liters per minute this means 1 pulse every 2.4 seconds so as long as the interval is set lower than the max flow you are save and will not miss a pulse (unless things happen at the exact same time. sofar this did not happen).

reset the pulsecounter every 24 hours, just to make sure the variable values do not get to big

set wifi to nosleep(not sure if this is mandatory but it helps)

use a dummy sensor which keeps the total meter value(pulsecounter has no realtime update, dummy device has)
use the pulsecounter to trigger a rule which increments this total meter value
send over the complete value(i used to sendover the delta but if anything goes wrong with the transmission you miss a pulse) using rules
use the http protocol to send it over(i used to use MQTT but this can cause missing/extra pulses)

with all this in place i still had an occasional missing/extra pulse.
i suspect that this was caused by either a timing problem in the rules(2 things happening at exact the same time) or a short burst in the waterpipe which causes the watermeter to move back/forth and caused a jitter in the sensor

what i did was adding a small but powerfull magnet to the side of the proximity sensor. The idea is to make the sensor more sensitive. by doing so i did not expect that this would solve the problem because when the senso is more sensitive there still is a border area where the sensor will go from 0 to 1 and back and this area is sensitive for jitter.
when you move the magnet around you notice that the sensor will become more sensitive while moving in another direction it will become less sensitive. let the watermeter run and move the magnet around. the idea is to widen the pulse so that it stays on longer)

Well with that final adjustment it is running stable for 7 days in a row now(fingers crossed)
I tried the magnet trick before but it did not solve my problem at that moment so whenever you notice problems with the pulsecounter missing/adding pulses it is not an easy fix.

things i tried which did not solve the problem.
change the protocol(HTTP/MQTT)
change MQTT timing
sendover data directly from pulsecounter instead of rules
let rules be triggered by gpio pin instead of pulsecounter variable
changed wifi from eco to no sleep
IMG_7806.JPG
IMG_7806.JPG (777.95 KiB) Viewed 13897 times

Code: Select all

On System#Boot do     						// When the ESP boots, do
	TaskValueSet 3,1,0					// TaskValueSet TASKnr,VARnr,Value, Reset the Flow counter to 0
	TaskValueSet 3,2,0					// TaskValueSet TASKnr,VARnr,Value, Reset the PreviousFlow counter to 0
	if [Water#CounterTotal] = 0
	TaskValueSet 3,4,1836974    				// TaskValueSet TASKnr,VARnr,Value, set watermeter start value in case of power failure
	endif
	TimerSet,1,30      					// Set Timer 1 for the next event in 30 seconds
EndOn

On Clock#Time=All,00:10 do					// Reset Pulsecounter and LitersToday
	if [Water#Flow] = 0
		TaskValueset 3,3,[Water#CounterTotal]		 // set Water#CounterOffset to correct value at start of day. just for testing purposes
		resetpulsecounter,1
	endif
EndOn
 
On Watermeter#Count do						// When Pulse is detected
	if [Watermeter#Count] = 1 				// if pulse is received
		TaskValueSet 3,4,[Water#CounterTotal]+1  	// increase Water#CounterTotal with 1 liter
		SendToHTTP 192.168.1.50,8084,/json.htm?type=command&param=udevice&idx=344&nvalue=0&svalue=[Water#CounterTotal]
		TaskValueSet 3,1,60000/[Watermeter#Time]	// set Water#Flow
		SendToHTTP 192.168.1.50,8084,/json.htm?type=command&param=udevice&idx=338&nvalue=0&svalue=[Water#Flow]
	endif
EndOn

On Rules#Timer=1 do  						// When Timer 1 expires, do
	if [Water#Flow] > 0 or [Water#PreviousFlow] > 0	// Only send value if flow and PreviousFlow > 0
		SendToHTTP 192.168.1.50,8084,/json.htm?type=command&param=udevice&idx=338&nvalue=0&svalue=[Water#Flow]
		TaskValueSet 3,2,[Water#Flow]			// set Flow to PreviousFlow
		TaskValueSet 3,1,0
	endif
	TimerSet,1,30      					// Set Timer 1 for the next event in 30 seconds
Endon
Attachments
devices.JPG
devices.JPG (70.89 KiB) Viewed 13897 times

ArieKanarie987
New user
Posts: 3
Joined: 11 Mar 2019, 14:41

Re: tips to create a really stable pulsecounter for watermeter

#2 Post by ArieKanarie987 » 20 Apr 2020, 08:22

I have been using the 5V proximity sensor for over a year now, without issues... until it failed. It just stopped detecting the watermeter. This weekend I replaced the sensor with the 12V version NPN NO I had laying around. After some googling I found on Youtube the exact setup as you have in your scheme. Some minor thinking is required - as the image doesnot show if the sensor is NPN or PNP.

My setup: wemos d1 + power shield (12V supply), 12V to NPN sensor, black lead of the sensor to emitter section of PC817 via a resistor (PC817 out to gnd). Custom shield for 2DS18B20's and the 3.3V pull up / pin D5 connected to the collected section of the PC817. All installed in a triple Wemos D1 shield.

I have never (not in the 5V setup or now in the 12V setup) have encountered wrong/false readings (unless the wemos d1 was out of power). I do not use rules, just the sensor data to domoticz via ESPEASY every 300 seconds. This is a simple but effective solution for me.

I can share pictures of my current setup and the link to the easyeda for the custom sensor shield of the Wemos D1 - currently at work ;) .

Did you check the online time of your ESP ? Does it reboot "often" ? That can cause a loss of data, of perhaps count a liter double or so.
Why do you use rules ? Does it have extra functionality compared to the devices settings ?

I don't expect Domoticz to be an exact digital copy of my watermeter (just want to see if there is a trend / leakage), eventhough it is so far. - with the current settings.

BR
Arie

edit: fixed last sentence.
Last edited by ArieKanarie987 on 20 Apr 2020, 14:30, edited 1 time in total.

martinus
Normal user
Posts: 129
Joined: 15 Feb 2020, 16:57

Re: tips to create a really stable pulsecounter for watermeter

#3 Post by martinus » 20 Apr 2020, 12:03

Cheapest levelshifter i use: simple diode between output/input, cathode connected to sensor output.
Ouput active low will pull ESP input to ground, Active high 5V will be blocked by diode.

martinus
Normal user
Posts: 129
Joined: 15 Feb 2020, 16:57

Re: tips to create a really stable pulsecounter for watermeter

#4 Post by martinus » 20 Apr 2020, 12:09

Unfortunately, the debounce feature will only prevent consecutive pulses to be counted incorrectly. In case of a very short single noise pulse, it will still be counted.

Because of the ISR handling even a microsecond pulse will trigger the counter.

When high frequency noise is an issue (so not the typical switch debounce effect), one would need a low-pass filter. Could be done digitally, but maybe a simple RC filter is much easier to try. (series R, Cap to ground on the input)

GravityRZ
Normal user
Posts: 206
Joined: 23 Dec 2019, 21:24

Re: tips to create a really stable pulsecounter for watermeter

#5 Post by GravityRZ » 20 Apr 2020, 12:42

the esp did not reboot so it showed it was online for many days.

maybe it was my bad luck that i encountered multiple problems(maybe the 5V sensor is lesss sensitive then the 12v variant)


indeed a diode as shifter will work to.

one very nice feature i implemented last week and had 100% succes
i am reading the domoticz watermeter value back in case of powerloss.

last night we had a total powerloss in Alkmaar, result everything went down an 02:00 in the night.
when i woke up my clockradio was blinking so i knew about the powerloss

checked my domoticz settings, everything correct with the correct watermeter value
checked the esp, everthing correct with the coorect watermeter value.

if i would not have implemented that feature my watermeter in domoticz would have read 0 since i send over the complete value(to avoid missing pulses)

so the setup is really stable now and powerloss proof without the need for a RTC backup or flash memory.

@Arie, please do share those pictures

ArieKanarie987
New user
Posts: 3
Joined: 11 Mar 2019, 14:41

Re: tips to create a really stable pulsecounter for watermeter

#6 Post by ArieKanarie987 » 21 Apr 2020, 08:42

Main setup, I didnot want to put components directly on the tripler base... well because I prefer not to do so ;-)

Image

Stacked protoshield on tripler board, with custom sensor shield
Image

PC817 details. I could easily have connected to D6 on the proto shield, but I want to have all connections (sensors) on a top shield. Hence, the dupont cable to the top shield.
Image

custom sensor shield for wemos D1 - this design is easy to make yourself.
Image

Cheers, have a great day and stay home !
Arie

GravityRZ
Normal user
Posts: 206
Joined: 23 Dec 2019, 21:24

Re: tips to create a really stable pulsecounter for watermeter

#7 Post by GravityRZ » 21 Apr 2020, 09:35

nice setup Arie.

one last question?
do you send over only the pulses(liters) or the total?)

in the beginning i did send over the pulses only but since i was missing a pulse once in a while i wanted to rule out transmission problems.
now i send over the total everytime and he total gets synced back from domoticz

please note my watermeter is realtime so i see instant changes in counters and flow.
this is ofcourse more sensitive for missing data.

the sync back from domoticz was only intended so that i could onplug the esp, plug it back in and it still have the correct meter value.
i never would have thought it would work with a complete power failure (synology offline, power outage etc)
looked at the timing

at 1:55 the esp came online
at 1:56:18 mosquitto(synology package) came online
at 1:56:20 domoticz started up
at 1:56:25 domoticz connected to mqtt
at 1:56:40 domoticz received an mqtt broadcast to send out the watermeter.

looks very fast since a synology takes alot of time to bootup
the esp waits for the mqttconnected singnal before it sends out a command so timing apartently is always right

ArieKanarie987
New user
Posts: 3
Joined: 11 Mar 2019, 14:41

Re: tips to create a really stable pulsecounter for watermeter

#8 Post by ArieKanarie987 » 21 Apr 2020, 10:21

Hi,

I have the wemos send the pulses (=liters) every 300 seconds. I've configured Domoticz to have an incremental meter for water. The meter offset in Domoticz can be used to align the meter total counter to match the actual water meter.

I'm actually more interested in the water consumption profile during the day. The incremental meter in Domoticz stores data per hour, hence the 5 minute update frequency of the wemos. My synology & domoticz is online within 5 minutes after a powerup.

Your configuration sounds good - it looks like it requires specific configuration. I'm not a big fan of rules - so if I can stay away from using the rules I usually will.

I assume my wemos is out of power more often than my Synology NAS with Domoticz (for the pictures I unplugged the wemos for a moment, just to get the proto/pc817 photo). As with all my sensors it is still "under construction" and I want to change/improve/upgrade. Hence I rely on Domoticz for the totals and I would like to have my sensor configuration as simple as possible (too many spare wemos :oops: laying around - swapping them when I've changed something :lol: ).

GravityRZ
Normal user
Posts: 206
Joined: 23 Dec 2019, 21:24

Re: tips to create a really stable pulsecounter for watermeter

#9 Post by GravityRZ » 24 Apr 2020, 17:07

it all depends on what functionality you want.

i can understand that a default setup is much easier.
I however wanted to do something nobody did before.
real time watermeter usage
realtime flow usage
persistant watermeter value without the need to adjust the watermeter if something happens(power outage consistent)

i have a settings backup of the module and the rules script so in case of damage, replace the unit,restore settings and copy/paste the rules and eveything is working again.

now that i have a stable solution i am testing things like eco instead of wifi no sleep

i can change sendtohttp back to mqtt but i will not gain anything.

with the latest firmware we can even set the pulsecontroller to a certain value so i could do without the dumy devices but then i loose flow control

Jieffe
New user
Posts: 6
Joined: 02 May 2021, 22:17

Re: tips to create a really stable pulsecounter for watermeter

#10 Post by Jieffe » 30 Jun 2021, 17:35

A big Thank You for your tutorial !

Until now, I had some false reading (a few litres here and there, especially at night, making me suspect a leak).

I applied some (not all) of your instructions and my counter is now spot on for more than a week !

The recommendations I applied were the 1 second debounce time and the opto-isolation of the sensor (I think the latter played the biggest role in the increased precision as I unsuccessfully played with the debounce time previously).

Didn't need the magnet or the use of scripts.

Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot] and 34 guests