Counting time, calculate and display?

[M*I*B]

Nah, must have been something we're missing here or else... you may have found a bug?

I think it is just missing documentation here

Assuming you didn't actually use GPIO-8 as that is connected to the flash

Hey, let me find a bug pls, let me

No, I have checked every single port from 3 to 14... Doesn't matter...
Just comes in mind: Maybe the option "don't wake up the display when writing to it" don't work right? I will test that and stop the writing... I come back with the result shortly...

EDIT:
Yes! That's the problem! Disabling "Wake display on receiving text:" has absolutely no effect...

[TD-er]

Congratulations, you found something that hasn't been documented... or is a bug.
Ton has to decide which one applies and fix it

[M*I*B]

HAH... Day is saved

[M*I*B]

Before I go out to repair my excavator, I thought about something that will make or break the whole project...

The data collected, at least the summit data, must somehow be saved and accessed absolutely safely, even after a power failure or even destruction of the ESP.
The original idea with the cache things actually falls out. Works in the event of a power failure, but not in the event of destruction.
What about an SD card? Could I recharge the previously stored data into the RAM after a power failure?
The SD could still be read out if the ESP is destroyed. Except the hut burns off, but then it doesn't matter anyway ...

[TD-er]

You can have a second ESP connected via p2p which can also store the data.
This one doesn't need to be close and you can even distribute the time via p2p so the GPS can be mounted elsewhere.

Not sure how bullet proof it needs to be, but to me it sounds a lot more reliable compared to a SD card which gets written to a lot.
And if you need to do forensics to retrieve the data, then a flash chip is probably easier to extract data from.
Just assuming it isn't embedded in the ESP chip itself.

[M*I*B]

... excavator is running again, finger jammed, car moved with excavator, lawn mower repaired... Closing time...

Back to the topic:
A second ESP for redundancy might not be a bad idea. Then it has to be on the same circuit so that under no circumstances can only the counting ESP be switched off.
However, it can still happen that both die at the same time, e.g. in the event of overvoltage (lightning strike, power failure) in the network.
I'll keep that in the back of my mind.
While I was working outside, I had another idea: On the one hand, an SD card might not be such a bad idea after all, since only then a single data set is appended to the log file on it (one file per day). On the other hand, this writing doesn't happen all the time, but a maximum of 4 times an hour in the winter when it's icy. And the card is changed once a year anyway.
And as additional redundancy, I could stick a purely mechanical pulse counter onto the board. It would have to have at least 7 digits, but the little Asian have something for little money and with 5V control.
And if I then want to add additional redundancy, I could use a 1sec clock generator (clock module) as a source for the mechanical counter in case the ESP fails, take over control via classical logic gates..

[TD-er]

Hmm reminds me of the time I built a MythTV HTPC computer with 4 TV tuner cards as a VCR back in 2005 I think.
I guess scheduling downtime on the main control PC of a nuclear power plant was easier than scheduling downtime on that PC.

I get the same vibe from this setup.

[M*I*B]

Well... In Germany this is now the order of the day after the green dirt shuts down one power plant after the other. Two digital three-phase meters died at the site last year due to network problems; They were both less than a year old. Then it caused real issues at the end of the year. But even before that, we often had problems with the power grid at the location. From strong fluctuations, short dips, individual phases to massive surges...
So my paranoia is not entirely unfounded. And experience also shows that you can often read mechanical meters even after a fire... That's something...

[TD-er]

Yep I know all about "smart" meters dying.
Over here the electricity grid company pushed out the wrong update to the meters here.
So all meters in our street had to be replaced.
This was mid october last year (meters were less than 2 years old).
I had proof of my meter counts.
The grid company made an estimate, which were the exact number of April 1st that year.
So they would have robbed me of over 2000 kWh of net feed in from my solar panels which I can deduct from the consumption in the winter months.
Then the Electricity company (separate companies for the grid and the ones who bill you the energy) agreed with my estimate but they wanted to send out the final bill of the year so I would hardly get anything for my negative feed-in kWhs and had to pay full price (0,40 euro per kWh) during the winter months.

Of course I made it very clear to them I would not like to be robbed due to a fault of someone else.
Took about 4 months to get it straightened out, but I got it reimbursed.

Anyway, maybe not a bad idea to have a mechanical meter in place
I guess it would make a stronger point compared to some piece of breadboard with wires sticking out and some software anyone could have written.

[M*I*B]

I understand you very well. I had similar experiences with the “energy rip-offs”.
In the meantime, I have decided not to sell my PV energy anymore. I still have an old contract until '27 that guarantees me 33.5C/kW (before tax, of course), but the purchase price is already over that. So I took the system off the grid and use the energy myself and currently feed excess energy into the heating/hot water.
Fortunately, I have so far been able to resist the installation of SmartMeters that communicate with the outside world in some way. That won't happen either. If necessary, we can now work completely self-sufficiently with a self-made diesel generator (8kW straight, 10kW short) to which I can connect the PV. The 5 day test run was only a month ago. Except for a few little things, it actually worked well...
Let's hope that the so-called red/green - dirty "traffic light" government here in Germany will soon be a thing of the past. Otherwise I see pretty black...

Well, let's leave it alone, otherwise I'll get a big neck again


I'm going to get some sleep. When you're over 60, a day like this is a bit tiring...

Good N8

[M*I*B]

Good morning all together,

In the meantime I have gotten around the problem with the display using rules. Not nice, but it works for now...
But for today I have "rebuild my 3D printer station to create a better WAF" on my note. And if the weather stays dry, I'll probably have to spend a few hours mowing the lawn before it grows over the house. Also not positive for the WAF

I think I can add a new variable to the GPS Plugin, providing that Fix value, it seems useful...

It is.... It is
While you're at it, maybe you can also take a short look at the display-sleep issue. Possibly it's just a small thing why this option is ignored...

[Ath]

I'm going to get some sleep. When you're over 60, a day like this is a bit tiring...

I can confirm that

While you're at it, maybe you can also take a short look at the display-sleep issue. Possibly it's just a small thing why this option is ignored...

This part is of bigger importance IMHO, and I've been looking at that already, but the code is 'somewhat confusing' to say the least, so no solution yet

[M*I*B]

A new problem that kills the whole project:

I've had two identical test systems running on breadboard for the past few days, completely identical in structure with identical display, GPS, firmware, rules, devices and so on.
Both systems are individually connected to a power bank, which are charged by a central power supply in order to avoid influences from the AC network.

With both systems I have noticed over the days that sometimes system 1 and sometimes system 2 generate a reboot completely unmotivated for no apparent reason, at least if you can trust the information in "Uptime". System 1 must have done this at some point last night (uptime: 11h37m) and just now System 2 did exactly that, so that I could see it out of the corner of my eye, which is the trigger for my text here.
And at the moment they don't do anything except writing fixed dummy data to the display every second. Of course, this is completely unusable for a system that is supposed to record oil consumption down to the second...

Is the problem known? Are there any possible solutions?

[TD-er]

Almost all powerbanks do turn off power every now and then to see if the device is still connected.

[M*I*B]

... nope, not this kind of PB: DIY

[TD-er]

What was the reboot reason and last action before reboot? (can be seen on sysinfo page)

[Ath]

What exact build are your ESP32-S3 running? If not on the build from https://td-er.nl/espeasy/latest (20240412 or later) you may have issues caused by wonky USBCDC software, that's finally fixed in the latest build (is merged yesterday into "mega")

[M*I*B]

This one on both:

Code: Select all

Build:⋄	20240331 - Mega32-s3
System Libraries:⋄	ESP32 SDK 5.1.3.240313
Git Build:⋄	mega-20240401
Plugin Count:⋄	147 ["Normal","MAX ESP32"]
Build Origin:	GitHub Actions
Build Time:⋄	Mar 31 2024 22:43:10
Binary Filename:⋄	ESP_Easy_mega_20240331_max_ESP32s3_16M8M_LittleFS_OPI_PSRAM_CDC_ETH
Build Platform:⋄	Linux-6.5.0-1016-azure-x86_64-with-glibc2.35
Git HEAD:⋄	HEAD_b5af85e

[TD-er]

Can you try the builds from here: https://td-er.nl/ESPEasy/latest/
Either flashed via serial via your browser, or you download the all.zip file and pick your bin (without "factory" in the name) and update firmware via the tools page.

[M*I*B]

Ok, will do so tomorrow and report then... Ty

[M*I*B]

... forget to ask: Any way to completely disable IPv6 (Stack)???

[Ath]

... forget to ask: Any way to completely disable IPv6 (Stack)???

Currently only as a compile-time option, and it's only enabled in a few builds, f.e. the MAX builds.
Why would that be needed, it is interfering with something?

[M*I*B]

I can't quite understand it yet, but sometimes IPv6-capable devices try to get a v6 from the DHCP instead of the v4 and then of course fail because v6 is deactivated in the DHCP (ISC DHCPd version 4.4.1). In my opinion, v6 makes absolutely no sense in the LAN unless you are running a server farm...

[TD-er]

ESPEasy isn't actually using IPv6.
It does have it present as a testing feature and the webserver is accepting requests via IPv6 also.
It might use IPv6 only if you're addressing something via a hostname which only has an AAAA record and no A record.

And the reason we will need IPv6 in the near future is to support Matter.

ESPEasy will first get an IPv4 address from DHCP and then a few seconds later a link-local IPv6 address via Neighbor Discovery.
A few seconds later it may receive a global scope IPv6 address via Router Discovery.
ESPEasy doesn't use DHCP6

[TD-er]

I made a new test build which has the option to disable IPv6 on the tools->Advanced page
See: https://td-er.nl/ESPEasy/latest/

[M*I*B]

thumbs-up-smiley-36.png (7.85 KiB) Viewed 792 times

[TD-er]

Does it fix stuff at your setup?

[M*I*B]

... I let you know tomorrow (Monday) evening

[M*I*B]

... yesterday evening I quickly installed the ...
- ESP_Easy_mega_20240412_max_ESP32s3_16M8M_LittleFS_OPI_PSRAM_CDC_ETH
and now just wanted to quickly install the
- ESP_Easy_mega_20240414_max_ESP32s3_16M8M_LittleFS_OPI_PSRAM_CDC_ETH
which was released shortly afterwards...
Doesn't work, at least not via the GUI (/Tools/Update). ESP stops and at some point the browser times out... Reboot reactivate the ESP so nothing is written...

And on this occasion: in the 20240412 the option to switch off v6 is not available...

EDIT:
Ok, now that was completely messed up!
I tried to transfer the 0414 with the FDT. It worked so far without any errors. Unfortunately, the ESP is now dead and no longer even outputs data to the terminal. Even downgrading to 0412 doesn't change anything...

Could it be that the ESP_Easy_mega_20240414_max_ESP32s3_16M8M_LittleFS_OPI_PSRAM_CDC_ETH.bin is defective?!?

[TD-er]

Sometimes an OTA update fails due to timeouts.
Temporary disable some tasks ("taskdisable,N" without save) will often help to successfully update the firmware.

Transfer with the "FTD" is via serial?
If via serial, you need the bin file with "factory" in the name as this also includes the bootloader.

[M*I*B]

Jepp, I have try it many times via OTA. Wonder why it don't like due yesterday with the 0412 it goes on 1st try without hassle...

Just some minutes ago I have flash it the 3rd times with the factory via serial. Now the ESP is reincarnated and talk to me again *puhhh* I have no idea what ghost like to kick my ass here...

OK, have to run now...

[TD-er]

... I let you know tomorrow (Monday) evening

And??? What is the verdict of the build with option to disable IPv6?

[M*I*B]

ahhh sorry ... Alzheimer... You know?! Was busy with my repetier server / crashed SSD and forget to report...

1st: Switch off v6 seems to work so far. I haven't noticed any v6 trash at the DHCPd from any ESP's and the ACL to the WLAN feels like it goes faster than before...
--- good job !

2nd: Since I have solved the flash issue and running the last build, I haven't noticed any reboot yet. Both run smart greased; like that
--- also good job !

If ATH sometimes fix the display-sleep issue then I can go future (ok, that was a wink with the whole fence *lol*)

[TD-er]

Well at least your setup had longer time to get tested, so no big deal
It has been merged so in a next build it will be included.

[M*I*B]

Great!
I let it run future and go back to my Repetier fu**. Have to print some and can't. That sucks... My gut feeling about the durability and reliability of SSD storage devices did not deceive me. Maybe I should install a 2.5" WD-RED instead of an SSD...

[TD-er]

Well I'm not that fond of how WD labels their drives.

A few weeks ago I was at a friend's house and he mentioned he had lots of issues with his NAS.
Like every week or so his RAID array was rebuilding and he already had replaced all his drives at least more than once.

The drives were WD RED (not sure about the color name), but more importantly they are labelled like designed for NAS use.

Now the really bad part...
Those were SMR drives.
So the tracks are "shingled" over eachother like roof tile shingles.
Meaning that if you have to change something, you need to read the entire block of N shingled tracks, convert them in memory and write them back as you can't change anything inside such a block without erasing other data.

I really hate the idea of class action lawsuits as they are used for just about anything, but this might be some use case where I think each and everyone should join class action lawsuits to make it very clear to hard drive vendors they should not mess around with user data by selling something which is absolutely not usable for what they market it for (WD is not alone in this)


Back to the SSD and reliability...
Just make sure not to buy the "cheapest" ones, avoid QLC flash and I always buy drives which have some power backup capacitors so they can store the last journal of changes when power goes down.
Those are harder to find for NVME drives, but for SATA drives they are easy to find.

And I have been buying those "enterprise" drives 2nd hand.
Usualy you should not (ever!) buy second hand drives from some unknown person as they might have been used for those crypto-like mining things where you get rewarded for storage space.
But those will not likely be using expensive enterprise drives.
So I have like several Samsung PM863a and Intel D3-S4510 drives of 2 or 4 TB with less than 100 TB written.


N.B. some SSDs also have had firmware issues.
Samsung had some where the drives age way too fast and Intel had an issue where the drive suddenly only claimed to have 8 MByte without ever being able to restore data.

[M*I*B]

As far as I know, the WD-RED SMR is only used on the very large plates.
I only installed WD-RED in my 16-port NAS, but only "small" ones with 4tb and 8tb. They have been running 24/7 without any problems for years.
In the middle of last year, a 4tb of them gave a SMART warning; nothing wild. So I left them in until the beginning of this year and didn't replace them until January. There were no additional errors added, so it would certainly have lasted another year.

My experiences with SSDs (SATA) are somewhat different. I didn't buy the cheapest ones, but by far not the most expensive ones either. Mostly Samsung or Sandisk. However, my experience is that defects in SSDs often have catastrophic consequences immediately, often without prior notice in SMART or elsewhere. With mechanical records, on the other hand, things like this happen slowly, so you still have time to react. It was the same here with the Repetier SSD: from now on it was as good as dead.

The small 2.5" WD-RED with 2tb does it for now, but it puts quite a lot of strain on the Raspberry in terms of power requirements. In addition, it is actually far too large in terms of memory. That's just the disadvantage of mechanical disks. I have this morning I bought a Fanxiang 256gb SSD. Let's see what it's like. The reviews are at least promising.

[TD-er]

As far as I know, the WD-RED SMR is only used on the very large plates.
[...]

Nope, those were "just" 8 TB ones and are still being sold (at least 1 month ago)

[M*I*B]

hmmm... My RED's are a lot older, the last ones I bought are probably 2 years old. And I've never had an unplanned rebuild... It may also depend on the NAS OS used; I use OMV

[TD-er]

Nah if you put SMR drives in some kind of RAID configuration you will end up loosing data.
Regardless of your NAS OS.

[M*I*B]

... so then I haven't any of them...

[TD-er]

You can often also hear the difference as they really don't sound healthy.

[M*I*B]

... then that is the 2nd mark that I haven't any of them. They all very quiet and also if the fans are in idle you can't hear any of them...

BTW: ESP #2 rebooted completely unmotivated again 15h05m ago...

[M*I*B]

Ok, time to make a decision...
I think I will stop the project. There are simply far too many instabilities, such as the occasional reboots and the constant Wi-Fi drops, which also affects the AP mode. Intervention by deleting it with "erasesdkwifi" and resetting the access data via the terminal doesn't help...
I have no idea whether this is a general hardware problem with the ESP's or a problem with the WLAN stack. But it makes no difference in the result.

All in all, ESP's are a great thing and so is your software. But the ESP's are obviously not usable for tasks like this where reliability is important...

I would see if there is a chip that can convert a GPS signal into a precise second cycle or use an Atmel for it (two redundant in parallel to classic Logic modules) and then count completely electromechanically.

[TD-er]

Turning the GPS in an exact pulse every second is quite easy when your GPS module has a PPS pin.
This pin will generate a pulse exactly at the start of a second... when it has a fix.
Depending on the GPS module, the behavior of this PPS pin may differ.
For example not outputting a PPS pulse, or at a lower (or at least significantly different) frequency.

[M*I*B]

... this GPS I have buy haven't a "tick" output. SO maybe a have to take a look around where to find one with it... Many thx for the hint...

Meanwhile, I try to run two ESP's as redundancy for each other as parallel project. The serial data from GPS goes via logic gate to both and I try to generate a "I'm alive" tick on a GPIO for the other ESP and vice versa. The display is also planned to work on both via logic gates so that the active ESP drive the display.
At the moment it's on hold due I'm searching for my big breadboard... shit Alzheimer

[TD-er]

Boards which do not have the PPS pin exposed, often do have some LED which is essentially internally connected to this pin.
I know the BN220 for example does have some (blue?) LED which is connected to the PPS pin.

In ESPEasy I do keep track of when the first message in a burst was received as this also coincides with the start of the second.
Meaning whenever I see the start of the line, I look at how many characters are left in the serial buffer to see how much time has passed since (taking the baud rate into account). This appears to be quite an accurate way of syncing to the exact second.
So far I have only seen GPS modules which do send all NMEA messages in a burst, only at the start of a second.

[M*I*B]

Boards which do not have the PPS pin exposed, often do have some LED which is essentially internally connected to this pin.

You are right! Why haven't noticed that by myself? (headbanging on table corner...)