Automatic backup of settings and rules?

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
User avatar
Andrew Mamohin
Normal user
Posts: 102
Joined: 12 Jun 2020, 08:30
Location: Belarus, Gomel
Contact:

Automatic backup of settings and rules?

#1 Post by Andrew Mamohin » 09 Oct 2023, 08:20

I have several ESP32 devices with EspEasy firmware. Sometimes I change the settings and rules. And I forget it. I would like to make an automatic copy of settings and rules from all such devices on my server. Maybe someone did this? Is it possible to take settings and rules from EspEasy with a command from the server?
Andy.

User avatar
Ath
Normal user
Posts: 3918
Joined: 10 Jun 2018, 12:06
Location: NL

Re: Automatic backup of settings and rules?

#2 Post by Ath » 09 Oct 2023, 09:07

There are several scripts floating around for doing that, f.e.:
This GH Issue thread suggests a script for updating, but also backing up, a series of ESPEasy units. I'm sure you can use that as an inspiration/starting point.

And next to that I've created PR #4777 to backup all configuration, or all files, of an ESPEasy unit in a .tar file.
/Ton (PayPal.me)

User avatar
Andrew Mamohin
Normal user
Posts: 102
Joined: 12 Jun 2020, 08:30
Location: Belarus, Gomel
Contact:

Re: Automatic backup of settings and rules?

#3 Post by Andrew Mamohin » 09 Oct 2023, 10:05

The script does not backup settings and rules. I'm not interested in updating the firmware.
Has PR #4777 not yet been implemented?
Currently there is no URL to download settings and rules?
I would make a script with searching through all addresses myself.
Andy.

User avatar
Ath
Normal user
Posts: 3918
Joined: 10 Jun 2018, 12:06
Location: NL

Re: Automatic backup of settings and rules?

#4 Post by Ath » 09 Oct 2023, 10:07

Andrew Mamohin wrote: 09 Oct 2023, 10:05 The script does not backup settings and rules. I'm not interested in updating the firmware.
A later version of the script, in one of the comments, does include that backup feature AFAICS, you can extend that to include rules1.txt .. rules4.txt.
/Ton (PayPal.me)

TD-er
Core team member
Posts: 9221
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Automatic backup of settings and rules?

#5 Post by TD-er » 09 Oct 2023, 10:08

You can just right-click on any file on the file browser page and use that URL via some tools like wget.

User avatar
Ath
Normal user
Posts: 3918
Joined: 10 Jun 2018, 12:06
Location: NL

Re: Automatic backup of settings and rules?

#6 Post by Ath » 09 Oct 2023, 10:13

Andrew Mamohin wrote: 09 Oct 2023, 10:05 Has PR #4777 not yet been implemented?
The latest available build with that feature is available from this GH Actions run, so you could help with testing that feature :D
/Ton (PayPal.me)

User avatar
Andrew Mamohin
Normal user
Posts: 102
Joined: 12 Jun 2020, 08:30
Location: Belarus, Gomel
Contact:

Re: Automatic backup of settings and rules?

#7 Post by Andrew Mamohin » 09 Oct 2023, 10:14

TD-er wrote: 09 Oct 2023, 10:08 You can just right-click
Wow! I didn't know about this possibility. But the config that I saved in this way is different from the config that I uploaded through the button in the web interface.

Code: Select all

09.10.2023  11:10           131 275 config.dat
09.10.2023  11:11           131 072 config_ZAL_U45_Build20230508_20231009111120.dat
Andy.

User avatar
Andrew Mamohin
Normal user
Posts: 102
Joined: 12 Jun 2020, 08:30
Location: Belarus, Gomel
Contact:

Re: Automatic backup of settings and rules?

#8 Post by Andrew Mamohin » 09 Oct 2023, 10:20

Sorry! This is strange behavior of wget for windows. The correct file is downloaded on the debian server.
Thank you, now I will make an automatic backup, just like I wanted!
Andy.

TD-er
Core team member
Posts: 9221
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Automatic backup of settings and rules?

#9 Post by TD-er » 09 Oct 2023, 10:20

If a file gets saved by ESPEasy, it will update the settings if the settings file was from a previous version.

So this will actually alter the file contents.

I have no idea why the file should be larger than a 128k, which is 131072 bytes.

User avatar
Andrew Mamohin
Normal user
Posts: 102
Joined: 12 Jun 2020, 08:30
Location: Belarus, Gomel
Contact:

Re: Automatic backup of settings and rules?

#10 Post by Andrew Mamohin » 09 Oct 2023, 12:56

I made a script to backup my modules (it turns out there are already 12 of them). Now backups are created every night and stored for a month. Thanks for the advice!
Andy.

TD-er
Core team member
Posts: 9221
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Automatic backup of settings and rules?

#11 Post by TD-er » 09 Oct 2023, 13:07

Maybe you can also consider to have the flow of data in the opposite direction.
There is already experimental support in ESPEasy for provisioning.
This means you can let ESPEasy fetch files from a HTTP server.

Just to give you some ideas, this is how I do it for some nodes.
Each node does have some specific settings like calibration data on ADC and the unit nr.
So Rules Set 1 is generic for all nodes and rules set 2 contains node specific info.
Rules set 3 contains specific info on how data can be provisioned.

At system boot I call this event: Event,LoadCalibration

Code: Select all


On LoadCalibration Do
  // Test units @Gijs
  if %cpu_id% = 0x1280E0
    config,task,bat,SetCalib,TwoPoint,392,13.9846,571,19.9924
    config,task,batBackup,SetCalib,TwoPoint,1785,10.000
    unit,2
  // Test units @Gijs
  elseif %cpu_id% = 0x0C17E4
    config,task,bat,SetCalib,TwoPoint,370.8615,13,520.857,18
    config,task,batBackup,SetCalib,TwoPoint,1785,10.000
    unit,3
    
  endif
  Save
EndOn
Rules3.txt:

Code: Select all

// Last edit: 2023-04-07 14:05

On updateSettings Do
  provisionprovision
  provisionconfig
Endon

On updateCredentials Do
  provisionsecurity  
Endon

On updateRules Do
  provisionrules,1
  provisionrules,2
  provisionrules,3
Endon

On updateRulesSettings Do
  AsyncEvent,updateSettings
  AsyncEvent,updateRules
  Reboot
Endon

// e.g.
// event,PerformFirmwareUpdate=firmware_max_ESP32_16M8M_LittleFS.bin
On PerformFirmwareUpdate=* Do
  pwm,2,100,0,8
  provisionfirmware,%eventvalue1%
Endon

On provisionfirmware#success=* Do
  gpio,2,0
  Reboot
Endon

On provisionfirmware#failure Do
  gpio,2,0
  Reboot  
Endon
You need to have provisioning support enabled in your build (it might be already present in the max builds, not 100% sure)
On the Settings Archive page you then get some extra checkboxes where you configure what is allowed to be fetched and what not and where it can be fetched from.

As you can see, you can even trigger commands via for example MQTT to update the firmware of a node.

User avatar
Andrew Mamohin
Normal user
Posts: 102
Joined: 12 Jun 2020, 08:30
Location: Belarus, Gomel
Contact:

Re: Automatic backup of settings and rules?

#12 Post by Andrew Mamohin » 09 Oct 2023, 15:09

TD-er wrote: 09 Oct 2023, 13:07 As you can see, you can even trigger commands via for example MQTT to update the firmware of a node.
This is all very interesting, thank you!
But I will never update the firmware automatically.
Andy.

TD-er
Core team member
Posts: 9221
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Automatic backup of settings and rules?

#13 Post by TD-er » 09 Oct 2023, 16:19

Not automatically, but via a command.
So it isn't a periodical pull, but more like a command triggered pull.
For ESP32 (and variants) there are quite a few safety checks built in as I have had some issues while remote updating nodes and those finally rebooted back into the old firmware. (no guarantees)
Those were some nerve wrecking minutes waiting till they came back online.

And it is also quite practical for initial deployment.
Someone I know is using an initial image of the entire flash (with some basic settings) to flash newly manufactured units and these then pull the full configuration and latest build from a local webserver.
This way he has pre-programmed all ESPs with a single image and then only needs to power them up and they do automatically fetch their own config and he also has some extra rules included to perform a HTTP call with the MAC address to "register" the device and let his deployment script generate a new rules file with some new parameters set based on that MAC address.

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 2 guests