ESP Easy Webdumper

[manjh]

The version I downloaded does not support the parameter, and the blue command window is the same size as I remember it.
So I think I donot have the latest version...

0.00.007 is not in releases only in the code section.

Ah, that explains it.
For speed of testing, could you send the exe to me via mail? You have my mail address...

[grovkillen]

True! Will make a check in the next release. Thanks

[grovkillen]

The version I downloaded does not support the parameter, and the blue command window is the same size as I remember it.
So I think I donot have the latest version...

0.00.007 is not in releases only in the code section.

Ah, that explains it.
For speed of testing, could you send the exe to me via mail? You have my mail address...

You can download the exe from the code section. The APP_DATA.zip is unchanged so just replace the exe.

[manjh]

The version I downloaded does not support the parameter, and the blue command window is the same size as I remember it.
So I think I donot have the latest version...

0.00.007 is not in releases only in the code section.

Ah, that explains it.
For speed of testing, could you send the exe to me via mail? You have my mail address...

Never mind, got it.....

[manjh]

OK, some quick (unstructured) test results.

1. when I start the tool it comes up with the default IP address. I changed that to one of mine and ran it, then closed the tool and restarted. The IP address is again your default (so my last used address is not remembered).
2. when I start the tool in batch mode with an IP address as parameter, it runs the tool but it fails to select all options. Result: no much is downloaded...
3. unit name and number is not found, I tried on build 147 and also on the Mega release.
4. output files are not clear to me, I tried to look at a few but I will need more time to study the structure. I was expecting simple screenshot files. But this could be my mistake, perhaps I opened the files in the wrong way?

[grovkillen]

You need to save the settings. Batch mode will use these default settings.

Regarding the output files and unit name. These should be straight on screen shots and the name and number should be found. If not that tells me you didn't have the correct IP number? Can you confirm that the IP is that of a ESP Easy unit?

[manjh]

You need to save the settings. Batch mode will use these default settings.

Regarding the output files and unit name. These should be straight on screen shots and the name and number should be found. If not that tells me you didn't have the correct IP number? Can you confirm that the IP is that of a ESP Easy unit?

I tried on a few different ESP's. In fact, am running one now: it complains about unit name/number, and then proceeds with the other screens with no problems.
I will compress a few of the outputs and send them to you.

One question stays in the back of my mind. When I browse into the ESP unit, it asks me to log on with userid/password (I configured it like that). How does your Web dumper pull the information out of the ESP's without knowledge of those credentials?

[manjh]

You need to save the settings. Batch mode will use these default settings.

Tested and confirmed.... works great!

[grovkillen]

One question stays in the back of my mind. When I browse into the ESP unit, it asks me to log on with userid/password (I configured it like that). How does your Web dumper pull the information out of the ESP's without knowledge of those credentials?

Whoooops! That's probably it. I guess some pages are not protected by the password (JSON, SDCARD, CONFIG etc).

Thanks for that input, and we need to go to the bottom on that one since downloading the config will let you clone a unit and get onto the network fairly easy. Lets try all the pages and file the findings to GitHub.

[manjh]

One question stays in the back of my mind. When I browse into the ESP unit, it asks me to log on with userid/password (I configured it like that). How does your Web dumper pull the information out of the ESP's without knowledge of those credentials?

Whoooops! That's probably it. I guess some pages are not protected by the password (JSON, SDCARD, CONFIG etc).

Thanks for that input, and we need to go to the bottom on that one since downloading the config will let you clone a unit and get onto the network fairly easy. Lets try all the pages and file the findings to GitHub.

I would expect that all pages are protected by the login credentials. If any of them are not, that is a security gap...

For the tool it can be solved by adding two fields (userid/password) and store them in the settings.

[grovkillen]

Yes I will add that. Thanks for the information.

Would you be able to test all the web pages manually?

[manjh]

Not really test findings, but more usability suggestions:

1. change the name of the tool (especially the exe file!) to "ESPDumper". It covers the nature of the tool and is easier.
2. when the tool is called in batch mode (so when the parameter is provided), run in "quiet mode". So donot create the popup window.

Reason for this second suggestion: I am now testing with a bat file, pulling info from five units (to start with). I can run this in backhround and do other things in Windows. But each time a new unit is started, I get that popup window taking over the focus... annoying!

[grovkillen]

Not really test findings, but more usability suggestions:

1. change the name of the tool (especially the exe file!) to "ESPDumper". It covers the nature of the tool and is easier.
2. when the tool is called in batch mode (so when the parameter is provided), run in "quiet mode". So donot create the popup window.

Reason for this second suggestion: I am now testing with a bat file, pulling info from five units (to start with). I can run this in backhround and do other things in Windows. But each time a new unit is started, I get that popup window taking over the focus... annoying!

1. I might add the possibility to rename the actual file but the name stands
2. I will add a /HEADLESS_MODE=1 switch for the command line. I totally understand you there

[manjh]

Yes I will add that. Thanks for the information.

Would you be able to test all the web pages manually?

I will see what I can do.

It seems you are depending on release level. Perhaps we should consider taking another route?
I can see three ways.
First: simply go along all possible page names and pull the complete page out as screenshot. I suspect you were already following this route.

Second, include a special interface in the next release, where the ESP unit simply spits out each and every data element it contains, identified by a data element key.
I realize this is not simple to do, and it may take a bit of storage that we don't have.

Third, again starting with the next release, have ESPEasy dump the entire content of the storage area that is used to store variables, along with the built number to identify how data is stored. Then in the ESPDump tool implement the intelligence to decypher the data area and create nice and readable output from that. Essentially the same as the second option, except that it will not take as much storage in the ESP unit.

Both second and third way has the downside that it will not work until the next release.
Taking screenshots will always work.

[manjh]

Not really test findings, but more usability suggestions:

1. change the name of the tool (especially the exe file!) to "ESPDumper". It covers the nature of the tool and is easier.
2. when the tool is called in batch mode (so when the parameter is provided), run in "quiet mode". So donot create the popup window.

Reason for this second suggestion: I am now testing with a bat file, pulling info from five units (to start with). I can run this in backhround and do other things in Windows. But each time a new unit is started, I get that popup window taking over the focus... annoying!

1. I might add the possibility to rename the actual file but the name stands
2. I will add a /HEADLESS_MODE=1 switch for the command line. I totally understand you there

Name is only relevant for the exe file.
As for the flag, I would suggest something that reflects the purpose, such as /QUIET_MODE,
But why not take quiet mode as default when running with IP as parameter?

[grovkillen]

Not really test findings, but more usability suggestions:

1. change the name of the tool (especially the exe file!) to "ESPDumper". It covers the nature of the tool and is easier.
2. when the tool is called in batch mode (so when the parameter is provided), run in "quiet mode". So donot create the popup window.

Reason for this second suggestion: I am now testing with a bat file, pulling info from five units (to start with). I can run this in backhround and do other things in Windows. But each time a new unit is started, I get that popup window taking over the focus... annoying!

1. I might add the possibility to rename the actual file but the name stands
2. I will add a /HEADLESS_MODE=1 switch for the command line. I totally understand you there

Name is only relevant for the exe file.
As for the flag, I would suggest something that reflects the purpose, such as /QUIET_MODE,
But why not take quiet mode as default when running with IP as parameter?

Good idea.

[grovkillen]

https://github.com/Grovkillen/ESP_Easy_ ... g/0.00.010

New release is out

[rol-rider]

Hello,
sorry for my limited english (google translate)
thank you for this work.

I wanted to test!
but I can not find the screenshot?
the LOG folder does not contain a screenshot. is this normal?
I wonder if I understood how it works?

CaptureLog.PNG (69.56 KiB) Viewed 24874 times

I only have 3 files in the LOG folder

I use windows 7 (64bits)

thank you

[grovkillen]

You need APP_DATA.zip to be in the same folder as the exe file.

I will add this as a warning. Thanks for the feedback

[grovkillen]

I have now updated with a new release.

https://github.com/Grovkillen/ESP_Easy_ ... r/releases

[rol-rider]

thank you for the answer
however, he is in the same folder

CaptureDossier.PNG (31.18 KiB) Viewed 8280 times

[grovkillen]

thank you for the answer
however, he is in the same folder
CaptureDossier.PNG

Do you mind testing current version 0.00.012 (found here): https://github.com/Grovkillen/ESP_Easy_ ... dumper.exe

[rol-rider]

Yessssssssss 0.00.012 works
It works. Still thank you.
I'm going to take advantage of the French jeedom

[grovkillen]

Great news

Thanks for the feedback.

[rol-rider]

Hello,
would it be possible to decrease the size of the files.
with webdumper .... 2.41M !
with capture tool .... 41K, better for forums.
thank

[grovkillen]

Hello,
would it be possible to decrease the size of the files.
with webdumper .... 2.41M !
with capture tool .... 41K, better for forums.
thank

I'm gonna look into it. I make uncompressed pictures, will fix it so they are compressed by default.

[Oxyandy]

Hi Grovkillen,
An idea, the total time taken !
This could be used as a measure of web-server performance to measure differences in firmwares
Cheers

[grovkillen]

Sure, that can be added.

[Oxyandy]

Excellent, I was awake for over 36 hours configuring my 'test environment' - I have an older firmware which has not yet failed,
so I have a base line, a few hours 'power sleep' and I am back.
Only 2 firmware versions higher & I see 'bad behaviour' - very pleased, but so much more to do before I track down the exact cause.
If those performance results could go into a file with the firmware version & time taken, that would save 'manual' effort keeping track.

Time to dump the current failing firmware & OTA update the next.

[grovkillen]

0.01.002 is now available. Fixed problem with not being able to fetch the unit name an number.
https://github.com/Grovkillen/ESP_Easy_ ... r/releases