json or xml parser feature request

[chromo23]

I am using espeasy for quite a long time now on many devices and since now i had no need for a broker/gateway/server.
I want to keep it that way. Not because i am unable to do so but i like it simple.
One thing i am missing is the possibility to receive data from somewhere far away. Like weather data from a weather service oder from another station.
Therefore the simplest solution in my opinion would be a simple json or xml parser.
Is it something other people need too and therefore worth implementing ?

[Ath]

Is this PR, at least partially, what you have in mind? https://github.com/letscontrolit/ESPEasy/pull/3424

Biggest challenge is the lack of available memory on the ESP8266 units, IMHO, that makes it very hard when receiving large(r) messages.

[chromo23]

I am no expert but requesting simple json messages doesn´t seem like a big memory eater.


Example output from a thingspeak channel.

Code: Select all

{"created_at":"2021-04-21T07:38:11Z","entry_id":36611,"field1":null,"field2":null,"field3":"19.11","field4":"55.37","field5":null,"field6":null,"field7":"1013.81","field8":"4.06"}

i request this by:

Code: Select all

http://api.thingspeak.com/channels/978931/feeds/last

i thought about it like

Code: Select all

do every minute a url request
write field1 to dummy device

i don’t have large messages in mind.

[Ath]

i thought about it like

Code: Select all

do every minute a url request
write field1 to dummy device

Currently there is no 'GetFromHTTP' command, that would have to be implemented first. And handling strings in rules is also a bit of a challenge, so that would either be quite specific, or needs some revision of ESPEasy internals (that has been thought/talked about before, but isn't picked up yet)

With the PR I linked implemented, you could receive that via MQTT to process on a unit.
Then also events can be generated, either for a selected field or for all fields received.

i don’t have large messages in mind.

Well, that's how we all start

[chromo23]

Thank you for the information!

MQTT is no option.... because the i would also need a server for it.
a url request is much simpler because you only need the esp and internet of course.
You can put the esp where ever you want and you don’t need to configure something extra.
Also i personally find the json way easier to share data with other esps that are not in the same network or even with friends.
E.g.: I have made an esp with a display and a sensor for a friend so he easily can see the inside temperature. But he would like to also see the outside temp.
since he is in a big city there is no need for an extra outside sensor. he could get the data from openweather
If i would do it the mqtt way i have to install a sever at his place and also have to maintain it. (Waste of energy an resources)

(One could ask... why is he not using his phone to see the outside temperature? answer: some people are not able or it’s harder for them...)

(Edit: for me personally i associate espeasy with being able to use an esp almost standalone. the ability of using a broker is a nice addition but was never the reason for me using espeasy.)

[Ath]

And another issue is that ESPEasy (at least on ESP8266) doesn't have the resources available (and hasn't been implemented because of that), to process HTTPS connections, and it is becoming more and more of a challenge to find external services that still accept HTTP requests.

Sharing your data with 'external parties' will be a challenge for the same reasons, I've seen other solutions that use stuff like an OpenHAB server or similar that has support for HTTPS, to send out local data, or use a MQTT server, either on a Raspberry Pi or out on the internet, to collect sensor data, to make that available to third parties.

[chromo23]

And another issue is that ESPEasy (at least on ESP8266) doesn't have the resources available (and hasn't been implemented because of that), to process HTTPS connections, and it is becoming more and more of a challenge to find external services that still accept HTTP requests.

Good Point.
All services i use can handle http (thingspeak and openweather). but it might be possible that that will change.

Sharing your data with 'external parties' will be a challenge for the same reasons,

Sharing means in this context: If you have the url, you have the data. Thats only a matter of copy’n’paste

[Ath]

Sharing means in this context: If you have the url, you have the data. Thats only a matter of copy’n’paste

Allowing access to the ESP directly from the i-net is a big no-no, as there is hardly any security, and the unit can probably be hacked quite easily, even for/by non-malicious hackers.

[chromo23]

You understood me wrong i guess...

I have an esp that sends data do thingspeak via sendtohttp.....

you want my data:

Code: Select all

http://api.thingspeak.com/channels/978931/feeds/last.json

you want to process it on your esp:

Code: Select all

do every minute a url request of http://api.thingspeak.com/channels/978931/feeds/last.json
write field1 to dummy device

Edit:
if you use this data to control something vital or hazardous it could be n issue... but just for displaying data its fine i guess
the esp doesn’t present itself to the internet and the hacking potential for a url request is small.. a hacker can only change the numbers on the server side

[chromo23]

Maybe implementing this in the core ist not a good idea for the reasons you stated.
It could be an optional thingspeak or openweather only plugin.
Maybe i have to start diving deeper into the rabbit hole and do it. But.....realistically .....i don´t...unfortunately ..... if only days and individual lifetime would be longer

[Ath]

Maybe implementing this in the core ist not a good idea for the reasons you stated.
It could be an optional thingspeak or openweather only plugin.

I'm thinking of a somewhat generic plugin that will support the GetJSONFromHTTP command, with optional credentials and similar processing options that I proposed in the PR for P037. It will involve some refactoring of the code, but code re-use should be very well possible.

[chromo23]

That would be it i guess.
I don`t like to ask..but could you do this.

It feels uncomfortable to ask because i could be the only one in need of this.

[Ath]

That would be it i guess.
I don`t like to ask..but could you do this.

I can, but I won't give an ETA

[chromo23]

I can, but I won't give an ETA

This is something i would never ask in this context....only if i would be your boss