Page 1 of 1

Send data to thingspeak

Posted: 23 Sep 2019, 21:34
by paulz
Hello guys!
In the past days I have been trying to figure out how to send data to thingspeak, but I must have been doing something wrong, because no data is visible on my feed... :(
So what I am trying to do: send analog values to my thingspeak chanel. I have set up a fresh install of the latest esp easy normal core on a wemos d1. Here I have a soil moisture sensor connected to the analog port. Data is visible on the ESP:
Screenshot 2019-09-23 at 21.18.31.png
Screenshot 2019-09-23 at 21.18.31.png (18.02 KiB) Viewed 35233 times
I would like to display the data on my TS channel.
For this I have added the ThingSpeak controller with the following:
Screenshot 2019-09-23 at 21.09.45.png
Screenshot 2019-09-23 at 21.09.45.png (93.91 KiB) Viewed 35233 times
Next I have configured the device and enabled the Data Acquisition:
Screenshot 2019-09-23 at 21.23.14.png
Screenshot 2019-09-23 at 21.23.14.png (18.87 KiB) Viewed 35233 times
Finally checked the cart on the TS. No data is visible...
Checking the log on the esp, I can see that there is a connection error:
Screenshot 2019-09-23 at 21.09.12.png
Screenshot 2019-09-23 at 21.09.12.png (35.41 KiB) Viewed 35233 times
I have checked the esp's configuration. It seems it is correctly connected to my network:
Screenshot 2019-09-23 at 21.11.36.png
Screenshot 2019-09-23 at 21.11.36.png (56.74 KiB) Viewed 35233 times
Also checked the ThingSpeak channel:
Screenshot 2019-09-23 at 21.26.52.png
Screenshot 2019-09-23 at 21.26.52.png (70.13 KiB) Viewed 35233 times

The write key seems to be right (just copy-pasted).
What I am unsure is the ThingHTTP name: field. What shall I write here? Now that is set to my channel ID, but also tried with my username.
Or have I miss-configured anything else?

Thank you for your help!
Cheers
Paul

Re: Send data to thingspeak

Posted: 23 Sep 2019, 21:45
by flimon
This is my settings and its work.

Re: Send data to thingspeak

Posted: 23 Sep 2019, 22:40
by ThomasB
FWIW, ThingSpeak's communication protocol changed Feb 2019. It now expects MustCheckReply=True, otherwise the communication may unreliable. Set the plugin's Check Reply to Check Acknowledgement.

Details are here:
https://github.com/letscontrolit/ESPEasy/issues/2273

- Thomas

Re: Send data to thingspeak

Posted: 23 Sep 2019, 22:54
by paulz
Changed to check acknowledgement, but still the error: HTTP : C004 connection failed printed into the log. Still no data on the TS channel :(

Re: Send data to thingspeak

Posted: 24 Sep 2019, 05:53
by ThomasB
Sorry, I don't have a magic solution. But here's what I would try.
1. Install the latest Mega build.
2. Remove and reinstall the Thingspeak controller. Reconfigure it.
2. Confirm your API Key is accurate. It must be 16 characters long.

- Thomas

Re: Send data to thingspeak

Posted: 24 Sep 2019, 05:56
by sobolkz
Same issue was expected in some last versions of ESPEasy. No issues in mega-20190830. I can use IP address AND name api.thingspeak.com.
Try this version.

Re: Send data to thingspeak

Posted: 24 Sep 2019, 08:22
by TD-er
I can imagine Thingspeak is getting data from all over the world, so I would not be surprised if they have some localization on DNS to IP,
Meaning that if you're trying to connect to the DNS name, you are actually connecting to a different IP compared to when using the IP mentioned here.
So also test on your computer to see to what IP address the DNS name resolves.

Re: Send data to thingspeak

Posted: 24 Sep 2019, 08:24
by TD-er
sobolkz wrote: 24 Sep 2019, 05:56 Same issue was expected in some last versions of ESPEasy. No issues in mega-20190830. I can use IP address AND name api.thingspeak.com.
Try this version.
Did you test with the same core library used?

Re: Send data to thingspeak

Posted: 24 Sep 2019, 09:44
by sobolkz
TD-er wrote: 24 Sep 2019, 08:24 Did you test with the same core library used?
Hmm...Hard to answer, because this issue wasn't in my investigation scope. It was expected, checked with different versions (which was tested for my main issue with Sensear S8) and closed for me with mega-20190830. Accordingly topic's issue - I can check my working config today's evening, when I will be in home (ESPEasy remote access wasn't allowed).

Re: Send data to thingspeak

Posted: 24 Sep 2019, 19:51
by paulz
Okay, so this has driven me crazy, but it seems I might have found the root cause. After:
continuously re-installing different versions, creating new TS channel, I have created a tethered WIFI from my mobile. Connected the ESP, and now data is visible on the TS channel. Due this, it seems the issue is somewhere in my home wifi network. Any advice how to look further to identify the exact cause? :D

Re: Send data to thingspeak

Posted: 24 Sep 2019, 20:14
by grovkillen
Could it be firewall related?

Re: Send data to thingspeak

Posted: 24 Sep 2019, 21:18
by paulz
grovkillen wrote: 24 Sep 2019, 20:14 Could it be firewall related?
Yeah, thought about that, but checked my Router, and no firewall is activated. DMZ is also disabled. For a quick test created a fresh guest wifi, but the connection error is still prompted. My connection looks like the following: I have a DOCSIS router, which is now set to modem mode, thus it is not handling any firewall configuration. Behind this I have my main router, which is handling everything.

Tomorrow I will give a try and check with an older Router. If it works, then there is some filtering somewhere in the router. If this does not help, I will re-activate the wifi mode of the docsis router. If that does not help, then maybe my ISP does some filtering....

Re: Send data to thingspeak

Posted: 25 Sep 2019, 00:07
by TD-er
Also check the returned IP address when doing a DNS lookup.
Make sure you use the same DNS and gateway as the ESP node.
You can see the used DNS and gateway on the sysinfo page.

Is it possible the ESP does not know how to communicate to the outside world? (e.g. when using the wrong gateway IP)

Re: Send data to thingspeak

Posted: 25 Sep 2019, 06:26
by sobolkz
paulz wrote: 24 Sep 2019, 21:18 Yeah, thought about that, but checked my Router, and no firewall is activated. DMZ is also disabled. For a quick test created a fresh guest wifi, but the connection error is still prompted. My connection looks like the following: I have a DOCSIS router, which is now set to modem mode, thus it is not handling any firewall configuration. Behind this I have my main router, which is handling everything.

Tomorrow I will give a try and check with an older Router. If it works, then there is some filtering somewhere in the router. If this does not help, I will re-activate the wifi mode of the docsis router. If that does not help, then maybe my ISP does some filtering....
Screenshot_14.jpg
Screenshot_14.jpg (52.9 KiB) Viewed 35093 times
Is thingspeak.com and api.thingspeak.com accessible from other home devices (for example from PC with IP 192.168.0.189)? If you can open it - it isn't firewall issue, because home routers firewall by default can block whole subnet instead one device.
As I can see in you first post - IP of ESPESay is 192.168.0.170, subnet is 255.255.255.0 - all correct. Gateway- 192.168.0.1, also correct.
IP and client PC IP are from same subnet.
DNS isn't important if you are using IP address of thingspeak instead it's name.

Re: Send data to thingspeak

Posted: 25 Sep 2019, 09:20
by TD-er
sobolkz wrote: 25 Sep 2019, 06:26 [...] Gateway- 192.168.0.1, also correct.
[...]
It is only correct if that's the address of his router/gateway.
Not all routers have their gateway on .1 of the subnet.
Having it given via DHCP makes it (a lot) less likely it is an incorrect setting, because if it was given incorrect information via DHCP, other devices on the network would have similar strange connection issues.

Re: Send data to thingspeak

Posted: 25 Sep 2019, 21:00
by paulz
TD-er wrote: 25 Sep 2019, 09:20
sobolkz wrote: 25 Sep 2019, 06:26 [...] Gateway- 192.168.0.1, also correct.
[...]
It is only correct if that's the address of his router/gateway.
Not all routers have their gateway on .1 of the subnet.
Having it given via DHCP makes it (a lot) less likely it is an incorrect setting, because if it was given incorrect information via DHCP, other devices on the network would have similar strange connection issues.
Yeah, this is correct. I am using DHCP, and the router is on the IP: 192.168.0.1, so this seems right.
However today I have checked in terminal if I can ping the api.thingspeak.com. It seems I have 100% packet loss :D The IP, which my PC was pinging is: 52.22.86.23.
Previously I have used the IP: 184.106.153.149 which I have found somewhere. My PC can ping this with only minor (2 out of 20) packet loss. Hence, if I set this IP directly in the ESP, the error is printed, and no data is visible on the TS channel. Now I am totally confused :D

Re: Send data to thingspeak

Posted: 26 Sep 2019, 04:51
by sobolkz
paulz wrote: 25 Sep 2019, 21:00 Yeah, this is correct. I am using DHCP, and the router is on the IP: 192.168.0.1, so this seems right.
However today I have checked in terminal if I can ping the api.thingspeak.com. It seems I have 100% packet loss :D The IP, which my PC was pinging is: 52.22.86.23.
Previously I have used the IP: 184.106.153.149 which I have found somewhere. My PC can ping this with only minor (2 out of 20) packet loss. Hence, if I set this IP directly in the ESP, the error is printed, and no data is visible on the TS channel. Now I am totally confused :D
Hmmm, let's check situation, there thingspeak potentially balancing traffic by geolocation.
There is results for nslookup api.thingspeak.com from Kazakhstan:
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: api.thingspeak.com
Address 1: 34.226.171.107
Address 2: 52.22.86.23
No difference between my and you IP information.
But I'm confusing with next:
paulz wrote: 25 Sep 2019, 21:00 However today I have checked in terminal if I can ping the api.thingspeak.com. It seems I have 100% packet loss :D
It's look like you have some problems with DNS service at you router. Try to run nslookup api.thingspeak.com and nslookup api.thingspeak.com 8.8.8.8 - second one should be done any way if there is no any issues with port 53 (DNS service) at you router. If both will be unsuccessful - try to check router's firewall rules.
As I remember (TD-er, correct me if I wrong) there is some issues in past with something like hardcoded "api.thingspeak.com" in thingspeak controller module code, which can block it's work even if you use IP address instead name in controller preferences

Re: Send data to thingspeak

Posted: 26 Sep 2019, 10:28
by TD-er
Is possible, that Thingspeak does require the full domain name in the request header.
This then eliminates the usability of setting the IP address in the Thingspeak address.

Re: Send data to thingspeak

Posted: 27 Sep 2019, 14:22
by paulz
sobolkz wrote: 26 Sep 2019, 04:51
paulz wrote: 25 Sep 2019, 21:00 Yeah, this is correct. I am using DHCP, and the router is on the IP: 192.168.0.1, so this seems right.
However today I have checked in terminal if I can ping the api.thingspeak.com. It seems I have 100% packet loss :D The IP, which my PC was pinging is: 52.22.86.23.
Previously I have used the IP: 184.106.153.149 which I have found somewhere. My PC can ping this with only minor (2 out of 20) packet loss. Hence, if I set this IP directly in the ESP, the error is printed, and no data is visible on the TS channel. Now I am totally confused :D
Hmmm, let's check situation, there thingspeak potentially balancing traffic by geolocation.
There is results for nslookup api.thingspeak.com from Kazakhstan:
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: api.thingspeak.com
Address 1: 34.226.171.107
Address 2: 52.22.86.23
No difference between my and you IP information.
But I'm confusing with next:
paulz wrote: 25 Sep 2019, 21:00 However today I have checked in terminal if I can ping the api.thingspeak.com. It seems I have 100% packet loss :D
It's look like you have some problems with DNS service at you router. Try to run nslookup api.thingspeak.com and nslookup api.thingspeak.com 8.8.8.8 - second one should be done any way if there is no any issues with port 53 (DNS service) at you router. If both will be unsuccessful - try to check router's firewall rules.
As I remember (TD-er, correct me if I wrong) there is some issues in past with something like hardcoded "api.thingspeak.com" in thingspeak controller module code, which can block it's work even if you use IP address instead name in controller preferences
Yeah, done that. My results are the same for nslookup. It seems the IP address is resolved correctly. Connected the ESP to my mobile again, and data are correctly transferred and visible on the TS channel.... Of course if ESP is connected to my home Wi-Fi, there are no data sent... Think that there is some firewall somewhere, but I have basically deactivated the whole feature in my router.

Re: Send data to thingspeak

Posted: 27 Sep 2019, 15:07
by paulz
Okay, so finally I have done some more tests today. I have turned off my Home Router and activated the Router function of my Modem provided from the ISP. I have configured the DHCP, deactivated the firewall and DMZ. Connected my PC. I was able to open thingspeak.com nslookup result is the following:

Non-authoritative answer:
Name: api.thingspeak.com
Address: 52.22.86.23
Name: api.thingspeak.com
Address: 34.226.171.107

This all seems to be right.
Finally, connected the ESP, and again nothing is visible on the channel :(

Connected the ESP back to the mobile hotspot, and the data are sent again.

Since everything seems to be correctly configured from my end, I believe there is some issue at my ISP's side. Any advice how I can verify this?

Re: Send data to thingspeak

Posted: 28 Sep 2019, 13:34
by sobolkz
Stupid question, but I can't find you ESPEasy actual version info.
Is it older than mega-20190830?
It's look like my issue with old releases and (or) TD-er explained issue with "named" request limitation.
If you version are older - try to update to mega-20190830 and check situation.

Re: Send data to thingspeak

Posted: 28 Sep 2019, 23:17
by miot
Important is:
- Controller Hostname: api.thingspeak.com
- and a Client Timeout of 1000 ms or larger

And make a reboot after finish config

My Firmware
Git Build: mega-20190903

Re: Send data to thingspeak

Posted: 11 Oct 2019, 07:01
by Temper
Hi

I have exactly the same problem.
I invested several days for this. Also all settings under controller and device tried. Different routers and IP settings. Wifi settings and time settings.
There must definitely be a problem in the build! Can someone fix this?

Regards

Tested with: mega-20190903 and mega-20190928

Re: Send data to thingspeak

Posted: 11 Oct 2019, 09:52
by TD-er
Temper wrote: 11 Oct 2019, 07:01 Hi

I have exactly the same problem.
I invested several days for this. Also all settings under controller and device tried. Different routers and IP settings. Wifi settings and time settings.
There must definitely be a problem in the build! Can someone fix this?

Regards

Tested with: mega-20190903 and mega-20190928
Not sure what exactly is the problem here.
- IP resolved incorrect when using DNS?
- Waiting for acknowledgement after sending?
- ISP provider specific issue? (test with using mobile as WiFi hotspot and ESP operating via that hotspot)

Re: Send data to thingspeak

Posted: 24 Feb 2020, 18:51
by spachal
Hi, just FYI, I had also problems with sending data to TS. I can't figure how to do that (" Error : HTTP : C004 connection failed " all the time), after I changed Client timeout from 100ms to 1000ms it starts to work!
Everything else I configured as described here:

newest build mega-20200222, 4M dev build, LOLIN D1 mini

hostname: api.thingspeak.com, port 80
Minimum Send Interval: 15000ms
ThingHTTP Name: [blank]
Tools > Advanced > Message Interval: 15000ms

Thingspeak is the 2nd controller (first is local mqtt broker), I'm sending data from two sensors (first three values with IDX1, second IDX4), interval on sensors is 60s.
Maybe you can update wiki for more informations.. (https://www.letscontrolit.com/wiki/inde ... ThingSpeak)

Re: Send data to thingspeak

Posted: 24 Feb 2020, 20:04
by TD-er
Yep, Thingspeak is some online service, so then 100 msec is a bit low.
Most ping times to local web services take roughly 50 - 100 msec, depending on your connection and speed of the machine running the ping, so one overseas will take more than 100 msec.

Re: Send data to thingspeak

Posted: 03 Jan 2022, 13:52
by simkob
Dear All,

I have problems with getting my ESP32 module and EspEasy work to send sensor data to ThingSpeak. I have tried a lot variations with the SW versions and ESP modules.
The main problem is that when the ThingSpeak controller "Check Reply" parameter setted to "Ignore Ack." it works (and the ESP modul also running), but the connection is not stable (values cannot arrives to ThingSpeak), but when I change to "Check Ack." and submit the setting the web interface is not working anymore and the ESP modul seems froozen (sensor data also not arrives to ThingSpeak). It happens when the "minimum send interval" spent after submitting the setting and one data packet sent to ThingSpeak because I can see one packet on ThingSpeak web interface. When I restart the ESP module it also begins.

I just have this problem with my ESP32 modules (Espressif ESP32-wroom-32 and WT32-S1), on ESP6288 it works correctly. I also tried to update the EspEasy version but no luck.
These are the versions that I have tried:
ESP_Easy_mega_20211224_custom_ESP32_4M316k
ESP_Easy_mega_20211224_normal_ESP32_4M316k
ESP_Easy_mega_20211224_normal_ESP32_4M316k_ETH
ESP_Easy_mega_20211105_custom_ESP32_4M316k_ETH
ESP_Easy_mega_20201227_normal_ESP32_4M316k_ETH

It is no matter that I use wifi connection or wired ethernet. The settings are the same for the ESP32 and ESP6288 because copy paste from the working one.

Could anyone offer something how can I find where is the problem? Or how can I start?

Thank you in advance!

Re: Send data to thingspeak

Posted: 03 Jan 2022, 14:01
by TD-er
How often do you send messages to thingspeak?
See: https://espeasy.readthedocs.io/en/lates ... figuration

You may also want to check the checkbox on tools->Advanced: SendToHTTP wait for ack
But that's only needed if you try to send it from the rules, if you create your own URL to send the values.

Re: Send data to thingspeak

Posted: 03 Jan 2022, 14:38
by simkob
Thank you for your quick reply!

For testing I only installed one dummy sensor and one ThingSpeak controller with the settings bellow:
espeasy_problem_1.png
espeasy_problem_1.png (16.72 KiB) Viewed 15584 times
Unfortunately its not working... When it reach the send interval the ESP web interface is not working anymore. (I checked the connection in the router settings with ESP and it exists but not responding.)

Re: Send data to thingspeak

Posted: 03 Jan 2022, 15:08
by TD-er
Is it also not responding to ping anymore (this is done at a lower level)

Can you also check with a "sensor" plugin like the sysinfo plugin?
Just to make sure that the dummy plugin isn't doing something silly here as it is a rather special one not acting the same as all other plugins regarding sending data to linked controllers.

Re: Send data to thingspeak

Posted: 03 Jan 2022, 15:42
by simkob
TD-er wrote: 03 Jan 2022, 15:08 Is it also not responding to ping anymore (this is done at a lower level)

Can you also check with a "sensor" plugin like the sysinfo plugin?
Just to make sure that the dummy plugin isn't doing something silly here as it is a rather special one not acting the same as all other plugins regarding sending data to linked controllers.
The ESP module respond to ping so its working (after the web interface freezed).

I tried to use the sysinfo plugin with uptime value but the result was the same, no respont of the web interface after the first sending of the ThingSpeak controller... (ping works fine).
Can be this bug caused by the router itself? (but any special firewall settings used at the moment...)

Re: Send data to thingspeak

Posted: 04 Jan 2022, 10:45
by simkob
simkob wrote: 03 Jan 2022, 15:42
TD-er wrote: 03 Jan 2022, 15:08 Is it also not responding to ping anymore (this is done at a lower level)

Can you also check with a "sensor" plugin like the sysinfo plugin?
Just to make sure that the dummy plugin isn't doing something silly here as it is a rather special one not acting the same as all other plugins regarding sending data to linked controllers.
The ESP module respond to ping so its working (after the web interface freezed).

I tried to use the sysinfo plugin with uptime value but the result was the same, no respont of the web interface after the first sending of the ThingSpeak controller... (ping works fine).
Can be this bug caused by the router itself? (but any special firewall settings used at the moment...)
I also tried to send data to ThingSpeak with rule:

Code: Select all

On System#Boot do    //When the ESP boots, do
  timerSet,1,60      //Set Timer 1 for the next event in 60 seconds
endon

On Rules#Timer=1 do  //When Timer1 expires, do
  SendToHTTP api.thingspeak.com,80,/update?api_key=**********************&field1=0
endon
Same problem, the ESP sends the first package and then freeze. "SendToHTTP wait for ack" was checked in advanced settings.

Re: Send data to thingspeak

Posted: 04 Jan 2022, 12:43
by TD-er
Hmm, maybe this is more like a DNS issue?
Or the gateway cannot be reached?

Re: Send data to thingspeak

Posted: 04 Jan 2022, 14:50
by simkob
TD-er wrote: 04 Jan 2022, 12:43 Hmm, maybe this is more like a DNS issue?
Or the gateway cannot be reached?
I also checked the serial output of the ESP. It behaves like the log on the web interface.
This is the last line that I got from ESP:
60383 : Info : ACT : SendToHTTP api.thingspeak.com,80,/update?api_key=***************************&field1=0<\r><\n>

I dont know why it is only happen with the ESP32 modules and on ESP8266 works OK.

Do you have any other idea what can I check?

Thank you!

Re: Send data to thingspeak

Posted: 04 Jan 2022, 15:08
by TD-er
Try sending a continuous ping to the node from any host in your network, while it tries to send to Thingspeak.

Re: Send data to thingspeak

Posted: 11 Jan 2022, 10:12
by simkob
I have tried to ping the ESP module continously while sending data to Thingspeak. The module responds to ping, there is no error in connection but the web interface stops.

I also tried the module with shared internet connection from my mobile. The result is the same. After the first packet, the module freeze so I think this problem is not related to my local network.

Re: Send data to thingspeak

Posted: 11 Jan 2022, 12:33
by TD-er
Just to see if it "unfreezes", can you uncheck the "wait for ACK" on the tools->Advanced page (and/or ignore ack on the controller page)
Sending to Thingspeak may not succeed, but if the unit responds after, we know where to look. (or at least know a bit more)

Re: Send data to thingspeak

Posted: 11 Jan 2022, 14:44
by simkob
TD-er wrote: 11 Jan 2022, 12:33 Just to see if it "unfreezes", can you uncheck the "wait for ACK" on the tools->Advanced page (and/or ignore ack on the controller page)
Sending to Thingspeak may not succeed, but if the unit responds after, we know where to look. (or at least know a bit more)
Yes. If the "wait for ACK" unchecked or the "ignore ACK" selected it works without "freezing" the web interface, but not all of the data (less than half) arrives to ThingSpeak.

Thank you!

Re: Send data to thingspeak

Posted: 11 Jan 2022, 14:49
by TD-er
OK, so it seems the timeout is not properly implemented on the ESP32 code.

Re: Send data to thingspeak

Posted: 11 Jan 2022, 14:59
by TD-er
One more thing to check...
Do you have Tools->Advanced "Periodical Scan WiFi" checked?
If so, uncheck it.
I already removed it in the latest code as it does cause a lot of strange issues.

Also check "Send with max TX power" and "Periodical send Gratuitous ARP"
Keep "CPU Eco Mode" unchecked.

Re: Send data to thingspeak

Posted: 11 Jan 2022, 15:22
by simkob
TD-er wrote: 11 Jan 2022, 14:59 One more thing to check...
Do you have Tools->Advanced "Periodical Scan WiFi" checked?
If so, uncheck it.
I already removed it in the latest code as it does cause a lot of strange issues.

Also check "Send with max TX power" and "Periodical send Gratuitous ARP"
Keep "CPU Eco Mode" unchecked.
None of them are checked. "Periodical Scan Wifi" option is not in the list. I think I am using the latest release (20211224).
espeasy_problem_2.png
espeasy_problem_2.png (19.07 KiB) Viewed 15145 times