HTTP Advanced to InfluxDB not working, error not visible

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
significant
New user
Posts: 8
Joined: 27 Sep 2021, 15:51

HTTP Advanced to InfluxDB not working, error not visible

#1 Post by significant » 27 Sep 2021, 20:41

I have compiled the latest version with HTTP Advanced plugin ("ESP_Easy_mega_20210802_test_A_ESP8266_4M1M Aug 2 2021") on a Wemos to monitor 8 DS18B20 temperature sensors. Unfortunatly, posting the measurement values to InfluxDB with HTTP Advanced does not work.

HTTP Advanced is configured as per example in the docs (https://espeasy.readthedocs.io/en/lates ... /C011.html).

Debugging is also not trivial as both the serial as well as the web monitoring do not provide the full error message (log is on debug dev level). Checking the page source does not reveal much more than is printed, namely "C011 POST ...".

Can anyone provide directions to a working configuration?

Image
Image

User avatar
Ath
Normal user
Posts: 3416
Joined: 10 Jun 2018, 12:06
Location: NL

Re: HTTP Advanced to InfluxDB not working, error not visible

#2 Post by Ath » 27 Sep 2021, 21:21

That 400 code in the message is already saying something, as it's not in the 100..299 range (OK), 400 means ERROR (Bad Request). Is the IP or Hostname correct?
/Ton (PayPal.me)

significant
New user
Posts: 8
Joined: 27 Sep 2021, 15:51

Re: HTTP Advanced to InfluxDB not working, error not visible

#3 Post by significant » 27 Sep 2021, 21:53

The host is correct as I can post using the Postman program.
Using this programme I can also verify that 400 errors are produced by influx, providing hints on what goes wrong.
As espeasy is not showing the error in full I can not deep dive in what I do wrong.

First step for me would to understand how the get the information after the ... (see screnshot).

User avatar
Ath
Normal user
Posts: 3416
Joined: 10 Jun 2018, 12:06
Location: NL

Re: HTTP Advanced to InfluxDB not working, error not visible

#4 Post by Ath » 27 Sep 2021, 22:19

There is nothing hidden behind those dots, they are just dots. The response from the http request is appended to the message, but it seems to be empty, as the outgoing request is already invalid.

Did you include http:// in the hostname or IP? That should not be there, as the plugin (and ESPEasy, currently) only supports the http protocol, so that's prefixed already by default.
/Ton (PayPal.me)

significant
New user
Posts: 8
Joined: 27 Sep 2021, 15:51

Re: HTTP Advanced to InfluxDB not working, error not visible

#5 Post by significant » 27 Sep 2021, 22:43

For the sake of completeness, a screenshot of the upper section of the parameters for HTTP Advanced. To my view, it is configured as pointed out in the docs.

Furthermore, the successful Postman request. The obtained value can also be found in the database.

Now the question remains: what do I do different in ESPEasy contrary to Postman...

Image
Image

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: HTTP Advanced to InfluxDB not working, error not visible

#6 Post by TD-er » 27 Sep 2021, 22:49

Also you're using a build with "test" in the name, so debug logs are not included.
If you build your own, you could make a "custom" build and set the debug log level.

You set it to "ignore acknowledgement". Any reason why?

Also increase the timeout during testing. Not sure how quickly the server replies, but it can be hard to find an error if you're running into that timeout.
Given you see a 400 reply, it is unlikely the timeout is an issue here though.

What does the log of the web server tell you?
And is the ESP in the same subnet as the web server? If not, make sure the gateway is set correct on the ESP.

significant
New user
Posts: 8
Joined: 27 Sep 2021, 15:51

Re: HTTP Advanced to InfluxDB not working, error not visible

#7 Post by significant » 29 Sep 2021, 08:40

To verify what is received on the InfluxDB side of things I have captured port 8086 as follows:
Image

As can be seen, another service is pushing data to InfluxDB which is correctly processed and received.

On suggestion of TD-er, I changed the acknowledgement parameter. Furthermore data is send binary (contrary to previous screenshot).

When I check sudo journalctl -u influxdb.service I only see the requests from the other service.

Based on these observations somehow the request is not successfully processed. Looking at the port dump it is noticed that the working service has different accept-encoding, authorization and connection settings. Any of these causing the bug, and if so, how to change?

Update
Reverting back to 20190301 version the magic works. Interesting observation is that we now get, as already thought, different encoding and authorization.

Code: Select all

23:03:48.303842 IP ESP-Easy.hidden.local.49164 > PImeterkast.hidden.local.                                                                                                             8086: Flags [P.], seq 13166:13404, ack 1986521097, win 2144, length 238
E....W....![..............3nvg. P..`.G..POST /write?db=home HTTP/1.1
Host: 192.168.1.12:8086
User-Agent: ESP Easy/20103/Mar  1 2019 03:18:48
Connection: close
Content-type: application/x-www-form-urlencoded
Content-Length: 42

underfloorHeating,group=groep_1 temp=19.50

significant
New user
Posts: 8
Joined: 27 Sep 2021, 15:51

Re: HTTP Advanced to InfluxDB not working, error not visible

#8 Post by significant » 02 Oct 2021, 13:57

Trying to find out what is changed between the versions on GitHub. Would like to use a more recent build, given all other improvements made.

Any suggestions on how to find this out?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: HTTP Advanced to InfluxDB not working, error not visible

#9 Post by TD-er » 02 Oct 2021, 14:22

See the release notes for builds:
https://github.com/letscontrolit/ESPEasy/releases

I wanted to fix an issue with WiFi, so that's why the new build was delayed a bit.
But now that I look at it, it is already 2 months since the last build.
Will try to make a new build today.

significant
New user
Posts: 8
Joined: 27 Sep 2021, 15:51

Re: HTTP Advanced to InfluxDB not working, error not visible

#10 Post by significant » 02 Oct 2021, 16:46

Much appreciated. Would be good to know why the HTTP advanced plugin does not seem to work in the latest build from Augustus. There's been many changes since 2019 version I am running at present. The release documentation is not clear when the additional parameters are added.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: HTTP Advanced to InfluxDB not working, error not visible

#11 Post by TD-er » 02 Oct 2021, 17:29

What is not working with that controller?

Maybe also try to enable "SendToHTTP wait for ack" on Tools -> Advanced page.

significant
New user
Posts: 8
Joined: 27 Sep 2021, 15:51

Re: HTTP Advanced to InfluxDB not working, error not visible

#12 Post by significant » 02 Oct 2021, 20:42

See above, posting to influxdb does work on the 2019 version but not on the latest release from Augustus.
Your suggestion was also taking into account.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: HTTP Advanced to InfluxDB not working, error not visible

#13 Post by TD-er » 02 Oct 2021, 21:04

Ah sorry, missed that one...


One of the major changes since 2019 is the "controller parameters".
See: https://espeasy.readthedocs.io/en/lates ... parameters

One of the things that could be causing issues here is the "timeout".
The default is 100 msec, which may be a bit too short when a request has to be processed, and/or the server is not local.

Maybe good to check all these controller parameters, as those were probably not present in your previous build.

significant
New user
Posts: 8
Joined: 27 Sep 2021, 15:51

Re: HTTP Advanced to InfluxDB not working, error not visible

#14 Post by significant » 02 Oct 2021, 22:17

No problem.

The timeout can be eliminated as the request and response are both captured with tcpdump. To double check I increased this to 2sec with no results.

As detailed in my previous post encoding and authorisation do differ (See pictures there).

Any idea how to influence these parameters such that they equal the 2019 build?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: HTTP Advanced to InfluxDB not working, error not visible

#15 Post by TD-er » 02 Oct 2021, 22:24

significant wrote: 02 Oct 2021, 22:17 [...]
Any idea how to influence these parameters such that they equal the 2019 build?
Not yet, as the 2019 build probably also used a different core version.
So it could even be something in the way how the core libs implement HTTP requests.

This makes it really hard to speculate on what's happening here.

I do see your logs mention a .local address.
Do you use mDNS?

I do know there are some reported issues on esp8266/Arduino related to mDNS.

n8tufts
New user
Posts: 1
Joined: 10 Jan 2022, 06:09

Re: HTTP Advanced to InfluxDB not working, error not visible

#16 Post by n8tufts » 10 Jan 2022, 06:33

Hi, new user to the forum. I came across this post after reading many others about this functionality. I struggled for quite a while to get this to work, and was eventually successful. I was able to debug my issue by sending the output to a machine on port 80 temporarily and reading the resulting HTTP POST command using netcat to listen to that port. I compared the output of espeasy's, versus a curl submission that was known to work from an influx example page. What I found was espeasy's output was missing a "/".

The URI field needed to look like this: /write?db=test (replace "test" with your existing database name)
All the other documentation I've seen omits this forward slash. I don't know if this is a bug, or something strange with my setup, but without the slash it does not work. I haven't tried any other builds, this happens to be the one I'm using:

ESP_Easy_mega_20211224_test_A_ESP8266_4M1M

FWIW, this is what is in the body field:
indata temp=%val1%%LF%indata humid=%val2%%LF%indata press=%val3%%LF%

It submits 3 values from one sensor organized within "indata". I'm still getting used to influx, and I'm sure there are other ways to do this, but this should hopefully help someone getting this working.

Also, I believe the "send binary" box needs to be ticked. In the log page, a successful submission will show a "204" code, "400" means that the inject to influx has failed.

Nate

User avatar
Ath
Normal user
Posts: 3416
Joined: 10 Jun 2018, 12:06
Location: NL

Re: HTTP Advanced to InfluxDB not working, error not visible

#17 Post by Ath » 10 Jan 2022, 08:01

n8tufts wrote: 10 Jan 2022, 06:33 The URI field needed to look like this: /write?db=test (replace "test" with your existing database name)
Thanks, Nate, for the comprehensive feedback :idea:
/Ton (PayPal.me)

agroszer
Normal user
Posts: 28
Joined: 02 Jan 2023, 16:36

Re: HTTP Advanced to InfluxDB not working, error not visible

#18 Post by agroszer » 21 Sep 2023, 16:47

ough yes, influxDB is picky on newlines and spacing

So the body needs to be exactly like this, no extra spaces, single line:

Code: Select all

esp,unit=%sysname%,device=%tskname%,sensor=%vname1% value=%val1% %unixtime%%LF%%2%esp,unit=%sysname%,device=%tskname%,sensor=%vname2% value=%val2% %unixtime%%/2%%LF%%3%esp,unit=%sysname%,device=%tskname%,sensor=%vname3% value=%val3% %unixtime%%/3%%LF%%4%esp,unit=%sysname%,device=%tskname%,sensor=%vname4% value=%val4% %unixtime%%/4%%LF%
Otherwise you get 400.
Of course use send binary.

With influx headers are:

Code: Select all

Content-Type: application/x-www-form-urlencoded
Authorization: Token <yourtoken>

Post Reply

Who is online

Users browsing this forum: No registered users and 20 guests