How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#1 Post by rai » 14 Oct 2016, 01:22

Hello from germany (sorry, my english is not perfect),

first of all many thanks to all developers of this useful piece of software. Great!
I managed to setup the Easy ESP part for my ESP8266 ESP-01 in my Arduino 1.6.12 IDE within 10 minutes. It took me only 25 minutes to include and setup my sensors and OLED display for a functional WiFi controled temperature, humidity and air pressure system. Please take into account, that i have only experience with generic HTTP with PHP, and that's the reason why i want to solve my localhost data backup via PHP and HTTP request. All other protocol solutions offered under esp-menu "Config" won't help me, simply because i have no idea about these protocols!!

Following is:
1.) what i have (hardware)
2.) what i did (setup)
3.) what i want (data backup on my localhost)

Infos to 1.) what i have (hardware)
-----------------------------------------
Windows 7 Pro
ApacheFriends XAMPP Version 1.7.7 with:
- Apache 2.2.21
- MySQL 5.5.16 (Community Server)
- PHP 5.3.8 (VC9 X86 32bit thread safe) + PEAR
ARDUINO IDE 1.6.12
ESP8266 Version 01 (GPIO0 and GPIO2 are used for my I2C sensors!)
I2C BMP085 temperature and air pressure
I2C Si7021 temperature and humidity
I2C OLED SH1106 1.3" display (with for SH1106 adapted "_P023_OLED.ino" sketch)

Infos to 2.) what i did (setup)
----------------------------------
BMP085 (Temperature & Pressure - BMP085)
Name: bmp085
Delay: 5
IDX / Var: 2
Send Data: yes
Value Name 1: temp
Value Name 2: press
Decimals: 1
Decimals: 1

Si7021 (Temperature & Humidity - SI7021)
Name: si7021
Delay: 5
IDX / Var: 3
Resolution (bits): Temp 14 bits / RH 12 bits
Send Data: yes
Value Name 1: temp
Value Name 2: hum
Decimals: 1
Decimals: 1

SH1106 OLED (Display - OLED SSD130)
Name: display
IDX / Var: 1
I2C Address: 3C
Rotation: Rotated
Line 1: %systime% <> Buero
Line 2: empty
Line 3: Temp [bmp085#temp] C
Line 4: empty
Line 5: Druck [bmp085#press] mb
Line 6: empty
Line 7: Feuchte [si7021#hum] %
Line 8: empty
Display Button: empty / none
Display Timeout: 0

This setup works perfectly! System time, Temperature, air pressure and humidity is displayed on the 1.3" OLED and all values will be refreshed every 5 seconds.

Infos to 3.) what i want (data backup on my localhost)
---------------------------------------------------------------
The temprature, air pressure and humidity sensor data should be saved on my localhost for further data history/analytics/graphs. I want to do that with PHP scripting language and a HTTP request.

How can i do that?
What are the correct "Generic HTTP" settings in menu "Config"?
What are the correct corresponding settings in menu "Tools" "Advanced" "Publish Template"?

My localhost has the IP-Address 192.168.178.21 in my WiFi network and uses the port 80!

According to webpage "http://www.esp8266.nu/index.php/EasyProtocols" section "Generic HTTP" I assembled the URL "http://www.homeserver.local/update.php? ... value=23.5)" for my test-application into the URL "192.168.178.21/demo.php?name=bmp085&id=2&value=temp", where demo.php is my test page on my localhost to call the requested data, and name is the esp-variable containing the sensor-name, id is the esp-variable containing the sensor-id, and value is the esp-variable containing the temperature-value. But this URL is not valid, so i cannot use it!

I tried to find the correct PHP HTTP request myself for a couple of days, but without success. I didn't find any hint or solution for my problem, neither with google in the big www, nor with the search function in the Easy ESP forum. I did my best, but now i am stucked.

Please help me to get the data from my Easy ESP sensors to my localhost. I would be very glad if somebody could guide me to a correct HTTP Request solution with PHP.

Many thanks in advance.
Regards from Germany.
rai

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#2 Post by moelski » 14 Oct 2016, 07:11

Hi !
what i want (data backup on my localhost)
I think you got the protocol wrong ... They don´t offer a server where you can connect. It´s a client which connects to a server like MQTT broker or FHEM or whatever.
Just looked into the code of the generic HTTP modul ... It´s also no server - only a client.

So you have to ensure that there is any server in your network / local environment which is able to receive the data from the ESPEasy.

I you need ESPEasy to be a server for your data things getting complicated :roll:
You have to write your own controller plugin and I think it´s not easy to get that running...
regards
Dominik

groovy
Normal user
Posts: 39
Joined: 14 Oct 2016, 11:32
Location: Chemnitz, Germany

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#3 Post by groovy » 14 Oct 2016, 11:52

@moelski:
rai surely has a server running, read his post:
ApacheFriends XAMPP Version 1.7.7 with:
- Apache 2.2.21
- MySQL 5.5.16 (Community Server)
- PHP 5.3.8 (VC9 X86 32bit thread safe) + PEAR
@rai:
I just flashed my first ESP with this firmware today and also want to send data with generic HTTP to my local server.
You have to use variables in your template which then are replaced by the corresponding values when the request is send.
For your example
it will be something like:

update.php?name=%sysname%&id=%tskname%&value=%value%

What I actually miss at the moment is a complete list of variables that can be set/read. Also, my needed template is actually longer than the allowed size and gets cut when sending.

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#4 Post by moelski » 14 Oct 2016, 12:29

Hi !
rai surely has a server running, read his post:
Ups ... Missed that. Sorry.

@groovy:
Just an idea ... since the HTTP General controller plugin uses only HTTP GET you could send the data with a POST and place the payload (sensor data) in the POST message body. Should be possible to transfer more data.
But for that you have to write an extended HTTP controller plugin I suppose.
regards
Dominik

groovy
Normal user
Posts: 39
Joined: 14 Oct 2016, 11:32
Location: Chemnitz, Germany

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#5 Post by groovy » 14 Oct 2016, 13:05

But for that you have to write an extended HTTP controller plugin I suppose.
Well, the main reason for choosing ESPEasy is, that my knowledge in programming tends to zero 8-)
So I'm lost with your idea. Thanks anyway ;)

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#6 Post by rai » 14 Oct 2016, 15:46

Hello again,

thanks to all who have send an answer so far.

@groovy
you write "What I actually miss at the moment is a complete list of variables that can be set/read. Also, my needed template is actually longer than the allowed size and gets cut when sending."
You are right. That's the reason why i got lost. Which variable names gives ESP Easy to my sensors? And my needed template should not be to long for just 3 variables and their values!
Can you please publish here your exact settings for all your "Generic HTTP" settings, what your query string in "Subcripe Template" looks like, and what your PHP update.php code looks like?
Maybe this would help me to continue my own experiences.

@all
Once again my sensor setup in menu "Devices" "Edit" for my 2 sensors:
1.) Name: bmp085
Value Name 1: temp
Value Name 2: press
From this sensor i will use both data values ("temp" and "press")

2.) Name: si7021
Value Name 1: temp
Value Name 2: hum
From this sensor i will use only 1 data value ("hum")

My OLED SH1106 accepts following variable names to display the sensor values correctly:
[bmp085#temp] -> for sensor BMP085 temperature value
[bmp085#press] -> for sensor BMP085 air pressure value
[si7021#hum] -> for sensor SI7021 humidity value

When i setup in menu "Config":
- "Protocol:" to 'Generic HTTP'
- "Locate Controller:" to 'Use IP address'
- "Controller IP:" to '192.168.178.21' (which represents my localhost in my WiFi network)
- "Controller Port:" to '80'

and then setup in menu "Tools" "Advanced":
- "Subscribe Template:" to empty, not used for Generic HTTP?!
- "Publish Template:" to the query string '/update.php?bmp085#temp=%value%&bmp085#press=%value%&si7021#hum%value%'

i guess that Easy ESP will assemble the URL for the request as follwing:
192.168.178.21/update.php?bmp085#temp=%value%&bmp085#press=%value%&si7021#hum%value%
or to
http://192.168.178.21/update.php?bmp085 ... #hum%value%

Is this correct, or do i misunderstand that? At least this query string won't work in my application!

This raises other questions like:
- if i use "Generic HTTP" which variable names are expected from Easy ESP to implement into my query string?
- what are the correct settings in "Config" for using the "Generic HTTP" protocol?
- what is the correct setting in "Tools" "Advanced" for "Publish Template:" for using the "Generic HTTP" protocol?
- what is the exact query string to write into the field "Subscribe Template:" for using the "Generic HTTP" protocol?

Last, but not least, it would be helpful if someone would show here a short example for the update.php to be placed on localhost which calls the sensor data from ESP Easy.

For all helpful replies many thanks in advance.
Regards from Germany.
rai

groovy
Normal user
Posts: 39
Joined: 14 Oct 2016, 11:32
Location: Chemnitz, Germany

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#7 Post by groovy » 14 Oct 2016, 17:40

The subscribe Template is empty on my system.
For the variables and the publish template have a look at my other post: http://www.esp8266.nu/forum/viewtopic.php?f=4&t=2139

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#8 Post by rai » 15 Oct 2016, 01:48

Hello,

@groovy
thanks for your short answer. But this does not help me, because your ESP is a NodeMCU, and mine is a simple ESP8266 Version 01 (just 2 I/Os - GPIO0 and GPIO2). Your NodeMCU is based on Lua Script Language, my application is based on Arduino Script Language. In conjunction to my application I don't understand your variable declaration at all. To bring your experience to my application it would be helpful if:

- you kindly could tell me how my variable-value pairs for my ESP query string should look like
- you kindly show me your ESP "Generic HTTP" settings including your settings for:
- Locate Controller (Use IP address or Use Hostname?)
- if i use IP address for the "Controller IP" i use my localhost IP address 192.168.178.21 (is this correct or should i use instead "Use Hostname" with localhost?)

For your help in case of my varable settings please take into account my following ESP sensor setup:

BMP085 (Temperature & Pressure - BMP085)
Name: bmp085
Delay: 5
IDX / Var: 2
Send Data: yes
Value Name 1: temp
Value Name 2: press
Decimals: 1
Decimals: 1

Si7021 (Temperature & Humidity - SI7021)
Name: si7021
Delay: 5
IDX / Var: 3
Resolution (bits): Temp 14 bits / RH 12 bits
Send Data: yes
Value Name 1: temp
Value Name 2: hum
Decimals: 1
Decimals: 1

What are the correct variable and value pair names for the a. m. sensors, and how should they be implemented into the ESP query string?

Hopefully you can help me, or at least you can show me the way to a helpful solution for my application problem. For your help,

many thanks in advance.
Regards from Germany.
rai

groovy
Normal user
Posts: 39
Joined: 14 Oct 2016, 11:32
Location: Chemnitz, Germany

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#9 Post by groovy » 15 Oct 2016, 09:13

But this does not help me, because your ESP is a NodeMCU, and mine is a simple ESP8266 Version 01 (just 2 I/Os - GPIO0 and GPIO2).
No, my ESP is a simple (AI Thinker) ESP8266-01, also with only 2 free GPIO (actually there are 4, RX and TX can be used as GPIO too) :)
you kindly could tell me how my variable-value pairs for my ESP query string should look like
So, for your URL 192.168.178.21/update.php?bmp085#temp=%value%&bmp085#press=%value%

the "Controller IP" would be 192.168.178.21, "Controller Port" would be 80 (assuming a standard apache installation)

Given the "Taskname" is "bmp085", the "Value Name 1" is "temp", the "Value Name 2" is "press", your template would look like this:

update.php?%tskname%#%valname%=%value%&%tskname%#%valname%=%value%

Please note that I actually don't know how more than one %valname%=%value%-pairs are handled, I asked this question in the other post mentioned above.

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#10 Post by rai » 15 Oct 2016, 15:42

@groovy
thanks for your answer. According to your hint I translated my query string as following (for test purpose) :
update.php?%bmp085%#%temp%=%value%
and this should send the data just for one name/value pair for temperature from my BMP085 sensor to my localhost 192.168.178.21. So my complete query string in my ESP Easy setup is:
192.168.178.21/update.php?%bmp085%#%temp%=%value%
My query string should be OK now, or is there still an error in it? Because when i call up my 'update.php' file from my server with the IP address 192.168.178.21 (localhost), my $_GET["bmp085#temp"] in my 'update.php' file has no value, it's empty! Following please find the very simple test code for my 'update.php' file:

Code: Select all

<?php
// refresh data every 5 seconds
echo '<meta http-equiv="refresh" content="5">';

$temp = $_GET["bmp085#temp"];
echo "Temperatur: " . $temp . "°C\n<br />";
?>
Do i understand the ESP Easy "sender/caller" principle correct, or am i completely wrong? Is the test-code in my 'update.php' file correct? If not, please give advise how to do it rigjht.
I feel sorry for my dumb questions for a normally easy task. In this moment my brain is like a black hole, and I turn around in circles.

I appreciate each answer/hint to solve my application problem.
Regards from Germany.
rai

groovy
Normal user
Posts: 39
Joined: 14 Oct 2016, 11:32
Location: Chemnitz, Germany

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#11 Post by groovy » 15 Oct 2016, 15:58

replace %bmp085% with %tskname% and name your task "bmp085", because %bmp085% is no recognized variable, but %tskname% is.

I'm new to this firmware too, just trying to get it work. Additionally, I'm no programmer, so I can't help with your backend.

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#12 Post by rai » 15 Oct 2016, 17:44

@groovy
replace %bmp085% with %tskname% and name your task "bmp085", because %bmp085% is no recognized variable, but %tskname% is.
that's what i did. my BMP085 sensor has the taskname bmp085!

sorry groovy, but from my point of view this can't be, because what happened if you have more than 1 task, like i do? I have 3 devices = 3 tasks:
display - this is task 1 with task name display
bmp085 - this is task 2 with task name bmp085
si7021 - this is task 32 with task name si7021
How can ESP Easy assign to the 3 different tasks 3 different values with one and the same %tskname% variable?
Regards from Germany.
rai

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#13 Post by rai » 16 Oct 2016, 00:55

sorry,
but i don't make any progress with my application because i cannot figure out how ESP Easy is naming/renaming the task- and value names behind the scenes. As result of that, i cannot compose the correct URL in menu "Tools" "Advanced" "Publish Template:". If i do not know the correct "Publish Template:" string with the included and correct named variable and value names, i cannot call up from my localhost the corresponding name/value pairs with my PHP $_GET[""] function.
is there really no one out there who can tell me how to use correctly the ESP Easy Generic HTTP settings/protocol for sending sensor data to a localhost, and how to request this sensor data from the localhost with PHP scripting langugae to store the requested data on the localhost? Maybe one of the brilliant developer of Easy ESP?
If i have a look to the number of views of my thread so far, i guess there must be a common interest in this matter.?.
Many thanks for all answers so far, especially to groovy. Still waiting for a practical solution.
Regards from Germany.
rai

Martinus

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#14 Post by Martinus » 16 Oct 2016, 13:11

Using the default template:

Code: Select all

demo.php?name=%sysname%&task=%tskname%&valuename=%valname%&value=%value%
On an ESP with name "ESPDemo" and running a DHT11 sensortask with name "DHT11" and default valuenames,
this would result in two http requests fired at your controller:

Code: Select all

demo.php?name=ESPDemo&task=DHT11&valuename=Temperature&value=21
demo.php?name=ESPDemo&task=DHT11&valuename=Humidity&value=65
Your demo.php file should process the valuepairs provided in the querystring and do something with them, like storing to a MySQL database.
Remember that each sensorvalue is processed as a single request. A BME280 would send 3 requests.

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#15 Post by rai » 17 Oct 2016, 15:22

@Martinus
Thanks for your answer. But if i adapt your example URL to my speficic application, i cannot call the data with a http request from my localhost with a PHP file. I guess i do not understand the "caller/receiver" concept of ESPs "Generic HTTP" protocol.

@all
How i have explained in detail in the very beginning of this thread, i have another sensor (Si7021) and a OLED-Display (SH1106 1.3") connected to my ESP, and all values are displayed correctly on the OLED!. But to make it short and easy for a helpful answer concening my"Generic HPPT" localhost problem, following i only show once again my setup for just one sensor, namely for the sensor BMP085:
- in menu "Config" in field "Name" i wrote "buero" - so my ESP is named buero, and the IP address from my ESP is 192.168.178.66!
- in menu "Config" in select field "Protocol:" i choosed "Generic HTTP".
- in menu "Config" in select field "Locate Controller:" i choosed "Use IP address".
- in menu "Config" in field "Controller IP:" i wrote "192.168.178.21", which is the IP-Address from my localhost!
- in menu "Devices" i made my sensor settings for "Temperature & Pressure - BMP085" as following:
Device: "Temperature & Pressure - BMP085"
Name: bmp085
Value Name 1: temp (for the Temperature)
Value Name 2: press (for the air pressure)
For both fields i choosed "Decimals: 1"
- in menu "Tools" "Advanced" in field "Publis Template:" i tried to adapt "Martinus" default URL as follwing:
update.php?name=%sysname%&task=%tskname%&valuename=%valname%&value=%value%
Then i wrote in PHP a short "update.php" script to get the valuepairs out of the query string send by my ESP "buero" for further data processing and storing the data to a MySQL database or to a *.csv file as following:

Code: Select all

<?php
//refresh the sensor data every 5 seconds
echo '<meta http-equiv="refresh" content="5">';

// my ESP variable for the temperature is "temp", so grab the temperature value from this variable
$temp = $_GET["temp"];
// show me the temperature
echo $temp;

// my ESP variable for the air pressure is "press", so grab the air pressure value from this variable
$press = $_GET["press"];
// show me the air pressure
echo $press;
?>
This very simple "update.php" file i placed for test purposes only into the root of my localhost (IP address 192.168.178.21) and i try to get my sensor data by calling the URL "192.168.178.21/update.php". This doesn't work at all. I do not get any server error message, but i do not get any sensor data as well.

@ Martinus and @ all:
I guess i missed the whole "Generic HTTP" ESP Easy protocol concept on how ESP Easy is sending the sensor data to the localhost and how the localhost then will request and receive the requested sensor data.
So who ever want to help me please take for your answer, instead of pseudo code, my above mentioned ESP system name with my a. m. ESP sensor and value names to show me the correct "Publish Template:" URL string. And please don't forget to show me a short example script for my PHP file to make a correct request to my ESP for the sensor data.
If i not only can display, but store the data values as well on my localhost for further data mining and data graphing, the Easy ESP framework would be the solution for my application. Please help to get me running, and for that many thanks in advance!
Regards from Germany.
rai

groovy
Normal user
Posts: 39
Joined: 14 Oct 2016, 11:32
Location: Chemnitz, Germany

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#16 Post by groovy » 17 Oct 2016, 17:32

First of all, set the "web log level" under tools in the advanced section to 4 and see, what the resulting URL-string actually is.

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#17 Post by rai » 17 Oct 2016, 18:33

@ groovy
thanks, but what do you mean with
First of all, set the "web log level" under tools in the advanced section to 4 and see, what the resulting URL-string actually is.
As you proposed I set the "web log level" to 4, but i cannot see any changes in the URL-String at all. Where and how should i see a change in the resulting URL?
The URL from my ESP stays always the same, namely at 192.168.178.66, and it only changes with the change of menu position, like:
- http://192.168.178.66/ -> if i click on the "Main" menu button
- http://192.168.178.66/config -> if i click on the "Config" menu button
- http://192.168.178.66/devices -> if i click on the "Devices" menu button
- http://192.168.178.66/tools-> if i click on the "Tools" menu button
- http://192.168.178.66/advanced -> if i click on the "Advanced" menu button
There is no visible URL from my http://192.168.178.66 ESP-system with any name/value pairs which sends any data to my http://192.168.178.21 localhost server, what raises for me the question:
does the word "Controller" in menu "Config" really mean my localhost server (192.168.178.21) or is it the name for the ESP itself with IP-address 192.168.178.66? If so, then i have to set the IP-address to my ESP IP-address 192.168.178.66, instead of IP-address 192.168.178.21 for my localhost?!
I tested that without any success by setting the "Controller" IP-address from 192.168.178.21 (localhost) to my ESP IP-address 192.168.178.66!
groovy: how did you setup your data request?
Beside the correct setup of my ESP "Publish Template" URL string, my main problem is the PHP script (e.g. update.php) to make the correct request to my ESP for the sensor data. Any helpful suggestions?
Regards from Germany.
rai

groovy
Normal user
Posts: 39
Joined: 14 Oct 2016, 11:32
Location: Chemnitz, Germany

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#18 Post by groovy » 17 Oct 2016, 18:54

Setting the "web log level" to 4 and watching the log (tools -> log), you should see something like this:

Code: Select all

102334782 : DHT : Humidity: 38.70
102334782 : EVENT: 30#v1=26.60
102334782 : EVENT: 30#v2=38.70
102334783 : HTTP : Delay 976 ms
102335759 : HTTP : connecting to 192.168.2.81 using port 80
102335763 : /shc/index.php?app=shc&a&ajax=pushsensorvalues&spid=3&sid=30&type=2&v1=26.60
102335965 : HTTP : closing connection
102336965 : HTTP : connecting to 192.168.2.81 using port 80
102336968 : /shc/index.php?app=shc&a&ajax=pushsensorvalues&spid=3&sid=30&type=2&v2=38.70
102337170 : HTTP : closing connection
As you can see, the variables are translated to the right values and sent to the server.

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#19 Post by rai » 18 Oct 2016, 17:13

Hello again,

@ groovy
thanks for your advice
Setting the "web log level" to 4 and watching the log (tools -> log), you should see something like this:
Your help has taken me a step further!


To make things easier, i now installed a BME280 sensor, because this sensor gives me all the data i need, namely Temperature, Air Pressure, and Humidity. Now i only have to take care about 1 task! I left my "Config" settings as mentioned in my previous post, but changed my "Tools" "Advanced" "Publish Template:" setting to:
"esp_easy/demo.php?task=%tskname%&valuename=%valname%&value=%value%"
I had to change the path because i made some changes in my localhost file structure.

In the next step i've scripted following PHP cURL code

Code: Select all

<?php
echo '<meta http-equiv="refresh" content="1">';

// Get cURL resource
$curl = curl_init();
// Set some options - we are passing in a useragent too here
curl_setopt_array($curl, array(
    CURLOPT_RETURNTRANSFER => 1,
    CURLOPT_URL => 'http://192.168.178.66/esp_easy/demo.php?task=bme280&valuename=Temperature&value=value',
    CURLOPT_USERAGENT => 'Codular Sample cURL Request'
));
// Send the request & save response to $resp
$resp = curl_exec($curl);
echo $resp;
// Close request to clear up some resources
curl_close($curl);
?>
into my "demo.php" file on my localhost server (192.168.178.21), which sends a cURL request for my sensor data to my ESP Easy (192.168.178.66).

With the a. m. settings and with the a. m. PHP cURL script i now can see in my Easy ESP log all my sensor data send to my server 192.168.178.21 (localhost) as following:
151991 : HTTP : Delay 820 ms
152811 : HTTP : connecting to 192.168.178.21 using port 80
152816 : /esp_easy/demo.php?task=bme280&valuename=Temperature&value=23.6
153213 : HTTP : closing connection
154213 : HTTP : connecting to 192.168.178.21 using port 80
154240 : /esp_easy/demo.php?task=bme280&valuename=Humidity&value=50.7
154680 : HTTP : closing connection
155680 : HTTP : connecting to 192.168.178.21 using port 80
155685 : /esp_easy/demo.php?task=bme280&valuename=Pressure&value=1021.6
156146 : HTTP : closing connection
But how to get my sensor data out of my cURL script? The data should be in variable $resp (update.php)!?. But this variable only contains information about the cURL request like:
URI: /esp_easy/demo.php Method: GET Arguments: 3 NAME:task VALUE:bme280 NAME:valuename VALUE:Temperature NAME:value VALUE:value
I figured out, that the cURL request has to go back to my ESP Easy (192.168.178.66), and not to my localhost (192.168.178.21), as i thought before! As soon as i set the request in my "demo.php" to my localhost IP address 192.168.178.21 the "demo.php" page hangs up, the sensor data in my ESP log stops reading, and the apache server port 80 goes crazy, because the port changes constantly to undefined values!

Resume so far:
- I figured out the "Controller" and the "Publish Template:" settings
- i figured out a HTTP cURL request (demo.php) which keeps my ESP sending and my localhost pulling my sensor data
But i still don't know how to get my sensor data out of the URL send from my ESP (192.168.178.66) to my localhost (192.168.178.21)!

groovy,
i feel sorry, that i cannot help you with your problem. If i look into your thread, i can see that you have the same information problem like i have. More or less we are talking to ourself!
I really hope, that another user, or maybe one of the developers, will help to find a solution for my "Generic HTTP" problem. There must be a solution for sending ESP sensor data to a private server (localhost).

Who ever can help, i kindly ask: please help! Many thanks in advance.
Regards from Germany.
rai

raptor
Normal user
Posts: 25
Joined: 25 Oct 2016, 10:39
Location: Montana / Bulgaria

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#20 Post by raptor » 25 Oct 2016, 22:18

Hello rai,

I will try to explain you in details how I get temperature and humidity values from my DHT11 sensor and write it to file or MySQL database on localhost.
First, go to config page and set these parameters
Untitled1.png
Untitled1.png (8.71 KiB) Viewed 36117 times
where Controller IP is your localhost ip, Controller Port is your localhost port and sensor delay to 60 sec. (in my case is 300 sec.)

Next, go to Tools->Advanced and set Publish Template like this:
esp/upload.php?name=%sysname%&task=%tskname%&valuename=%valname%&value=%value%
If your upload.php file is in C:/xampp/htdocs/esp_easy/ your publish template must be like this:
esp_easy/upload.php?name=%sysname%&task=%tskname%&valuename=%valname%&value=%value%

Reboot your module.

Go to localhost and create upload.php with this code:

<?php

//EXPERIMENTAL USE ONLY!

$date = date('l jS \of F Y h:i:s A');
error_log($date."\n", 3, "C:/xampp/htdocs/esp_easy/test_log.txt");

$valuename = $_GET['valuename'];
$value = $_GET['value'];

switch ($valuename) {
case 'Temperature':
$temp = $value;
error_log('Temperature - '.$temp.' C'."\n", 3, "C:/xampp/htdocs/esp_easy/test_log.txt");
break;
case 'Humidity':
$humidity = $value;
error_log('Humidity - '.$humidity.' %'."\n", 3, "C:/xampp/htdocs/esp_easy/test_log.txt");
break;
}

?>

This code get temp and humidity values from HTTP GET request and write them in test_log.txt

Now when esp module get sensor data it will make HTTP GET request to the upload.php on your localhost for every value of every sensor. I have one DHT11 sensor with two values (temperature and humidity), so when my module get sensor data it will send two HTTP GET request to upload.php one for temp and one for humidity. If you have 3 sensors with two values each you will have 6 HTTP GET requests to upload.php and etc...
The idea is that upload.php listen for HTTP GET requests with sensor values from Espeasy and when Espeasy make HTTP GET request to upload.php, upload.php get and processing this data.

This is how my Apache access log looks like:

192.168.100.2 - - [25/Oct/2016:21:26:32 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Temperature&value=25.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:26:33 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Humidity&value=61.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:31:33 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Temperature&value=25.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:31:34 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Humidity&value=62.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:36:32 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Temperature&value=25.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:36:33 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Humidity&value=62.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:41:33 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Temperature&value=25.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:41:34 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Humidity&value=62.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:46:33 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Temperature&value=25.00 HTTP/1.1" 200 - "-" "-"
192.168.100.2 - - [25/Oct/2016:21:46:34 +0300] "GET /esp/upload.php?name=SonoffExperiment&task=TempHumidity&valuename=Humidity&value=61.00 HTTP/1.1" 200 - "-" "-"

You can see that espeasy make HTTP GET request for every value of my DHT11 sensor.

This is my test_log.txt file:

Tuesday 25th of October 2016 09:26:32 PM
Temperature - 25.00 C
Tuesday 25th of October 2016 09:26:33 PM
Humidity - 61.00 %
Tuesday 25th of October 2016 09:31:33 PM
Temperature - 25.00 C
Tuesday 25th of October 2016 09:31:34 PM
Humidity - 62.00 %
Tuesday 25th of October 2016 09:36:32 PM
Temperature - 25.00 C
Tuesday 25th of October 2016 09:36:33 PM
Humidity - 62.00 %
Tuesday 25th of October 2016 09:41:33 PM
Temperature - 25.00 C
Tuesday 25th of October 2016 09:41:34 PM
Humidity - 62.00 %
Tuesday 25th of October 2016 09:46:33 PM
Temperature - 25.00 C
Tuesday 25th of October 2016 09:46:34 PM
Humidity - 61.00 %

This is some images from my configuration:
Untitled2.png
Untitled2.png (28.12 KiB) Viewed 36117 times
Untitled3.png
Untitled3.png (21.46 KiB) Viewed 36117 times
Untitled4.png
Untitled4.png (20.65 KiB) Viewed 36117 times
I'm beginner in IoT world. I hope I've helped you in some way. (sorry for my english :) )
Regards from Bulgaria.
raptor

sexton
New user
Posts: 1
Joined: 27 Oct 2016, 12:43

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#21 Post by sexton » 27 Oct 2016, 12:47

Thank you Raptor.Where there was a dark void the is now light.

raptor
Normal user
Posts: 25
Joined: 25 Oct 2016, 10:39
Location: Montana / Bulgaria

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#22 Post by raptor » 27 Oct 2016, 15:01

I'm glad to be helpful :)
Regards from Bulgaria.
raptor

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#23 Post by rai » 28 Oct 2016, 00:00

Hello again,
since i was out of home for a couple of days i couldn't get back earlier to my thread. Sorry for that, but finally i found a sufficiant solution for me.

@ raptor
Thanks a lot for your answer. I could understand every word of your post, so your english is OK. Your answer really gives some clear application advices in this matter.

Anyway, i had already found my own solution, which i will herewith present and explain step by step.
First try:
My apache "access log" showed up the correct GET request as following:
192.168.178.66 - - [22/Oct/2016:15:26:53 +0200] "GET /webdev/myhome/update.php?sl=buero&s=bme280&st=Temperature&sv=21.9 HTTP/1.1" 200 1790 "-" "-"
192.168.178.66 - - [22/Oct/2016:15:26:54 +0200] "GET /webdev/myhome/update.php?sl=buero&s=bme280&st=Humidity&sv=51.7 HTTP/1.1" 200 1790 "-" "-"
192.168.178.66 - - [22/Oct/2016:15:26:56 +0200] "GET /webdev/myhome/update.php?sl=buero&s=bme280&st=Pressure&sv=1015.0 HTTP/1.1" 200 1790 "-" "-"
This "access.log" clearly shows, that my ESP Easy sends the correct values. I didn't want to hack my server "acces.log" file, so i updated my "httpd.conf" file as following:
Go to section
<IfModule log_config_module>
..........
..........
..........
</IfModule>
and make a new common LogFormat entry like
LogFormat "%t %q" common
and a new path and log-filename entry like
CustomLog "D:/xampp/htdocs/home/logs/espeasy-data.log" common
All together it looks similar to something like
<IfModule log_config_module>
..........
LogFormat "%t %q" common
..........
CustomLog "D:/xampp/htdocs/home/logs/espeasy-data.log" common
..........
</IfModule>
This will save the ESP Easy Data in path "D:/xampp/htdocs/home/logs/" in file "espeasy-data.log" as following:
[22/Oct/2016:16:16:44 +0200] ?name=Buero&task=bme280&valuename=Temperature&value=21.2
[22/Oct/2016:16:16:45 +0200] ?name=Buero&task=bme280&valuename=Humidity&value=49.9
[22/Oct/2016:16:16:46 +0200] ?name=Buero&task=bme280&valuename=Pressure&value=1032.5
This is much shorter and more convinient as the "access.log" part! To save the data like this for further data-handling would be sufficiant.

Second and final try:
But for the long run i've looked for a more professional way to save my data into a CSV-File (Comma Separated Value). I ended up with following solution, which now represents my final version for the handling of reading and writing my sensor data to my localhost (XAMPP Version 5.6.24). The basic settings for "Generic HTTP", "Devices" (BME280) and "Publish Template:" are all the same as described before in this thread!

My sensor data is accessible in following PHP GET variables (which are all ESP Easy standard variable names):
$_GET['name'];
$_GET['task'];
$_GET['valuename'];
$_GET['value'];
The data, stored in this GET variables, will be parsed as described in the following "get_data.php" file:

Code: Select all

<?php

// Unix Zeitstempel
$unix_stamp = time();
// Datum und Uhrzeit
$date_and_time = date('d.m.Y-H:i:s');

// Sensor Ort
$location = $_GET['name'];
// Sensortyp
$sensor = $_GET['task'];
// Wertname (entweder Temperatur oder Luftdruck oder Luftfeuchte)
$valuename = $_GET['valuename'];
// Messwert von Temperatur oder Luftdruck oder Luftfeuchte
$value = $_GET['value'];

// Pfad zur CSV-Datendatei
$csv_path = "data/";
// Dateiname der CSV-Datendatei
$csv_filename = $location . ".csv";
// Pfad und CSV-Dateiname
$csv_path_and_filename = $csv_path.$csv_filename;

// Kopfzeile - Spaltennamen der CSV-Datendatei
$csv_header = "Nr.\t;Sensor-Ort\t;Unix Timestamp\t;Datum u. Uhrzeit\t;Sensortyp\t;Temperatur C\t;Luftfeuchte %\t;Luftdruck mb (NN)\n";

// Wenn die CSV-Datendatei nicht existiert, dann erstelle sie mit der Kopfzeile neu
if (!file_exists($csv_path_and_filename)) {
	$datei = fopen($csv_path_and_filename,"w");
	fwrite($datei, $csv_header);
	fclose($datei);	
}
// Zeilenanzahl der CSV-Datendatei ermitteln - Achtung: 0-basiert (Null-basiert)!
// Kopfzeile ist die Zeile 0. Die Zeile 1 fängt also wie gewuenscht mit 1 an!
$anzahl = count(file($csv_path_and_filename));
  
// Die nacheinander folgenden Sensorwerte auslesen und in eigene Variablen einlesen
switch ($valuename) {
	case 'Temperature':
		$temp = $value;
	 	error_log($anzahl."\t;"."\t".$location."\t;"."\t".$unix_stamp."\t;".$date_and_time.';'."\t".$sensor."\t;"."\t".$temp."\t\t;", 3, $csv_path_and_filename);
		break;
	case 'Humidity':
		$humidity = $value;
		error_log("\t".$humidity."\t\t;", 3, $csv_path_and_filename);
		break;
	case 'Pressure':
		$pressure = $value;
		error_log("\t".$pressure."\n", 3, $csv_path_and_filename);
		break;
	Default:
		error_log("Messfehler!", 3, $csv_path_and_filename);		
}	

?>

Sorry, but due to lack of time i will leave the comments in german language. 
OK, the result is a nice and clean csv data file with a clear data structure inside. ESP Easy will update the sensor data every 5 minutes, and my CSV data now looks like:
Nr. ;Sensor-Ort ;Unix Timestamp ;Datum u. Uhrzeit ;Sensortyp ;Temperatur C ;Luftfeuchte % ;Luftdruck mb (NN)
1 ; Buero ; 1477600137 ;27.10.2016-22:28:57; bme280 ; 24.2 ; 51.0 ; 1034.6
2 ; Buero ; 1477600438 ;27.10.2016-22:33:58; bme280 ; 24.1 ; 50.8 ; 1034.7
3 ; Buero ; 1477600738 ;27.10.2016-22:38:58; bme280 ; 24.1 ; 50.6 ; 1034.7
4 ; Buero ; 1477601039 ;27.10.2016-22:43:59; bme280 ; 24.1 ; 51.0 ; 1034.7
5 ; Buero ; 1477601339 ;27.10.2016-22:48:59; bme280 ; 24.2 ; 51.5 ; 1034.6
6 ; Buero ; 1477601640 ;27.10.2016-22:54:00; bme280 ; 24.2 ; 50.6 ; 1034.7
7 ; Buero ; 1477601940 ;27.10.2016-22:59:00; bme280 ; 24.1 ; 50.0 ; 1034.7
Please note:
others as shown above, due to the "\t" signs (Tabulator) in the "get_data.php" file all columns/values of the head line and all following data lines are set exactly under each other (in the stored CSV data file)!

The main goal is, that there are tons of PHP CSV functions for further data handling! And if your CSV data file gets to big, you easyly can migrate to a MySQL Data Base!

If nobody comes up with a better/easier way of how to do it, i will stay with my solution as described before. For each room now i have a little cheap WiFi sensor for measuring and displaying Temperature, Humidity and Air Pressure as well as for further data handling/research grabbed from my "Big Data" stored on my localhost.

There are now round about 1100 views for this thread. So there must be a certain common interest in this "Generic HTTP" matter. That's the reason why i presented my final solution to the public. I would be glad, if the two solutions (raptor or mine) in this thread would be for any help for some people in the future.
Regards from Germany.
rai

rai
Normal user
Posts: 11
Joined: 14 Oct 2016, 00:54

Re: How to send Sensor Data from Easy ESP to my localhost with generic HTTP?

#24 Post by rai » 28 Oct 2016, 15:50

Hello,

in addition to my yesterdays post please find following screenshot of my csv datafile created with my "get_data.php" file.

Image

The above screenshot shows, what i meant in my yesterdays post with
Please note:
... all columns/values of the head line and all following data lines are set exactly under each other (in the stored CSV data file)!
P. S.
Sorry. The image (sreenshot) i put here will only be displayed for about 15-30 minutes. After this short time the image is gone, and only the word "Image" is left (see above). I tried it with images stored on "Dropbox" or "MS One Drive". Both won't work, and I don't know why!?
Regards from Germany.
rai

Post Reply

Who is online

Users browsing this forum: No registered users and 25 guests