Logging Data to an SD card

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Logging Data to an SD card

#1 Post by moelski » 03 Oct 2016, 20:45

Hi @all,

today I want to show you my latest developement with / for ESPEasy ... You may have noticed that I have done a new plugin for the SPI adapter MAX6675 / MAX31855. After I finished the plugin I thought it could be nice to add a SD card interface since it uses also the SPI interface. ;)

After some grabbing in my components I found an adapter from CATALEX. See the pictures in the attachement. It´s connected with 5V, GND and SPI.

It needs some SPI additions in ESPEasy (which I hope will be merged ... :roll: ) to get the SD card working.
But after 3-4h of testing it´s working :ugeek:

Now you may ask why we need a SD card controller if we can send the data wireless ... Well there are situations where logging to the network is not the best option.
Think about time limited loggings or loggings at places where you don´t have access to your network.

Anyway I think this could be a great addition to ESPEasy. If someone is interested in this plugin I can add it to the Playground.

Find attached some pictures and a sample file.
Attachments
DATA.zip
Sample Data File ...
(696 Bytes) Downloaded 423 times
Data computation is very easy with EXCEL and some filtering ...
Data computation is very easy with EXCEL and some filtering ...
Capture.PNG (133.08 KiB) Viewed 18204 times
I used a Thermocouple via SPI, SI7021 I2C Sensor and the SD Card and there are still pins left :-D
I used a Thermocouple via SPI, SI7021 I2C Sensor and the SD Card and there are still pins left :-D
IMG-20161003-WA0016.jpg (175.88 KiB) Viewed 18204 times
CATALEX Adapter.jpg
CATALEX Adapter.jpg (14.79 KiB) Viewed 18204 times
regards
Dominik

stefbo
Normal user
Posts: 19
Joined: 24 Apr 2016, 15:35
Location: Germany

Re: Logging Data to an SD card

#2 Post by stefbo » 04 Oct 2016, 23:06

Hi Dominik,
this is something I was also thinking about. I would be very interested in your code. I ordered a SD-card interface some days ago. As soon as it arrives I will go testing
Stefan

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: Logging Data to an SD card

#3 Post by moelski » 05 Oct 2016, 07:38

Hi Stefan,
I can add the code as soon as I have finished it.
What you see here in the post was the first working example ;)
So I need to implement some settings and then you can give it a try.

Oh and you need the R134 from GIT because it includes the basic SPI configuration / Init.
regards
Dominik

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: Logging Data to an SD card

#4 Post by moelski » 06 Oct 2016, 05:33

Work in progress ;)

I just want to show you the settings page for the SD Carc Controller Plugin.
Now there are some file handling and IF conditions left to be coded :P

If you have any missing additions please let me know ...
Attachments
Settings on Config page
Settings on Config page
Settings.PNG (14.87 KiB) Viewed 17959 times
regards
Dominik

JR01
Normal user
Posts: 260
Joined: 14 Feb 2016, 21:04
Location: South Africa

Re: Logging Data to an SD card

#5 Post by JR01 » 06 Oct 2016, 16:38

Very very very cool, well done. Will for sure use it.
-----------
IOTPLAY. Tinkerer, my projects are @ http://GitHub.com/IoTPlay, and blog https://iotplay.org. Using RPi, Node-Red, ESP8266 to prove Industry 4.0 concepts.

timsson
Normal user
Posts: 77
Joined: 25 Mar 2016, 22:00

Re: Logging Data to an SD card

#6 Post by timsson » 07 Oct 2016, 01:01

:shock: cool!
maybe file save or sendto....

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: Logging Data to an SD card

#7 Post by moelski » 07 Oct 2016, 06:38

maybe file save or sendto....
great idea 8-)

But first let me finish the "normal" SD Card writing. I think it will be done this weekend.

And then I thought it could be cool to have a very small kind of file browser.
With a limited set of options:
* single file delete
* File download
* File View
* create / delete Directory
* maybe file move

But I have to think about that first. Have no idea at the moment where to place such peace of candy .. sorry code :D
regards
Dominik

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: Logging Data to an SD card

#8 Post by moelski » 08 Oct 2016, 20:57

Hi @all,

I have just finished the Controller. It was more work then i expected before :roll:
Anyway it´s done and now I will do some tests this night.

Can upload it tomorrow here as code or I place a pull request in the ESPEasy repo. :ugeek:
regards
Dominik

JR01
Normal user
Posts: 260
Joined: 14 Feb 2016, 21:04
Location: South Africa

Re: Logging Data to an SD card

#9 Post by JR01 » 08 Oct 2016, 22:10

+1 !
-----------
IOTPLAY. Tinkerer, my projects are @ http://GitHub.com/IoTPlay, and blog https://iotplay.org. Using RPi, Node-Red, ESP8266 to prove Industry 4.0 concepts.

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: Logging Data to an SD card

#10 Post by moelski » 09 Oct 2016, 08:54

Hi !

I added a pull request for the new controller.
https://github.com/ESP8266nu/ESPEasy/pull/83

Let´s see if it is ok for the devs :ugeek:
regards
Dominik

pm_cz
Normal user
Posts: 10
Joined: 16 Mar 2016, 10:56

Re: Logging Data to an SD card

#11 Post by pm_cz » 09 Oct 2016, 09:16

So if I understand correctly from the code, you may either send data to some server or write them to SD card but not both at the same time? I guess there is a reason why it has not been possible to implement is as an output plugin (like _P012_LCD.ino or _P023_OLED.ino). It would be interesting if you could share it since I was playing with the idea of SD plugin myself.

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: Logging Data to an SD card

#12 Post by moelski » 09 Oct 2016, 15:26

So if I understand correctly from the code, you may either send data to some server or write them to SD card but not both at the same time?
Correct. This is a controller plugin. So you can use it like MQTT or FHEM. Nut as per default ESPEasy can handle only one controller at a time.
I guess there is a reason why it has not been possible to implement is as an output plugin
It would be possible to use this as an output plugin. But the problem is that you have to check all the configured sensors when you wan´t to write a value to the SD card.
And you can´t use the default sensor delays in an output - because they won´t get triggered.
It would be interesting if you could share it since I was playing with the idea of SD plugin myself.
Take a look at the github pull request. If the devs find it ok - maybe you can donwload it directly in R137 ... ;)
regards
Dominik

pm_cz
Normal user
Posts: 10
Joined: 16 Mar 2016, 10:56

Re: Logging Data to an SD card

#13 Post by pm_cz » 09 Oct 2016, 15:39

moelski wrote:
It would be interesting if you could share it since I was playing with the idea of SD plugin myself.
Take a look at the github pull request. If the devs find it ok - maybe you can donwload it directly in R137 ... ;)
Sorry for misuderstanding, what I meant here was that I wanted to know the reason why not use the output plugin way. I did look into the diffs when you anounced the pull request. It seems that the best way would be to allow multiple output plugins, but this would probably mean rewriting quite a lot of the code, or to integrate the SD output in another way.

We will see what martinus thinks soon anyway...

Regards,
Pavel

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: Logging Data to an SD card

#14 Post by moelski » 09 Oct 2016, 16:09

what I meant here was that I wanted to know the reason why not use the output plugin way
I told you why ... ;)
But the problem is that you have to check all the configured sensors when you wan´t to write a value to the SD card.
And you can´t use the default sensor delays in an output - because they won´t get triggered.
regards
Dominik

timsson
Normal user
Posts: 77
Joined: 25 Mar 2016, 22:00

Re: Logging Data to an SD card

#15 Post by timsson » 15 Oct 2016, 19:01

... for me nearly perfect in a standalone network without openhab!
I use the wemos mini with sdboard...
perhaps a possibility to reading the files?
Thanks al lot

User avatar
moelski
Normal user
Posts: 161
Joined: 31 Aug 2016, 06:33
Location: Germany - NRW
Contact:

Re: Logging Data to an SD card

#16 Post by moelski » 15 Oct 2016, 23:11

Hi !
perhaps a possibility to reading the files?
Where do you want to read a file? In ESPEasy?
For which reason?
regards
Dominik

timsson
Normal user
Posts: 77
Joined: 25 Mar 2016, 22:00

Re: Logging Data to an SD card

#17 Post by timsson » 16 Oct 2016, 00:44

moelski wrote: Where do you want to read a file? In ESPEasy?
For which reason?
no, the best way for me, file download from ESPEasy someway
why? - because i can't remove my sd-card - its in a case...
Thanks

joshi
New user
Posts: 5
Joined: 16 Oct 2016, 18:36

Re: Logging Data to an SD card

#18 Post by joshi » 16 Oct 2016, 18:40

Hi,
this logging feature is exactly what I need - thanks a lot. But where I can find it ? It has not been added to the latest releases and I was not able to find it in the playground.
May I ask a kind soul ;) for guidance ?
TIA
Joshi

5ko
Normal user
Posts: 33
Joined: 03 Jan 2016, 17:58
Contact:

Re: Logging Data to an SD card

#19 Post by 5ko » 27 Nov 2016, 20:58

+1
Regards

Drum
Normal user
Posts: 300
Joined: 07 Feb 2016, 11:56

Re: Logging Data to an SD card

#20 Post by Drum » 07 Dec 2016, 13:35

moelski wrote:Hi !
perhaps a possibility to reading the files?
Where do you want to read a file? In ESPEasy?
For which reason?
Reading a file to upgrade the firmware without a wifi connection would be a nice feature, for a remote datalogger. Maybe update settings via "restore" from file as well.
If using as a remote data logger you could also disable the wifi and save power, unless you are war-walking..... ;)

Post Reply

Who is online

Users browsing this forum: No registered users and 8 guests