How to make online logging from scratch

Published on December 09, 2014

How to make online logging from scratch

    Described below is a weekend design. A couple of nights were spent from the idea to implementation. The description provides the necessary minimum for implementing an information collection system, with access to the latter in Online mode.

    Already in the process of writing this note, an article appeared: Wintering cacti with online temperature control . In the beginning, he was afraid that they were ahead of the curve, however, although the problem being solved is close in both cases, the implementation is radically different.
    The architecture of the resulting system is presented in the diagram:
    As you can see, the client part is fully implemented in the microcontroller. The system is easily scalable and customizable to any needs.


    Keil MCB2300, which was idle for a long time, was taken as the customer’s base. The board is built on the LPC2368 microcontroller, it is quite old (at the moment it is more reasonable to use the LPC1768 instead), but it is crammed with everything necessary. There is an Ethernet interface on the board, which in this case is a window to the world. As a data source, I took BMP085, long familiar to me, connected via the I 2 C interface . The
    software for testing the server and checking the operation of HTTP requests was written as part of a Win32 application:
    Subsequently, the same logic passed into the microcontroller. Currently, the request is addressed to a specific IP address, but it is better to use a DNS client, as in theory, a hosting operator can transfer a site within its address space. You can also add an NTP client to send data to the server immediately with a time stamp.
    This project is my first test of working with the TCP / IP protocol on a microcontroller. The example is based on EasyWeb, which is offered for many boards with Ethernet as a starting one. The main program cycle turned out to be crooked, however, I do not want to comb it, because the necessary functionality has already been implemented, but I want to try moving this application to the uIP stack, because I like it more in its internal structure.
    The main cycle of the program:
    1. We read the data from the sensor.
    2. Open the connection.
    3. Sends an HTTP GET request to the server.
    4. If 10 seconds have passed, and the protocol for communicating with the server has not passed, we break the connection by timeout.
    5. We are waiting for 1-5 minutes. This wait sets the data storage interval.
    And here is the appearance of the whole device:


    The server part was deployed on a shared hosting. The main requirement for it: PHP support. Data is saved as CSV files. The scripts are as simple as possible, the functionality is divided into 2 files:
    • add.php - script to add data to the log file. Gets the file parameter with the file name for writing and the data parameter with data for writing. The first column of the script adds a timestamp. There is also a clear function that allows you to clear the log file.
    • get.php - display data from a log file. Gets the only file parameter with the file name.

    All data is transmitted to the server as part of GET requests:
    GET http: //server/add.php? File = temp & data = 23.4; 99809 HTTP / 1.0 \ r \ n \ r \ n

    And here is the contents of the CSV file:
    1418069433; 23.4; 99809
    1418069443; 23.4; 99811
    1418069453; 23.4; 99818
    1418069464; 23.4; 99801

    Each line contains a time stamp, temperature and pressure in Pascals.

    Graphical output

    To make everything look beautiful, other forms of data display can be organized on the server. I tried to use Highcharts as a library of charts / graphs . You can make such a design with it:
    This is not a live schedule, this is a screenshot of its appearance. In live view, the graph can be viewed at the links: Temperature and Pressure .
    UPD: Combined both graphs on one page: Temperature / Pressure .