The term "smart home" combines the features that allow the user to control home devices from their smartphone, tablet or laptop. For example, remotely control the heating of a room from a telephone. Nevertheless, many believe that there are two main obstacles to the widespread adoption of smart homes: there is no single unified solution and they are still not practical enough. The Newton project, one of Intel's latest groundbreaking projects, can help end these disputes.

Problems

• “There is no single, unified solution.” Users want to make their own choices. They want to buy smart TVs, refrigerators, air conditioners from Apple, Samsung, Tencent, Xiaomi and many others. But each manufacturer has their own solution for a smart home, for example, they have different exchange protocols. This leads to the fact that it is almost impossible to somehow standardize the ecosystem of a smart home.
• "They are still not practical enough." Previous smart home management solutions had some drawbacks. Voice control can be complicated by ambient noise. Managing from the phone is inconvenient because it requires removing the phone, launching the corresponding application and choosing the device you want to control. Gesture control through the camera depends on the ambient lighting and your position in the room.

Newton Project

The Newton project is designed for a complete, improved smart home management solution. His tasks:

• Connect any smart home devices.
• Do not depend on wearable devices.
• Have a low price.
• Independent of environmental conditions, lighting level, noise level or user position.

We developed an improved IoT-solution for managing a smart home, which was called the "Newton project." It implements the connection of all major platforms (Intel Core, Atom, Centrino processor and the ARM mobile platform) and all IoT platforms (Intel Edison, Intel Galileo, Raspberry, Spark, Mbed, Freescale, Arduino Uno and so on). Thus, the Newton project can connect platforms on which all major operating systems (Windows, Linux, Android) and specialized operating systems without a user interface (Mbed, Contiki, RIOT, Spark, OpenWRT, Yocto, WindRiver, VxWorks, Raspbian and so on) are executed Further).
Smart home providers typically define only protocols for communication at the application level, but these protocols are relatively closed. In the Newton project, we use the CoAP protocol (Constrained Application Protocol - a restricted protocol, application level), which is a software protocol designed to communicate simple electronic devices via the Internet. The CoAP protocol is based on RESTful and makes it easy to build smart gateways. Here is the basic prototype design of the CoAP protocol in the Newton project.



We list all supported operating systems. The Intel Galileo, Intel Edison, UNO, and Spark motherboards use Arduino programming standards. The popular LPC1768 IoT board supports Mbed and RIOT operating systems. Mbed is a platform and operating system for ARM devices connected to the Internet. It is convenient for managing hardware resources by calling the corresponding objects. RIOT is an open source operating system that supports multi-threading and several different development boards. You can use the C language to work with it. The LPC1768 chip will need an external Wi-Fi module. Windows, Linux and Android can also be easily adapted to the CoAP protocol. The architecture of the Newton project is shown below.

CoAP Implementation

The open source implementation of CoAP for microcontrollers is called MicroCoAP . This small library is written in the standard C language, so it can simply be ported to various platforms. MicroCoAP contains four files (CoAP.h, CoAP.c, endpoints.c, and main.c).
The CoAP protocol is implemented in the CoAP.h and CoAP.c files, which is shown below, and the endpoints.c file includes response functions associated with special nodes.



The main.c file creates a CoAP server. We mainly use coap_packet_t, coap_parse, coap_handle_req and coap_build. coap_packet_t defines the data structure of CoAP packets. The coap_parse function parses the hexadecimal data received from the network or serial port and translates it into the coap_packet_t structure. The coap_handle_req function analyzes the received CoAP packet and creates suitable responses. The coap_build function translates the response packet into hexadecimal format. The key source code for creating a CoAP server is shown below.

Arduino implementation

Arduino is an open source prototyping platform that defines the hardware and software standards of Arduino. Intel Edison, Intel Galileo, and Spark boards follow these standards. The program for Arduino consists of two functions: setup and loop. The setup function initializes the hardware and is called only once before the loop function. The loop function is an endless loop that can be used to perform the main task.
To add CoAP support to the Arduino board, just add three files (CoAP.h, CoAP.c, endpoints.c) to the project and change the setup and loop functions to match main.c in microCoAP.

Mbed implementation

Mbed is an object-oriented C ++ library designed for the ARM Cortex-M processor. We can manage general purpose pins (GPIOs) and other hardware resources using the appropriate classes. However, there is no standard Wi-Fi module or library. Therefore, we will use the UART-to-WIFI module to add the ability to work with Wi-Fi to Mbed. The WIFI class is based on the documentation for the UART-WIFI module, and each function will send a corresponding line to the UART-WIFI module.
You need to add three files to the project (CoAP.h, CoAP.c, endpoints.c) and change the main.c file in accordance with the main.c file in the microCoAP project. To compile, just run the make command - everything in the Makefile is already configured correctly.

RIOT implementation

RIOT is an developer-friendly, open source operating system for IoT that supports multiple boards, such as the Mbed LPC1768 and Spark Core development kit. Development can be done in C and C ++. The tests folder contains many sample APIs for connecting hardware. As with Mbed, the CoAP server can be implemented by changing the Wi-Fi class from Mbed to a C function and changing the serial port API. However, in our experiments there were frequent drops, so for now it can only be used for testing. To compile, use the command in the root folder:

BOARD=Mbed_lpc1768 make clean all flash

To monitor the serial port and get output from Mbed, use:

BOARD=Mbed_lpc1768 make term

The picture below shows the structure of the CoAP server code for RIOT.

Contiki implementation

Contiki is a compact, multi-tasking, portable, open source operating system. The example folder contains several examples of using the API to manage hardware. There is also no standard Wi-FI module. To implement a CoAP server, use similar functions from RIOT and rewrite the example with UART. To compile, use the “TARGET = cc2530dk make” command and load the binary using jlink.

OpenWRT implementation

OpenWRT is an embedded Linux-based operating system used for routers. It supports the standard API for Linux. Thus, you can directly use the microCoAP code. In the package folder, install the microCoAP project and change the Makefile to match the Makefile of your project. Then run the make command in the root directory. The compiled application will be in the bin folder. The application can be downloaded to the board via ftp or via usb stotage. Finally, use the command to install the CoAP application in OpenWRT:

opkg command install CoAP*.ipk

Add “/ usr / bin / CoAP &” to the /etc/rc.local file. The code structure of the CoAP server for OpenWRT is shown below.

Ease of use in the Newton project

For an IoT solution to be convenient, you need to be able to control any IoT device using ordinary gestures, without using any wearable devices, and it should not depend on environmental conditions, such as lighting, noise, etc. One way to this solution can be implemented using a 9-axis gyro to determine the movement of the user's hand.

Example of a 9-axis gyro with a Wi-Fi module:



The recognition algorithm analyzes data in real time and determines the user's position, direction and gestures. By calculating the relative position and direction between the user and the IoT device, the system can determine which IoT device the user is directed to. In the future, calculating the direction of the hand and gestures, the IoT device will be selected and controlled.

Newton Project Demonstration

This is a demonstration in our laboratory where the Newton project was used to control various devices with different operating systems.
The whole set of devices located in our laboratory:



Selection (blue color) and control (green color) of the Intel Edison board in the Newton project:



Control of the Android device (tablet) in the Newton project:



Robot arm



control: Machine control:

Plans

The Newton project is a good complete solution for managing a smart home. It is easy to use, it allows you to connect any device and does not depend on any external parameters, such as lighting, noise level and so on.
However, like any other open source projects, the Newton project can be improved. The next step we want to improve gesture recognition, increase productivity and connect other wearable devices. For example, they say that the next-generation Google Glass glasses can boast a larger prism for the eyes and will be based on the Intel Atom processor, and this may be the best management solution in the Newton project.