September 8, 2017 at 16:59Using Python to process real-time information from sensors working with Arduino
Formulation of the problem
Digital and analog sensors connected to the Arduino generate large amounts of information that requires real-time processing [1].
Currently, data from Arduino is printed from the command line or displayed in the GUI with a delay. Therefore, the data in real time are not stored, which makes their further analysis impossible.
This publication is devoted to a software solution to the problem of storing information from sensors working with Arduino and its graphical representation in real time. The examples are used by well-known sensors such as a potentiometer and a PIR motion sensor.
Using CSV files to store data received from sensors working with Arduino
- To write data to a CSV file, you can use a simple listing:
import csv data = [[1, 2, 3], ['a', 'b', 'c'], ['Python', 'Arduino', 'Programming']] with open('example.csv', 'w') as f: w = csv.writer (f) for row in data: w.writerow(row) - The following listing can be used to read data from a CSV file:
import csv with open('example.csv', 'r') as file: r = csv.reader(file) for row in r: print(row)
Consider saving Arduino data using two sensors as an example: a potentiometer with an analog output signal and motion (PIR) with a digital output signal.
The potentiometer is connected to analog output 0, and the PIR motion sensor to digital output 11, as shown in the following diagram:
For this circuit to work, you need to load the pyFirmata module and the StandardFirmata sketch into the Arduino board in Python.
In any Python file, place the following code, which starts and stops writing data from both sensors to the SensorDataStore.csv file:
Listing No. 1
#!/usr/bin/python
import csv
import pyfirmata
from time import sleep
port = 'COM3'
board = pyfirmata.Arduino(port)
it = pyfirmata.util.Iterator(board)
it.start()
pirPin = board.get_pin('d:11:i')
a0 = board.get_pin('a:0:i')
print(pirPin)
with open('SensorDataStore.csv', 'w+') as f:
w = csv.writer(f)
w.writerow(["Number", "Potentiometer", "Motion sensor"])
i = 0
pirData = pirPin.read()
m=25
n=1
while i < m:
sleep(1)
if pirData is not None:
i += 1
potData = a0.read()
pirData = pirPin.read()
row = [i, potData, pirData]
print(row)
w.writerow(row)
print ("Done. CSV file is ready!")
board.exit()
As a result of Listing 1, we’ll get a record of the data in the file 'SensorDataStore.csv':
Consider the code associated with the storage of sensor data. The first line of the CSV file entry is the header line that explains the contents of the columns: w.writerow ([“Number”, “Potentiometer”, “Motion sensor”]) .
When the dynamics of data changes appears, which for the given circuit can be artificially created by turning the knob of the potentiometer or by moving a hand near the motion sensor, the number of records in the data file becomes critical. To restore the waveform, the frequency of writing data to a file should be twice the frequency of the signal. To adjust the recording frequency, the step - n can be used, and to adjust the measurement time, the number of cycles - m. However, the limitations on n from below are imposed by the speed of the sensors themselves. The following code fragment performs this function in the above program:
m=25
n=1
while i < m:
sleep(n)
if pirData is not None:
i += 1
row = [i, potData, pirData]
w.writerow (row)
The given program can be changed in accordance with the design requirements as follows:
- You can change the pin numbers of the Arduino and the number of pins to be used. This can be done by adding in the Python code and the StandardFirmata sketch in Arduino strings of additional values for the new sensors.
- CSV file: the file name and its location can be changed from SensorDataStore.csv to the one that relates to your application.
- The frequency of records m in the SensorDataStore.csv file can be changed by changing the step, and the recording duration by changing the number of cycles with a constant step.
Graphical analysis of data from a CSV file
Using the SensorDataStore.csv file, we will create a program for learning data arrays from a potentiometer and a motion sensor and plotting graphs for these arrays:
Listing # 2
import sys, csv
import csv
from matplotlib import pyplot
i = []
mValues = []
pValues = []
with open('SensorDataStore.csv', 'r') as f:
reader = csv.reader(f)
header = next(reader, None)
for row in reader:
i.append(int(row[0]))
pValues.append(float(row[1]))
if row[2] == 'True':
mValues.append(1)
else:
mValues.append(0)
pyplot.subplot(2, 1, 1)
pyplot.plot(i, pValues, '-')
pyplot.title('Line plot - ' + header[1])
pyplot.xlim([1, 25])
pyplot.xlabel('X Axis')
pyplot.ylabel('Y Axis')
pyplot.subplot(2, 1, 2)
pyplot.bar(i, mValues)
pyplot.title('Bar chart - ' + header[2])
pyplot.xlim([1, 25])
pyplot.xlabel('X Axis')
pyplot.ylabel('Y Axis')
pyplot.tight_layout()
pyplot.show()
As a result of Listing 2, we get two graphs on the same form to display the output data of the potentiometer and motion sensor in real time.
In this program, we created two arrays of sensor values - pValues and mValues - by reading the SensorDataStore.csv file line by line. Here pValues and mValues represent the sensor data for the potentiometer and motion sensor, respectively. Using the matplotlib methods , we construct two graphs on the same form.
The generated code after each cycle of generating pValues and mValues data arrays received from Arduino sensors updates the interface, which allows us to conclude that data is received in real time.
However, the data storage and visualization method has the following features - the entire data set is first written to the SensorDataStore.csv file (Listing No. 1), and then read from this file (Listing No. 2). Graphs should be redrawn every time new values from Arduino arrive. Therefore, it is necessary to develop a program in which the planning and updating of schedules takes place in real time, and the entire set of sensor values is not built, as in listings No. 1.2. [2].
We are writing a program for a potentiometer creating the dynamics of the output signal by changing its active resistance to direct current.
Listing 3
import sys, csv
from matplotlib import pyplot
import pyfirmata
from time import sleep
import numpy as np
# Associate port and board with pyFirmata
port = '/dev/cu.usbmodemfa1321'
board = pyfirmata.Arduino(port)
# Using iterator thread to avoid buffer overflow
it = pyfirmata.util.Iterator(board)
it.start()
# Assign a role and variable to analog pin 0
a0 = board.get_pin(''a:0:i'')
# Initialize interactive mode
pyplot.ion()
pData = [0] * 25
fig = pyplot.figure()
pyplot.title(''Real-time Potentiometer reading'')
ax1 = pyplot.axes()
l1, = pyplot.plot(pData)
pyplot.ylim([0,1])
# real-time plotting loop
while True:
try:
sleep(1)
pData.append(float(a0.read()))
pyplot.ylim([0, 1])
del pData[0]
l1.set_xdata([i for i in xrange(25)])
l1.set_ydata(pData) # update the data
pyplot.draw() # update the plot
except KeyboardInterrupt:
board.exit()
break
As a result of Listing 3, we’ll get a schedule.
Real-time planning in this exercise is achieved using a combination of the pyplot ion (), draw (), set_xdata (), and set_data () functions . The ion () method initializes the pyplot interactive mode . Interactive mode helps to dynamically change the x and y values of the graphs in the pyplot.ion () figure .
When the interactive mode is set to True , the chart will only appear when the draw () method is called . We initialize the Arduino board using the pyFirmata module and input pins to obtain sensor values.
As you can see in the next line of code, after configuring the Arduino board and the pyplot interactive mode, we initialized the chart with a set of empty data, in our case 0: pData = [0] * 25 .
This array for y, pData values is then used to add values from the sensor in a while loop . The while loop continues to add new values to this data array and redraws the graph with these updated arrays for the x and y values.
In Listing 3, we add new sensor values at the end of the array, while removing the first element of the array to limit the size of the array:
pData.append(float(a0.read()))
del pData[0]
The set_xdata () and set_ydata () methods are used to update the x and y axis data from these arrays. These updated values are applied using the draw () method on each iteration of the while loop :
l1.set_xdata([i for i in xrange(25)])
l1.set_ydata(pData) # update the data
pyplot.draw() # update the plot
You will also notice that we use the xrange () function to generate a range of values according to the provided length, which is 25 in our case. The code snippet [i for i in xrange (25)] will generate a list of 25 integers that start gradually from 0 and end with 24.
Chart Integration in the Tkinter Window
Thanks to the powerful Python integration capabilities, it is very convenient to link the graphics created by the matplotlib library with the Tkinter GUI . We will write a program for the connection between Tkinter and matplotlib .
Listing 4
import sys
from matplotlib import pyplot
import pyfirmata
from time import sleep
import Tkinter
def onStartButtonPress():
while True:
if flag.get():
sleep(1)
pData.append(float(a0.read()))
pyplot.ylim([0, 1])
del pData[0]
l1.set_xdata([i for i in xrange(25)])
l1.set_ydata(pData) # update the data
pyplot.draw() # update the plot
top.update()
else:
flag.set(True)
break
def onPauseButtonPress():
flag.set(False)
def onExitButtonPress():
print "Exiting...."
onPauseButtonPress()
board.exit()
pyplot.close(fig)
top.quit()
top.destroy()
print "Done."
sys.exit()
# Associate port and board with pyFirmata
port = 'COM4'
board = pyfirmata.Arduino(port)
# Using iterator thread to avoid buffer overflow
it = pyfirmata.util.Iterator(board)
it.start()
# Assign a role and variable to analog pin 0
a0 = board.get_pin('a:0:i')
# Tkinter canvas
top = Tkinter.Tk()
top.title("Tkinter + matplotlib")
# Create flag to work with indefinite while loop
flag = Tkinter.BooleanVar(top)
flag.set(True)
pyplot.ion()
pData = [0.0] * 25
fig = pyplot.figure()
pyplot.title('Potentiometer')
ax1 = pyplot.axes()
l1, = pyplot.plot(pData)
pyplot.ylim([0, 1])
# Create Start button and associate with onStartButtonPress method
startButton = Tkinter.Button(top,
text="Start",
command=onStartButtonPress)
startButton.grid(column=1, row=2)
# Create Stop button and associate with onStopButtonPress method
pauseButton = Tkinter.Button(top,
text="Pause",
command=onPauseButtonPress)
pauseButton.grid(column=2, row=2)
# Create Exit button and destroy the window
exitButton = Tkinter.Button(top,
text="Exit",
command=onExitButtonPress)
exitButton.grid(column=3, row=2)
top.mainloop()
As a result of Listing 4, we get the Tkinter built into the window. graph with button interface elements - startButton, pauseButton and exitButton.
The Start and Exit buttons provide breakpoints for matplotlib operations , such as updating the schedule and closing the schedule using the corresponding functions onStartButtonPress () and onExitButtonPress () . The onStartButtonPress () function also consists of a point of interaction between the matplotlib and pyFirmata libraries . As you can see from the following code fragment, we will start updating the graph using the draw () method and the Tkinter window using the update () method for each observation from the analog output a0 , which is obtained using the read () method .
The onExitButtonPress () function implements the exit function, as described in the name itself. It closes the pyplot figure and Tkinter window before disconnecting the Arduino board from the serial port.
After making the appropriate changes to the Arduino port parameter, we will start listing No. 4. You should see a window on the screen similar to the one shown in the previous screenshot. With this code, you can now manage real-time charts using the “Start” and “Pause” buttons. Press the “Start” button and begin to turn the potentiometer knob. When you press the Pause button, you may notice that the program has stopped building new values. When you press the “Pause” button, even turning the knob will not lead to any changes in the schedule.
As soon as you click on the “Start” button again, you will again see that the graph is updated in real time, discarding the values generated during the pause. Click the Exit button to safely close the program.
In the preparation of the materials of this publication took part Misov O. P.
conclusions
This publication introduces two main Python programming paradigms: creating, reading, and writing files using Python, as well as storing data in these files and plotting sensor values and real-time graphing. We also learned how to store and display Arduino sensor data in real time. In addition to helping you with your Arduino projects, these methods can also be used in everyday Python projects.