OptionParser and UnitTest in python scripts

In this article I want to ask the public if I am correctly implementing the capabilities of these two wonderful python modules, which have long been included in the standard (vital) set of python guides for the entire planet.

Background

A script is being written to process and graphically display some research data. But the story is not about that. I would like to show how the combination of OptionParser and UnitTest modules is used. And for one, learn ways to improve the code and its readability. So Python gurus, I will be very grateful for your criticism and suggestions.

Modules

Everyone has heard about the Test Driven Development (TDD) method of software product development at least once in their life. Immediately with the implementation of this approach for python, I came across in the book of Macra Pilgrim's "Diving into Python 3". In the ninth chapter of his book, Mark describes in detail how to implement unit tests for his module for converting Roman numbers. The basis of this principle can be called writing a test to verify the correct execution of the code before writing the code itself.

I would like to add from myself that I knew about this method of programming for a long time, but I had never used it before for a banal reason - the time it takes to write the code almost doubles. And thus, it becomes completely inappropriate to use this approach for short scripts that perform a specific task. With this kind of scripts, it’s clear if there is an error in the code, because approximately you know from what range of distributions (statistical processing) you need to expect output data.

Designing a new task showed that the script will be quite complex. And writing Unit tests to it will be justified. Because subsequent refactoring and debugging will be greatly simplified.

Next in line is OptionParser, which I have been using for a long time, and it looks like it will be used for a long time. Code readability increases when using it. There are several parsers like this one. And at one time active holivars were conducted, about which one is better. There were accusations that he was imposing his philosophy on organizing and processing options. Honestly, I have not noticed anything “strange” in this organization. And again, this will primarily depend on the programmer how he implements the readability of one or another option. So, let us leave the holivar aside.

Source code

Let's go straight to the source code. There is only one readin_monitor (monitor) working function in the executable module so far.

Copy Source | Copy HTML

1. #!/usr/bin/env python
2. # -*- coding: utf-8 -*- 
3.  
4. version = '0.0.1'
5. version_name = 'gamma'
6. modify_date = '2009-12-26'
7.  
8. from optparse import OptionParser
9.  
10. import matplotlib
11. import numpy as np
12. import scipy.stats as stats
13.  
14. import warnings
15. warnings.filterwarnings('ignore', '', DeprecationWarning)
16. # turning off deprecation warning in python 2.6
17.  
18. kB = 8.31441e-3 / 4.184
19.  
20. def readin_monitor(monitor):
21.     '''Read in monitor file. Ignoring all strings starting with # symbol.
        Function returns all stored data from the strings as list of lists of 
        floats.'''
22.     num =  0
23.     data = []
24.     for line in open(monitor, 'r'):
25.         try:
26.             if line[ 0] != '#':
27.                 data.append([float(i) for i in line.split()])
28.                 num = num + 1
29.         except:
30.             pass
31.     if options.verbose:
32.         print('Read in %i data points from monitor file %s' % (num, monitor))
33.     return data
34.  
35. def main():
36.     return  0
37.  
38.  
39. global options
40. global args
41. parser = OptionParser("usage: %prog [options] [monitors]",
42.                   version='%prog ' +version+ ' from '+modify_date)
43. parser.add_option("-v", "--verbose",
44.                   action="store_true", dest="verbose", default=False,
45.                   help="Print status messages to stdout")
46. parser.add_option("-C", "--combine", dest="combine",
47.                   action="store", default="",
48.                   help='Combine all monitor files passed as arguments \
                      to the UC.py script to one COMBINE file')
49. parser.add_option('-D', '--dimentions', dest='dimentions',
50.                   default = '1:2',
51.                   help='String of DIMENTIONS for monitor files to be \
                      read in. (defaut = %default)')
52. (options, args) = parser.parse_args()
53. if __name__ == '__main__':
54.     main()

Of the features of the location of the code, I would like to note the definition of parser options at the end of the module itself. Those. this piece of code will always be executed, even if the module is called by another script. Thus, in the globally defined variables options and args, there will be default values, args will be empty. Because Since global variables, then access to them will be possible from any environment.

Running the script with the -h option will give detailed help on using the options:

Copy Source | Copy HTML

1. Usage: UC.py [options] [monitors]
2.  
3. Options:
4.   --version show program's version number and exit
5.   -h, --help show this help message and exit
6.   -v, --verbose Print status messages to stdout
7.   -C COMBINE, --combine=COMBINE
8.                         Combine all monitor files passed as arguments
9.                         to the UC.py script to one COMBINE file.
10.                         (defaut = out)
11.   -D DIMENTIONS, --dimentions=DIMENTIONS
12.                         String of DIMENTIONS for monitor files to be
13.                         read in. (defaut =  0:1:2)

Next, unit tests themselves:

Copy Source | Copy HTML

1. #!/usr/bin/env python
2. # -*- coding: utf-8 -*-
3. '''Unit tests for UC.py module.'''
4.  
5. import UC
6. import unittest
7.  
8. global monitor
9. monitor = '''#
   # MD time (ps), CV #1, CV #2
   #
         0.9990     9.2349535263     7.7537518211
         1.9990     9.4331321327     7.9555258177
         2.9990     9.5368308183     8.1341402536
         3.9990     9.4468066031     7.9086253193
         4.9990     9.1565151681     8.0027457962
         5.9990     9.2310306859     7.9872398398
         6.9990     9.1540695183     7.5236796623
         7.9990     9.0727576308     7.8499035889
         8.9990     9.3113419250     8.1227557439
         9.9990     8.9597834513     8.3754973753
        10.9990     9.5761421491     8.3053224696
        11.9990     9.5178829977     8.1660258902'''
10.  
11. class Combine_monitors(unittest.TestCase):
12.     def test_readin_monitor(self):
13.         with open('test_mon', 'w') as MON:
14.             MON.write(monitor)
15.         UC.options.verbose = False
16.         self.assertEqual([[ 0.999, 9.2349535263, 7.7537518210999998],
17.             [1.9990000000000001, 9.4331321327000008, 7.9555258176999999],
18.             [2.9990000000000001, 9.5368308183000003, 8.1341402536],
19.             [3.9990000000000001, 9.4468066031000006, 7.9086253192999996],
20.             [4.9989999999999997, 9.1565151681000003, 8.0027457961999993],
21.             [5.9989999999999997, 9.2310306859000004, 7.9872398398],
22.             [6.9989999999999997, 9.1540695183, 7.5236796623000002],
23.             [7.9989999999999997, 9.0727576308, 7.8499035889000002],
24.             [8.9990000000000006, 9.3113419250000007, 8.1227557439000009],
25.             [9.9990000000000006, 8.9597834512999999, 8.3754973753000002],
26.             [10.999000000000001, 9.5761421491000007, 8.3053224696000001],
27.             [11.999000000000001, 9.5178829976999992, 8.1660258902000002]],
28.             UC.readin_monitor('test_mon'))
29.  
30. def main():
31.     unittest.main()
32.     return  0
33.  
34. if __name__ == '__main__':
35.     main()
36.  

It is worth adding deletion of temporary files to the written test. And of course, increase the number of tests as new script functions are implemented. Running the script leads to this conclusion: To write this small piece of code, I had to “deceive everyone” a bit (it seems like you can translate the verb cheat). The test was written after writing the readin_monitor () function itself from the main module. The result of the function was simply thrown out by print in stdout. And from there I uploaded the test module to the source code.

$ ./test-UC.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK




What does not like - seem to be fooling ourselves. First we write the code, then the test, thereby violating the philosophy of TDD development. Also, the output results, due to the specifics of the language, were not accurate (meaning 5.9989999999999997 = 5.9990 rounding). If you run the same unit test in a different version of python, you may get a test error. For Python 3.1, the test was passed positively, but I still care about such accuracy tests. You can, of course, organize rounding yourself to, say, the 5th decimal place, and compare the already rounded data. But this is fraught with a strong weighting of the code, and, as a result, poor readability thereof.

Total

Using two short scripts as an example, we showed how you can use the options of OptionParser and UnitTest modules. The purpose of the article was not a full description of all their capabilities, so the curious reader was given the opportunity to understand them myself, using the links from the beginning of the article.

Well, to the main question. What can be improved in this code / approach? Waiting for your answers.

Thanks for attention.