July 24, 2012 at 21:18

PyBrain working with neural networks in Python


    Within the framework of one project, I was faced with the need to work with neural networks, considered several options, I liked PyBrain most of all . I hope many will be interested in reading its description.

    PyBrain is one of the best Python libraries for learning and implementing a wide variety of neural network-related algorithms. It is a good example of combining compact Python syntax with a good implementation of a large set of various algorithms from the field of machine intelligence.

    Created for:

    • Researchers - provides a uniform environment for implementing various algorithms, eliminating the need to use dozens of different libraries. Allows you to focus on the algorithm itself and not on the features of its implementation.
    • Students - using PyBrain it is convenient to implement homework, course project or calculations in thesis. The flexibility of the architecture allows you to conveniently implement a variety of complex methods, structures and topologies.
    • Lecturers - Learning Machine Learning techniques was one of the main goals when creating the library. The authors will be happy if the results of their work will help in the preparation of competent students and specialists.
    • Developers are an Open Source project, so new developers are always welcome.


    About the library


    PyBrian is a modular library designed to implement various machine learning algorithms in Python. Its main goal is to provide the researcher with flexible, easy-to-use, but at the same time powerful tools for realizing tasks from the field of machine learning, testing and comparing the effectiveness of various algorithms.
    The name PyBrain is an acronym for English: Python-Based Reinforcement Learning, Artificial Intelligence and Neural Network Library.
    As stated on one site: PyBrain - swiss army knife for neural networking (PyBrain is a Swiss army knife in the field of neural network computing).

    The library is built on a modular basis, which allows it to be used by both students to learn the basics and researchers who need to implement more complex algorithms. The general structure of the procedure for its use is shown in the following diagram:


    The library itself is an open source product and is free for use in any project with one caveat; when used for scientific research, they ask to be added to the list of cited information sources (which people do) following book:
    Tom Schaul, Justin Bayer, Daan Wierstra, Sun Yi, Martin Felder, Frank Sehnke, Thomas Rückstieß, Jürgen Schmidhuber. PyBrain To appear in: Journal of Machine Learning Research, 2010.

    Key features


    The main features of the library (for version 0.3) are:

    Networks


    PyBrain operates with network structures that can be used to build almost all complex algorithms supported by the library. An example is:

    Instruments


    Additionally, there are software tools that allow you to implement related tasks:
    • Building / Visualizing Charts
    • NetCDF support
    • Writing / Reading XML

    Library installation


    Before installing Pybrain, the creators recommend installing the following libraries:
    Setuptools is a package manager for Python that greatly simplifies the installation of new libraries. To install it, it is recommended to download and execute (python ez_setup.py) this script.
    After installation, you will have the opportunity to use the command
    easy_install
    to install new libraries.
    We will immediately use them and install the two necessary packages:
    $ easy_install scipy
$ easy_install matplotlib

    Next, PyBrain itself is installed.

    • Or use the repository with github

      git clone git://github.com/pybrain/pybrain.git

    • Or download the latest currently stable version here . And install in the standard way:
      $ python setup.py install



    Library Basics


    Neural network

    Creating a neural network with two inputs, three hidden layers and one output:
    >>> from pybrain.tools.shortcuts import buildNetwork
>>> net = buildNetwork(2, 3, 1)

    As a result, the created neural circuit, initialized by random weights, is located in the net object.

    Activation function

    The activation function is defined as follows:
    net.activate([2, 1])

    The number of elements transmitted to the network should be equal to the number of inputs. The method returns a singular answer if the current circuit has one output, and an array in case of more outputs.

    Retrieving Network Information

    In order to obtain information about the current network structure, each of its elements has a name. This name can be given automatically, or by other criteria when creating a network.
    For example, for net, names are given automatically:
    >>> net['in']

>>> net['hidden0']

>>> net['out']

    Hidden layers are named with the layer number added to the name.

    Networking Features

    Of course, in most cases, the created neural network should have other characteristics than the default ones. There are various possibilities for this. For example, by default, a hidden layer is created using the sigmoid activation function , to specify another type of it, it is possible to use the following constants:
    • Biasunit
    • GaussianLayer
    • Linearlayer
    • LSTMLayer
    • MDLSTMLayer
    • Sigmoidlayer
    • Softmaxlayer
    • StateDependentLayer
    • TanhLayer

    >>> from pybrain.structure import TanhLayer
>>> net = buildNetwork(2, 3, 1, hiddenclass=TanhLayer)
>>> net['hidden0']


    It is also possible to specify the type of output layer:
    >>> from pybrain.structure import SoftmaxLayer
>>> net = buildNetwork(2, 3, 2, hiddenclass=TanhLayer, outclass=SoftmaxLayer)
>>> net.activate((2, 3))
array([ 0.6656323,  0.3343677])

    Additionally, bias can be used. 
    >>> net = buildNetwork(2, 3, 1, bias=True)
>>> net['bias']


    Data Operations (Building a DataSet)

    The created network should process the data that this section is dedicated to working with. A typical data set is a set of input and output values. To work with them, PyBrain uses the pybrain.dataset module, and the SupervisedDataSet class is also used later.

    Data setting

    The SupervisedDataSet class is used for typical teacher training. It supports arrays of output and output. Their sizes are set when creating an instance of the class:
    Record of the form:
    >>> from pybrain.datasets import SupervisedDataSet
>>> ds = SupervisedDataSet(2, 1)

    means that a data structure is being created to store two-dimensional input data and one-dimensional output.

    Adding Samples

    A classic task in training a neural network is learning the XOR function, the following shows the data set used to create such a network.
    >>> ds.addSample((0, 0), (0,))
>>> ds.addSample((0, 1), (1,))
>>> ds.addSample((1, 0), (1,))
>>> ds.addSample((1, 1), (0,))


    Study of sample structure

    To obtain data arrays in their current set, it is possible to use standard Python functions for working with arrays.
    >>> len(ds)

    will output 4, since this is the number of elements.
    Iteration over the set can also be organized in the usual way for arrays:
    >>> for inpt, target in ds:
	print inpt, target


    ...
[ 0.  0.] [ 0.]
[ 0.  1.] [ 1.]
[ 1.  0.] [ 1.]
[ 1.  1.] [ 0.]

    Also, each set of fields can be directly accessed using its name:
    >>> ds['input']


    array([[ 0.,  0.],
       [ 0.,  1.],
       [ 1.,  0.],
       [ 1.,  1.]])



    >>> ds['target']


    array([[ 0.],
       [ 1.],
       [ 1.],
       [ 0.]])


    You can also manually free the memory occupied by the sample by completely deleting it:
    >>> ds.clear()
>>> ds['input']


    array([], shape=(0, 2), dtype=float64)


    >>> ds['target']


    array([], shape=(0, 1), dtype=float64)


    Network training on samples

    PyBrain uses the concept of trainers to train networks with a teacher. The trainer receives an instance of the network and an instance of the set of samples and then trains the network according to the set received.
    A classic example is backpropagation. To simplify the implementation of this approach, PyBrain has a BackpropTrainer class.
    >>> from pybrain.supervised.trainers import BackpropTrainer

    The training set of samples (ds) and the target network (net) are already created in the examples above, now they will be combined. 
    >>> net = buildNetwork(2, 3, 1, bias=True, hiddenclass=TanhLayer)
>>> trainer = BackpropTrainer(net, ds)

    The trainer received a link to the network structure and can train it. 
    >>> trainer.train()


    0.31516384514375834

    The call to the train () method performs one iteration (era) of training and returns the value of the quadratic error (double proportional to the error).
    If there is no need to organize a cycle for each era, then there is a training network method to convergence:
    >>> trainer.trainUntilConvergence()

    This method will return an array of errors for each era.

    More examples of implementing different networks


    In the article
    Tom Schaul, Martin Felder, et.al. PyBrain, Journal of Machine Learning Research 11 (2010) 743-746.
    an example of creating a network with loading data from a .mat file is given.

    # Load Data Set. 
ds = SequentialDataSet.loadFromFile(’parity.mat’) 
# Build a recurrent Network. 
net = buildNetwork(1, 2, 1, bias=True, 
	hiddenclass=TanhLayer, 
	outclass=TanhLayer, 
	recurrent=True) 
recCon = FullConnection(net[’out’], net[’hidden0’]) 
net.addRecurrentConnection(recCon) 
net.sortModules() 
# Create a trainer for backprop and train the net. 
trainer = BackpropTrainer(net, ds, learningrate=0.05) 
trainer.trainEpochs(1000)


    Some links:




    Conclusion


    In conclusion, I want to say that this library makes a very good impression, it’s convenient to work with it, the descriptions of the algorithms are compact, but do not lose their clarity in the wilds of code.

    PS If there are corrections in the names of some terms, then I am ready to listen, I'm not sure of the 100% accuracy of a couple of translations, maybe there are already well-established names.
    Tags:

    Also popular now: