Install MPB and MEEP on CentOS 7 and Ubuntu 14.04

  • Tutorial
MPB (MIT Photonic Bands) is a free open source program that was originally developed to calculate the dispersion diagrams of photonic crystals .

MEEP is the same free program with source code that is used to simulate the behavior of electromagnetic waves in various environments (photonic crystals, waveguides, resonators, etc.).

Both programs were developed at the Massachusetts Institute of Technology (MIT) and both are constantly gaining new opportunities. The MPB was written by Steven G. Johnson during his graduate work. MEEP was written a little later with Steven.

Both programs calculate the distribution of electrical and magnetic components of the electromagnetic field using a combination of numerical and analytical methods for solving the Maxwell system of equations (in one-, two- or three-dimensional structures), but each of them does it in its own way. If the MPB is designed to be applied to periodic and quasiperiodic structures and to calculate the frequencies of standing waves (modes) in these structures, then MEEP is designed to simulate the propagation of electromagnetic waves through the same photonic crystals , dielectric mirrors , along waveguides and inside resonators. It allows one to calculate the same dispersion diagrams of photonic crystals, the frequencies of standing waves both in photonic crystals and non-periodic structures, the transmission and reflection spectra of various structures, the loss on the bends of waveguides, and much more. For this, MEEP uses a whole arsenal of different radiation sources, boundary conditions and radiation absorbers ( PML ).

Recent versions of MPB and MEEP can interact with each other. For example, it is possible to write a program for MEEP, which will ask MPB to calculate the field components for the main waveguide mode, and then use these components to excite this mode in an optical waveguide fiber. As a result, it will be possible to simulate the propagation of the main mode along the waveguide and display the result of calculations in third-party programs. An example is shown below, where the result of calculating the components of the wave that leaves the optical fiber is visible. To display this result, the free program Paraview was used .

I have to use these programs in my work, install and help other people with the installation. The mailing lists of these programs occasionally skip questions about installing these programs from Russian-speaking users. With surprise for myself, I did not find installation instructions in the Russian-language part of the Internet and decided to publish them here.


general information


In the world of commercial software products, these programs have competitors. The analogue of the MPB program is BandSOLVE , the analogue of MEEP is FullWAVE . BandSOLVE and FullWAVE have a convenient graphical interface, but they are worth the money. MPB and MEEP, unlike BandSOLVE and FullWAVE, are free, do not have a graphical interface, use the Guile scripting language and are distributed under the GNU license along with their source codes.

You can install MEEP and MPB from the repositories if you use Debian or Ubuntu. This makes life easier for those users who have not yet learned how to compile and install these programs on their own. At times, these programs from repositories work fine, but it also happens that some functions do not work or the version of programs in the repositories is simply old. Therefore, the best way is to install programs from source.

These programs can be installed on Windows computers using Cygwin.. If anyone is interested in learning how to do this, then I can further tell. But this is a laborious business and it was relevant 10-15 years ago. Now it’s easier to install one of the Linux distributions on your computer and use these programs in their native environment. Which Linux distribution you choose for your work is up to you.

In this note, CentOS 7 will be used . The repositories of this distribution already have the necessary HDF5 libraries compiled with support for parallel computing. This will facilitate our task, since often this library is the source of problems if it does not work as expected. There are other necessary libraries in the repositories, for example, fftw . But the latter does not supportThe MPI . Therefore, it will need to be compiled independently.

The main sources of information for this article are instructions for installing MPB and MEEP , but much will be simplified in this article. All programs will be installed in the / usr / local / directory.

Step 1: install compilers and libraries


In the terminal window, execute:
sudo yum install libtool* mpich-devel.* lapack* guile guile-devel readline-devel hdf5-* gcc-c++ scalapack-* paraview*

Ubuntu 14.04 LTS
sudo apt-get install libtool* mpich-dev* lapack* guile-2.0 guile-2.0-dev readline-dev fftw3-* paraview hdf5-* gcc-c++ scalapack-* paraview*



I suspect that unnecessary packages will be installed this way, but this method just works. And the most attentive of you will always tell you exactly what you can not install (I suspect that scalapack- * could not be installed).

Step 2: add the necessary environment variables to .bashrc (this is true for both CentOS 7 and Ubuntu 14.04 LTS)


For this purpose, you can use any favorite text editor. But if you use vim, then in the terminal window we type:
vim .bashrc

After that, press the i key to enter the editing mode, move the cursor down the file and add there:
LDFLAGS="-L/usr/local/lib -lm" export LDFLAGS
CPPFLAGS="-I/usr/local/include" export CPPFLAGS
LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH" export LD_LIBRARY_PATH
PATH=/lib64/mpich/bin:$PATH export PATH

Press the Esc key, type on the keyboard: wq and press Enter to save the changes to .bashrc and exit the editor. After that, create a temporary directory t, in which temporary files will be stored, and "enter" it:
mkdir t
cd t


Step 3: download, compile and install the FFTW library with MPI support


To do this, in the terminal window, execute:
wget http://www.fftw.org/fftw-3.3.4.tar.gz
tar -xf fftw-3.3.4.tar.gz
cd fftw-3.3.4
CC=/lib64/mpich/bin/mpicc CXX=/lib64/mpich/bin/mpicxx F77=/lib64/mpich/bin/mpif77 ./configure --enable-mpi --enable-openmp
make -j4
sudo make install
cd ..

Ubuntu 14.04 LTS
This library was already installed in step 1, but the hdf5 library for some reason does not work correctly. Therefore, you have to compile and install it from the source:
wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.14.tar.gz
tar -xf hdf5-1.8.14.tar.gz
cd hdf5-1.8.14
CC=mpicc CXX=mpicxx F77=mpif77 ./configure –-enable-parallel –prefix=/usr/local
make -j4
sudo make install
cd ..

All of the following instructions work for both CentOS 7 and Ubuntu 14.04.


Step 4: Libctl Library


In the same place we execute:
wget http://ab-initio.mit.edu/libctl/libctl-3.2.2.tar.gz
tar -xf libctl-3.2.2.tar.gz
cd libctl-3.2.2
CC=/lib64/mpich/bin/mpicc CXX=/lib64/mpich/bin/mpicxx F77=/lib64/mpich/bin/mpif77 ./configure
make -j4
sudo make install
cd ..


Step 5: MPB


We compile and install without support for MPI and OpenMP:
wget http://ab-initio.mit.edu/mpb/mpb-1.5.tar.gz
tar -xf mpb-1.5.tar.gz
cd mpb-1.5/
CC=/lib64/mpich/bin/mpicc CXX=/lib64/mpich/bin/mpicxx F77=/lib64/mpich/bin/mpif77 ./configure
make -j4 
sudo make install
make distclean

Support MPI and OpenMP
CC=/lib64/mpich/bin/mpicc CXX=/lib64/mpich/bin/mpicxx F77=/lib64/mpich/bin/mpif77 ./configure --with-mpi --with-openmp
make -j4 
sudo make install
cd ..


Step 6: Harminv


We carry out:
wget http://ab-initio.mit.edu/harminv/harminv-1.4.tar.gz
tar -xf harminv-1.4.tar.gz
cd harminv-1.4/
CC=/lib64/mpich/bin/mpicc CXX=/lib64/mpich/bin/mpicxx F77=/lib64/mpich/bin/mpif77 ./configure
make
sudo make install
cd ..


Step 7: MEEP


Without support for MPI and OpenMP:
wget http://ab-initio.mit.edu/meep/meep-1.3.tar.gz
tar -xf meep-1.3.tar.gz
cd meep-1.3/
CC=/lib64/mpich/bin/mpicc CXX=/lib64/mpich/bin/mpicxx F77=/lib64/mpich/bin/mpif77 ./configure
make -j4
sudo make install
make distclean

With support for MPI and OpenMP:
CC=/lib64/mpich/bin/mpicc CXX=/lib64/mpich/bin/mpicxx F77=/lib64/mpich/bin/mpif77 ./configure --with-mpi
make -j4
sudo make install


Step 8: h5utils


MPB and MEEP programs save the calculation results to files with the extension .h5. This package (h5utils) contains a set of programs for working with h5 files, such as h5topng (for converting h5 files to png graphic format), h5tovtk (converting to vtk format, convenient for displaying using Paraview) and h5totxt ( conversion to text format). If you do not install and use these programs, then many calculation results will simply not be available for viewing.
wget http://ab-initio.mit.edu/h5utils/h5utils-1.12.1.tar.gz
tar -xf http://ab-initio.mit.edu/h5utils/h5utils-1.12.1.tar.gz
cd h5utils-1.12.1
CC=/lib64/mpich/bin/mpicc CXX=/lib64/mpich/bin/mpicxx F77=/lib64/mpich/bin/mpif77 ./configure
make -j4 
sudo make install
cd ..

If it happened that the execution of the “make -j4” command was interrupted with the error “[writepng.o] Error 1”, then instead of the last three commands we execute:
make h5totxt
make h5tovtk
sudo mv h5tovtk /usr/local/bin/
sudo mv h5totxt /usr/local/bin/
cd ..


Check the operation of MEEP


We move to the directory with examples:
cd meep-1.3/examples/

First, run one of the examples (ring resonator model) without using MPI:
meep ring.ctl

If MEEP was installed normally, then after the calculations are completed, you will see this text in the terminal window:
creating output file "./ring-ez-000403.50.h5" ...
creating output file "./ring-ez-000403.85.h5" ...
creating output file "./ring-ez-000404.20.h5" ...
creating output file "./ring-ez-000404.55.h5" ...
creating output file "./ring-ez-000404.90.h5" ...
creating output file "./ring-ez-000405.25.h5" ...
creating output file "./ring-ez-000405.60.h5" ...
creating output file "./ring-ez-000405.95.h5" ...
creating output file "./ring-ez-000406.30.h5" ...
creating output file "./ring-ez-000406.65.h5" ...
run 1 finished at t = 406.70000000000005 (8134 timesteps)
Elapsed run time = 4.02319 s

In the last line is the time spent on calculation (in seconds).

Now you can test how much the use of MPI can reduce the calculation time. To do this, run meep-mpi (MEEP with MPI support) on one processor core:
mpirun -np 1 meep-mpi ring.ctl

After the calculations we see:
creating output file "./ring-ez-000405.95.h5" ...
creating output file "./ring-ez-000406.30.h5" ...
creating output file "./ring-ez-000406.65.h5" ...
run 1 finished at t = 406.70000000000005 (8134 timesteps)
Elapsed run time = 3.81012 s

With two cores:
mpirun -np 2 meep-mpi ring.ctl

After the calculations we see:
creating output file "./ring-ez-000405.25.h5" ...
creating output file "./ring-ez-000405.60.h5" ...
creating output file "./ring-ez-000405.95.h5" ...
creating output file "./ring-ez-000406.30.h5" ...
creating output file "./ring-ez-000406.65.h5" ...
run 1 finished at t = 406.70000000000005 (8134 timesteps)
Elapsed run time = 3.20775 s

With three cores:
mpirun -np 3 meep-mpi ring.ctl

After the calculations we see:
creating output file "./ring-ez-000405.95.h5" ...
creating output file "./ring-ez-000406.30.h5" ...
creating output file "./ring-ez-000406.65.h5" ...
run 1 finished at t = 406.70000000000005 (8134 timesteps)
Elapsed run time = 4.67524 s

That is, with an increase in the number of cores to 2, acceleration of calculations is observed. But with a further increase in the number of involved nuclei, the calculation time increases. Two - the optimal number of cores for this example, which gives the maximum increase in calculation speed. But in the case of using mpb-mpi (MPB with MPI support, which we also installed), the picture is usually different and it is possible to achieve a better speed increase.

In the case of using computing clusters and supercomputers, the optimal number of processors will be different - 8-12. You may ask: “Why use MPI and OpenMP if the achieved increase in calculation speed is not significant?” First of all, the increase in speed depends on the model itself and on how the performed calculations can be parallelized. The point is that in supercomputers and clusters, the amount of memory is tied to the number of nodes involved in the calculation. For example, 2 GB per node. This means that by involving 2 nodes in the calculation, the program gets access to 4GB of memory. Using 10 nodes, the program gets access to 20 GB. Thus, the use of both MPI and OpenMP (this was included above with the keys --with-mpi --with-openmp) allows you to slightly speed up the calculations and at the same time use more memory for calculations. AND,

Now we can take a look at the calculation results. First, we convert all the files with the extension .h5 in the working directory that were created as a result of calculations into the .vtk format with the command:
h5tovtk *.vtk

After that, you can run the Paraview program (by executing paraview on the command line), and open the calculation results for viewing. To look at the distribution of dielectric constant in the simulated structure of a ring resonator, you need to open the file ring-eps-000000.00.vtk. The electric field distributions are stored in files of the form ring-ez-000400.35.vtk.

Script to automatically perform operations 3-8 (written and tested for Scientific Linux 6.5)
#!/bin/bash
# 1. Build and install FFTW 
wget http://www.fftw.org/fftw-3.3.4.tar.gz
tar -xf fftw-3.3.4.tar.gz
cd fftw-3.3.4
CC=mpicc CXX=mpicxx F77=mpif77 ./configure --enable-mpi --enable-openmp
make -j4
sudo make install
cd ..
# 2. Libctl
wget http://ab-initio.mit.edu/libctl/libctl-3.2.2.tar.gz
tar -xf libctl-3.2.2.tar.gz
cd libctl-3.2.2
CC=mpicc CXX=mpicxx F77=mpif77 ./configure
make -j4
sudo make install
cd ..
# 3. MPB
# 3.1 without MPI support
wget http://ab-initio.mit.edu/mpb/mpb-1.5.tar.gz
tar -xf mpb-1.5.tar.gz
cd mpb-1.5/
CC=mpicc CXX=mpicxx F77=mpif77 ./configure
make -j4 
sudo make install
make distclean
# 3.2 with MPI support
CC=mpicc CXX=mpicxx F77=mpif77 ./configure --with-mpi --with-openmp
make -j4 
sudo make install
cd ..
# 4. Harminv
wget http://ab-initio.mit.edu/harminv/harminv-1.4.tar.gz
tar -xf harminv-1.4.tar.gz
cd harminv-1.4/
CC=mpicc CXX=mpicxx F77=mpif77 ./configure
make
sudo make install
cd ..
# 5. Meep
# 5.1 without MPI 
wget http://ab-initio.mit.edu/meep/meep-1.3.tar.gz
tar -xf meep-1.3.tar.gz
cd meep-1.3/
CC=mpicc CXX=mpicxx F77=mpif77 ./configure
make -j4
sudo make install
make distclean
# 5.2 with MPI
CC=mpicc CXX=mpicxx F77=mpif77 ./configure --with-mpi
make -j4
sudo make install
# 6. h5utils
wget http://ab-initio.mit.edu/h5utils/h5utils-1.12.1.tar.gz
tar -xf h5utils-1.12.1.tar.gz
cd h5utils-1.12.1
CC=mpicc CXX=mpicxx F77=mpif77 ./configure
make -j4 
sudo make install
cd ..



PS Added instructions for Ubuntu 14.04 LTS (not always the latest versions of these programs are available in the repositories).

Also popular now: