Set the virtual machine IP over MAC without using DHCP

The article discusses the use of scripts for CentOS and Windows XP that set IP in accordance with the MAC of the VM network interface, as well as the complexities of managing the network interface in Windows.

The system we are developing works very actively with virtual machines. When the kernel of the system needs another machine, the template image is copied and this copy is launched. Thus, at the same time can work a lot of copies of essentially the same machine.

Of course, at the time of launch, each virtual machine is identical to the template. Including installed network parameters are inherited. All virtual machines work on the same subnet, which means that they should not use the static IP that they got from the template machine - otherwise conflicts will arise. That is, each machine must get its own IP. It would seem that the solution is very simple - use a DHCP server and dynamic IP.

However, there is another option, which I will discuss in this article.

1. Why do we need virtual machines
2. The problem of the same static addresses when running several VM clones
3. We encode IP into MAC
4. IP allocation by MAC: DHCP
5. IP allocation by MAC: mac2ip script
a) Linux
b) Windows
6. Comments

Update
Update 2

Spoiler: in fact, the most interesting is in paragraphs 3) and 5), the rest is for those who want to see the whole picture.

1. Why do we need virtual machines

Our project, Nerrvana , performs functional tests of sites in different browsers. These tests work with a special framework for functional testing - Selenium, which allows you to emulate user actions in the browser (clicks on elements, mouse movements, entering characters, reading text), taking screenshots of pages and some other things.
A site test is a sequence of actions that a user could take on the site, and checks that the result of these actions is exactly as expected. For example, the simplest test is the login to the site. You need to open the login page, enter the login, password, press "Enter", and make sure that we are logged in - say, seeing a standard greeting. Everyone knows that browsers can display and work with the same page in completely different ways, and therefore it makes sense to run the same tests in the most popular browsers. As already mentioned, this is what our system does.

Tests in selected browsers are performed simultaneously and completely independently of each other. We called the test execution in one of the browsers speck. That is, for example, if I want to perform a login test on browsers IE 8 and FF 3.6, our system will make two independent specs - it will execute the test code using the selected browsers. Without going into too much detail, I’ll say that for each spec, we create at least two virtual machines. One machine, the “hub”, will be engaged in the actual execution of tests - there is Java and PHP, on which tests should be written. On the second machine, the “tester”, Selenium RC and the desired browser are running. Through Selenium RC there is an interaction between the tests and the browser. After each spec, the virtual machines that he worked on are destroyed.

Since there can be many tests running simultaneously, and everyone can use several specs, relatively many virtual machines can also work - 50, for example.

2. The problem of the same static addresses when running multiple VM clones

For each type of virtual machine there is a template image. When the kernel of the system needs another machine, this image is copied and a copy of the image is launched - the image itself remains unchanged. Thus, at the same time can work a lot of copies of essentially the same machine.

And then there is some problem.
Naturally, at the time of launch, each virtual machine is absolutely identical to the template one - after all, all data is stored in an image that cannot be changed. Including installed network parameters are inherited. Since all virtual machines work on the same subnet, it is obvious that they should not use the static IP that they got from the template machine — otherwise address conflicts will occur. That is, each machine must dynamically obtain its own IP.
It would seem that the solution is very simple - use a DHCP server, which will give each machine a unique address.

However, not all so simple. The fact is that the core of the system actively interacts with virtual machines. It should just do a lot of work with them: for example, make sure that the virtual machines start successfully, run Selenium RC on the tester, upload to the hub and run the tests themselves, monitor their execution, and then get the results back (screenshots, logs, etc.) d.). Work is done through ssh.
Those. The kernel of the system, whatever one may say, should know the IP addresses of machines that have just been launched at its request.

We saw two approaches to solving the problem:
1) After the virtual machine has risen, it receives a random address from DHCP, and then somehow registers itself in the database - i.e. indicates that I am a machine of such and such a type, received such and such an address from DHCP.
2) In some way, the kernel makes it clear to the virtual machine which address it should use, i.e. IP compliance - a virtual machine exists even before the machine starts.

The first option seemed rather complicated for us for several reasons. This option made the VM too independent of the kernel, there was an extra connection - from the virtual machine to the base, difficulties appeared in comparing the requested and registered VMs, and some problems related to the kernel architecture.
We liked the second option much more, because the kernel immediately got full control over the VM, and did not find itself in a suspended state, waiting for the VM to register itself.

3. We encode IP in MAC

How to work on a machine that is not yet running in order to tell it the future ip-address? We found this way: when starting the virtual machine, you can set the MAC address of the virtual network card. And the virtual machine itself can recognize it at any time. That is, the MAC can be used as a carrier of information about IP (or anything else).

How exactly will we encode the IP in the mac address? For virtualization, we use Xen , and xen virtual network cards must have a MAC that looks like this: 00: 16: 3E: XX: XX: XX, where 00: 16: 3E is the code for the manufacturer of the network card. We can use the last three bytes at our discretion (of course, bearing in mind that the MAC must be unique).
Suppose that our system will work on the 10.0.0.0/8 subnet, and therefore the logical solution is to use the values ​​corresponding to the last three bytes of the IP address that we want to encode for the last three bytes of the MAC.
That is, for the address 10.1.1.3 we will use MAC 00: 16: 3E: 01: 01: 03.

It remains to force the machine to obtain the desired IP corresponding to its MAC. This again can be done in two ways.

4. IP Allocation by MAC: DHCP

The first way is pretty obvious. We will start the DHCP server, which will be configured to issue IP by MAC address in accordance with the described encoding method.

The scenario will look like this:
1) The kernel requires a VM.
2) The kernel scans the pool of IP addresses, selects the first unoccupied IP (for example, 10.4.0.15), and converts it to MAC (00: 16: 3E: 04: 00: 0F)
4) The kernel copies the template image of the VM of the desired type, prepares VM configuration file (in which it indicates, including the received MAC)
5) The kernel starts the VM
6) The VM contacts DHCP for the address, it checks the MAC / IP mapping table and issues IP 10.4.0.15.
7) The kernel periodically pings 10.4.0.15 at this time, and, after receiving the response, starts working with the virtual machine (of course, after waiting for sshd to start)

- a DHCP server
is required - if we move to a different subnet or get a different IP address pool, change the configuration of not only the kernel, but also DHCP
+ the standard approach to obtaining IP is used

5. IP Allocation by MAC: mac2ip script

The second method is less obvious and requires some additional work.
It consists in the fact that the VM at startup will execute a special script that will receive the MAC, convert it to IP, and assign a network card. Thus, the scenario will be practically the same - only point 6 will change. It will look like this:

6) VM calculates its IP based on its MAC, and sets it before starting the interface.

+ no additional link is required in the form of DHCP and storage of the MAC-IP correspondence table there.
- each OS will need its own mac2ip script.

We have implemented this option.
We work with virtual machines with CentOS 5.6 and Windows XP PRO SP3, and therefore we needed two mac2ip scripts - for each of the systems.
Consider both scripts.

a) Linux

#!/bin/bash
# первый байт всегда будет равен 10
IP1=10
IFCFG=/etc/sysconfig/network-scripts/ifcfg-eth0
NETWORK=/etc/sysconfig/network
case "$1" in
*start)
;;
*)
exit
;;
esac
# получаем MAC и проверяем, что мы его таки получили
MAC=$(ifconfig eth0|grep HWaddr|awk '{print $NF}'|grep ^00:16:3E)
if [[ -z "$MAC" ]] ; then
echo "Can't determine MAC address" >&2
exit 1
fi
# преобразуем MAC в IP
set -- $(echo $MAC|awk -F: '{print $4,$5,$6}')
IPADDR=${IP1}.$((0x$1)).$((0x$2)).$((0x$3))
# меняем настройки интерфейса
sed -i -e "s/^IPADDR.*/IPADDR=$IPADDR/" $IFCFG
sed -i -e "/^HWADDR/d" $IFCFG
sed -i -e "s/^HOSTNAME.*/HOSTNAME=localhost/" $NETWORK

And make this script run before running network.

b) Windows

The same work in Windows XP is done in much more abstruse ways. Perhaps over time there will be a more efficient way to convert MAC to IP.

The first problem that I encountered was the inability to change the address of the interface with relatively easy ways BEFORE it is turned on. Thus, the template VM must have “network connection” turned off by default, otherwise two simultaneously running copies of Windows will immediately try to use the same address immediately after startup (it is static, because we do not use DHCP for the VM).

Ok, this is not a problem - turn off the interface. However getmac utility, which we will use to obtain the net address of the net, cannot return the MAC for the interface that is turned off! Therefore, we will first have to assign a random IP to the interface, turn it on, find out the MAC, and then set the desired IP.

To manipulate devices, the devcon utility is used .
Here's what it looks like:

@echo off
SET MAC=
SET IP=
SET MASK=255.255.255.0
SET GATEWAY=
rem получаем случайный IP. Этот IP будет использоваться в течении пары секунд,
rem и поэтому для обеспечения уникальности достаточно сгенерировать случайно два последних байта.
rem 100 - смещение, которое гарантирует, что мы в любом случае
rem не воспользуемся реально используемыми IP.
set /A TEMP_THIRD_BYTE=100+%RANDOM:~0,2%
set /A TEMP_FOURTH_BYTE=100+%RANDOM:~0,2%
set TEMP_IP="192.168.%TEMP_THIRD_BYTE%.%TEMP_FOURTH_BYTE%"
set TEMP_GATEWAY="192.168.%TEMP_THIRD_BYTE%.1"
rem устанавливаем параметры интерфейса
netsh interface ip set address local static %TEMP_IP% %MASK% %TEMP_GATEWAY% 1
rem включаем интерфейс. для этого мы используем
C:\devcon\i386\devcon enable *VEN_10E*
rem получаем три последних байта MAC
FOR /F "Tokens=4-6 Delims=- " %%d in ('getmac^|find "Device\Tcpip_"') do (
set /a dec_d=0x%%d
set /a dec_e=0x%%e
set /a dec_f=0x%%f
)
rem подготавливаем нужный IP и шлюз
SET IP=10.%dec_d%.%dec_e%.%dec_f%
SET GATEWAY=10.%dec_d%.0.1
rem меняем параметры интерфейса на реальные
netsh interface ip set address local static %IP% %MASK% %GATEWAY% 1
netsh interface ip set dns local static %GATEWAY%

Add this script to autoload - for example, like this (reboot required):

reg ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run" /v "mac2ip" /t REG_SZ /d "c:\init\mac2ip.bat"

Thus, after executing these scripts, the virtual machine will receive the address specified by the kernel, and will be ready to work.
The kernel learns about it from the earned ping to this address.

Perhaps this scheme will add clarity (by clicking - full size in a new window):

6. Notes

The method we used - with the mac2ip script at startup - turned out to be rather slow in Win XP. In any case, its current implementation does its work for 10-15 seconds. At the same time, the time it takes from the start of the virtual machine to the start of the mac2ip script is 15-20 seconds.

However, we are in no hurry to switch to using the first method (DHCP with IP binding to MAC), because:

- firstly, VMs do not end with us in the usual way for them (i.e. centos / windows do not complete the shutdown), but with virsh destroy for the VM (anyway, pull the power out). This saves a lot of time, and the integrity of the used VM is still not of interest to us - it will be immediately deleted after use. So, the leased VM address will not be released immediately, but will be released after the default-lease-time DHCP expires. This means that we cannot immediately start the VM with the same MAC (and the same IP). It is unlikely that setting default-lease-time too small (seconds) is a good idea. A more realistic option is to learn and use OMAPI / omshell, and use them to remove unnecessary DHCP entries immediately after the VM stops.

- secondly, for Linux, receiving an address from DHCP will be slower than the current option with assigning a static address.

- thirdly - of course, other pitfalls will also be revealed.

So the relatively slow operation of the current version of the script on Windows is not enough reason to switch to using a variant with DHCP.

An ideal solution would be to speed up the mac2ip script for Windows. I will be glad to advice - since this is my first experience in managing the windows network interface from scripts, then perhaps there is already a bicycle.


Update: Firstly, The_Kf opened its eyes to the banal way to get the MAC on the turned off interface using ipconfig.
Secondly, gribozavr conducted an experiment that showed that lease-time of the leased IP is not necessary at all, because when binding IP to MAC, DHCP will give IP in any case to the machine with the same MAC, even if the lease did not expire. He also pointed to stateless autoconfiguration from IPv6.
Thirdly, when dealing with akshakirov, I suddenly realized why they didn’t immediately look more closely at DHCP.

Update 2: Fourth, amarao suggested using xenstore to transfer IP to the VM. In a guest Linux machine, you just need to install xenstore-utils, but for Windows, you may need to write a utility to read from xenstore.