Using Acronis Products in Test Automation
Hello! In this article I will tell you about how you can use our products to prepare the environment for testing your programs. In most software testing scenarios, to achieve repeatable results, the environment in which it is run must be identical. We will call the preparation of this environment before the test deployment (“deployment”).
The whole complex of automated testing is as follows: the starting point is the server with Jenkins installed on it ; through it we set the parameters for automatic test run:
- on which server or machine we will run our test;
- how to configure a raid controller on it; how, in what order and with what parameters to split disks;
- which operating system to install
- which products must be installed on this server to run the test scenario.
Jenkins via SSH connects to another server on which all the necessary scripts for deployment are located, and runs these scripts with the parameters passed to it. The server, which directly performs all the actions, prepares the configuration for loading the selected computer through PXE .
As what PXE will load, we chose our Acronis Bootable Media, because with its help you can do everything we need. Acronis Bootable Media is a stripped-down version of Linux (BusyBox) with the executable files of our product, packed in ramdisk . You can obtain the ramdisk of our media by installing Agent For Windows from the Acronis Backup distribution. The ramdisk will be located in the folder with the product installed:
it will be called
Appearance of media downloaded from ramdisk
In the loaded media, we will execute all the scripts for preparing the raid controller, and through it we will restore the image of one or another operating system - on top of the newly configured iron. The first step is to configure the raid controller. To configure disks, we use the storcli utility , which is pre-packaged in ramdisk.
Here is an example of a shell script that packs the files we need into ramdisk:
After we packed the utility in ramdisk, it can be used in the media downloaded by Acronis. To boot through custom ramdisk from under PXE, you need to create the correct configuration file for the boot machine. As the PXE server, we use the same machine to which Jenkins transfers control. On it we have a DHCP server. We keep all test machines in our own virtual network so as not to disturb other employees of the company.
The deployment process begins with the preparation of a configuration file for PXE: which is generated by the name of the machine. Since the same server also performs the DHCP function, the name is enough for us to find out its mac address, create the correct configuration file and put it in / var / lib / tftpboot. An example boot configuration file is shown below. After that, we reboot the machine through IPMI (if the machine has one).
After the machine boots from ramdisk, the autostart.sh script will start running in the media. First, we prescribe in it how to configure disks through storcli, and then we set the parameters for restoring a particular operating system image to the machine through our product. These images are made by our own Acronis product.
Backup are * .tib files and lie on a ball on the same network.
Here is an example autostart.sh script:
We pass the parameters for this script through / proc / cmdline, which we take the configuration file from PXE. We look at which raid configuration we should take, and call storcli directly with the parameters we need. Now we have everything configured for certain glands, in the future there is a desire to make these operations more intelligent.
After configuring the raid, we start the operation to restore the operating system. To do this, we again extract the recovery command from the / proc / cmdline passed to us and execute it.
Here is an example configuration file generated to boot the machine through PXE:
From it, we set which archive to restore - just that in the append parameter then gets into proc / cmdline inside the media.
The ramdisk should be in / var / lib / tftpboot / on the PXE server. In our case, it is called ramdisk64.dat. After our media has worked out and the system is restored, we may encounter a problem: after setting up the raid using storcli, volumes are not created on the virtual disks of the raid. Therefore, in the highest-level script that prepares the configuration for PXE boot, reboots the machine, and waits for the restored system to boot, we added the following script:
It creates NTFS volumes on every disk found (except the first). Since diskpart cannot accept parameters as input, it can only use a script, we generate a file with a command for listing disks on the fly and then format the file system using the disks from the listing.
After restoring the OS, you can proceed to install the products. Scripts and other actions are run on the test machine from the management server via an SSH connection. To do this, the test machine has an SSH server that is already installed in the image, so after restoring the machine, we just connect to it via SSH. We install the product by copying to the machine the msi files of our product that we need, which we simply install via msiexec. Now you can start the testing itself.
Tests are “wrappers” in Python that perform certain operations through the Acrocmd utility command line. Acrocmd allows us to perform all the same operations as through the GUI of the main product.
The launch of these files (implemented in Python) on the machine under test is handled by a special system installed on another management server. The management server over the SSH connection uploads all the scripts to the machine and runs them, collects their output and combines the results of all tests into one final report. In the future, it can be used to build a beautiful report on the test scenario run. To do this, we use a simple web server written in Python, which converts the results into beautiful html-pages.
Here is an example of the report that we get in the end. Comparison of backup speed in three different versions of the product. The abscissa axis is the iteration number in the run, and the ordinate axis is the backup speed.
Everyone solves the task of deployment automation in their own way. We have chosen the path using our own products, because according to their capabilities they are fully suited to our goals.
And how do you automate testing and preparing the environment for it?
The whole complex of automated testing is as follows: the starting point is the server with Jenkins installed on it ; through it we set the parameters for automatic test run:
- on which server or machine we will run our test;
- how to configure a raid controller on it; how, in what order and with what parameters to split disks;
- which operating system to install
- which products must be installed on this server to run the test scenario.
Jenkins via SSH connects to another server on which all the necessary scripts for deployment are located, and runs these scripts with the parameters passed to it. The server, which directly performs all the actions, prepares the configuration for loading the selected computer through PXE .
As what PXE will load, we chose our Acronis Bootable Media, because with its help you can do everything we need. Acronis Bootable Media is a stripped-down version of Linux (BusyBox) with the executable files of our product, packed in ramdisk . You can obtain the ramdisk of our media by installing Agent For Windows from the Acronis Backup distribution. The ramdisk will be located in the folder with the product installed:
C:\Program Files (x86)\Common Files\Acronis\BackupAndRecoveryAgent\
it will be called
agent_ramdisk.dat
for 32-bit media and agent_ramdisk64.dat
for 64-bit. Appearance of media downloaded from ramdisk
In the loaded media, we will execute all the scripts for preparing the raid controller, and through it we will restore the image of one or another operating system - on top of the newly configured iron. The first step is to configure the raid controller. To configure disks, we use the storcli utility , which is pre-packaged in ramdisk.
Here is an example of a shell script that packs the files we need into ramdisk:
# Распаковка ramdisk.
extract_ramdisk() {
rm -rf .ramdisk
mkdir .ramdisk
pushd .ramdisk
gzip -dc < $1 | cpio -i -d --no-absolute-filenames
popd
}
# Подкладывание нужных файлов в Acronis Media.
copy_extra_files() {
mkdir -p .ramdisk/tmp/root/.ssh
cp -f files/id_rsa .ramdisk/tmp/root/.ssh
cp -f files/autostart.sh .ramdisk/bin/autostart_dbg
cp -f files/libstorelibir-2.so.14.07-0 .ramdisk/bin
cp -f files/storcli64 .ramdisk/bin
}
# Запаковка ramdisk’a.
pack_ramdisk() {
mkdir -p out
pushd .ramdisk
/cygdrive/c/cygwin/bin/find . | cpio -H newc -o > ../out/agent_ramdisk_echo.dat.initrd
popd
gzip -c9 out/agent_ramdisk_echo.dat.initrd > out/ramdisk.dat
rm -f out/agent_ramdisk_echo.dat.initrd
rm -rf .ramdisk
}
# Вызов методов.
echo Extracting ramdisk...
extract_ramdisk $1
echo Copying extra files...
copy_extra_files
echo Packing ramdisk...
pack_ramdisk
After we packed the utility in ramdisk, it can be used in the media downloaded by Acronis. To boot through custom ramdisk from under PXE, you need to create the correct configuration file for the boot machine. As the PXE server, we use the same machine to which Jenkins transfers control. On it we have a DHCP server. We keep all test machines in our own virtual network so as not to disturb other employees of the company.
The deployment process begins with the preparation of a configuration file for PXE: which is generated by the name of the machine. Since the same server also performs the DHCP function, the name is enough for us to find out its mac address, create the correct configuration file and put it in / var / lib / tftpboot. An example boot configuration file is shown below. After that, we reboot the machine through IPMI (if the machine has one).
After the machine boots from ramdisk, the autostart.sh script will start running in the media. First, we prescribe in it how to configure disks through storcli, and then we set the parameters for restoring a particular operating system image to the machine through our product. These images are made by our own Acronis product.
Backup are * .tib files and lie on a ball on the same network.
Here is an example autostart.sh script:
configureRaidDisk() {
conf_id=$(cat /proc/cmdline | grep -o 'raid_configuration_number=[^ ]*' | sed 's/\(^raid_configuration_number=\)\(.*\)/\2/')
echo Raid configuration: $conf_id
# Default value not to reconfigure raid controller disk state.
if [ "$conf_id" == "0" ]; then
return
elif [ "$conf_id" == "1" ]; then
(/bin/storcli64 /c1/vall delete force && echo Delete all virtual drives.) || exit 1
(/bin/storcli64 /c1 add vd type=raid0 drives=252:3 wt nora direct strip=64 && echo Create system partition.) || exit 1
(/bin/storcli64 /c1 add vd type=raid0 drives=252:0,252:1,252:2 wt nora direct strip=64 && echo Create first virtual drive.) || exit 1
(/bin/storcli64 /c1 add vd type=raid0 drives=252:4,252:5,252:6 wt nora direct strip=64 && echo Create second virtual drive.) || exit 1
(/bin/storcli64 /c1 add vd type=raid0 drives=252:7 wt nora direct strip=64 && echo Create third virtual drive.) || exit 1
(/bin/storcli64 /c1/v0 set bootdrive=on && echo Set system disk as bootable.) || exit 1
fi
}
execCmd() {
tmp=$(cat /proc/cmdline | grep -oe acrocmd.*\) )
cmd=${tmp%%\)}
echo Recovering OS image...
($cmd >> /dev/null && echo Done.) || exit 1
}
configureRaidDisk
startProduct
execCmd
reboot now
We pass the parameters for this script through / proc / cmdline, which we take the configuration file from PXE. We look at which raid configuration we should take, and call storcli directly with the parameters we need. Now we have everything configured for certain glands, in the future there is a desire to make these operations more intelligent.
After configuring the raid, we start the operation to restore the operating system. To do this, we again extract the recovery command from the / proc / cmdline passed to us and execute it.
Here is an example configuration file generated to boot the machine through PXE:
timeout=1
default=media
image=kernel64.dat
label=media
initrd=ramdisk64.dat
append="ramdisk_size=100000 quiet vga=791 recover=(acrocmd recover disk --loc=smb://10.250.114.44/raid/images --credentials=image,qweqweqwe --arc=win7_64srv --disk=1 --target_disk=1) bootfile=root@10.250.114.101:/tmp/uefi/0AFA7218.conf#end raid_configuration_number=1"
From it, we set which archive to restore - just that in the append parameter then gets into proc / cmdline inside the media.
The ramdisk should be in / var / lib / tftpboot / on the PXE server. In our case, it is called ramdisk64.dat. After our media has worked out and the system is restored, we may encounter a problem: after setting up the raid using storcli, volumes are not created on the virtual disks of the raid. Therefore, in the highest-level script that prepares the configuration for PXE boot, reboots the machine, and waits for the restored system to boot, we added the following script:
echo list disk > list.txt
for /f "usebackq tokens=1,2" %%a in (`diskpart /s list.txt ^| findstr /r /c:"Disk [1-99]"`) do (
echo sel %%a %%b>script.txt
echo clean>>script.txt
echo create part primary>>script.txt
echo format FS=NTFS quick>>script.txt
echo assign>>script.txt
echo rescan>>script.txt
diskpart /s script.txt
)
del list.txt, del script.txt
It creates NTFS volumes on every disk found (except the first). Since diskpart cannot accept parameters as input, it can only use a script, we generate a file with a command for listing disks on the fly and then format the file system using the disks from the listing.
After restoring the OS, you can proceed to install the products. Scripts and other actions are run on the test machine from the management server via an SSH connection. To do this, the test machine has an SSH server that is already installed in the image, so after restoring the machine, we just connect to it via SSH. We install the product by copying to the machine the msi files of our product that we need, which we simply install via msiexec. Now you can start the testing itself.
Tests are “wrappers” in Python that perform certain operations through the Acrocmd utility command line. Acrocmd allows us to perform all the same operations as through the GUI of the main product.
The launch of these files (implemented in Python) on the machine under test is handled by a special system installed on another management server. The management server over the SSH connection uploads all the scripts to the machine and runs them, collects their output and combines the results of all tests into one final report. In the future, it can be used to build a beautiful report on the test scenario run. To do this, we use a simple web server written in Python, which converts the results into beautiful html-pages.
Here is an example of the report that we get in the end. Comparison of backup speed in three different versions of the product. The abscissa axis is the iteration number in the run, and the ordinate axis is the backup speed.
Everyone solves the task of deployment automation in their own way. We have chosen the path using our own products, because according to their capabilities they are fully suited to our goals.
And how do you automate testing and preparing the environment for it?