December 12, 2014 at 09:37Transferring Windows OS to another computer using Linux OS
After reading the article “Frivolous cloning of MS Windows XP / Server 2003 OS with your own hands, GNU / Linux tools”, there was a natural desire to supplement it in accordance with the realities of the end of 2014.
In general, the problem of deploying a particular version of Windows consists of two parts - firstly, you need to copy the partition data from the reference PC to a new one, and secondly, you need to ensure that Windows boots on the changed hardware.
Let's start by copying the data. What data do we need to copy? For Windows XP, these are MBR (446 bytes at the beginning of the disk, the main partition table and the final signature 55AAh), data on extended partitions, and the actual data on the Windows system partition. For Vista and Windows 7, a 100-megabyte system partition is also added (GPT is not considered). It should be remembered that the first partition in Windows XP starts with sector 63 by default, and in Windows Vista / 7 - with sector 2048. Naturally, we will carry out all operations on an unstarted Windows, booting from the System Resque CD .
So, do not copy the MBR “forehead”, for this there is ms-sys utility in Linux, which writes the bootloader of any version of Windows from Win95 to Win7 in MBR. To save and restore the partition structure there is a sfdisk utility.
For example, like this:
Added:
In order for WIndows to boot, it is also necessary to generate a disk signature. To do this, read the contents of the registry key HKLM \ SYSTEM \ MountedDevices \ <\ DosDevices \ "The name of the partition on the disk with a colon">, and select the first 8 bytes (in hexadecimal notation), for example: 4F BE 4F BE. You can do this on a Linux mounted NTFS volume using the reged utility described below. Please note that in reged the key name containing the slash must be specified through a double slash, for example: \\ DosDevices \\ C :. Then, the specified 8 bytes should be written to the disk starting at offset 0x01B8 with hexedit. After that, the disk is ready to boot.
After copying the MBR and the partition structure, you can copy the actual partition data using partclone. The partclone tool allows you to create images of partitions ext2 / 3/4, fat, ntfs, xfs, btrfs, reiserfs, etc., analyzing the internal structure of the FS and including only used sectors in the image. Create a partition image on the reference PC using partclone, the size of the image will be equal to the size of the space used on the volume.
and deploy it on the target computer
or
It is also possible to directly copy data from one device to another using the -b switch:
Now consider the issue of data recovery to a partition of a size smaller than the reference. To do this, you need to mount a partclone image and either copy files from it to the target volume with the attributes, permissions, hardlinks and other properties of NTFS saved, or reduce the size of the NTFS partition inside the partclone image, and then copy this reduced partition to the target volume with the command partclone.ntfs with the -b option.
I don’t know the tools that could be used in Linux to copy files and hardlink from one NTFS volume to another with all attributes and streams (if anyone knows them, I ask in the comments), so we only have a reduction option volume size. Fortunately, there is an imagemount tool that exports a partclone image as a nbd block device and supports mounting the image including recording, the changes are saved in a separate file (key -c). Exporting a volume for recording
Now change the size of the FS on the NTFS volume. The test run will be done first (-n switch)
and then the real one:
Clone the section:
Unfortunately, partclone does not support image compression, but this problem can be circumvented with crutches and backups so beloved in Linux. Create an empty qcow2 format image with a virtual size obviously exceeding our needs (for example, 1 Tb). Its real volume will increase depending on the amount of data that we write there.
Mount it as a block device
Now copy the file system to the virtual device
unmount the device and as a result we get a qcow2 file with a size slightly larger than the size of the original NTFS volume. Now you can apply internal compression to the file:
It is easy to notice that this method requires doubled disk space for storing the original and compressed images, as well as additional time for compression. If we apply another crutch, we will get rid of these shortcomings. After mounting the qcow2 image (using the qcow-nbd -c / dev / nbd0 <image file> command), format it into the btrfs file system
and now mount with compression:
and create an NTFS file system image
After that, unmount everything.
As a result, we get an image file with compression, which includes only used sectors and which can be mounted via imagemount as a file system and read its contents.
For example, I was able to reduce the size of the image with a freshly installed Windows XP from 2.7 Gb to 587 Mb. In this case, you can mount the image without unpacking and view the files inside.
Much better than dd | gzip, right?
Now we are faced with the second task - to make Windows NTFS installed on that volume able to start. In the case of Windows 7, we also copy the boot partition, for Windows Vista - we additionally do (with Windows Vista Live CD) rewriting the BCD so that the data in this area corresponds to the serial number of the hard drive, check that the boot partition has the corresponding boot flag (you can do via parted) and reboot. Then with some probability we get a blue screen with the inscription 0x0000007B Inaccesseble boot device. In this case, it is recommended to set the correct HAL for Windows XP, but it’s a bad luck, I haven’t seen single-core PCs without APIC and especially ACPI for a long time, so our reference image most likely already contains the correct multiprocessor HAL. Therefore, do not hope that replacing the HAL will fix our problem right away. Most likely, the reason is that Windows did not recognize the hard disk controller, and, accordingly, could not continue to boot from it.
Suppose we did not use sysprep when creating the standard (otherwise it would be unsportsmanlike because sysprep will do all the work for us). Let's try to figure out why Windows does not find the disk controller on the new hardware, despite the fact that in the case of a “clean” installation, it sees this controller. The fact is that at the final stage of the installation, Windows disables (supposedly to speed up the download) all the drivers of the disk controllers on which there are no system partitions. It turns out like with a gopher in the movie "DMB": the system does not see it, but it is.
For Windows XP, there is a well-known MergeIDE patch , which, in fact, includes IDE device drivers back in the registry, simultaneously copying driver files from driver.cab to \ system32 \ drivers. The problem is that this patch is designed to run from under Windows. Let's try to impose it under Linux manually. Let the partition with Windows XP be mounted in / mnt, then:
In order to apply the patch MergeIDE.reg to the registry, you need to edit it by removing line breaks with the "\" character in the last lines of the file, and then use the reged utility with the -I switch
It may be necessary to also fix the CurrentControlSet key, which is sometimes not available on Windows that is not running, on ControlSet001 / 002.
After that, you can reboot. I must say that in my practice there were several cases when it was not possible to launch the image on other equipment either by the method indicated above or by using paid tools like Acronis Universal Restore.
Looking at the contents of the MergeIDE.reg file, you can see that all the magic of starting the boot driver consists in adding entries to two registry branches - HKLM \ SYSTEM \ CurrentControlSet \ Control \ CriticalDeviceDatabase and HKLM \ SYSTEM \ CurrentControlSet \ Services.
The CriticalDeviceDatabase branch contains a list of devices necessary for a successful boot of the system and sets the correspondence between the device class and the service (driver) that is responsible for it, and the Services branch contains the operation parameters of specific drivers. Of particular interest is the Start parameter - startup level:
0 - loading before the kernel starts with the OS loader
1 - loading at the time of kernel initialization
2 - loading during OS
3 startup - loading on demand
4 - do not load
obviously if our driver has a different launch level 0, then we will not see the corresponding controller at boot time.
MergeIDE.reg operates with the class identifier 4d36e96a-e325-11ce-bfc1-08002be10318 - ATA / ATAPI disk controllers. ClassGUIDs for various classes of devices are known, they can be found, for example, here .
Thus, you can force the system to start any driver at the moment the kernel is loaded into memory, for example, for a network card, if you boot via AoE or iSCSI.
For Windows Vista / 7, you can simply go through the Services branch in search of drivers containing the word IDE in the name, and then change the startup type (Start key) from “3-delayed launch” to “0-load at kernel boot time”. For these OSes, there is another way - we find msachi and iaStorV entries in the Services branch (universal AHCI driver and AHCI driver for Intel devices) and also change the startup type from “3” to “0”, and then go to the BIOS and change there AHCI type of disk controller. Unfortunately, this trick will not work with Windows XP, due to the lack of a built-in universal AHCI driver.
Therefore, let's pay attention to the absolutely magical driver of the UniATA disk controller from Alexander Telyatnikov aka Alter. This driver initializes any controller that considers itself IDE- or AHCI-compatible, and therefore is great for our purposes. The installation procedure is exactly the same - copy the driver file, register in CriticalDeviceDatabase and in Services.
Now we integrate the file my_uniata_inst.reg into the image with the following contents
Here ven_8086 & dev_2922 is the identifier of the installed SATA controller. We also believe that we do not have a CurrentControlSet and use the active branch - ControlSet001, whose number is specified in HKLM \ SYSTEM \ Select \ Current.
If this method does not work, you can try the more "dirty" installation methods - using the uniata_w2k.reg file from the driver archive or following the instructions on the developer's website.
The basic set of utilities can be found in the System Rescue CD . This includes ms-sys, mkfs.btrfs, sfdisk, partclone and reged. Imagemount can be taken from the partclone-utils package by simply unpacking it to the root systemresquecd file system. With qemu-utils, things are more complicated. I created a minimal installation of Debian in a separate directory using debootstrap, then chrooted there and then installed qemu-utils via apt-get. Before starting qemu-nbd, you need to remember to mount / proc, / sys and / dev inside the chroot.
In general, the problem of deploying a particular version of Windows consists of two parts - firstly, you need to copy the partition data from the reference PC to a new one, and secondly, you need to ensure that Windows boots on the changed hardware.
MBR, partition table and disk signature
Let's start by copying the data. What data do we need to copy? For Windows XP, these are MBR (446 bytes at the beginning of the disk, the main partition table and the final signature 55AAh), data on extended partitions, and the actual data on the Windows system partition. For Vista and Windows 7, a 100-megabyte system partition is also added (GPT is not considered). It should be remembered that the first partition in Windows XP starts with sector 63 by default, and in Windows Vista / 7 - with sector 2048. Naturally, we will carry out all operations on an unstarted Windows, booting from the System Resque CD .
So, do not copy the MBR “forehead”, for this there is ms-sys utility in Linux, which writes the bootloader of any version of Windows from Win95 to Win7 in MBR. To save and restore the partition structure there is a sfdisk utility.
For example, like this:
sfdisk -d /dev/sda > ./parttable.txt (сохраняем структуру разделов)
sfdisk -f /dev/sda < ./parttable.txt (восстанавливаем на другом ПК)
ms-sys -mbr /dev/sda (записываем MBR от Windows XP)Added:
In order for WIndows to boot, it is also necessary to generate a disk signature. To do this, read the contents of the registry key HKLM \ SYSTEM \ MountedDevices \ <\ DosDevices \ "The name of the partition on the disk with a colon">, and select the first 8 bytes (in hexadecimal notation), for example: 4F BE 4F BE. You can do this on a Linux mounted NTFS volume using the reged utility described below. Please note that in reged the key name containing the slash must be specified through a double slash, for example: \\ DosDevices \\ C :. Then, the specified 8 bytes should be written to the disk starting at offset 0x01B8 with hexedit. After that, the disk is ready to boot.
NTFS Partition Data
After copying the MBR and the partition structure, you can copy the actual partition data using partclone. The partclone tool allows you to create images of partitions ext2 / 3/4, fat, ntfs, xfs, btrfs, reiserfs, etc., analyzing the internal structure of the FS and including only used sectors in the image. Create a partition image on the reference PC using partclone, the size of the image will be equal to the size of the space used on the volume.
partclone.ntfs -c -s /dev/sda1 -o <файл образа>and deploy it on the target computer
partclone.ntfs -r -s <файл образа> -o /dev/sda1or
partclone.restore -s <файл образа> -o /dev/sda1It is also possible to directly copy data from one device to another using the -b switch:
partclone.ntfs -b -s /dev/sda1 -o /dev/sdb1Now consider the issue of data recovery to a partition of a size smaller than the reference. To do this, you need to mount a partclone image and either copy files from it to the target volume with the attributes, permissions, hardlinks and other properties of NTFS saved, or reduce the size of the NTFS partition inside the partclone image, and then copy this reduced partition to the target volume with the command partclone.ntfs with the -b option.
I don’t know the tools that could be used in Linux to copy files and hardlink from one NTFS volume to another with all attributes and streams (if anyone knows them, I ask in the comments), so we only have a reduction option volume size. Fortunately, there is an imagemount tool that exports a partclone image as a nbd block device and supports mounting the image including recording, the changes are saved in a separate file (key -c). Exporting a volume for recording
modprobe nbd max_part=16
imagemount -w -D -d /dev/nbd0 -f /mnt/images/winxp.img -c ./cow.binNow change the size of the FS on the NTFS volume. The test run will be done first (-n switch)
ntfsresize -s <новый размер> -n /dev/nbd0and then the real one:
ntfsresize -s <новый размер> /dev/nbd0Clone the section:
partclone.ntfs -b -s /dev/nbd0 -o /dev/sda1And now - a pinch of pichalka
Imagemount, even the latest version - 0.3.1, does not work with write disks on the x86_64 architecture (hangs tight) and practically does not work on the i386 architecture (it writes the image with errors), so resizing the partition is currently not possible. I will write partclone-utils in the bugtaker.
Unfortunately, partclone does not support image compression, but this problem can be circumvented with crutches and backups so beloved in Linux. Create an empty qcow2 format image with a virtual size obviously exceeding our needs (for example, 1 Tb). Its real volume will increase depending on the amount of data that we write there.
qemu-img create -f qcow2 <файл образа> 1024GMount it as a block device
modprobe nbd max_part=16
qemu-nbd -c /dev/nbd0 <файл образа>Now copy the file system to the virtual device
partclone.ntfs -b -s /dev/sda1 -o /dev/nbd0unmount the device and as a result we get a qcow2 file with a size slightly larger than the size of the original NTFS volume. Now you can apply internal compression to the file:
qemu-img convert -c <файл образа> <сжатый файл образа>It is easy to notice that this method requires doubled disk space for storing the original and compressed images, as well as additional time for compression. If we apply another crutch, we will get rid of these shortcomings. After mounting the qcow2 image (using the qcow-nbd -c / dev / nbd0 <image file> command), format it into the btrfs file system
mkfs.btrfs /dev/nbd0and now mount with compression:
mount -o compress-force=zlib /dev/nbd0 /mntand create an NTFS file system image
partclone.ntfs -c -s /dev/sda1 -o /mnt/ntfs-image.binAfter that, unmount everything.
As a result, we get an image file with compression, which includes only used sectors and which can be mounted via imagemount as a file system and read its contents.
For example, I was able to reduce the size of the image with a freshly installed Windows XP from 2.7 Gb to 587 Mb. In this case, you can mount the image without unpacking and view the files inside.
Much better than dd | gzip, right?
Running Windows on new hardware
Now we are faced with the second task - to make Windows NTFS installed on that volume able to start. In the case of Windows 7, we also copy the boot partition, for Windows Vista - we additionally do (with Windows Vista Live CD) rewriting the BCD so that the data in this area corresponds to the serial number of the hard drive, check that the boot partition has the corresponding boot flag (you can do via parted) and reboot. Then with some probability we get a blue screen with the inscription 0x0000007B Inaccesseble boot device. In this case, it is recommended to set the correct HAL for Windows XP, but it’s a bad luck, I haven’t seen single-core PCs without APIC and especially ACPI for a long time, so our reference image most likely already contains the correct multiprocessor HAL. Therefore, do not hope that replacing the HAL will fix our problem right away. Most likely, the reason is that Windows did not recognize the hard disk controller, and, accordingly, could not continue to boot from it.
Suppose we did not use sysprep when creating the standard (otherwise it would be unsportsmanlike because sysprep will do all the work for us). Let's try to figure out why Windows does not find the disk controller on the new hardware, despite the fact that in the case of a “clean” installation, it sees this controller. The fact is that at the final stage of the installation, Windows disables (supposedly to speed up the download) all the drivers of the disk controllers on which there are no system partitions. It turns out like with a gopher in the movie "DMB": the system does not see it, but it is.
Enable IDE Controllers: MergeIDE Patch
For Windows XP, there is a well-known MergeIDE patch , which, in fact, includes IDE device drivers back in the registry, simultaneously copying driver files from driver.cab to \ system32 \ drivers. The problem is that this patch is designed to run from under Windows. Let's try to impose it under Linux manually. Let the partition with Windows XP be mounted in / mnt, then:
cabextract -d /tmp/drivers "/mnt/WINDOWS/system32/Driver Cache/i386/driver.cab"
cd /tmp/drivers
cp atapi.sys intelide.sys pciide.sys pciidex.sys /mnt/WINDOWS/system32/driversIn order to apply the patch MergeIDE.reg to the registry, you need to edit it by removing line breaks with the "\" character in the last lines of the file, and then use the reged utility with the -I switch
reged -I /mnt/WINDOWS/system32/config/system HKEY_LOCAL_MACHINE\\SYSTEM ./MergeIDE.regIt may be necessary to also fix the CurrentControlSet key, which is sometimes not available on Windows that is not running, on ControlSet001 / 002.
After that, you can reboot. I must say that in my practice there were several cases when it was not possible to launch the image on other equipment either by the method indicated above or by using paid tools like Acronis Universal Restore.
Looking at the contents of the MergeIDE.reg file, you can see that all the magic of starting the boot driver consists in adding entries to two registry branches - HKLM \ SYSTEM \ CurrentControlSet \ Control \ CriticalDeviceDatabase and HKLM \ SYSTEM \ CurrentControlSet \ Services.
The CriticalDeviceDatabase branch contains a list of devices necessary for a successful boot of the system and sets the correspondence between the device class and the service (driver) that is responsible for it, and the Services branch contains the operation parameters of specific drivers. Of particular interest is the Start parameter - startup level:
0 - loading before the kernel starts with the OS loader
1 - loading at the time of kernel initialization
2 - loading during OS
3 startup - loading on demand
4 - do not load
obviously if our driver has a different launch level 0, then we will not see the corresponding controller at boot time.
MergeIDE.reg operates with the class identifier 4d36e96a-e325-11ce-bfc1-08002be10318 - ATA / ATAPI disk controllers. ClassGUIDs for various classes of devices are known, they can be found, for example, here .
Thus, you can force the system to start any driver at the moment the kernel is loaded into memory, for example, for a network card, if you boot via AoE or iSCSI.
For Windows Vista / 7, you can simply go through the Services branch in search of drivers containing the word IDE in the name, and then change the startup type (Start key) from “3-delayed launch” to “0-load at kernel boot time”. For these OSes, there is another way - we find msachi and iaStorV entries in the Services branch (universal AHCI driver and AHCI driver for Intel devices) and also change the startup type from “3” to “0”, and then go to the BIOS and change there AHCI type of disk controller. Unfortunately, this trick will not work with Windows XP, due to the lack of a built-in universal AHCI driver.
Boot from any IDE or SATA controller: UniATA driver
Therefore, let's pay attention to the absolutely magical driver of the UniATA disk controller from Alexander Telyatnikov aka Alter. This driver initializes any controller that considers itself IDE- or AHCI-compatible, and therefore is great for our purposes. The installation procedure is exactly the same - copy the driver file, register in CriticalDeviceDatabase and in Services.
wget http://alter.org.ua/ru/soft/win/uni_ata/BusMaster_v45d.tgz
tar -xzvf ./BusMaster_v45d.tgz
cp uniata.sys /mnt/WINDOWS/system32/drivers
cp atactl.exe /mnt/WINDOWS/system32Now we integrate the file my_uniata_inst.reg into the image with the following contents
UniATA_Inst.reg
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\CriticalDeviceDatabase\pci#ven_8086&dev_2922]
"ClassGUID"="{4D36E96A-E325-11CE-BFC1-08002BE10318}"
"Service"="uniata"
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\UniATA]
"Type"=dword:00000001
"Start"=dword:00000000
"ErrorControl"=dword:00000000
"Group"="System Bus Extender"
"ImagePath"=hex(2):53,00,79,00,73,00,74,00,65,00,6d,00,33,00,32,00,5c,00,44,00,52,00,49,00,56,00,45,00,52,00,53,00,5c,00,75,00,6e,00,69,00,61,00,74,00,61,00,2e,00,73,00,79,00,73,00,00,00
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\UniATA\Parameters\PnpInterface]
"1"=dword:0x00000000
"5"=dword:0x00000000
"SkipRaids"=dword:0x00000001Here ven_8086 & dev_2922 is the identifier of the installed SATA controller. We also believe that we do not have a CurrentControlSet and use the active branch - ControlSet001, whose number is specified in HKLM \ SYSTEM \ Select \ Current.
reged -I /mnt/WINDOWS/system32/config/system HKEY_LOCAL_MACHINE\\SYSTEM ./my_uniata.regIf this method does not work, you can try the more "dirty" installation methods - using the uniata_w2k.reg file from the driver archive or following the instructions on the developer's website.
Another pichalka
As it turned out, UniATA cannot initialize the Intel 8086: 2922 virtual controller, which is emulated by QEMU. The "neighboring" virtual IDE controller initializes wonderfully. Whose bug it is - QEMU or UniATA, is not entirely clear.
About Tools
The basic set of utilities can be found in the System Rescue CD . This includes ms-sys, mkfs.btrfs, sfdisk, partclone and reged. Imagemount can be taken from the partclone-utils package by simply unpacking it to the root systemresquecd file system. With qemu-utils, things are more complicated. I created a minimal installation of Debian in a separate directory using debootstrap, then chrooted there and then installed qemu-utils via apt-get. Before starting qemu-nbd, you need to remember to mount / proc, / sys and / dev inside the chroot.
Conclusion
Hidden text
Buy Better Acronis Universal Restore