Run ReactOS from the BTRFS partition
Hi, Habr!
My name is Victor, and this year I am the only student in the Google Summer of Code program on the ReactOS project. Today I will talk a little about what I am doing as part of an internship.
ReactOS supports a bunch of different file systems for reading and writing (fat32, ext2, ReiserFS , BTRFS ), but still can only boot from a partition formatted with fat32. This spring, I decided that it was time to start correcting this situation, and applied for GSoC. And now, after a few months I am writing this post :)
Why BTRFS? The answer is simple - WinBtrfs file system drivercurrently the most stable and fully functional of all that are included in the Reactor code. At this stage, we want to fix exactly the kernel bugs that prevent us from using other filesystems for booting, so the FS driver bugs are completely useless to us.
But I had to start not with the OS kernel, but with the installer. Fortunately for the installer, almost everything was ready: all you had to do was enable the download of the WinBtrfs driver in our installer (usetup), and add a couple of lines of code to support formatting in the correct file system. After that, I managed (almost) without problems to copy ReactOS files to a partition formatted in BTRFS.
The installer was dealt with quickly, but the next task is much more interesting. The ReactOS loader - FreeLdr supports almost only two file systems - fat32 and iso (there is code for ext2 and ntfs, but nobody has been trying to run it for 5 years). Since FreeLdr repeats the principle of operation of the ntldr bootloader from MS, it consists of two parts - the boot sector at the beginning of the section where the MBR of the disk transfers control, and the main part, which puts the processor in protected mode, loads the ntoskrnl.exe kernel into memory, and does a whole bunch.
(this is how the ReactOS boot process looks)
Thus, to support the new file system, you need to write a boot record (VBR), the task of which is to find the executable file of the main part of the bootloader (here we call it freeldr.sys) in the root directory of the disk, load it into memory and transfer control to it. But this is not all, in freeldr.sys itself, a practically complete read-only file system driver is needed in order to read configuration files, the kernel, registry hives, etc.
First you had to deal with the BTRFS file system itself. Before that, the most difficult things that I picked were fat32 and ext2, so it took me a lot of time to study the BTRFS combine harvester. Documentation on wiki.kernel.orgit helps to figure it out, but for complete understanding it was not enough - you had to go to the sources of grub, u-boot and other downloaders. A python utility that I wrote to display the file system structures to the console turned out to be very useful for studying the file system structure. With it, I wrote the first prototype of the boot sector, which pulls the bootloader from a binary file with a disk image with the BTRFS file system.
(elements of the root directory are visible in the picture)
Now it's time for the real boot sector. Its writing is complicated by the fact that here we are working in real processor mode with all the ensuing consequences (~ 1 MB of memory, segment addressing and work with the disk through BIOS interrupts). Razdol'e for old school lovers like me :)
In the structures of BTRFS, almost all fields have a 64-bit size, which is quite “bloated” the code, since it was necessary to actively use 32-bit x86 instructions. Often you have to use constructions like:
Writing a b-tree bypass procedure turned out to be the most time-consuming, debugging took the most time. And after several sleepless nights, I managed to get the cherished error message from the second loading stage:
freeldr.sys was successfully loaded into memory, and I didn’t even need to use magic like Unreal Mode . 640kb is enough for everyone!
The boot sector code can be viewed in my repository on github (it is still awaiting refactoring), and all the work on BTRFS in this thread.
Now it's the turn of the second part of the bootloader - you need to teach it to read the configuration file from the new file system. Follow the news!
My name is Victor, and this year I am the only student in the Google Summer of Code program on the ReactOS project. Today I will talk a little about what I am doing as part of an internship.
ReactOS supports a bunch of different file systems for reading and writing (fat32, ext2, ReiserFS , BTRFS ), but still can only boot from a partition formatted with fat32. This spring, I decided that it was time to start correcting this situation, and applied for GSoC. And now, after a few months I am writing this post :)
Why BTRFS? The answer is simple - WinBtrfs file system drivercurrently the most stable and fully functional of all that are included in the Reactor code. At this stage, we want to fix exactly the kernel bugs that prevent us from using other filesystems for booting, so the FS driver bugs are completely useless to us.
But I had to start not with the OS kernel, but with the installer. Fortunately for the installer, almost everything was ready: all you had to do was enable the download of the WinBtrfs driver in our installer (usetup), and add a couple of lines of code to support formatting in the correct file system. After that, I managed (almost) without problems to copy ReactOS files to a partition formatted in BTRFS.
The installer was dealt with quickly, but the next task is much more interesting. The ReactOS loader - FreeLdr supports almost only two file systems - fat32 and iso (there is code for ext2 and ntfs, but nobody has been trying to run it for 5 years). Since FreeLdr repeats the principle of operation of the ntldr bootloader from MS, it consists of two parts - the boot sector at the beginning of the section where the MBR of the disk transfers control, and the main part, which puts the processor in protected mode, loads the ntoskrnl.exe kernel into memory, and does a whole bunch.
(this is how the ReactOS boot process looks)
Thus, to support the new file system, you need to write a boot record (VBR), the task of which is to find the executable file of the main part of the bootloader (here we call it freeldr.sys) in the root directory of the disk, load it into memory and transfer control to it. But this is not all, in freeldr.sys itself, a practically complete read-only file system driver is needed in order to read configuration files, the kernel, registry hives, etc.
First you had to deal with the BTRFS file system itself. Before that, the most difficult things that I picked were fat32 and ext2, so it took me a lot of time to study the BTRFS combine harvester. Documentation on wiki.kernel.orgit helps to figure it out, but for complete understanding it was not enough - you had to go to the sources of grub, u-boot and other downloaders. A python utility that I wrote to display the file system structures to the console turned out to be very useful for studying the file system structure. With it, I wrote the first prototype of the boot sector, which pulls the bootloader from a binary file with a disk image with the BTRFS file system.
(elements of the root directory are visible in the picture)
Now it's time for the real boot sector. Its writing is complicated by the fact that here we are working in real processor mode with all the ensuing consequences (~ 1 MB of memory, segment addressing and work with the disk through BIOS interrupts). Razdol'e for old school lovers like me :)
In the structures of BTRFS, almost all fields have a 64-bit size, which is quite “bloated” the code, since it was necessary to actively use 32-bit x86 instructions. Often you have to use constructions like:
mov si, SOME_OFFSET
lea si, [esi+ecx*8]
lea si, [esi+ecx*8]
lea si, [esi+ecx*8] // one element is 24 bytes longWriting a b-tree bypass procedure turned out to be the most time-consuming, debugging took the most time. And after several sleepless nights, I managed to get the cherished error message from the second loading stage:
freeldr.sys was successfully loaded into memory, and I didn’t even need to use magic like Unreal Mode . 640kb is enough for everyone!
The boot sector code can be viewed in my repository on github (it is still awaiting refactoring), and all the work on BTRFS in this thread.
Now it's the turn of the second part of the bootloader - you need to teach it to read the configuration file from the new file system. Follow the news!