August 16, 2015 at 19:13Exploring OpenWRT: How UImage and Sysupgrade Images Differ
- Tutorial
In the comments to the article “Upgrading the Upvel UR-313N4G router on OpenWRT”, a dispute arose between your humble servant and respected Maysoft about differences in the structure of the uImage and sysupgrade OpenWRT firmware images. I promised Maysoft to sort out the problem, and this article is before you.
As you know, in the OpenWRT download directory, for the most part, two types of firmware are available - uImage and sysupgrade, for example:
openwrt-15.05-rc3-ramips-rt305x-dir-320-b1-initramfs-uImage.bin
openwrt-15.05-rc3- ramips-rt305x-dir-320-b1-squashfs-sysupgrade.bin
The official FAQ writes very little about their differences:
What is the difference between the different image formats?
a factory image is one built for the bootloader flasher or stock software flasher
a sysupgrade image (previously named trx image) is designed to be flashed from within openwrt itself
The two have the same content, but a factory image would have extra header information or whatever the platform needs. Generally speaking, the factory image is to be used with the OEM GUI or OEM flashing utilities to convert the device to OpenWrt. After that, use the sysupgrade images.
According to the documentation, the content of the images is identical, except that there are additional headers in the factory image so that this image can be flashed through the Web interface of the original firmware.
Well, compare the size of the firmware:
openwrt-15.05-rc3-ramips-rt305x-dir-320-b1-initramfs-uImage.bin - 3253035 bytes.
openwrt-15.05-rc3-ramips-rt305x-dir-320-b1-squashfs-sysupgrade.bin - 3407876 bytes.
Wow, the sysupgrade firmware is almost 140 KB larger than uImage, and according to the documentation they should be about the same size, and uImage due to this very “extra header information” is a bit bigger.
Of course, it is enough to look at the assembly scripts to understand how uImage and sysupgrade images differ, but this, you must admit, is unsportsmanlike. Today we will analyze the firmware “forehead”, as if we do not have the source code, and already at the end we will look at the assembly scripts to confirm our guesses.
The main tool for analyzing firmware at the moment is the binwalk utility, available under Linux. Rename the shorter firmware files so that it is more convenient for us, and begin the analysis.
> binwalk ./uImage.bin
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
0 0x0 uImage header, header size: 64 bytes, header CRC: 0x19DE1499, created: Fri Jul 3 22:16:00 2015, image size: 3252971 bytes, Data Address: 0x80000000, Entry Point: 0x80000000, data CRC: 0x886ADE01, OS: Linux, CPU: IPS, image type: OS Kernel Image, compression type: lzma, image name: "MIPS OpenWrt Linux-3.18.17"
64 0x40 LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 5479932 bytes
It seems that the whole firmware is an uImage image - at the beginning there is a header with a length of 64 (0x40) bytes, and after us - a data stream compressed by the LZMA algorithm, size 3252971 bytes. Add 64 and 3252971, we get 3253035 bytes, that is, the size of the downloaded image. Therefore, apart from the uImage image, there is nothing else in the file. Binwalk can decompress found LZMA streams. In principle, you can manually cut off the first 64 bytes from the file and unpack the remainder with the lzma -d command, but why, when is there such a convenient tool?
> binwalk -e ./uImage.bin
Let's see what we got
> ls -l ./_uImage.bin.extracted/
итого 8532
-rw-r--r-- 1 user user 5479932 авг 8 23:10 40
-rw-r--r-- 1 user user 3252971 авг 8 23:10 40.7z
A file named 40 (offset in the source file) is the unpacked stream. Let's set binwalk on it:
binwalk ./_uImage.bin.extracted/40
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
2812692 0x2AEB14 Linux kernel version "3.18.17 (buildbot@builder1) (gcc version 4.8.3 (OpenWrt/Linaro c version 4.8.3 (OpenWrt/Linaro GCC 4.8-2014.04 r46018) ) #2 Fr"
2932132 0x2CBDA4 LZMA compressed data, properties: 0x5D, dictionary size: 16777216 bytes, missing uncompressed size
2936592 0x2CCF10 xz compressed data
3400392 0x33E2C8 LZMA compressed data, properties: 0x6D, dictionary size: 1048576 bytes, uncompressed size: -1 bytes
And here we have something, at first glance, incomprehensible - binwalk discovered the Linux kernel at offset 0x2AEB14 and three compressed data streams following the kernel. The fact is that binwalk uses heuristics for analysis and what it outputs is not the ultimate truth, but information for consideration.
Common sense dictates that the kernel should start at offset 0, and the compressed stream should be one and contain initramfs - the initial file system loaded into RAM. The kernel documentation says the same thing :
What is initramfs?
- All 2.6 Linux kernels contain a gzipped "cpio" format archive, which is extracted into rootfs when the kernel boots up. After extracting, the kernel checks to see if rootfs contains a file "init", and if so it executes it as PID 1.
and further
Populating initramfs:
- The 2.6 kernel build process always creates a gzipped cpio format initramfs archive and links it into the resulting kernel binary. By default, this archive is empty (consuming 134 bytes on x86).
The stream format is also mentioned here - CPIO.
Let's see what binwalk can extract from our image:
> binwalk -e ./_uImage.bin.extracted/40
ls -l ./_uImage.bin.extracted/_40.extracted/
итого 14028
-rw-r--r-- 1 user user 2547808 авг 8 23:25 2CBDA4.7z
-rw-r--r-- 1 user user 2543340 авг 8 23:25 2CCF10.tar
-rw-r--r-- 1 user user 7186944 авг 8 23:25 33E2C8
-rw-r--r-- 1 user user 2079540 авг 8 23:25 33E2C8.7z
So, only the stream at offset 33E2C8 was successfully unpacked. If we do everything right, then this should be a CPIO container with a file system:
> binwalk _uImage.bin.extracted/_40.extracted/33E2C8
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
0 0x0 ASCII cpio archive (SVR4 with no CRC), file name: "dev", file name length: "0x00000004", file size: "0x00000000"
116 0x74 ASCII cpio archive (SVR4 with no CRC), file name: "dev/console", file name length: "0x0000000C", file size: "0x00000000"
240 0xF0 ASCII cpio archive (SVR4 with no CRC), file name: "lib", file name length: "0x00000004", file size: "0x00000000"
356 0x164 ASCII cpio archive (SVR4 with no CRC), file name: "lib/netifd", file name length: "0x0000000B", file size: "0x00000000"
480 0x1E0 ASCII cpio archive (SVR4 with no CRC), file name: "lib/netifd/netifd-wireless.sh", file name length: "0x0000001E", file size: "0x00001638"
***********Куча файлов***********
7186416 0x6DA7F0 ASCII cpio archive (SVR4 with no CRC), file name: "dev/urandom", file name length: "0x0000000C", file size: "0x00000000"
7186540 0x6DA86C ASCII cpio archive (SVR4 with no CRC), file name: "dev/pts", file name length: "0x00000008", file size: "0x00000000"
7186660 0x6DA8E4 ASCII cpio archive (SVR4 with no CRC), file name: "TRAILER!!!", file name length: "0x0000000B", file size: "0x00000000"
At the end of the archive we see a file named TRAILER !!! .. which. according to the documentation , this is the end of the archive label.
So, the structure of the initramfs-uImage firmware is as follows:
Now let's take on the squashfs-sysupgrade image. The name implies that the image contains (except for the kernel) the squashfs file system. Let's see if this is so:
> binwalk -e ./sysupgrade.bin
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
0 0x0 uImage header, header size: 64 bytes, header CRC: 0x66CC90D2, created: Mon Jul 6 21:54:35 2015, image size: 1142606 bytes, Data Address: 0x80000000, Entry Point: 0x80000000, data CRC: 0x91B77696, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "MIPS OpenWrt Linux-3.18.17"
64 0x40 LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 3396940 bytes
1142670 0x116F8E Squashfs filesystem, little endian, version 4.0, compression:lzma (non-standard type definition), size: 2221946 bytes, 1132 inodes, blocksize: 262144 bytes, created: Mon Jul 6 21:54:02 2015
Let's take the arithmetic: 64 + 1142606 (image size) = 1142670, just at this offset the squashfs image starts, and it ends at the offset 1142670 + 2221946 = 3364616. The size of the image, meanwhile, is 3407876 bytes, which means we have 3407876 more - 3364616 = 43260 bytes of unidentified information. Let's see what's there, hexdump
> hexdump -s 3364616 ./sysupgrade.bin
0335708 ffff ffff ffff ffff ffff ffff ffff ffff
*
0335ff8 ffff ffff ffff ffff adde dec0 ffff ffff
0336008 ffff ffff ffff ffff ffff ffff ffff ffff
*
0337ff8 ffff ffff ffff ffff adde dec0 ffff ffff
0338008 ffff ffff ffff ffff ffff ffff ffff ffff
*
033fff8 ffff ffff ffff ffff adde dec0
0340004
There is clearly some kind of stub. We will come back to her later.
Let's see what we have in the directory with the unpacked image:
> ls -l _sysupgrade.bin.extracted/
итого 8820
-rw-r--r-- 1 user user 2221946 авг 8 23:40 116F8E.squashfs
-rw-r--r-- 1 user user 3396940 авг 8 23:40 40
-rw-r--r-- 1 user user 3407812 авг 8 23:40 40.7z
Unpack the LZMA stream at offset 40:
binwalk -e _sysupgrade.bin.extracted/40
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
2812644 0x2AEAE4 Linux kernel version "3.18.17 (buildbot@builder1) (gcc version 4.8.3 (OpenWrt/Linaro c version 4.8.3 (OpenWrt/Linaro GCC 4.8-2014.04 r46018) ) #1 Fr"
2932068 0x2CBD64 LZMA compressed data, properties: 0x5D, dictionary size: 16777216 bytes, missing uncompressed size
2936444 0x2CCE7C xz compressed data
3396424 0x33D348 ASCII cpio archive (SVR4 with no CRC), file name: "dev", file name length: "0x00000004", file size: "0x00000000"
3396540 0x33D3BC ASCII cpio archive (SVR4 with no CRC), file name: "dev/console", file name length: "0x0000000C", file size: "0x00000000"
3396664 0x33D438 ASCII cpio archive (SVR4 with no CRC), file name: "root", file name length: "0x00000005", file size: "0x00000000"
3396780 0x33D4AC ASCII cpio archive (SVR4 with no CRC), file name: "TRAILER!!!", file name length: "0x0000000B", file size: "0x00000000"
Here we have the Linux kernel and a small initramfs image with four files. The rest, apparently, has moved to the squashfs image:
> unsquashfs -i ./_sysupgrade.bin.extracted/116F8E.squashfs
Parallel unsquashfs: Using 4 processors
1033 inodes (1034 blocks) to write
squashfs-root
squashfs-root/bin
squashfs-root/bin/ash
squashfs-root/bin/board_detect
squashfs-root/bin/busybox
***********Куча файлов***********
squashfs-root/www/luci-static/resources/load.svg
squashfs-root/www/luci-static/resources/wifirate.svg
squashfs-root/www/luci-static/resources/wireless.svg
squashfs-root/www/luci-static/resources/xhr.js
created 848 files
created 99 directories
created 184 symlinks
created 0 devices
created 0 fifos
Indeed, the main file system is contained in the squashfs image.
Now we will deal with the stub at the end of the image. There is a suspicion that it somehow relates to the JFFS2 FS, because when flashing the sysupgrade image and then loading it into dmesg, the lines appear:
[ 29.550000] jffs2_scan_eraseblock(): End of filesystem marker found at 0x0
[ 29.580000] jffs2_build_filesystem(): unlocking the mtd device... done.
[ 29.590000] jffs2_build_filesystem(): erasing all blocks after the end marker...
and when flashing and loading the uImage image, these lines are not. A search in the “vanilla” kernel for these lines yields nothing, but in the OpenWRT source tree there are such lines:
--- a/fs/jffs2/build.c
+++ b/fs/jffs2/build.c
@@ -114,6 +114,16 @@ static int jffs2_build_filesystem(struct
dbg_fsbuild("scanned flash completely\n");
jffs2_dbg_dump_block_lists_nolock(c);
+ if (c->flags & (1 << 7)) {
+ printk("%s(): unlocking the mtd device... ", __func__);
+ mtd_unlock(c->mtd, 0, c->mtd->size);
+ printk("done.\n");
+
+ printk("%s(): erasing all blocks after the end marker... ", __func__);
+ jffs2_erase_pending_blocks(c, -1);
+ printk("done.\n");
+ }
+
dbg_fsbuild("pass 1 starting\n");
c->flags |= JFFS2_SB_FLAG_BUILDING;
/* Now scan the directory tree, increasing nlink according to every dirent found. */
--- a/fs/jffs2/scan.c
+++ b/fs/jffs2/scan.c
@@ -148,8 +148,14 @@ int jffs2_scan_medium(struct jffs2_sb_in
/* reset summary info for next eraseblock scan */
jffs2_sum_reset_collected(s);
- ret = jffs2_scan_eraseblock(c, jeb, buf_size?flashbuf:(flashbuf+jeb->offset),
- buf_size, s);
+ if (c->flags & (1 << 7)) {
+ if (mtd_block_isbad(c->mtd, jeb->offset))
+ ret = BLK_STATE_BADBLOCK;
+ else
+ ret = BLK_STATE_ALLFF;
+ } else
+ ret = jffs2_scan_eraseblock(c, jeb, buf_size?flashbuf:(flashbuf+jeb->offset),
+ buf_size, s);
if (ret < 0)
goto out;
@@ -561,6 +567,17 @@ full_scan:
return err;
}
+ if ((buf[0] == 0xde) &&
+ (buf[1] == 0xad) &&
+ (buf[2] == 0xc0) &&
+ (buf[3] == 0xde)) {
+ /* end of filesystem. erase everything after this point */
+ printk("%s(): End of filesystem marker found at 0x%x\n", __func__, jeb->offset);
+ c->flags |= (1 << 7);
+
+ return BLK_STATE_ALLFF;````
+ }
+
/* We temporarily use 'ofs' as a pointer into the buffer/jeb */
ofs = 0;
max_ofs = EMPTY_SCAN_SIZE(c->sector_size);
Yeah, here is our 0xDEADCODE stub. If the JFFS2 driver finds this label, then it considers it the end of the previous file system and erases everything from the label zero byte to the end of the drive. Thus, the label itself is also overwritten.
After that, the driver creates a new instance of JFFS2 at this location.
So, the following image structure is obtained:
Thus:
- uImage contains the minimum necessary functionality to run OpenWRT and due to this, its structure can be easily changed so that it passes validation checks in the Web interfaces of the original firmware.
- Sysupgrade is more complex and uses Linux-specific tools - SquashFS and JFFS2.
- Both types of images do not contain a bootloader (and should not)
- When flashing through the bootloader (via UART or disaster recovery), you can immediately sew sysuprade.
- When flashing through mtd_write, if this utility is available via telnet from the official firmware, you can also flash sysupgrade right away.
In conclusion, take a look at the OpenWRT build scripts to make sure of our conclusions. Let's start with the file / target / linux / ramips / Makefile . But if you think this is a regular GNU Makefile, then it is not. Here is how the developers themselves describe the improved Makefile:
Looking at one of the package makefiles, you'd hardly recognize it as a makefile. Through what can only be described as blatant disregard and abuse of the traditional make format, the makefile has been transformed into an object oriented template which simplifies the entire ordeal.
The image build seems to run on line 564:
Image/Build/Profile/DIR-320-B1=$(call BuildFirmware/Default8M/$(1),$(1),dir-320-b1,DIR-320-B1)
The purpose of BuildFirmware / Default8M is described in lines 195 and 196:
BuildFirmware/Default8M/squashfs=$(call BuildFirmware/OF,$(1),$(2),$(3),$(ralink_default_fw_size_8M),$(4))
BuildFirmware/Default8M/initramfs=$(call BuildFirmware/OF/initramfs,$(1),$(2),$(3),$(4))
It consists of two positions: squashfs and initramfs. The goals of BuildFirmware / OF and BuildFirmware / OF / initramfs are described in the same file just above.
149 # $(1), Rootfs type, e.g. squashfs
150 # $(2), lowercase board name
151 # $(3), DTS filename without .dts extension
152 # $(4), maximum size of sysupgrade image
153 # $(5), uImage header's ih_name field
154 define BuildFirmware/OF
155 $(call MkImageLzmaDtb,$(2),$(3),$(5))
156 $(call MkImageSysupgrade/$(1),$(1),$(2),$(4),$(6))
157 endef
and
169 # $(1), squashfs/initramfs
170 # $(2), lowercase board name
171 # $(3), DTS filename without .dts extension
172 # $(4), ih_name field of uImage header
173 define BuildFirmware/OF/initramfs
174 $(call MkImageLzmaDtb,$(2),$(3),$(4),-initramfs)
175 $(CP) $(KDIR)/vmlinux-$(2)-initramfs.uImage $(call imgname,$(1),$(2))-uImage.bin
176 endef
By the name of the target MkImageLzmaDtb, you can guess that it creates an uImage image from the description from the DTS file. For the purpose of BuildFirmware / OF / initramfs, an initramfs section with the main file system is added to this image, after which the image is copied to the output directory. For the BuildFirmware / OF target, the source image is processed by the MkImageSysupgrade function, which, using the “cat” command, attaches the squashfs section to it, and then calls the prepare_generic_squashfs function defined in the image.mk file .
# pad to 4k, 8k, 16k, 64k, 128k, 256k and add jffs2 end-of-filesystem mark
110 define prepare_generic_squashfs
111 $(STAGING_DIR_HOST)/bin/padjffs2 $(1) 4 8 16 64 128 256
112 endef
And the padjffs2 utility, written in C , writes the 0xDEADCODE mark to the end of the image file:
static unsigned char eof_mark[4] = {0xde, 0xad, 0xc0, 0xde};
***
static unsigned char *pad = eof_mark;
***
/* write out the JFFS end-of-filesystem marker */
t = write(fd, pad, pad_len);
if (t != pad_len) {
ERRS("Unable to write to %s", name);
goto close;
}
In general, the OpenWRT world is beautiful and amazing, especially comments like:
# The real magic happens inside these templates
or
/ * vodoo from original driver * /
In conclusion, I want to say that when building OpenWRT from source, you need to pay attention to the build environment, for example, in this repository on my Debian 8 + Sid, even a set of tools for cross-compilation is not going to be built. But on Debian 7, all this wealth is going to be just amazing, if you do not edit the initial configuration. If you correct it, it may turn out that some Web sources from which the assembly script downloads the source code are already “rotten”, you need to look for new sources, fix the script, and so on.