How Android works, part 4


    Hello! We took the time to continue the series of articles about the internal device Android. In this article I will talk about the Android boot process, the file system contents, how user data and applications are stored, root access, the portability of Android assemblies, and the problem of fragmentation.


    Articles series:



    Packages


    As I said before, the Android architecture is built around applications. It is applications that play a key role in the device of many parts of the system, the model of activity and intent is built for the harmonious interaction of applications, and the Android security model is based on the isolation of applications. And if the activity manager is in charge of orchestrating the interaction of application components, then the package manager is responsible for installing, updating, and managing application rights (package manager — you can call it in the shell with a command pm).


    As the name “package manager” suggests, at this level applications are often called packages . Packages are distributed in the format of APK (Android package) - special zip-archives. Each package has a name (also known as an application ID ) that uniquely identifies this application (but not its specific version — on the contrary, the names of different versions of the package must match, otherwise they will be considered separate packages). Packet names are usually written in reverse DNS name notation - for example, the YouTube application uses the package namecom.google.android.youtube. Often, the package name matches the namespace used in its Java code, but Android does not require this (besides, application APK files usually include third-party libraries whose namespace naturally has nothing to do with the package names, who use them).


    Each APK must be signed by the developer using a digital signature when building . Android checks for the presence of this signature when installing the application, and when updating an already installed application, it additionally compares the public keys with which the old and new versions are signed; they must match, which ensures that the new version was created by the same developer as the old one. (If this check were not available, the attacker could create a package with the same name as an existing application, convince the user to install it by “updating” the application, and access the data of this application.)


    The package update itself is the installation of its new version instead of the old one, preserving the data and permissions received from the user. It is also possible to “roll back” (downgrade) applications to older versions, but by default Android deletes the data saved by the new version, since the old version may not be able to work with the data formats used by the new version.


    As I already said, usually the code of each application is executed under its own Unix-user (UID), which ensures their mutual isolation. Multiple applications can explicitly ask Android to use a common UID for them, which will allow them to directly access each other’s files and even, if they wish, run in one process.


    Although usually one package corresponds to one APK file, Android supports packages consisting of several APKs (this is called split APK, or split APK ). This is the basis of such "magic" features of Android as dynamic loading of additional modules of the application ( dynamic feature modules ) and Instant Run in Android Studio (automatically updating the code of the running application without reinstalling it completely and, in many cases, even without restarting).



    File system


    The device of the file system is one of the most important and interesting questions in the architecture of the operating system, and the device of the file system in Android is no exception.


    Of interest is, firstly, which file systems are used, that is, in what format the contents of the files are stored on the conditional disk (in the case of Android, this is usually flash-memory and SD-cards) and how this format is supported from the kernel . The Linux kernel used in Android to a varying degree supports a large number of very different file systems - from the FAT and NTFS used in Windows and the infamous HFS + and modern APFS used in Darwin to the 9pfs network from Plan 9. There are many “native” "For Linux file systems - for example, Btrfs and the ext family.


    The de facto standard for Linux for a long time has been ext4 , used by default by most popular Linux distributions. Therefore, there is nothing surprising in the fact that it is used in Android. Some builds (and some enthusiasts) also use the F2FS (Flash-Friendly File System), optimized specifically for flash-memory (however, with its advantages, things are not so straightforward ).


    Secondly, of interest is the so-called filesystem layout - the location of system and user folders and files in the file system. Filesystem layout in "normal Linux" deserves a more detailed description (which can be found, for example, by this link ); I will mention here only a few of the most important directories:


    • /homestores user home folders; here, in a variety of hidden folders ( .var.cache.configetc.), software programs store their settings, data and cache, user-specific,
    • /boot stores the Linux kernel and the image of the initramfs (special boot file system),
    • /usr(it would be more logical to call it /system) stores the main part of the system itself, including libraries, executable files, configuration files, as well as resources - themes for the interface, icons, contents of the system manual, etc.,
    • /etc(it would be more logical to call it /config) stores system-wide settings,
    • /devstores device files and other special files (for example, a socket /dev/log),
    • /var stores data to be changed - logs, system cache, database contents, etc.

    Android uses a similar, but noticeably different filesystem layout. Here are some of the most important parts of it:


    • /data stores changeable data,
    • the kernel and the initramfs image are stored on a separate flash partition (partition) that is not mounted on the main file system,
    • /systemmatches /usrand stores the system
    • /vendor- an analogue /systemintended for files specific to this Android build, and not included in the “standard” Android,
    • /dev, as in “regular Linux”, stores device files and other special files.

    The most interesting of these directories are /dataand /system. Content /systemdescribes the system and contains the majority of its component files. /systemis located on a separate section of flash-memory, which by default is mounted in read-only mode; usually the data on it is changed only when the system is updated. /dataIt is also located on a separate section and describes the changing state of a specific device, including user settings, installed applications and their data, caches, etc. Clearing all user data, the so-called factory reset, with such a scheme is simply to clear the contents of the data section ; The untouched system remains installed in the system section .


    # mount | grep /system
    /dev/block/mmcblk0p14 on /system type ext4 (ro,seclabel,relatime,data=ordered)
    # mount | grep /data
    /dev/block/mmcblk0p24 on /data type ext4 (rw,seclabel,nosuid,nodev,noatime,noauto_da_alloc,data=writeback)

    Applications — namely, their APKs, odex files (compiled ahead-of-time Java code), and ELF libraries — are installed in /system/app(for applications that come with the system) or in /data/app(for user-installed applications). When you create an Android assembly, each preinstalled application is allocated a folder with a view name /system/app/Terminal, and for user-installed applications during installation, folders are created whose names begin with their package name. For example, the YouTube application is saved in a folder with a name like /data/app/com.google.android.youtube-bkJAtUtbTuzvAioW-LEStg==/.


    About this suffix

    Суффикс в названии папок приложений — 16 случайных байт, закодированных в Base64. Использование такого суффикса не позволяет другим приложениям «угадать» путь к приложению, о существовании которого им знать не следует. В принципе, список установленных на устройстве приложений и путей к ним не является секретом — его можно получить через стандартные API — но в некоторых случаях (а именно, для Instant apps) на доступ к этим данным накладываются ограничения.


    Этот суффикс служит и другой цели. Каждый раз при обновлении приложения новая APK устанавливается в папку с новым суффиксом, после чего старая папка удаляется. До версии 8.0 Oreo в этом и состояло назначение суффиксов, и вместо случайных байт поочерёдно использовались-1 и -2 (например, /data/app/com.google.android.youtube-2 для YouTube).


    Полный путь к папке приложения в /system/app или /data/app можно получить с помощью стандартного API или команды pm path org.example.packagename, которая выводит пути всех APK-файлов приложения.


    # pm path com.android.egg
    package:/system/app/EasterEgg/EasterEgg.apk

    Since the pre-installed applications are stored in the system section (the contents of which, I recall, only change when the system is updated), they cannot be deleted (instead of this, Android provides the ability to "disable" them). Nevertheless, the update of the pre-installed applications is supported - at the same time a folder for the new version is created /data/appand the version supplied with the system remains in /system/app. In this case, the user has the opportunity to "remove updates" of such an application by returning to the version from /system/app.


    Another feature of pre-installed applications is that they can receive special “system” permissions . For example, third-party applications do not get permissions DELETE_PACKAGES, which allows you to remove other applications REBOOT, allowing you to restart the system, and READ_FRAME_BUFFERallowing direct access to the contents of the screen. These permissions have a signature protection level , that is, an application trying to access them must be signed with the same key as the application or service in which they are implemented - in this case, since these permissions are implemented by the system, the key that signed itself build Android.


    Each application is allocated a folder in /data/data(for example, /data/data/com.google.android.youtubefor YouTube) for storing changeable data . Only the application itself has access to this folder — that is, only the UID under which this application runs (if the application uses multiple UIDs, or several applications use a common UID, things can be more complicated). In this folder, the applications save the settings, the cache (in subfolders shared_prefsand cacherespectively) and any other data they need:


    # ls /data/data/com.google.android.youtube/
    cache  code_cache  databases  files  lib  no_backup  shared_prefs
    # cat /data/data/com.google.android.youtube/shared_prefs/youtube.xml<?xml version='1.0' encoding='utf-8' standalone='yes' ?><map>    <string name="user_account">username@gmail.com</string>    <boolean name="h264_main_profile_supported7.1.2" value="true" />    <int name="last_manual_video_quality_selection_max" value="-2" />    <...></map>

    The system is aware of the existence of the folder cacheand can clean it itself if there is not enough space. When you delete an application, the entire folder of this application is completely deleted, and the application leaves no trace behind it. Alternatively, both can be explicitly done in the settings:



    It is generated for each application data store called internal storage (internal storage).


    In addition, there is another type of storage in Android - the so-called external storage (external storage - this name reflects the original idea that the external storage should be located on the external SD card inserted into the phone). In fact, the external storage plays the role of the user's home folder - it is there that folders such as Documents, Download, Music and Pictures are located, the external storage is opened by the file managers as the default folder, it is to the contents of the external Android storage that the computer can access when it is connected by cable.


    # ls /sdcard
    Alarms      Download        Podcasts
    Android     Movies          Ringtones
    Books       Music           Subtitles
    DCIM        Notifications   bluetooth
    Documents   Pictures

    Unlike internal storage, divided into folders of individual applications, external storage is a “common zone”: it has full access to any application that has received the appropriate permission from the user. As I mentioned in the last article, this permission should be requested by applications such as the file manager; and for most other applications it is better to use intent with an action ACTION_GET_CONTENT, allowing the user to select the desired file in the system file manager.


    Many applications prefer to save some of their internal files that are large (for example, the cache of downloaded images and audio files) in external storage. For this, Android allocates a folder with the names of the form to applications in the external storage Android/data/com.google.android.youtube. The application itself does not need to access this folder .permission to access the entire external storage (because its UID is set as the owner of this folder), but any other application that has this permission can access this folder, so it really should only be used to store public and non-critical data. When you delete an application, the system will delete its special folder in the external storage; but files created by applications in external storage outside their special folder are considered owned by the user and remain in place after the application that created them is deleted.


    As I mentioned above, the initial assumption was that the external storage would actually be located on an external SD card, because at that time the volume of SD cards was significantly higher than the amount of memory embedded in the phones (in the very same HTC Dream there were only 256 megabytes, of which about 90 megabytes were allocated to the data section ). Since then, many conditions have changed; In modern phones, there is often no slot for an SD card, but a huge amount of internal memory is installed by mobile standards (for example, in Samsung Galaxy Note 9 it can be up to 512 gigabytes ).


    Therefore, in modern Android, almost always both internal and external storage are located in the internal memory. The actual path that the external storage is located in the file system is of the form /data/media/0(for each user of the device , a separate external storage is created , and the number in the path corresponds to the user number). For compatibility to external storage can also be reached by paths /sdcard, /mnt/sdcard, /storage/self/primary, /storage/emulated/0, multiple paths, starting with /mnt/runtime/, and some others.


    On the other hand, many devices still have a slot for an SD card. An SD card inserted into an Android device can be used as a regular external drive (without turning it into internal or external storage of the system) - save files to it, open files stored on it, use it to transfer files to other devices, etc. In addition, Android allows you to "borrow" the SD card and place the internal and external storage on it (this is called the adopted storage ). At the same time, the system reformats the SD card and encrypts its contents - the data stored on it cannot be read by connecting it to another device.



    You can read more about all this, for example, in this post and in the official documentation for application developers and creators of Android assemblies .


    Loading


    The traditional approach to computer security is limited to protecting the system from software attacks. It is believed that if the attacker has physical access to the computer, the game is already lost : he can get full access to any data stored on it. To do this, it is sufficient, for example, to launch an arbitrary operating system controlled by it on this computer, which allows it to bypass any restrictions on rights imposed by the “primary” system, or directly connect the data disk to another device. If desired, an attacker can leave the computer in a healthy state, but patch the system installed on it, install arbitrary backdoors, keyloggers, etc.


    The Unix user restriction model (and the app sandbox technology in Android based on it) is focused on protection against software attacks; the restriction of rights in Unix itself does not protect the system from a user who has entered the server and has received physical access to the computer. And if serious multi-user servers can and should be protected from unauthorized physical access to personal computers - and even more so mobile devices - this approach is simply not applicable.


    You can try to improve the situation with protection from an attacker who received physical access to the device in two ways:


    • First, you can encrypt the data stored on the disk , thereby preventing the attacker from gaining access to the data itself, even if it has access to the contents of the disk.
    • Secondly, it is possible in one way or another to limit the ability to load arbitrary operating systems on a device , forcing an attacker to undergo authentication and authorization procedures in the installed system.

    It is with these two areas of protection that the secure boot model in Android is associated.


    Verified Boot


    The Android boot process is designed so that, on the one hand, it does not allow attackers to download an arbitrary OS on the device, on the other hand, it can allow users to install customized Android assemblies (and other systems).


    First of all, Android devices, unlike desktop computers, usually do not allow the user (or an attacker) to boot from external media; instead, the bootloader installed on the device starts up immediately . Bootloader is a relatively simple program whose tasks (when booting in ordinary mode) include:


    • Initialization and configuration of the Trusted Execution Environment (for example, ARM TrustZone),
    • Finding the internal memory partitions in which Linux kernel and initramfs images are stored,
    • checking their integrity and integrity (integrity) - otherwise the download is interrupted with an error message - by verifying the digital signature of the manufacturer,
    • loading the kernel and initramfs into memory and transferring control to the kernel.

    Flashing, unlocking, fastboot and recovery


    In addition, the bootloader supports additional functionality for updating and reinstalling the system.


    First, it is the ability to download instead of the main system (Android) a special minimum system, called recovery . The recovery version installed on most Android devices by default is very minimal and only supports installing system updates automatically, but many Android enthusiasts install custom recovery .


    This is done by using the second bootloader feature to update and reinstall the system — supporting flashing the contents and partition structure on commands from a computer connected via a cable. To do this, the bootloader is able to boot into another special mode, which is called fastboot mode (or sometimes just bootloader mode), since usually the fastboot protocol (and the corresponding tool fastbootfrom the Android SDK from the computer side ) is used for communication between the computer and the bootloader. ).


    Some bootloader implementations use other protocols. This mainly concerns devices manufactured by Samsung, where a special bootloader implementation (Loke) communicates with a computer using its own proprietary protocol (Odin). To work with Odin on the computer side, you can use either the implementation from Samsung itself (which is also called Odin) or a free implementation called Heimdall .


    Specific details depend on the implementation of the bootloader (that is, differ depending on the device manufacturer), but in many cases installing recovery and Android assemblies, signed by the device manufacturer's key, simply works without additional difficulties:


    $ fastboot flash recovery recovery.img
    $ fastboot flash boot boot.img
    $ fastboot flash system system.img

    In this way, you can manually update the system; but installing an older version will not work: a function known as rollback protection will not allow the bootloader to download an older version of Android than it was loaded last time, even if it is signed by the manufacturer’s key, since downloading older versions opens the way to use published vulnerabilities that are fixed in newer versions.


    In addition, many devices support bootloader unlocking (unlocking the bootloader, also known as OEM unlock) - disabling the bootloader’s verification of the system signature and recovery, which allows you to install arbitrary assemblies of the one and the other (for some manufacturers this voids the warranty). This is how commonly installed such popular Android distributions as LineageOS (formerly CyanogenMod ), Paranoid Android , AOKP , OmniROM and others.


    Since unlocking the bootloader still allows you to load your own version of the system onto the device, for security reasons, when unlocking, all user data (from the data section ) is forcibly deleted . If the system is reinstalled by the user himself, and not by the attacker, after reinstalling, he can restore his data from the backup (for example, from a cloud backup on Google servers or from backup on external media), if the attacker - he gets a working system, but cannot steal the owner’s data devices.



    After installing the preferred recovery builds and systems, the bootloader should be locked back to protect your data again if the device falls into the hands of intruders.


    To unlock the bootloader, you may also need to additionally enable it from the system settings:



    Popular third-party recovery, which is installed by most flashaholics - TWRP (Team Win Recovery Project). It contains a touch interface and many advanced features, including the ability to install parts of the system from assemblies as zip archives, built-in support for backups, and even a full terminal emulator with a virtual keyboard:



    Disk encryption


    Modern versions of Android use file-based encryption . This mechanism is based on the ext4 encryption support implemented in the Linux kernel ( fscrypt ) and allows the system to encrypt different parts of the file system with different keys.


    By default, the system encrypts most user data located on the data section with a key that is created based on the user's password and is not saved to disk (credential encrypted storage). This means that when booting, the system should ask the user to enter their password in order to use it to decrypt the data. That is why the first time after switching on the device, the user meets the requirement to enter a full password or a pattern, and not just pass authentication by placing a finger on the finger scanner.



    In addition to credential encrypted storage, Android also uses device encrypted storage, which is key encryption based on data stored on the device (including in the Trusted Execution Environment). Files encrypted in this way can be decrypted by the system before the user enters a password. This is the basis of the feature known as Direct Boot.: the system is able to boot into a working state without entering a password; at the same time, applications can explicitly ask the system to save the (least private) part of their data in device encrypted storage, which allows them to begin to perform their basic functions without waiting for the full unlocking of the device. For example, Direct Boot allows the alarm to be triggered before the first password is entered, which is especially useful if the device unexpectedly restarts at night due to a temporary power outage or system failure.


    Root


    The so-called root access is the ability to execute code as a “root user” (UID 0, also known as superuser ). Let me remind you that root is a dedicated Unix user who, with a few interesting exceptions, is allowed full access to everything in the system and to which no restrictions of rights apply.


    Like most other modern operating systems, Android is designed with the expectation that an ordinary user does not need to use root access for anything . Unlike more closed operating systems, whose users call the destruction of the restrictions imposed on them literally “jailbreak”, in Android it is possible to “out of the box”, without the need to get root access and install special third-party “tweaks”, it is possible to:


    • install arbitrary applications — both from a variety of existing app stores, and arbitrary APKs from any sources,
    • choose default applications: web browser, email client, camera, file manager, launcher, call application, SMS application, contact application and so on (this works due to the beautiful system of activity and intent, which I described in the last article)
    • install and use icon packs,
    • get direct access to the file system, store arbitrary files in it,
    • connect the device to a computer (or even to another Android device) and directly transfer files between them via cable,
    • and even, in many Android distributions, customize the colors and fonts of the system theme.


    So, for ordinary tasks that a simple user may encounter, root access is not needed. At the same time, the use of root access is inevitably associated with many security problems (more about them below), so in most cases Android does not allow the user to work as root . Applications, including terminal emulators, run on behalf of their limited Unix users; and the shell, which starts when using the command adb shell, works on behalf of the shell specifically for this intended Unix user .


    However, it happens that root is available to the user:


    • First, root access is usually allowed on Android builds for emulators — QEMU-based virtual machines that come with Android Studio, which depict real devices and are usually used by developers for debugging and testing applications.
    • Secondly, root access is enabled by default on many third-party Android distributions ( pre-rooted ROMs).
    • Thirdly, when the bootloader is unlocked, root access on any Android build can be enabled directly by simply installing the executable file that implements the command suvia the bootloader.
    • And finally, fourthly, on older versions of a system containing known vulnerabilities , you can often get root access by exploiting them (usually several vulnerabilities are used in different layers and components of the system to get root access). For example, if the system has not been updated since the first half of 2016, you can use the widely known Dirty Cow vulnerability to get root access .

    Why it may be necessary? Of course, root access is useful for debugging and investigating the operation of the system. In addition, with root access, you can customize the system indefinitely, changing its themes, behavior, and many other aspects. You can forcefully replace code and application resources — for example, you can remove an advertisement from an application or unlock its paid or hidden functionality. You can install, change and delete arbitrary files, including in the system section (although this is almost certainly a bad idea).


    From a more philosophical point of view , root-access allows the user to fully control his device and his system - instead of the device and software developers controlling the user.


    Root access issues


    With great power comes great responsibility.

    With great opportunities comes a big responsibility, and not always this responsibility can be entrusted to the user and applications. In other words, a large number of security and stability issues are associated with root access.


    Let me remind you that the user almost always interacts with the device not directly, but through applications. And this means that it will use root access mainly through applications that, hopefully, will use root access in good faith for good purposes. But if root is available on the device, this, in principle, means that applications can use it for bad purposes - to harm the system, steal valuable data, infect the system with a virus, install a keylogger, etc.


    As I said in the second article of the series, in contrast to the traditional model of trusting programs in classic Unix, Android is designed for the fact that the user cannot trust third-party applications - that is why they are placed in a sandbox. Moreover, you can not trust the root-access applications , hoping that they will use it only for good. In fact, root access destroys the neatly built Android security model by removing all restrictions from applications and opening them up with the right to access everything in the system.


    On the other hand, applications cannot trust the device on which root access is connected , since on such a device, the user and other applications can interfere with his work in unforeseen ways. For example, developers of an application containing paid functionality, of course, do not want the check to make a purchase can be turned off. Many applications that work with particularly valuable data - for example, Google Pay (formerly Android Pay) - clearly refuse to work on devices with root access , considering them to be not safe enough.



    Similarly, online games like the popular Pokémon GO also refuse to work on root devices, fearing that the player will be able to easily break the rules of the game.


    Google Play and root access


    Google allows manufacturers to pre-install the Google Play Store and Google Play Services only on devices where root access is disabled, thereby making Android a more attractive platform for developers, who naturally prefer to invest resources in development for platforms that do not allow users to easily interfere with the work of their applications. And since the Play Store is the most famous and popular (both among application developers and users) an Android application store, most manufacturers prefer to pre-install it on their devices. (There are exceptions - for example, Amazon uses its own Android distribution called Fire OS for its devices - Echo, Fire TV, Fire Phone, Kindle Fire Tablet - and does not pre-install any applications from Google).


    Despite the restriction for manufacturers, Google allows users of assemblies that allow root access to independently install Google Play (usually this is done by installing ready-made packages from the Open GApps project ); at the same time, Google requires that, after installation, the user manually registers the device on a dedicated registration page .


    Android apps with Google apps installed. To be certified, a device must pass Android compatibility tests. If you are unable to add a Google Account, you can’t get it. As a result, your device is uncertified. This means that your device might not be secure.

    If you are a user, please go to your Google Services Framework Android ID below.

    Root Access Restriction


    In “normal” Linux, as in other Unix systems, to get root access (using the sudoand commands su), the user is required to enter a password (either his or the Unix root user password, depending on the system setting and the specific command) or log in another way (for example, using a fingerprint). In addition to the actual authorization, this serves as confirmation that the user trusts this program and agrees to allow it to use root access.


    The standard version of the command sufrom the Android Open Source Project does not explicitly request user confirmation, but it is available only to the Unix shell user (and root itself), which completely cuts off the ability of applications to get root-rights. Many third-party implementations suallow any application to gain superuser rights, which, as I explained above, is very bad in terms of security.


    As a compromise, many users use special programs that control access to su. When an application attempts to call, suthey request confirmation from the user, who can allow or deny the application to get root access. A few years ago, one such program, SuperSU , was popular ; Recently, it has been supplanted by a new open source project called Magisk .



    Magisk


    The main feature of Magisk is the ability to conduct a wide class of system modifications, including those related to changing files located in a folder /system, without actually changing the system partition itself (this is called systemless-ly ) by using several advanced Linux features . In conjunction with Magisk Hide- the ability not only to not allow some applications root access, but to completely hide from them the very fact of root access and installing Magisk in the system - this allows the device to still receive system updates from the manufacturer and use applications like the same Google Pay, which refuse to work on root-based systems. Magisk Hide is able to bypass even quite advanced root detection technologies like SafetyNet from Google.


    Although this is really incredibly cool , you need to realize that being able to use applications like Google Pay on root devices, despite the checks built into them, is not a solution to root-related security problems. Applications that the user has entrusted root access to can still decide to intervene in the system and steal the user's money through Google Pay. Security problems remain, we only manage to close their eyes to them.


    Magisk supports systemless installation of ready-made system tweaks in the form of Magisk Modules , special packages for installation using Magisk. In this form , for example , the well-known root framework Xposed and ViPER , a set of advanced drivers and settings for playing sound, are available.


    SoC, drivers and fragmentation


    Systems on chip


    The hardware of traditional desktop and server computers is built around the motherboard , on which the most important hard components are installed, such as the central processor and RAM, and to which additional boards — for example, graphic and network — that contain the remaining components (respectively, graphic processor and Wi-Fi adapter).


    Unlike such a scheme, most Android devices use so-called systems on a chip (SoC). A system on a chip is a set of computer components — a central processor, a block of RAM, input / output ports, a graphics processor, LTE, Bluetooth, and Wi-Fi modems, etc. - fully implemented and integrated within a single microchip. This approach not only reduces the physical size of the device so that it fits in your pocket, and improves its performance due to greater locality and better integration between the components, but also significantly reduces its power consumption and heat dissipation, which is especially important for mobile devices powered by the built-in battery. and not having active cooling systems.


    But, of course, systems on a crystal also have their drawbacks. The most obvious drawback is that such a system cannot be “upgraded” by purchasing, for example, additional RAM, and it is impossible to replace a poorly functioning component. This, in principle, is beneficial to manufacturing companies, because it encourages people to buy new devices when existing ones become obsolete or fail, instead of updating or repairing them.


    About drivers


    The other - much more important in the context of Android - the disadvantage of SoC lies in the associated difficulties with the drivers. As in traditional systems based on separate boards, each component of the system on a chip — from the camera to the LTE modem — requires the use of special drivers to work. But unlike traditional systems, these drivers are usually developed by manufacturers of systems on a chip and specific to their specific models; In addition, the source codes of such drivers are usually not disclosed , and binary assemblies are not always laid out by manufacturers for free.


    Instead, the SoC manufacturer (for example, Qualcomm) gives a ready-made driver assembly to manufacturers of Android devices (for example, Sony or LG), which include it in their Android assembly, based on code from the Android Open Source Project. So it turns out that the assembly of Android, pre-installed on the device by the manufacturer, contains all the drivers necessary for this device, and it is impossible to directly use the assembly for one device on another.


    As a result, the authors of Android assemblies have to make separate efforts to support each family of models of Android devices. Device manufacturers themselves are not always interested in allocating resources for porting their assemblies to new versions of Android. At the same time, developers of third-party Android distributions have to extract drivers from assemblies pre-installed by manufacturers, which is associated with additional difficulties and does not always lead to a good result. Many third-party distributions only have enough resources to support the most popular device models.


    Fragmentation


    This leads to a problem known as fragmentation : the Android ecosystem consists of a large number of different assemblies - both official assemblies from device manufacturers and versions of third-party distributions. Many of them are also based on older versions of Android, because for many devices, update builds from the manufacturer come out slowly or not at all.


    Of course, a slow ecosystem update means that not all users get small interface updates and other improvements that are visible to the user that new versions of Android bring. But it brings two more serious problems.


    First, security suffers . Although modern systems use advanced protection mechanisms, they constantly reveal new vulnerabilities. Typically, these vulnerabilities are promptly corrected by developers, but this does not help users if system updates containing patches never reach them.


    Secondly, fragmentation affects the developers of applications for Android - the so-called developer experience (DX, similar to user experience / UX) suffers . In theory, despite the internal differences in the drivers and user interface of different versions and assemblies of Android, used by application developers API - Android Framework, OpenGL / Vulkan and others - should be portable and work the same. In practice, of course, this is not always the case, and developers have to test and run their applications on many Android versions and assemblies - on different devices, different system versions, third-party distributions, and so on.



    Don't Stop Thinking About Tomorrow


    Not all manufacturers of Android devices consider it important to release updates for their devices in a timely manner. For example, Google releases updates for its Nexus and Pixel lines as well as the release of the new version of Android. For many other manufacturers, porting Android builds to its new version takes months, but they nonetheless try to release monthly security updates.


    In addition, not all devices that use Android, are based on SoC. Android can be easily installed on a regular “desktop” computer (for example, assemblies from Android-x86 and RemixOS projects are suitable ). A special Android build is built into ChromeOS, which allows Chromebooks to run Android apps along with Linux and web apps. A similar approach - launching a special Android assembly in a container - is followed by the Anbox project , which allows using Android applications on “ordinary” Linux systems. (Let me remind you that Android applications are so easily transferred to x86 architectures, without requiring recompilation, thanks to the use of the Java virtual machine, which I mentioned in the second article of the series.)



    Nevertheless, the poor portability of Android assemblies between different devices and the associated fragmentation of the ecosystem is a serious problem that impedes the development and promotion of Android as a platform.


    The most direct way to deal with fragmentation is to convince device manufacturers in every possible way not to abandon support for their Android builds. As practice shows, it works, but it does not work well enough. It would be much better if a technical solution could be developed to increase the portability of Android assemblies and thereby greatly facilitate the task of the authors of the assemblies and third-party distributions.


    And such a solution already exists. In 2017, Google announced Project Treble - a new, even more modular (compared to the existing HAL, hardware abstraction layer) architecture for the interaction of drivers (and other software specific to a particular device) with the rest of the system. Treble allows you to install the main system and device-specific drivers on different partitions of the file system and update — or modify as you like — the system separately from the installed drivers.


    Treble is fundamentally changing the situation with the slow release of updates and poor portability of assemblies. Thanks to Treble, devices from seven different companies (Sony, Nokia, OnePlus, Oppo, Xiaomi, Essential and Vivo - not just Google themselves) were able to participate in the Android Pie beta program. Treble let Essential release an Android Pie upgrade for their Essential Phone right on the day the Android Pie was released. The same Android build - the same binary file without recompilation or any changes - can now be run on any device that supports Treble, despite the fact that they can be based on completely different SoCs.


    Treble's influence is really hard to overestimate. Java brought the “write once, run everywhere” feature for high-level code — including the ability to run Android applications on computers with virtually any processor architecture. Treble is a similar breakthrough that allows you to use the once-written and compiled Android build on devices with completely different SoCs. Now it's up to the manufacturers who need to convert their drivers to a Treble compatible format. It is hoped that in a few years the problems with updating Android devices will disappear completely.




    In the next article, I plan to talk about the low-level userspace of Android: the init process, Zygote, Binder, services, and props.


    Also popular now: