TrustZone: hardware implementation in ARMv7A
Today we begin to explore the internal device TrustZone (this is a trademark of ARM).
The name itself is commercial, it was invented by marketers to inform the whole world about the key property of this technology. According to their idea, we should provide some kind of trusted, protected, very reliable place. For example, a house where we, having closed the doors and turned on the light, feel comfortable and safe.
Therefore, I will start with the fact that TrustZone is in no way a “place” in the processor . It can not be found on the chip, like cache or ALU. And trusted programs, in fact, are not executed in some physically allocated zone of the processor.
Even if we looked into the source code of the ARM core, we could not clearly identify TrustZone. Rather, by analogy with programs, TrustZone is a few modules and a set of patches for almost all other parts of the processor.
In this article, we will look at how TrustZone is implemented at the hardware level of ARM Cortex-A processors (ARMv7A).
In ARMv8A, it will be about the same, but in ARMv7M, everything is completely different. For the sake of marketing, there is also TrustZone, but another.
The first component of TrustZone is processor mode. It is specified by the NS (Non-Secure) bit in the SCR (Secure Configuration Register). If NS = 1, we are in Non-Secure mode, if NS = 0, we are in trusted, that is, Secure mode.
Cortex-A5 SCR register
Regardless of NS, all the usual processor modes remain in place. The most popular of them:
Thanks to NS, we have Secure User, Non-Secure User, Secure Supervisor, Non-Secure Supervisor and so on.
The NS bit affects the execution of individual processor functions, prohibits access to individual blocks and changes the behavior of part of the registers, both the processor core and peripheral devices.
Moreover, it turns out that it is impossible to take and change the value of the NS bit in any of the usual processor modes - this is prohibited. To change the NS value, a ceremonial is provided with the processor entering a separate Secure Monitor mode, which does not belong unambiguously to either Secure or Non-Secure. But we will talk about this in the next article.
It turns out that NS bifurcates the processor, creates two unequal operating modes: Secure and Non-Secure. In each mode, however, there is everything you need to run the OS and programs, just the privileges for accessing some of the functions of the CPU and peripherals are different.
We continue to remove the curtains.
A trusted program execution mode is one where NS = 0, that's it!
There is no additional pipeline of commands, ALU, a separate program memory - nothing that can be imagined by hearing the name TrustZone. There is no boundary to this zone; teams of intruders do not seek to "crawl" into the trusted zone, like viruses through a cell membrane.
In the general case, the pipeline executed the commands of an untrusted program (NS = 1), and then (bang!) An interruption occurred, the processor went into trusted mode (NS = 0) and immediately executed the trusted code.
In fact, TrustZone technology gives us the tools to take a number of measures (share the memory of trusted and untrusted programs, share access to peripherals) to create a reliable barrier between Secure and Non-Secure. But the reliability of this barrier will depend on the quality and completeness of the implementation of trusted software.
The end of the removal of the curtains.
The NS bit does not just tell the processor core what mode it should operate in. It is also an external signal connected from the processor to almost the entire periphery.
How to present it? In the general case, we imagine that peripherals are connected to the CPU by address, data, and control buses. NS is part of the control signals for those processors where TrustZone is implemented. Thus, from the CPU to the device are not just Read, Write, but Secure Read, NonSecure Read, Secure Write, NonSecure Write commands.
Cortex-A is alittle more often than always delivered as System On Chip (SoC), so all these buses are hidden from us inside the chip. However, a number of SoCs allow you to bring the NS signal out, in case you connect external peripherals that support safe mode.
Which peripherals support Secure / NonSecure access? For example, this is a GIC interrupt controller - in ARM it is a peripheral device as part of SoC. In Secure-mode, it allows you to configure the delivery of some interrupts to Secure FIQ mode and prohibit changing this software setting from NonSecure-mode.
This is what happens when the CPU works with the GIC: when the GIC register is written in Secure mode from the CPU, the signal NS = 0 goes along with the register address and data. GIC understands that the entry is trusted, and gives full access. If NS = 1, the GIC restricts access to part of the registers, both for writing and reading.
Other processor units that support the NS signal: memory controllers, real-time clock (RTC), key storage, reset controller and power management.
Note that in ARMv7A TrustZone support is optional, and when creating SoC, the Secure Extensions option (read: TrustZone) can be disabled. At the same time, unnecessary blocks and communications are removed from the chip, in particular, there is no need to trace the NS line throughout the chip. In this case, the inputs of the NS peripherals are connected to 0 (at least we can imagine it this way). The chip topology is getting easier.
What happens when an SoC contains multiple processor cores? Each core (usually the core is called a CPU in the ARM documentation) can work in Secure or Non-Secure modes. At any given time, it may turn out that some cores are Secure, while others are not.
Let's look at the internals of the modern ARM to understand how TrustZone will work in this case.
In ARM processors, all processor cores, memory and peripherals are connected by an internal bus called AMBA ( https://en.wikipedia.org/wiki/Advanced_Microcontroller_Bus_Architecture ). Beginning with approximately ARMv4, there is a switching unit in the AMBA bus, it connects units called Bus Master to various Slave devices.
Only a really tough nut will understand the details of the work of AXI and AMBA, but for the full picture you need to add AHB, APB and take into account the implementation details in different architectures. But the general idea is captured very quickly.
For example, the processor core (or rather, the D-cache and I-cache of this processor) is the Bus Master, and some I2C controller is the Slave. Bus Master starts the transaction on the bus, that is, read or write. Slave - this is the block where they write or where they read. From here, by the way, the set of masters follows: processor cores, DMA controllers and peripherals with built-in DMA (such as USB host).
We will consider the Master Slave switching unit in more detail. In ARMv7A, it is called Interconnect and is an implementation element of AXI (Advanced eXtensible Interface). In ARM926, this unit was called Bus Matrix and was part of the implementation of the AHB (Advanced High-Performance Bus) internal bus interface. In fact, this is the same.
We have M × Master and N × Slave, and there is a switching matrix connecting the first to the second. At each point in time, each Master can be connected to one Slave or disabled at all. But several Master can be simultaneously active, if connected to different devices.
In general, not all connections are possible. In particular, the system designer can eliminate unnecessary connections - for example, if there is no reason for the Ethernet controller (Master), you can write directly to the I2C controller (Slave).
In addition, some devices can be either Master or Slave. For example, USB Host, when it saves data via DMA to memory, is Master, and when we configure its registers, it is Slave.
Moreover, each Master is also a source of the NS signal, and Slave is the recipient of this signal. AXI transmits NS signals from Master to the corresponding Slave via Interconnect, and due to this, both Secure and NonSecure transactions can simultaneously occur in SoC.
Now we see how the ARM Cortex-A supports the simultaneous operation on the internal bus of several processor cores and many peripheral devices, simultaneously in Secure and Non-Secure modes. Make it a little more complicated?
When creating SoC, the developer takes blocks from ARM, blocks from third-party manufacturers and blocks of his own design, combines them into a single system.
From ARM are taken, including
All of them have TrustZone support and internally share NS access into trusted and untrusted.
For example, the cache controller knows which lines were stored in trusted mode, and which ones in untrusted mode, and will perform the corresponding AXI transactions to dump data to physical memory.
Further, many processor units are purchased from third-party (reliable and well-known) developers; they are the same even in processors of different manufacturers. This is, for example, USB host, SDHC host. The SoC developer uses other blocks in all of his processors, almost without changing. This, for example, Ethernet MAC, I2C, UART, SPI controllers.
These purchased and their units may not have TrustZone support at all. This is understandable - we cannot imagine why it is necessary to share access to UART between Secure and Non-Secure. But the question of integrating such devices into TrustZone hangs in the air.
The integration of these devices is decided by the manufacturer SoC independently. In fact, the manufacturer must solve two problems:
Let's see what this means for Bus Master using an example of a video controller that takes data from memory and transfers it directly to HDMI.
We want to provide the notorious DRM: the encrypted video stream will come from Linux to Secure OS, where it will be decrypted and displayed on the screen. The decrypted data will be located in a memory area accessible only for Secure Read / Write, reading this area from Linux (Non-Secure) will give an access error. Thus, we will not let Linux copy the decrypted stream. A video adapter with the right of Secure access will freely read the decrypted video data and display it on the screen.
In order for the video adapter to receive data from Secure-memory via AXI, it must access with NS = 0. However, if our DRMdisgusted do not need to provide video controller we privileged access not want.
In order for the controller to work this way and that way, a setting is introduced in the system: the type of access for each Bus Master that does not support TrustZone. That is, at least 1 bit per Bus Master. Perhaps this is just one register - but this is the work for the creator of SoC, his responsibility. And this, of course, is the source of incompatibility between processors from different manufacturers.
For each Slave device, it is wise to determine the following access rights when working with AXI:
This set stems from a superposition of Read / Write operations and Secure / Non-Secure modes.
In fact, how to share rights in this case is decided by the SoC manufacturer independently. For example, you can reduce the number of settings by always allowing Secure access. And you can increase it by adding a breakdown of User / Supervisor access types.
For such access control, a register can be provided for each Bus Slave with 2-4-8 bits that allow or prohibit access to the device, depending on the access mode.
And here we come to another topic: what will happen if Bus Master started access, but Bus Slave did not allow it?
If there is a restriction, then there will be a violation. If some type of access to the device is prohibited, something must happen if it is implemented.
In fact, not always. For example, in the same GIC (interrupt controller), write operations prohibited for Non-Secure are not performed (quietly and calmly), and read operations return zeros. Nothing happens, and it’s specially conceived - it allows you to run the same OS (for example, Linux) in both Secure- and Non-Secure-modes.
In Secure mode, Linux will configure everything on its own, in Non-Secure - the controller will be pre-configured, and Linux will be able to configure only what is left to it. But she will not blink an eye, she will not notice a catch, because the GIC will not give out any errors when writing to a forbidden area.
What if we use less sophisticated smart devices? Then, for example, when Non-Secure writes to the Secure memory region, Abort will happen. Abort is an ARM exception type that occurs when it is not possible to access some device or memory area.
Most often, Asynchronous Data Abort, or in Russian, asynchronous abortion, will occur.You should not discuss this at lunch.
Data Abort - because it occurred while reading / writing data, not processor instructions. It is asynchronous because it does not occur immediately at the time of the error, but some time after it. And from this place will be even more detailed.
In general, when access is violated, both synchronous and asynchronous abortions can occur.
For example, when Linux downloads an application, it may not load the whole application, placing only part of the pages in physical memory, and configure the rest to generate Abort at the time of access. The application will start, and when it comes to the page that is not loaded into the physical memory, a synchronous abortion will occur. It is synchronous because it will happen exactly on the instruction that made the memory access. When the processor enters Abort mode, Linux will load the desired memory page and return control to the same instruction that called Abort. Result - the program will continue to work "as it never happened."
But in the case of TrustZone, everything is not so smooth. Some processors will throw synchronous exceptions, but most will throw asynchronous Abort for most access errors.
We will answer two questions:
To begin with, ARMv7A is an architecture with a command pipeline, where instructions are pre-broken by the processor and executed not strictly sequentially. Part of the instructions may be executed in parallel with others. For example:
Here, the first command stores r1 at the address r2, and the second increases r2. After the first command is executed, in general, saving to memory will only begin , and perhaps not yet, when the second instruction is fully executed.
Further, the processor has a cache in which the recorded cell will be stuck for an indefinite amount of time, and an access error will potentially occur only when the cache is synchronized with the memory.
Then, even if the memory area is not cached: the memory in ARM is divided into Normal, Strongly Ordered and Device Memory, allowing different liberties on the part of the processor to change the order of real accesses to memory and devices via AXI. As a result, a transaction through AXI may not occur immediately due to the fact that access to the device is busy with another call.
And finally, if Abort caused access to the usual Bus Slave, then it will be a logical signal external to the processor core. The kernel does not expect that this signal is synchronized with what is happening now in the command pipeline, and this is absolutely true: the kernel cannot even 100% determine the cause of such an abortion.
Under any of these circumstances, ARM will generate an Asynchronous Abort, telling us that there was an attempt to block access, but how many clock cycles or instructions ago it does not know.
Yes, because we cannot determine the point of failure and cannot fix anything. After an erroneous access, a program can execute more than a dozen cycles and during this time it can be removed so far from its proper functioning that it only remains to be stopped and restarted. Perhaps with a complete reset of the processor, if any peripherals or internal structures of the OS suffer from the work of the program after Abort.
When working with TrustZone, at first it is tempting to use this technology as a hardware virtualization technology. But due to Asynchronous Abort, this will not work.
Indeed, there are two modes: Secure and Non-Secure. Secure mode can create an analogue of a sandbox for Non-Secure, restrict access to peripherals.
However, the next step will be the virtualization of part of the periphery, for example, Flash-memory, which both the guest OS and the hypervisor work with. And here we come across the fact that it is impossible to just take and close access to the device for the guest OS.
How I would like:
And here's how it goes:
Conclusion: TrustZone technology alone cannot be used for hardware virtualization.
You can force the guest OS to knock on Secure OS to access prohibited devices, and this is the main way to split devices between Secure OS and the guest OS. But we will talk about this next time.
And what about access to regular memory? Is it possible to allocate part of system DDRAM for Secure access?
ARM took care of this less than you might expect!
Memory controllers are different, for example,
All of these controllers are typical Bus Slave. ARM does not develop them, so the separation of access to Secure / Non-Secure falls on the shoulders of the SoC developer, according to the above scheme.
The most basic option is almost always - access to the built-in SRAM is configured as Secure, and to DDR - as Non-Secure.
This is a fairly safe method, because all Secure data is stored inside the chip, do not leave its perimeter. But the built-in SRAM is pathetic tens or hundreds of kilobytes, and this may not be enough for a full-fledged Secure OS and protected data.
A more flexible way appears if the SoC manufacturer, at its discretion, implemented a DDR controller with support for memory zoning according to the NS = 0/1 criterion. In fact, there may be many options for implementation, but this does not change the essence.
In general, such a memory offers a minimum of the following:
Fortunately, manufacturers do include such controllers in their SoCs.
It is a pity that ARM did not take care of this, and we have a variety of solutions.
This implementation has a minus: since the usual program and data memory in ARM is cached, and the memory controller is a regular Bus Slave, we can not immediately find out that there was a write to a forbidden address. Asynchronous Abort will happen, and we will only have to clean up the fragments of the program.
In this article, we reviewed the hardware implementation of TrustZone in ARMv7A and dispelled some of the misconceptions associated with this technology.
Considered:
We can say that we figured out under the hood, but the ignition has not yet been turned on. In the next article, we will launch the processor, consider its operation in the Secure, Non-Secure modes and switch between them through the Secure Monitor mode.
The name itself is commercial, it was invented by marketers to inform the whole world about the key property of this technology. According to their idea, we should provide some kind of trusted, protected, very reliable place. For example, a house where we, having closed the doors and turned on the light, feel comfortable and safe.
Therefore, I will start with the fact that TrustZone is in no way a “place” in the processor . It can not be found on the chip, like cache or ALU. And trusted programs, in fact, are not executed in some physically allocated zone of the processor.
Even if we looked into the source code of the ARM core, we could not clearly identify TrustZone. Rather, by analogy with programs, TrustZone is a few modules and a set of patches for almost all other parts of the processor.
In this article, we will look at how TrustZone is implemented at the hardware level of ARM Cortex-A processors (ARMv7A).
In ARMv8A, it will be about the same, but in ARMv7M, everything is completely different. For the sake of marketing, there is also TrustZone, but another.
Mode
The first component of TrustZone is processor mode. It is specified by the NS (Non-Secure) bit in the SCR (Secure Configuration Register). If NS = 1, we are in Non-Secure mode, if NS = 0, we are in trusted, that is, Secure mode.
Cortex-A5 SCR register
Regardless of NS, all the usual processor modes remain in place. The most popular of them:
- User - application command execution mode;
- Supervisor - operating mode of the OS kernel;
- IRQ - mode when processing interrupts.
Thanks to NS, we have Secure User, Non-Secure User, Secure Supervisor, Non-Secure Supervisor and so on.
Hidden text
The User / Supervisor names given here are used for all 32-bit ARMs, up to and including ARMv7. ARMv8 uses other designations: EL0 / EL1 and PL0 / PL1. This does not change the essence.
The NS bit affects the execution of individual processor functions, prohibits access to individual blocks and changes the behavior of part of the registers, both the processor core and peripheral devices.
Moreover, it turns out that it is impossible to take and change the value of the NS bit in any of the usual processor modes - this is prohibited. To change the NS value, a ceremonial is provided with the processor entering a separate Secure Monitor mode, which does not belong unambiguously to either Secure or Non-Secure. But we will talk about this in the next article.
It turns out that NS bifurcates the processor, creates two unequal operating modes: Secure and Non-Secure. In each mode, however, there is everything you need to run the OS and programs, just the privileges for accessing some of the functions of the CPU and peripherals are different.
Mode, not a zone!
We continue to remove the curtains.
A trusted program execution mode is one where NS = 0, that's it!
There is no additional pipeline of commands, ALU, a separate program memory - nothing that can be imagined by hearing the name TrustZone. There is no boundary to this zone; teams of intruders do not seek to "crawl" into the trusted zone, like viruses through a cell membrane.
In the general case, the pipeline executed the commands of an untrusted program (NS = 1), and then (bang!) An interruption occurred, the processor went into trusted mode (NS = 0) and immediately executed the trusted code.
In fact, TrustZone technology gives us the tools to take a number of measures (share the memory of trusted and untrusted programs, share access to peripherals) to create a reliable barrier between Secure and Non-Secure. But the reliability of this barrier will depend on the quality and completeness of the implementation of trusted software.
The end of the removal of the curtains.
NS signal
The NS bit does not just tell the processor core what mode it should operate in. It is also an external signal connected from the processor to almost the entire periphery.
How to present it? In the general case, we imagine that peripherals are connected to the CPU by address, data, and control buses. NS is part of the control signals for those processors where TrustZone is implemented. Thus, from the CPU to the device are not just Read, Write, but Secure Read, NonSecure Read, Secure Write, NonSecure Write commands.
Cortex-A is a
Which peripherals support Secure / NonSecure access? For example, this is a GIC interrupt controller - in ARM it is a peripheral device as part of SoC. In Secure-mode, it allows you to configure the delivery of some interrupts to Secure FIQ mode and prohibit changing this software setting from NonSecure-mode.
This is what happens when the CPU works with the GIC: when the GIC register is written in Secure mode from the CPU, the signal NS = 0 goes along with the register address and data. GIC understands that the entry is trusted, and gives full access. If NS = 1, the GIC restricts access to part of the registers, both for writing and reading.
Other processor units that support the NS signal: memory controllers, real-time clock (RTC), key storage, reset controller and power management.
Note that in ARMv7A TrustZone support is optional, and when creating SoC, the Secure Extensions option (read: TrustZone) can be disabled. At the same time, unnecessary blocks and communications are removed from the chip, in particular, there is no need to trace the NS line throughout the chip. In this case, the inputs of the NS peripherals are connected to 0 (at least we can imagine it this way). The chip topology is getting easier.
Multiprocessing
What happens when an SoC contains multiple processor cores? Each core (usually the core is called a CPU in the ARM documentation) can work in Secure or Non-Secure modes. At any given time, it may turn out that some cores are Secure, while others are not.
Hidden text
And yet, kernels that do not support TrustZone can be combined with kernels that support it in one SoC.
Let's look at the internals of the modern ARM to understand how TrustZone will work in this case.
In ARM processors, all processor cores, memory and peripherals are connected by an internal bus called AMBA ( https://en.wikipedia.org/wiki/Advanced_Microcontroller_Bus_Architecture ). Beginning with approximately ARMv4, there is a switching unit in the AMBA bus, it connects units called Bus Master to various Slave devices.
Only a really tough nut will understand the details of the work of AXI and AMBA, but for the full picture you need to add AHB, APB and take into account the implementation details in different architectures. But the general idea is captured very quickly.
For example, the processor core (or rather, the D-cache and I-cache of this processor) is the Bus Master, and some I2C controller is the Slave. Bus Master starts the transaction on the bus, that is, read or write. Slave - this is the block where they write or where they read. From here, by the way, the set of masters follows: processor cores, DMA controllers and peripherals with built-in DMA (such as USB host).
We will consider the Master Slave switching unit in more detail. In ARMv7A, it is called Interconnect and is an implementation element of AXI (Advanced eXtensible Interface). In ARM926, this unit was called Bus Matrix and was part of the implementation of the AHB (Advanced High-Performance Bus) internal bus interface. In fact, this is the same.
We have M × Master and N × Slave, and there is a switching matrix connecting the first to the second. At each point in time, each Master can be connected to one Slave or disabled at all. But several Master can be simultaneously active, if connected to different devices.
In general, not all connections are possible. In particular, the system designer can eliminate unnecessary connections - for example, if there is no reason for the Ethernet controller (Master), you can write directly to the I2C controller (Slave).
In addition, some devices can be either Master or Slave. For example, USB Host, when it saves data via DMA to memory, is Master, and when we configure its registers, it is Slave.
Moreover, each Master is also a source of the NS signal, and Slave is the recipient of this signal. AXI transmits NS signals from Master to the corresponding Slave via Interconnect, and due to this, both Secure and NonSecure transactions can simultaneously occur in SoC.
Periphery
Now we see how the ARM Cortex-A supports the simultaneous operation on the internal bus of several processor cores and many peripheral devices, simultaneously in Secure and Non-Secure modes. Make it a little more complicated?
When creating SoC, the developer takes blocks from ARM, blocks from third-party manufacturers and blocks of his own design, combines them into a single system.
From ARM are taken, including
- processor cores, for example, Cortex-A, Cortex-M4, or the entire multiprocessor system, for example, Cortex-A9 MPCore;
- GIC interrupt controller, for example, PL390;
- cache controller, for example, L2C-310.
All of them have TrustZone support and internally share NS access into trusted and untrusted.
For example, the cache controller knows which lines were stored in trusted mode, and which ones in untrusted mode, and will perform the corresponding AXI transactions to dump data to physical memory.
Further, many processor units are purchased from third-party (reliable and well-known) developers; they are the same even in processors of different manufacturers. This is, for example, USB host, SDHC host. The SoC developer uses other blocks in all of his processors, almost without changing. This, for example, Ethernet MAC, I2C, UART, SPI controllers.
These purchased and their units may not have TrustZone support at all. This is understandable - we cannot imagine why it is necessary to share access to UART between Secure and Non-Secure. But the question of integrating such devices into TrustZone hangs in the air.
The integration of these devices is decided by the manufacturer SoC independently. In fact, the manufacturer must solve two problems:
- for Bus Master without TrustZone support, substitute the correct NS-bit;
- for Bus Slave, configure and verify permissions.
Access Bus Master without TrustZone support
Let's see what this means for Bus Master using an example of a video controller that takes data from memory and transfers it directly to HDMI.
We want to provide the notorious DRM: the encrypted video stream will come from Linux to Secure OS, where it will be decrypted and displayed on the screen. The decrypted data will be located in a memory area accessible only for Secure Read / Write, reading this area from Linux (Non-Secure) will give an access error. Thus, we will not let Linux copy the decrypted stream. A video adapter with the right of Secure access will freely read the decrypted video data and display it on the screen.
In order for the video adapter to receive data from Secure-memory via AXI, it must access with NS = 0. However, if our DRM
In order for the controller to work this way and that way, a setting is introduced in the system: the type of access for each Bus Master that does not support TrustZone. That is, at least 1 bit per Bus Master. Perhaps this is just one register - but this is the work for the creator of SoC, his responsibility. And this, of course, is the source of incompatibility between processors from different manufacturers.
Access Bus Slave without TrustZone support
For each Slave device, it is wise to determine the following access rights when working with AXI:
- whether Secure Read access is allowed;
- whether Secure Write access is allowed;
- whether access is allowed Non-Secure Read;
- whether Non-Secure Write access is allowed.
This set stems from a superposition of Read / Write operations and Secure / Non-Secure modes.
In fact, how to share rights in this case is decided by the SoC manufacturer independently. For example, you can reduce the number of settings by always allowing Secure access. And you can increase it by adding a breakdown of User / Supervisor access types.
For such access control, a register can be provided for each Bus Slave with 2-4-8 bits that allow or prohibit access to the device, depending on the access mode.
And here we come to another topic: what will happen if Bus Master started access, but Bus Slave did not allow it?
access error
If there is a restriction, then there will be a violation. If some type of access to the device is prohibited, something must happen if it is implemented.
In fact, not always. For example, in the same GIC (interrupt controller), write operations prohibited for Non-Secure are not performed (quietly and calmly), and read operations return zeros. Nothing happens, and it’s specially conceived - it allows you to run the same OS (for example, Linux) in both Secure- and Non-Secure-modes.
In Secure mode, Linux will configure everything on its own, in Non-Secure - the controller will be pre-configured, and Linux will be able to configure only what is left to it. But she will not blink an eye, she will not notice a catch, because the GIC will not give out any errors when writing to a forbidden area.
What if we use less sophisticated smart devices? Then, for example, when Non-Secure writes to the Secure memory region, Abort will happen. Abort is an ARM exception type that occurs when it is not possible to access some device or memory area.
Most often, Asynchronous Data Abort, or in Russian, asynchronous abortion, will occur.
Data Abort - because it occurred while reading / writing data, not processor instructions. It is asynchronous because it does not occur immediately at the time of the error, but some time after it. And from this place will be even more detailed.
In general, when access is violated, both synchronous and asynchronous abortions can occur.
For example, when Linux downloads an application, it may not load the whole application, placing only part of the pages in physical memory, and configure the rest to generate Abort at the time of access. The application will start, and when it comes to the page that is not loaded into the physical memory, a synchronous abortion will occur. It is synchronous because it will happen exactly on the instruction that made the memory access. When the processor enters Abort mode, Linux will load the desired memory page and return control to the same instruction that called Abort. Result - the program will continue to work "as it never happened."
But in the case of TrustZone, everything is not so smooth. Some processors will throw synchronous exceptions, but most will throw asynchronous Abort for most access errors.
Hidden text
In principle, ARMv7A processors with both Security Extensions and Virtualization Extensions can be configured to generate synchronous Abort. This is, for example, Cortex-A17, but the bulk of ARMv7A (by the number of chips produced) does not have virtualization.
We will answer two questions:
- Why does asynchronous abortion occur?
- Why is this bad?
Why is asynchronous?
To begin with, ARMv7A is an architecture with a command pipeline, where instructions are pre-broken by the processor and executed not strictly sequentially. Part of the instructions may be executed in parallel with others. For example:
STR r1, [r2] // * r2 = r1; ADD r2, r2, # 16 // r2 = r2 + 16;
Here, the first command stores r1 at the address r2, and the second increases r2. After the first command is executed, in general, saving to memory will only begin , and perhaps not yet, when the second instruction is fully executed.
Further, the processor has a cache in which the recorded cell will be stuck for an indefinite amount of time, and an access error will potentially occur only when the cache is synchronized with the memory.
Then, even if the memory area is not cached: the memory in ARM is divided into Normal, Strongly Ordered and Device Memory, allowing different liberties on the part of the processor to change the order of real accesses to memory and devices via AXI. As a result, a transaction through AXI may not occur immediately due to the fact that access to the device is busy with another call.
And finally, if Abort caused access to the usual Bus Slave, then it will be a logical signal external to the processor core. The kernel does not expect that this signal is synchronized with what is happening now in the command pipeline, and this is absolutely true: the kernel cannot even 100% determine the cause of such an abortion.
Under any of these circumstances, ARM will generate an Asynchronous Abort, telling us that there was an attempt to block access, but how many clock cycles or instructions ago it does not know.
Why is Asynchronous Abort Bad?
Yes, because we cannot determine the point of failure and cannot fix anything. After an erroneous access, a program can execute more than a dozen cycles and during this time it can be removed so far from its proper functioning that it only remains to be stopped and restarted. Perhaps with a complete reset of the processor, if any peripherals or internal structures of the OS suffer from the work of the program after Abort.
... and which of this can be concluded
When working with TrustZone, at first it is tempting to use this technology as a hardware virtualization technology. But due to Asynchronous Abort, this will not work.
Indeed, there are two modes: Secure and Non-Secure. Secure mode can create an analogue of a sandbox for Non-Secure, restrict access to peripherals.
However, the next step will be the virtualization of part of the periphery, for example, Flash-memory, which both the guest OS and the hypervisor work with. And here we come across the fact that it is impossible to just take and close access to the device for the guest OS.
How I would like:
- the guest OS accesses the device, Abort (synchronous) occurs;
- The hypervisor understands what happened.
- The hypervisor emulates the expected operation of the device by the guest OS;
- the hypervisor returns control to the guest OS, it continues to work, as if nothing had happened.
And here's how it goes:
- the guest OS is accessing the device, conditions are being created for Asynchronous abortion;
- the guest OS continues to work without suspecting it;
- all of a sudden Abort is generated by the system;
- The hypervisor understands that Abort is asynchronous, and it cannot calculate what instruction caused it, at what address and to which device access was;
- The hypervisor stops the guest OS.
Conclusion: TrustZone technology alone cannot be used for hardware virtualization.
You can force the guest OS to knock on Secure OS to access prohibited devices, and this is the main way to split devices between Secure OS and the guest OS. But we will talk about this next time.
And memory, memory?
And what about access to regular memory? Is it possible to allocate part of system DDRAM for Secure access?
ARM took care of this less than you might expect!
Memory controllers are different, for example,
- a static memory controller, SRAM, often an internal SoC memory;
- dynamic memory controller, for example, DDR3;
- universal controller for access to parallel memory, can be used for SRAM, NOR Flash.
All of these controllers are typical Bus Slave. ARM does not develop them, so the separation of access to Secure / Non-Secure falls on the shoulders of the SoC developer, according to the above scheme.
The most basic option is almost always - access to the built-in SRAM is configured as Secure, and to DDR - as Non-Secure.
This is a fairly safe method, because all Secure data is stored inside the chip, do not leave its perimeter. But the built-in SRAM is pathetic tens or hundreds of kilobytes, and this may not be enough for a full-fledged Secure OS and protected data.
A more flexible way appears if the SoC manufacturer, at its discretion, implemented a DDR controller with support for memory zoning according to the NS = 0/1 criterion. In fact, there may be many options for implementation, but this does not change the essence.
In general, such a memory offers a minimum of the following:
- There are zones with different access rights, a number from 3.
- One zone can be configured as Non-Secure, Linux or another guest OS will work there. This is the largest part of the memory.
- The other zone can be configured as Secure, there will be Secure OS data. This area is much smaller.
- The third zone is configurable with access both Secure and Non-Secure. It is used to exchange large amounts of data between Linux and Secure OS, it is only a few MB.
- More flexible settings allow you to make areas Secure Write / Non-Secure Read and, conversely, for unidirectional data exchange.
Fortunately, manufacturers do include such controllers in their SoCs.
It is a pity that ARM did not take care of this, and we have a variety of solutions.
This implementation has a minus: since the usual program and data memory in ARM is cached, and the memory controller is a regular Bus Slave, we can not immediately find out that there was a write to a forbidden address. Asynchronous Abort will happen, and we will only have to clean up the fragments of the program.
Conclusion
In this article, we reviewed the hardware implementation of TrustZone in ARMv7A and dispelled some of the misconceptions associated with this technology.
Considered:
- Secure and Non-Secure modes
- operation of one or several cores;
- work with peripherals through AXI;
- work with peripherals developed without TrustZone support;
- types of access errors that occur;
- differentiation of access to physical memory.
We can say that we figured out under the hood, but the ignition has not yet been turned on. In the next article, we will launch the processor, consider its operation in the Secure, Non-Secure modes and switch between them through the Secure Monitor mode.
