February 7, 2015 at 13:27OpenOCD, ThreadX and your processor
This note may be useful for people who write bare-metal code and use ThreadX in their tasks (of their own choice or by imposing an SDK). The problem is that in order to effectively debug code under ThreadX or another multi-threaded operating system, you need to be able to see these very threads, be able to see the stack trace, the state of the registers for each thread.
OpenOCD ( Open On Chip Debugger ) claims support for ThreadX , but does not explicitly specify its breadth. And normally, at the time of writing, in version 0.8.0, there are only two cores: Cortex M3 and Cortex R4. I, by the will of fate, had to work with the Cypress FX3 chip which is based on the ARM926E-JS core.
Under the cat, consider what needs to be done to add support for your version of ThreadX for your CPU. The emphasis is on ARM, but, theoretically, it may well be suitable for other processors. In addition, we consider the case where access to the ThreadX source is not and is not expected.
From the first lines, I’ll disappoint right away: without assembler, nowhere. No, we won’t have to write on it, but yes, to read the code.
Let's start by introducing ThreadX support in OpenOCD. This is just one file: src / rtos / ThreadX.c .
The supported system is described by the ThreadX_params structure , which contains information about the name of the target, the "width" of the pointer in bytes, a set of offsets in the TX_THREAD structureto the necessary service fields, as well as information about how the context of the stream is preserved during the switch (the so-called stacking info). Supported systems themselves are registered using the ThreadX_params_list array .
There are no problems with all parameters except the last one: the width of the pointer is usually equal to the processor capacity, the offsets are considered handles (and even then, they are almost always the same).
An interesting question: where to get information on stacking? But there is a lot of information:
This last one is the most complex and incomprehensible. I can immediately convince you - there is no standard approach here. It is extremely difficult, if not impossible, to pick these values.
Moreover, looking ahead, as it turned out for Cortex M3 / R4 cores, one stacking scheme is used, and for ARM926E-JS - two! All for the sake of economy.
Briefly (and also very rude and inaccurate) how the sheduler works in ThreadX: it simultaneously provides a cooperative and crowding out approach to multitasking.
The cooperative approach works for threads of the same priority that do not have a time slice (0). Those. if stream A and B have the same priority, stream A has started, then stream B will not receive control until A:
If a time slice is set, then upon completion, the flow will be interrupted and control will be transferred to the next one in the Ready state (for the case when the flow falls asleep, but has not developed its own slice, the cooperative approach will also work). A preemptive approach is already working here. For its operation, a timer is needed and interrupts from it with a certain periodicity. Also, stream A from the example above can be replaced by stream B if its priority is higher.
It is clear that the context of the stream is preserved when it transfers control to someone and is restored when it receives control. We will understand how this happens - we will understand what needs to be described in an array of register offsets.
I won’t go into details about how I found out where and how the main parts of the scheduler were hidden, a lot came up here: savvy, and luck, and Google, and a disassembler. But I will give the main components thereof:
I studied the listings of all these functions, but if you again need to screw support for an unsupported processor, I will focus on the last three. But I will start with the latter, and only after that (if there is not enough information) will I study others.
Let's look at its listing (I replaced some indirect addressing with real characters, the characters themselves are
looked in the elf file using arm-none-eabi-nm):
The function is crazy to simple:
But starting with 40004cb8, there is code that, in fact, restores the context of the new stream.
First, two values are read into the registers r0 , r1 :
Next is a comparison of r0 with zero:
Obviously, these values, at least r0 , are part of the context (after all, the stack register is already configured on the stack of the restored thread), but it does not quite look like these are registers. A comparison with zero implies some kind of branching. Continuing the analysis, we see that if r0! = 0 , then the code is executed:
Actually, this is similar to restoring the context. Moreover, the value of register r1 is the saved value of the status register CPSR. If the line 40004ccc is executed, then control will not go further: pc ( r15 ) register will be restored and the program after this point will return to the place from where it was interrupted.
Ok, now we can write such a tablet:
Each register and each flag is 32 bits or 4 bytes, respectively 17 * 4 = 68 bytes are needed for this context. It is logical that the stack goes on, as it was at the time of the interruption.
But, as we see, this is part of the work. We have this very flag. And if its value is 0, then the code is executed:
Apparently, this is also a context, only slightly reduced. Moreover, returning from it occurs as from a normal function, and not by recovering the pc register . Rewriting the plate above, we get:
For this context, only 11 * 4 = 44 bytes are needed.
Using Google, viewing disassembler listings, and also studying the conventions for calling procedures, we come to understand that this type of context is used when cooperative multitasking works: i.e. when we called tx_thread_sleep () or others like them. And since such a switch, in fact, is just a function call, then the context can be saved according to the calling conventions, according to which, we have the right not to save the values of the registers r0-r3, r12 between calls . Moreover, we do not need to save the pc - all the necessary information is already contained in the rl - return address from tx_thread_sleep (). Benefit on the face. Cortexes are usually used on systems with more memory than ARM9E, they do not resort to such tricks and use one type of stacking.
According to information from the Internet, I found out that the first type of context is called interrupt, and is used when the stream is interrupted by interruption, in other words, it can be interrupted anywhere, so you need to save all possible registers. The second type of context is called solicited and is used when the thread is interrupted by a system call, which leads to resheduling.
Here, in fact, everything is ready to understand what alterations are needed in OpenOCD:
I will not give the code for the first paragraph, look at the patch. For point two, I’ll explain a little how to compose a displacement label for OpenOCD-friendly ones.
First of all, we look at the output of the 'info registers' command, we look at how many registers and in what order is displayed, we make such a fish:
Here 32 is the bit size of the register. For ARM, it is always 32. The first column is filled with the plates that we wrote above when analyzing the restoration of context. We take into account special values: -1 - this register is not saved, -2 - stack register, is restored from the stream structure.
A filled fish for a solicited context is as follows:
For interrupt context, try to write yourself or look at the source.
What will it give:
Commands given for gdb.
All in all, happy debugging!
Resources:
PS there is not enough “Reverse development” hub and highlighting for different assemblers ;-)
UPD / 2015-08-15 /: Changes are in the main OpenOCD branch: openocd.zylin.com/#/c/2848
OpenOCD ( Open On Chip Debugger ) claims support for ThreadX , but does not explicitly specify its breadth. And normally, at the time of writing, in version 0.8.0, there are only two cores: Cortex M3 and Cortex R4. I, by the will of fate, had to work with the Cypress FX3 chip which is based on the ARM926E-JS core.
Under the cat, consider what needs to be done to add support for your version of ThreadX for your CPU. The emphasis is on ARM, but, theoretically, it may well be suitable for other processors. In addition, we consider the case where access to the ThreadX source is not and is not expected.
From the first lines, I’ll disappoint right away: without assembler, nowhere. No, we won’t have to write on it, but yes, to read the code.
Let's start by introducing ThreadX support in OpenOCD. This is just one file: src / rtos / ThreadX.c .
The supported system is described by the ThreadX_params structure , which contains information about the name of the target, the "width" of the pointer in bytes, a set of offsets in the TX_THREAD structureto the necessary service fields, as well as information about how the context of the stream is preserved during the switch (the so-called stacking info). Supported systems themselves are registered using the ThreadX_params_list array .
There are no problems with all parameters except the last one: the width of the pointer is usually equal to the processor capacity, the offsets are considered handles (and even then, they are almost always the same).
An interesting question: where to get information on stacking? But there is a lot of information:
- stack growth direction (well, it's easy)
- the number of registers in the system (this is also simple, run “info registers” on the existing OpenOCD version and count the number of lines).
- frame alignment on the stack, I got this value by chance, for Cortex M3 / R4 it is specified 8 bytes, for ARM926E-JS - 0 (i.e. without alignment). In fact, the alignment is 4, but the memory allocated with tx_byte_alloc () is already aligned, and the stack usage is always a multiple of 4. In general, try the values 0, 4, and so on.
- an array of offsets in the stack (relative to the current vertex) along which the values of specific registers lie (the size of the array is equal to the number of registers in the output of “info registers").
This last one is the most complex and incomprehensible. I can immediately convince you - there is no standard approach here. It is extremely difficult, if not impossible, to pick these values.
Moreover, looking ahead, as it turned out for Cortex M3 / R4 cores, one stacking scheme is used, and for ARM926E-JS - two! All for the sake of economy.
Briefly (and also very rude and inaccurate) how the sheduler works in ThreadX: it simultaneously provides a cooperative and crowding out approach to multitasking.
The cooperative approach works for threads of the same priority that do not have a time slice (0). Those. if stream A and B have the same priority, stream A has started, then stream B will not receive control until A:
- won't end
- will not call a function that leads to resheduling (sleep, waiting in line, mutex, semaphore, etc.)
If a time slice is set, then upon completion, the flow will be interrupted and control will be transferred to the next one in the Ready state (for the case when the flow falls asleep, but has not developed its own slice, the cooperative approach will also work). A preemptive approach is already working here. For its operation, a timer is needed and interrupts from it with a certain periodicity. Also, stream A from the example above can be replaced by stream B if its priority is higher.
It is clear that the context of the stream is preserved when it transfers control to someone and is restored when it receives control. We will understand how this happens - we will understand what needs to be described in an array of register offsets.
I won’t go into details about how I found out where and how the main parts of the scheduler were hidden, a lot came up here: savvy, and luck, and Google, and a disassembler. But I will give the main components thereof:
- _tx_timer_interrupt () - the function is called from the context of the timer interrupt, in fact it is responsible for the crowding out part of the scheduler.
- _tx_thread_context_save () (or _tx_thread_vectored_context_save () ) and _tx_thread_context_restore () are a pair of functions designed to be called from interrupts to save and restore the context. When restoring the context, an attempt is made to re-seduling.
- _tx_thread_system_return () is part of a cooperative approach. Called at the end of any chain of calls that results in resolving.
- and finally, _tx_thread_schedule () is the most important function for analysis and perhaps the simplest of all the above.
I studied the listings of all these functions, but if you again need to screw support for an unsupported processor, I will focus on the last three. But I will start with the latter, and only after that (if there is not enough information) will I study others.
Let's look at its listing (I replaced some indirect addressing with real characters, the characters themselves are
looked in the elf file using arm-none-eabi-nm):
40004c7c <_tx_thread_schedule>:
40004c7c: e10f2000 mrs r2, CPSR
40004c80: e3c20080 bic r0, r2, #128 ; 0x80
40004c84: e12ff000 msr CPSR_fsxc, r0
40004c88: e59f104c ldr r1, [pc, #76] ; 40004cdc <_tx_thread_schedule+0x60>
40004c8c: e5910000 ldr r0, [r1]
40004c90: e3500000 cmp r0, #0
40004c94: 0afffffc beq 40004c8c <_tx_thread_schedule+0x10>
40004c98: e12ff002 msr CPSR_fsxc, r2
40004c9c: e59f103c ldr r1, [pc, #60] ; 40004ce0 <_tx_thread_schedule+0x64>
40004ca0: e5810000 str r0, [r1]
40004ca4: e5902004 ldr r2, [r0, #4]
40004ca8: e5903018 ldr r3, [r0, #24]
40004cac: e2822001 add r2, r2, #1
40004cb0: e5802004 str r2, [r0, #4]
40004cb4: e59f2028 ldr r2, [pc, #40] ; 40004ce4 <_tx_thread_schedule+0x68>
40004cb8: e590d008 ldr sp, [r0, #8]
40004cbc: e5823000 str r3, [r2]
40004cc0: e8bd0003 pop {r0, r1}
40004cc4: e3500000 cmp r0, #0
40004cc8: 116ff001 msrne SPSR_fsxc, r1
40004ccc: 18fddfff ldmne sp!, {r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, sl, fp, ip, lr, pc}^
40004cd0: e8bd4ff0 pop {r4, r5, r6, r7, r8, r9, sl, fp, lr}
40004cd4: e12ff001 msr CPSR_fsxc, r1
40004cd8: e12fff1e bx lr
40004cdc: 4004b754 .word 0x4004b754 ; _tx_thread_execute_ptr
40004ce0: 4004b750 .word 0x4004b750 ; _tx_thread_current_ptr
40004ce4: 4004b778 .word 0x4004b778 ; _tx_timer_time_slice
The function is crazy to simple:
- enable interrupts (lines 40004c7c-40004c84)
- wait for someone to cock _tx_thread_execute_ptr (40004c88-40004c94) - the next thread to execute
- prohibit interruptions, or rather, restore the status register (40004c98)
- save _tx_thread_current_ptr pointer to r0 (40004c9c-40004ca0)
- increase the value of tx_thread_run_count of the current thread by 1 (40004ca4, 40004cac-40004cb0)
- get the value tx_thread_time_slic e of the current thread and assign it _tx_timer_time_slice (40004ca8, 40004cb4, 40004cbc)
- set a new pointer to the stack stored in the thread structure (read tx_thread_stack_ptr ) (40004cb8)
But starting with 40004cb8, there is code that, in fact, restores the context of the new stream.
First, two values are read into the registers r0 , r1 :
40004cc0: e8bd0003 pop {r0, r1}
Next is a comparison of r0 with zero:
40004cc4: e3500000 cmp r0, #0
Obviously, these values, at least r0 , are part of the context (after all, the stack register is already configured on the stack of the restored thread), but it does not quite look like these are registers. A comparison with zero implies some kind of branching. Continuing the analysis, we see that if r0! = 0 , then the code is executed:
40004cc8: 116ff001 msrne SPSR_fsxc, r1
40004ccc: 18fddfff ldmne sp!, {r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, sl, fp, ip, lr, pc}^
Actually, this is similar to restoring the context. Moreover, the value of register r1 is the saved value of the status register CPSR. If the line 40004ccc is executed, then control will not go further: pc ( r15 ) register will be restored and the program after this point will return to the place from where it was interrupted.
Ok, now we can write such a tablet:
Offset Register -------- ------- 0 flag 4 CPSR 8 r0 12 r1 16 r2 20 r3 24 r4 28 r5 32 r6 36 r7 40 r8 44 r9 48 sl (r10) 52 fp (r11) 56 ip (r12) 60 lr (r14) 64 pc (r15)
Each register and each flag is 32 bits or 4 bytes, respectively 17 * 4 = 68 bytes are needed for this context. It is logical that the stack goes on, as it was at the time of the interruption.
But, as we see, this is part of the work. We have this very flag. And if its value is 0, then the code is executed:
40004cd0: e8bd4ff0 pop {r4, r5, r6, r7, r8, r9, sl, fp, lr}
40004cd4: e12ff001 msr CPSR_fsxc, r1
40004cd8: e12fff1e bx lr
Apparently, this is also a context, only slightly reduced. Moreover, returning from it occurs as from a normal function, and not by recovering the pc register . Rewriting the plate above, we get:
Offset Register -------- ------- 0 flag 4 CPSR 8 r4 12 r5 16 r6 20 r7 24 r8 28 r9 32 sl (r10) 36 fp (r11) 40 lr (r14)
For this context, only 11 * 4 = 44 bytes are needed.
Using Google, viewing disassembler listings, and also studying the conventions for calling procedures, we come to understand that this type of context is used when cooperative multitasking works: i.e. when we called tx_thread_sleep () or others like them. And since such a switch, in fact, is just a function call, then the context can be saved according to the calling conventions, according to which, we have the right not to save the values of the registers r0-r3, r12 between calls . Moreover, we do not need to save the pc - all the necessary information is already contained in the rl - return address from tx_thread_sleep (). Benefit on the face. Cortexes are usually used on systems with more memory than ARM9E, they do not resort to such tricks and use one type of stacking.
According to information from the Internet, I found out that the first type of context is called interrupt, and is used when the stream is interrupted by interruption, in other words, it can be interrupted anywhere, so you need to save all possible registers. The second type of context is called solicited and is used when the thread is interrupted by a system call, which leads to resheduling.
Here, in fact, everything is ready to understand what alterations are needed in OpenOCD:
- it is necessary to refine the mechanism for registering targets, so that it would be possible to use several stacking options for one target;
- actually make a description of the target.
I will not give the code for the first paragraph, look at the patch. For point two, I’ll explain a little how to compose a displacement label for OpenOCD-friendly ones.
First of all, we look at the output of the 'info registers' command, we look at how many registers and in what order is displayed, we make such a fish:
static const struct stack_register_offset rtos_threadx_arm926ejs_stack_offsets_solicited[] = {
{ , 32 }, /* r0 */
{ , 32 }, /* r1 */
{ , 32 }, /* r2 */q
{ , 32 }, /* r3 */
{ , 32 }, /* r4 */
{ , 32 }, /* r5 */
{ , 32 }, /* r6 */
{ , 32 }, /* r7 */
{ , 32 }, /* r8 */
{ , 32 }, /* r9 */
{ , 32 }, /* r10 */
{ , 32 }, /* r11 */
{ , 32 }, /* r12 */
{ , 32 }, /* sp (r13) */
{ , 32 }, /* lr (r14) */
{ , 32 }, /* pc (r15) */
{ , 32 }, /* xPSR */
};
Here 32 is the bit size of the register. For ARM, it is always 32. The first column is filled with the plates that we wrote above when analyzing the restoration of context. We take into account special values: -1 - this register is not saved, -2 - stack register, is restored from the stream structure.
A filled fish for a solicited context is as follows:
static const struct stack_register_offset rtos_threadx_arm926ejs_stack_offsets_solicited[] = {
{ -1, 32 }, /* r0 */
{ -1, 32 }, /* r1 */
{ -1, 32 }, /* r2 */
{ -1, 32 }, /* r3 */
{ 8, 32 }, /* r4 */
{ 12, 32 }, /* r5 */
{ 16, 32 }, /* r6 */
{ 20, 32 }, /* r7 */
{ 24, 32 }, /* r8 */
{ 28, 32 }, /* r9 */
{ 32, 32 }, /* r10 */
{ 36, 32 }, /* r11 */
{ -1, 32 }, /* r12 */
{ -2, 32 }, /* sp (r13) */
{ 40, 32 }, /* lr (r14) */
{ -1, 32 }, /* pc (r15) */
{ 4, 32 }, /* xPSR */
};
For interrupt context, try to write yourself or look at the source.
What will it give:
- list of threads by "info threads"
- individually for the thread: "thread apply all bt"
- switching between threads: "thread 3"
- switching between frames: "frame 5"
- individual viewing of the status of the registers of each thread
Commands given for gdb.
All in all, happy debugging!
Resources:
- Patch: ThreadX-arm926ejs.diff
- Build for Win32 / 64, patched sources, patch, auxiliary scripts: openocd-0.8.0-20150206-win.tar.xz
- Discussion on the Cypress forum: www.cypress.com/?app=forum&id=167&rID=106353
- OpenOCD mailing list: sourceforge.net/p/openocd/mailman/message/33287429
PS there is not enough “Reverse development” hub and highlighting for different assemblers ;-)
UPD / 2015-08-15 /: Changes are in the main OpenOCD branch: openocd.zylin.com/#/c/2848
