Why enable the interrupt so early on threadx scheduler on M4 port? - scheduler

On threadx M4 port of pendsv interrupt implementation, which do the actual context switch of the multithread support of threadx, it shows all the during the procedure, the interrupt all opened without disabled like other rtos, like ucos, rtthread, etc, so is this cause somethings that unpredictable at running time ?
like below code, the "cpsie i" was executed before the next ready to run thread restored
the line 202 eanbled the interrupt, but the actual context swithc has not finished yet, this is not like ucos does on m4 port, why is this safe? thank you!
**202 CPSIE i** # Enable interrupts
203 #
204 # /* Increment the thread run count. */
205 #
206 __tx_ts_restore:
207 LDR r7, [r1, #4] # Pickup the current thread run count
208 LDR r4, =_tx_timer_time_slice # Build address of time-slice variable
209 LDR r5, [r1, #24] # Pickup thread's current time-slice
210 ADD r7, r7, #1 # Increment the thread run count
211 STR r7, [r1, #4] # Store the new run count
212 #
213 # /* Setup global time-slice with thread's current time-slice. */
214 #
215 STR r5, [r4] # Setup global time-slice
216
217 #ifdef TX_ENABLE_EXECUTION_CHANGE_NOTIFY
218 #
219 # /* Call the thread entry function to indicate the thread is executing. */
220 #
221 PUSH {r0, r1} # Save r0/r1
222 BL _tx_execution_thread_enter # Call the thread execution enter function
223 POP {r0, r1} # Recover r3
224 #endif
225 #
226 # /* Restore the thread context and PSP. */
227 #
228 LDR r12, [r1, #8] # Pickup thread's stack pointer
229 LDMIA r12!, {LR} # Pickup LR
230 #ifdef TX_ENABLE_FPU_SUPPORT
231 TST LR, #0x10 # Determine if the VFP extended frame is present
232 BNE _skip_vfp_restore # If not, skip VFP restore
233 VLDMIA r12!, {s16-s31} # Yes, restore additional VFP registers
234 _skip_vfp_restore:
235 #endif
236 LDMIA r12!, {r4-r11} # Recover thread's registers
237 MSR PSP, r12 # Setup the thread's stack pointer
238 #
239 # /* Return to thread. */
240 #
241 BX lr # Return to thread!

ThreadX is designed to run as fast as possible and minimize critical sections. Enabling interrupt before restoring context is safe.
More details:
The scheduler is running in the PendSV interrupt, only an interrupt with higher priority than the PendSV can preempt the scheduler. The PendSV cannot preempt itself. ThreadX does not call the scheduler directly(except the first time). So, there is no re-entrancy for the scheduler. The context restoring process can be interrupted by a higher priority interrupt, the hardware will take care of interrupt nesting and saving/restoring caller saved registers etc. When the high priority interrupt is returned, the context restoring process continues as if nothing happened. If a high priority interrupt makes another higher priority thread ready, ThreadX will record the new thread as next thread to execute, but it will not call the scheduler directly. Instead, it sets the PendSV interrupt to pending state. Another context switch happens after the current context restoring process is finished.
As other interrupts do not affect the process of restoring context, it is safe to enable interrupt during context restore process.
By enabling interrupt earlier, other interrupts can be served with less delay. It is better to have less time in interrupt disabled state.

The short answer is that it's safe because beyond that point the system can handle an interrupt without leaving its critical data structures in an inconsistent state.
More specifically, after that point ThreadX's internal data structures are in a consistent state such that an ISR can invoke permitted ThreadX services safely - whereas before that point they are not, and so ISRs must be prevented from running.
Note that the M4's exception system - especially the SVCall, PendSV and interrupt exceptions and their relative priorities - facilitates implementation of a preemptive multitasking system without requiring explicit disabling and enabling of interrupts at all.

Related

arm64 ptrace SINGLESTEP: are the steps described in this paper correct?

I was reading the paper Hiding in the Shadows: Empowering ARM for Stealthy Virtual Machine Introspection and I was wondering whether the steps they are described in paragraph "2.3 Debug Exceptions" were correct or not:
AArch64 allows to generate Software Step exceptions by setting the SS
bit of the Monitor Debug System Control MDSCR_EL1 and Saved Program
Status Register SPSR of the target exception level. For instance, to
single-step a hit breakpoint in EL1 the monitor must set the
MDSCR_EL1.SS and SPSR_EL1.SS bits. After returning to the trapped
instruction, the SPSR will be written to the process state PSTATE
register in EL1. Consequently, the CPU executes the next instruction
and generates a Software Step exception.
I have tried to understand how single-stepping happens in freeBSD, and I am noticing a mismatch.
I am basing the following lines of code to the release 12.3.0 of freeBSD (4 December 2021), commit: 70cb68e7a00ac0310a2d0ca428c1d5018e6d39e1. I chose to base this question on freeBSD because, in my opinion, following its code is easier than Linux, but the same principles shall be common to both families.
According to my understanding, this is what happens in freeBSD:
1- Ptrace single step is invoked, arriving in the architecture-independent code proc_sstep(), in sys_process.c:
int proc_sstep(struct thread *td)
{
PROC_ACTION(ptrace_single_step(td));
}
2- Architecture-dependent code ptrace_single_step()is called, in arm64/ptrace_machdep.c:
int ptrace_single_step(struct thread *td)
{
td->td_frame->tf_spsr |= PSR_SS;
td->td_pcb->pcb_flags |= PCB_SINGLE_STEP;
return (0);
}
Here single step bit (number 21) is set in the "Process State" of the tracee (tracee = thread that is traced) and a flag is set.
3- After a while, the traced task will be selected for scheduling. In cpu_throw() of swtch.S (where the new thread takes place), the flags of the new thread are checked, to see if it must single step:
/* If we are single stepping, enable it */
ldr w5, [x4, #PCB_FLAGS]
set_step_flag w5, x6
4- set_step_flag macro in defined in the same swtch.S:
.macro set_step_flag pcbflags, tmp
tbz \pcbflags, #PCB_SINGLE_STEP_SHIFT, 999f
mrs \tmp, mdscr_el1
orr \tmp, \tmp, #1
msr mdscr_el1, \tmp
isb
999:
.endm
Here, if the single-step flag is set, it sets the single step bit of register MDSCR_EL1 (bit in position 0).
4- To the best of my understanding, the combination of single step bit on SPSR_EL1 of the "Pstate" + single step bit on MDSCRL_EL1 implies that the tracee execute 1 instruction and it traps.
5- Trap is recognized as a EXCP_SOFTSTP_EL0 and it is handled in do_el0_sync() function of trap.c:
case EXCP_SOFTSTP_EL0:
td->td_frame->tf_spsr &= ~PSR_SS;
td->td_pcb->pcb_flags &= ~PCB_SINGLE_STEP;
WRITE_SPECIALREG(mdscr_el1,
READ_SPECIALREG(mdscr_el1) & ~DBG_MDSCR_SS);
call_trapsignal(td, SIGTRAP, TRAP_TRACE,
(void *)frame->tf_elr, exception);
userret(td, frame);
break;
Here, all the flags are reset and the traced thread receives a SIGTRAP (sent by itself, I think). Being traced, it will stop. And the tracer, at this point, can return from a possible waitpid().
What I could observe differs from the paper explanation. Can you check and correct the steps that I listed, please ?

How to know which code caused a trap in xv6 when debugging?

Here is what I suppose: When the code causes a trap (system call or exceptions), xv6 will replace the registers with certain values to transfer the control to alltraps(), in which trap() is called.
But sometimes xv6 runs into trap() out of my expectation, and I want to know why it got into this trap. When debugging, after I set a break point in trap() and xv6 stopped here, I can only see this in the debugger (I'm using CLion). In the call stack, the bottom stack frame is alltraps() so I can't find out when and why the trap is caused.
I want to find out in which file, at which line the trap is caused for a certain call of trap(). Is this possible?
If you will examine trap() code more carefully, you will see that it also handles hardware interrupts (timer, ide and so on).
36 void
37 trap(struct trapframe *tf)
38 {
39 if(tf->trapno == T_SYSCALL){
...
47 }
48
49 switch(tf->trapno){
50 case T_IRQ0 + IRQ_TIMER:
51 if(cpuid() == 0){
52 acquire(&tickslock);
53 ticks++;
54 wakeup(&ticks);
55 release(&tickslock);
56 }
57 lapiceoi();
58 break;
59 case T_IRQ0 + IRQ_IDE:
...
So what are you seeing is hardware interrupt coming and processor transfers control to one of the IDT vectors then to alltraps then to trap(). Most likely you are facing timer interrupt, which is used for context switch.
I want to find out in which file, at which line the trap is caused for a certain call of trap(). Is this possible?
No it is not possible, because this is a hardware event and it has no relation to source code.

arm cortex m4 xPSR change when branching

I am writing an operating system... I am working on context switching
..
I can switch the kernel into user program and go back. but SVC call seems not work well.
syscall:
svc SYSCALL_SVC_NUMBER
bx lr
when calling svc it trigger interrupt, I can see the control flow go back to kernel. The hard fault arise when it gets back to user program.
around here
--> bx lr
I've checked that all the registers are correctly loaded, except that xPSR lacks of thumb bit. That's why the hard fault comes.
But I have no idea why xPSR is clear to zero...
.global activate
activate:
/* save kernel state in ip register */
mrs ip, psr
push {r4, r5, r6, r7, r8, r9, r10, r11, ip, lr}
/* switch to process stack */
msr psp, r0
mov ip, #2
msr control, ip
ldr ip, [sp, #0x38]
msr psr_nzcvq, ip
/* load user state */
pop {r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, lr}
add sp, #0x8
ldr ip, [sp, #-0x8]
/* this line can branch correctly */
bx ip
Ahh, yes, you are right, but you dont need to modify anything at the same time.
Using this code for example
mov r1,#0x22
mov r2,#0x33
mov r3,#0x44
mov r0,#0x55
mov r12,r0
mov r0,#0x66
mov r14,r0
mov r0,#0x11
svc #1
The stack looks like this when it hits the svc handler
Fairly certain this is documented
20000FD4 FFFFFFF9 return address
20000FD8 00000011 r0
20000FDC 00000022 r1
20000FE0 00000033 r2
20000FE4 00000044 r3
20000FE8 00000055 r12
20000FEC 00000066 r14
20000FF0 0100009E r15
20000FF4 21000000 xPSR
On an exception on a cortex-m they behave in hardware in a way that can call a C function directly per their calling convention. Understand that the program counter itself has an lsbit of 0, the lsbit of 1 thing is for BX, BLX and POP, it is used by those instructions to determine ARM or THUMB mode, the bit is stripped then used as the PC.
The return from an SVC can/should look like the above. if you want to use svc or any other interrupt to do a context switch you need to build the stack to match.
there is a bit of a chicken and egg problem of course, for each thread you build a stack image like the above
20000FD4 FFFFFFF9
20000FD8 00000011
20000FDC 00000022
20000FE0 00000033
20000FE4 00000044
20000FE8 00000055
20000FEC 00000066
20000FF0 01001234
20000FF4 21000000
but you can put dont cares in the registers except for the pc which you set to the entry point of your thread. with the lsbit not set. Also you need some structure where you keep the state of the other registers.
then when you context switch you save the registers other than the above in a structure somewhere, which includes the sp, you then fill in those registers from the next thread including its sp. then you bx lr to return.
there is a little more to it than that, see atomthreads or chibios or other open source, functional OS.
You are correct the address that was interrupted or in this case the address after the svc, the program counter, is on the stack without the lsbit set, but at the same time that is correct. The actual lr used to return from the exception (svc or timer interrupt or whatever) is the special "exception return" 0xFFFFFFxx which has the lsbit set.

How could one interrupt handler go until the same source is free?

Note that a single interrupt source (timer, keyboard, etc.)
will not signal a new interrupt to the processor until
the processor has indicated that handling of the previous interrupt from that source is ``done'', even if the system-wide interrupt-enable flag is on.
Who tells the PIC the current interrupt is over, and what does "systerm-wide interrupt-enable flag" mean?
That has been covered in my comment in the other question. :)
OK... Some more details...
If we're talking about the PIC and its usual operation (as in the BIOS and DOS), there are 16 IRQ lines. They are mapped (in the PIC) to interrupt vectors 8 through 0Fh (IRQ0 through IRQ7) and 70h through 77h (IRQ8 through IRQ15).
By reprogramming the PIC you can change this assignment (see the PIC (8259 chip) documentation). Changing this assignment is often more than just desirable in protected mode because various important exceptions are hardwired to interrupt vectors from 0 to around 1Fh (e.g. the general protection exception (AKA #GP) is at vector 0Dh, which is IRQ5 in this default assignment).
IRQ0 is the periodic timer (AKA PIT)
IRQ1 is the keyboard
IRQ2 is used to chain 2nd PIC (every PIC handles at most 8 IRQs, so you have 2 for 16 IRQs; IRQ8 thorugh 15 are, in fact, delivered through this IRQ2)
IRQ3 and IRQ4 are used for COM1 and COM2 serial ports
IRQ6 is used for the FDD
IRQ7 is used for the parallel port (where we used to connect our printers, and now it's usually the USB port)
IRQ8 is used for another timer, the real-time clock (AKA RTC)
IRQ12 is normally used for the PS/2 mouse
IRQ14 and IRQ15 are used for HDDs/CDROMs
Other IRQs aren't very fixed.
The PIC itself is connected to the CPU at I/O ports 20h and 21h (PIC1) and 0A0h and 0A1h (PIC2).
The CPU signals completion of IRQ handling by sending the EOI command to the corresponding PIC, from where this IRQ has come.
Thus, for IRQ0 through IRQ7 the ISR typically ends with this code:
...
mov al, 20h
out 20h, al ; send EOI to PIC1
; restore al using pop or mov
iret
For IRQ8 through IRQ15 the same thing looks like this:
mov al, 20h
out 0a0h, al ; send EOI to PIC2
out 20h, al ; send EOI to PIC1
; restore al using pop or mov
iret
In this latter case every PIC gets an EOI because, as I mentioned it earlier, PIC2 doesn't deliver IRQs directly to the CPU, but rather through PIC1 (on PIC1's IRQ2; this effectively limits the number of IRQs to 15), so both PICs are involved. And PIC2 is an interrupt source to PIC1 just like, say, the keyboard. So, 2 EOIs.
Further, some devices (may) have their own equivalents of EOI. For example, XT keyboards waited for a bit pulse (from 1 to 0) in one of their registers as an indication that the keyboard interrupt handling is complete. In such cases you send EOIs to the device and PIC(s).
EDIT: Most likely the text you're referring to means FLAGS.IF by "system-wide interrupt-enable flag".

change local stack variable value

Using Windbg/SOS, it possible to change value of a local varible on stack? If so how?
The short answer is: It depends.
Per default local value types are stored on the stack but due to optimization they will often be stored only in registers as needed. Reference types are stored on the heap, with a reference to the instance on the stack (or in a register).
I am going to assume that you're looking to change a local value type. Let's look at a simple example.
[MethodImpl(MethodImplOptions.NoInlining)] // avoid inlining of short method
public static void Method(int x) {
Console.WriteLine("The answer is {0}", x + x);
}
Assuming we set a breakpoint on Method and run until the breakpoint is hit, the stack looks like this:
0:000> !clrstack -a
OS Thread Id: 0x1abc (0)
Child SP IP Call Site
0035f290 003600e0 TestBench2010.Program.Method(Int32)*** WARNING: Unable to verify checksum for C:\workspaces\TestBench2010\TestBench2010\bin\Release\TestBench2010.exe
[C:\workspaces\TestBench2010\TestBench2010\Program.cs # 17]
PARAMETERS:
x (<CLR reg>) = 0x00000002
0035f294 003600a2 TestBench2010.Program.Main(System.String[]) [C:\workspaces\TestBench2010\TestBench2010\Program.cs # 24]
PARAMETERS:
args = <no data>
0035f4c0 636221bb [GCFrame: 0035f4c0]
Notice that the local x is listed as , but it doesn't tell us which register. We could look at the registers and find the one with the value 2, but there could be more than one. Instead let's look at the JIT compiled code for the method.
0:000> !u 001c37f0
Normal JIT generated code
TestBench2010.Program.Method(Int32)
Begin 003600e0, size 32
C:\workspaces\TestBench2010\TestBench2010\Program.cs # 17:
003600e0 55 push ebp
003600e1 8bec mov ebp,esp
003600e3 56 push esi
003600e4 8bf1 mov esi,ecx
*** WARNING: Unable to verify checksum for C:\windows\assembly\NativeImages_v4.0.30319_32\mscorlib\658bbc023e2f4f4e802be9483e988373\mscorlib.ni.dll
003600e6 b9302be004 mov ecx,offset mscorlib_ni+0x322b30 (04e02b30) (MT: System.Int32)
003600eb e8301fe5ff call 001b2020 (JitHelp: CORINFO_HELP_NEWSFAST)
003600f0 8bd0 mov edx,eax
003600f2 03f6 add esi,esi <==== This is x + x
003600f4 897204 mov dword ptr [edx+4],esi
003600f7 8bf2 mov esi,edx
003600f9 e882709d04 call mscorlib_ni+0x257180 (04d37180)(System.Console.get_Out(), mdToken: 060008cd)
003600fe 56 push esi
003600ff 8bc8 mov ecx,eax
00360101 8b1534204c03 mov edx,dword ptr ds:[34C2034h] ("The answer is {0}")
00360107 8b01 mov eax,dword ptr [ecx]
00360109 8b403c mov eax,dword ptr [eax+3Ch]
0036010c ff5018 call dword ptr [eax+18h]
C:\workspaces\TestBench2010\TestBench2010\Program.cs # 18:
0036010f 5e pop esi
00360110 5d pop ebp
00360111 c3 ret
Looking at the code, we see that the only add instruction uses the esi register, so our value is stored here prior to the calculation. Unfortunately, esi doesn't hold the correct value at this point, but looking backwards we find mov esi,ecx. I.e. the value is initially stored in ecx.
To change the value of ecx use the r command. E.g. to set the value to 0x15 do the following:
0:000> r ecx=15
The output of the method is now:
The answer is 42
Please keep in mind that the example above is only one of many possible scenarios. Locals are handled differently depending on debug/release build as well as 32/64 bit. Also, for complex methods it may be a bit harder tracking the exact location of the value.
To change the state of an instance, you have to locate the reference on the stack (e.g. using !clrstack or !dso). Once located you can use the offsets to find the memory, that holds the data and use the e* commands to change the values as needed. Let me know if you want an example for that as well.