What are some of the important do's and dont's inside a kernel mode and ISR Routine ?
For example -
Is context-switching disabled while running an interrupt handler ?
Can a context switch happen when a process is inside a critical
section ?
What circumstances inside kernel mode merit disabling of further interrupts ?
How come a process switch can occur on a page-fault, where a process fetches data from the disk, but not happen during other occurences of interrupts.
How do you classify if a executable path can be interrupted/rescheduled/pre-empted ?
What are the other things one has to remember when process is in kernel mode or handling ISR routine ?
In short: NO CONTEXT SWITCH, EVER.
This means:
No preemption
No locks on mutexes (use spin locks instead and ensure your non-ISR counterparts acquire them with spin_lock_irqsave to disable IRQs)
No call to any kernel function that can sleep (check the function's documentation, some functions also have _cansleep variants).
A process switch can occur on a page fault, but it happens after the corresponding ISR has been processed. Basically a path can be scheduled if it is not an ISR and if you do not have a spinlock locked. If you hold a spinlock, you must avoid sleeping until it is released.
Since ISRs are very restrained, then handling of IRQs is usually split between a top-half (that runs in ISR context and does the critical job) and a bottom-half (that runs later as a kernel thread and does whatever can be delayed) which can sleep. See this page for more information:
http://www.makelinux.net/ldd3/chp-10-sect-4
Related
in a non preemptive system, after an ISR finishes execution, will the interrupted task continue execution even if a higher priority task was activated?
This answer is specific to FreeRTOS, and may not be relevant to other RTOS'es.
FreeRTOS is preemptive by default. However, it can also be configured to be non-preemptive by the config option in FreeRTOSConfig.h
#define configUSE_PREEMPTION 0
Normally, a return from ISR does not trigger a context switch. But in preemptive systems it's often desirable, so in most FreeRTOS examples, you see portYIELD_FROM_ISR(xHigherPriorityTaskWoken); at the end of the ISR, which triggers a context switch if xHigherPriorityTaskWoken is pdTRUE.
xHigherPriorityTaskWoken is initialized to pdFALSE at the start of the ISR (manually by the user), and operations which can cause a context switch, such as vTaskNotifyGiveFromISR() , xQueueSendToBackFromISR() etc. , take it as an argument and set it to pdTRUE if a context switch is required after the system call.
In a non-preemptive configuration, you simply pass NULL to such system calls instead of xHigherPriorityTaskWoken, and do not call portYIELD_FROM_ISR() at the end of the ISR. In this case, even if a higher priority task is awakened by the ISR, execution returns to the currently running task and remains there until this task yields or makes a blocking system call.
You may mix ISR yield mechanism with preemption method. For example, you can force a context switch (preemption) from ISR even when configUSE_PREEMPTION is 0, but this may cause problems if the interrupted/preempted task doesn't expect it to happen, so I don't recommend it.
I was reading the OS textbook, in the synchronization chapter,it says :
In particular,
most implementations of thread systems enforce the invariant that a thread
always disables interrupts before performing a context switch
Hence when writing Aquire() before go to sleep it will first disable interrupt.
My question is why interrupt disable is needed before context switch, is it use to protect the registers and keep the Aquire() atomic?
Aquire() is used before the critical section as:
Aquire(){
disable interrupt;
if (is busy){
put on wait queue;
sleep();
}
else set_busy;
enable interrupt;
}
Go to sleep will implement context switch,why should we disable interrupt during context switch?Can we change the code to :
Aquire(){
disable interrupt;
if (is busy){
enable interrupt;
put on wait queue;
sleep();
}
else set_busy;
enable interrupt;
}
That is enables interrupt in thread A instead of letting other thread B after context switch(after A go to sleep) enable interrupt?
Typically, a synchronization primitive requires updating multiple data locations simultaneously. For example, a semaphore Acquire might require changing the state of the current thread to blocked, updating the count of the semaphore, removing the current thread from a queue and placing it on another queue. Since simultaneously isn't really possible(*), it is necessary to devise an access protocol to simulate this. In a single cpu system, the easiest way to do this is disable interrupts, perform the updates, then re-enable interrupts. All software following this protocol will see the updates at once.
Multi-cpu systems typically need something extra to synchronize threads on separate cpus from interfering. Disabling interrupts is insufficient, since that only affects the current cpu. The something extra is typically a spin lock, which behaves much like a mutex or binary semaphore, except that the caller sits in a retry loop until it becomes available.
Even in the multi-cpu system, the operation has to be performed with interrupts disabled. Imagine Thread#0 has acquired a spinlock on cpu#0; then an interrupt on cpu#0 causes Thread#1 to preempt, and Thread#1 then attempts to acquire the same spinlock. There are many scenarios which amount to this.
(*) Transaction-al Memory provides something like this, but with limited applicability, and the implementation has to provide an independent implementation to ensure forward progress. Also, since transactions do not nest, they really need to disable interrupts as well.
I have read some books about os kernel recently. I knew that when an event (like clock ticks) happens, it will trigger an interruption then the kernel's specified routine response.
So my questions are:
1)When an interruption was triggered and its corresponding kernel routine was still running, then another interruption was triggered for some sort of reason. How will the kernel response? Will it mask the second interruption when it was handling the first interruption? Or the first interruption's corresponding routine was interrupted by the second one? If the second condition was true, how the kernel make sure the routines are "reentrance"?
2)Does the kernel multithreaded or multiprocess? I mean when things go like the first question, the kernel will use CPU's extra cores to handle interruptions? If it did, how can the kernel make sure everything works correctly just like running on a single-core CPU?
1) If an interruption is triggered and its corresponding kernel routine is still running, then another interruption is triggered for some sort of reason; how will the kernel respond? Will it mask the second interruption when it was handling the first interruption? Or the first interruption's corresponding routine was interrupted by the second one?
Yes; different operating systems may either:
mask other IRQs while an IRQ is being handled
allow different IRQs to nest (interrupt each other)
allow all IRQs to nest (including the same IRQ interrupting itself)
mask some IRQs and allow other IRQs to nest
not use more than one IRQs (e.g. only use a timer IRQ, and poll everything else)
If the second condition was true, how does the kernel make sure the routines are "reentrant"?
If the OS designer decided that (some or all) IRQs may interrupt others; then they'll need to figure out how reentrancy will work for whatever cases they allowed. This can be "do nothing that causes a problem" (e.g. maybe IRQ handler just sends a notification to a task that does the real work later), and could be further restrictions (e.g. temporarily acquire a lock that prevents further IRQs for pieces of the IRQ handler that might cause a reentrancy problem but not other pieces that don't).
2) Does the kernel multithreaded or multiprocess? I mean when things go like the first question, the kernel will use CPU's extra cores to handle interruptions?
Yes; different operating systems may either use multi-threading or multi-processing (or both or neither); and may or may not use other cores to handle interrupts.
If it did, how can the kernel make sure everything works correctly just like running on a single-core CPU?
If a kernel does use other cores to handle interrupts; it will also do something to ensure everything works correctly. "Something" could be a system of locks, or transaction memory, or lock-free/block-free algorithms, or a "shared nothing" approach, or a combination of these things.
Why are interrupts disabled when the kernel is currently handling an interrupt ?
What if an interrupt carrying an important message is missed ?
This prevents "stacked interrupts" that can overflow the kernel stack. It also can also prevent deadlocks and/or "pinning".
Most hardware doesn't "lose" an interrupt. During an interrupt, the CPU's "interrupt flag" is cleared, but the interrupt controller [a separate beast] is still available/enabled to note new interrupts. If the CPU is processing an interrupt for hardware_A (in "interrupt service routine" ISR_A), the interrupt for hardware_B can still be asserted. It will be remembered [by the interrupt controller], it just won't interrupt the CPU at that time. When ISR_A returns, the interrupt flag is reenableed on exit, and now, immediately, ISR_B will be entered (and its call stack frame will start at the same exact memory location as for ISR_A).
While interrupts won't be missed/dropped, ISRs should be short [execute quickly] to minimize latency. In other words, ISR_A should not take so long that hardware_B will overflow some internal state/buffer [as it continues to accumulate data while waiting for ISR service].
Minimizing latency is a part of careful kernel design and ISR design. In Linux, ISRs can be broken down into the ISR part and a "bottom half" or "tasklet" part. The ISR [with interrupts disabled] does the minimum needed to service/quiesce the device (e.g. clear a bit in the device to prevent it from reasserting the interrupt immediately).
It then enables its corresponding tasklet [which runs with interrupts enabled] to do the more laborous operations that may take longer. Tasklets, despite the name, aren't like full blown tasks/processes that show up in "ps". They're a [very] lightweight/efficient way of splitting up the work the ISR must do, to minimize latency.
Let's take each question.
Why are interrupts disabled when the kernel is currently handling an interrupt ?
Though there are many types of interrupts, such as I/Os, timers, watchdogs, serial ports,peripherals and DMA, let's take example of I/Os. We'll talk a raw case and extend it into kernel.
Imagine a fire-alarm/sensor bit 0/1 connected to your CPU's particular interrupt pin. 0 is normal state and 1 is fire! Then one would configure that input's interrupt as "level triggered". The moment the sensor fires 1, the ISR has to execute a relevant code to siren or automatically dial fire department. An interrupt should be generally cleared the moment you enter ISR. If it's not cleared, the hardware keeps interrupting the CPU and the safety code inside ISR will never execute.
Also the CPU needs to maintain a stack of its current execution state. A recurring interrupt makes it a complex situation.
Second example with a "edge triggered" or "transition triggered" Imagine a series of bits are coming on an input line/pin (NRZ coded). If the ISR's job is assembling those bits into words (8,16,32 whatever length), we need to clear the interrupt, assemble the bit into buffer, and enable the interrupt again, in cycles. Not clearing the interrupt would cause a glitch-y transition to mistake just 1 bit as 2 bits.
So the practice is setup and enable the interrupts, if interrupted, clear it, execute the ISR code and enable the interrupts back wherever relevant.
Kernel
Kernel itself is a critical piece of code and scheduler (also OS timer, OS Clock) depend on a hardware timers interrupt. The hardware part of the CPU that contains interrupt logic maintains state-transition. It also has hardwired logic for enabling, disabling, masking and setting interrupt priorities.
If a kernel module or diver module should execute code safely, a deterministic behavior can only be obtained by disabling the interrupt before executing the handler.
What if an interrupt carrying an important message is missed ?
The interrupt handling shouldn't be too long (before enabling back the interrupts). One should design code properly depending on the frequency of interrupts and complexity of handler.
I was reading up on interrupts. It is possible to suspend non-critical interrupts via a special interrupt mask. This is called interrupt masking. What i dont know is when/why you might want to or need to temporarily suspend interrupts? Possibly Semaphores, or programming in a multi-processor environment?
The OS does that when it prepares to run its own "let's orchestrate the world" code.
For example, at some point the OS thread scheduler has control. It prepares the processor registers and everything else that needs to be done before it lets a thread run so that the environment for that process and thread is set up. Then, before letting that thread run, it sets a timer interrupt to be raised after the time it intends to let the thread have on the CPU elapses.
After that time period (quantum) has elapsed, the interrupt is raised and the OS scheduler takes control again. It has to figure out what needs to be done next. To do that, it needs to save the state of the CPU registers so that it knows how to undo the side effects of the code it executes. If another interrupt is raised for any reason (e.g. some async I/O completes) while state is being saved, this would leave the OS in a situation where its world is not in a valid state (in effect, saving the state needs to be an atomic operation).
To avoid being caught in that situation, the OS kernel therefore disables interrupts while any such operations that need to be atomic are performed. After it has done whatever needs doing and the system is in a known state again, it reenables interrupts.
I used to program on an ARM board that had about 10 interrupts that could occur. Each particular program that I wrote was never interested in more than 4 of them. For instance there were 2 timers on the board, but my programs only used 1. I would mask the 2nd timer's interrupt. If I didn't mask that timer, it might have been enabled and continued making interrupts which would slow down my code.
Another example was that I would use the UART receive REGISTER full interrupt and so would never need the UART receive BUFFER full interrupt to occur.
I hope this gives you some insight as to why you might want to disable interrupts.
In addition to answers already given, there's an element of priority to it. There are some interrupts you need or want to be able to respond to as quickly as possible and others you'd like to know about but only when you're not so busy. The most obvious example might be refilling the write buffer on a DVD writer (where, if you don't do so in time, some hardware will simply write the DVD incorrectly) versus processing a new packet from the network. You'd disable the interrupt for the latter upon receiving the interrupt for the former, and keep it disabled for the duration of filling the buffer.
In practise, quite a lot of CPUs have interrupt priority built directly into the hardware. When an interrupt occurs, the disabled flags are set for lesser interrupts and, often, that interrupt at the same time as reading the interrupt vector and jumping to the relevant address. Dictating that receipt of an interrupt also implicitly masks that interrupt until the end of the interrupt handler has the nice side effect of loosening restrictions on interrupting hardware. E.g. you can simply say that signal high triggers the interrupt and leave the external hardware to decide how long it wants to hold the line high for without worrying about inadvertently triggering multiple interrupts.
In many antiquated systems (including the z80 and 6502) there tends to be only two levels of interrupt — maskable and non-maskable, which I think is where the language of enabling or disabling interrupts comes from. But even as far back as the original 68000 you've got eight levels of interrupt and a current priority level in the CPU that dictates which levels of incoming interrupt will actually be allowed to take effect.
Imagine your CPU is in "int3" handler now and at that time "int2" happens and the newly happened "int2" has a lower priority compared with "int3". How would we handle with this situation?
A way is when handling "int3", we are masking out other lower priority interrupters. That is we see the "int2" is signaling to CPU but the CPU would not be interrupted by it. After we finishing handling the "int3", we make a return from "int3" and unmasking the lower priority interrupters.
The place we returned to can be:
Another process(in a preemptive system)
The process that was interrupted by "int3"(in a non-preemptive system or preemptive system)
An int handler that is interrupted by "int3"， say int1's handler.
In cases 1 and 2, because we unmasked the lower priority interrupters and "int2" is still signaling the CPU: "hi, there is a something for you to handle immediately", then the CPU would be interrupted again, when it is executing instructions from a process, to handle "int2"
In case 3, if the priority of “int2” is higher than "int1", then the CPU would be interrupted again, when it is executing instructions from "int1"'s handler, to handle "int2".
Otherwise, "int1"'s handler is executed without interrupting (because we are also masking out the interrupters with priority lower then "int1" ) and the CPU would return to a process after handling the “int1” and unmask. At that time "int2" would be handled.