What are the differences between scheduling of process and threads and difference between methods of IPC's between thread and process. Do all IPC's mechanism like semaphores, mutex, spinlock etc can be applied for the scheduling of process and threads.???
The process is the circle, representing the "container" concept (the address space), and the three squigley lines are the threads.
We are scheduling threads
A process is a program in execution, whereas a thread is a path of execution within a process.
Related
Having just finished a book on comp. architecture, I find myself not completely clarified on where the scheduler is running.
What I'm looking to have clarified is where the scheduler is running - does it have it's own core assigned to run that and nothing else, or is the "scheduler" in fact just a more ambiguous algorithm, that it implemented in every thread being executed - ex. upon preemption of thread, a swithToFrom() command is run?
I don't need specifics according to windows x/linux x/mac os x, just in general.
No the scheduler is not run in it's own core. In fact multi-threading was common long before multi-core CPUs were common.
The best way to see how scheduler code interacts with thread code is to start with a simple, cooperative, single-core example.
Suppose thread A is running and thread B is waiting on an event. thread A posts that event, which causes thread B to become runnable. The event logic has to call the scheduler, and, for the purposes of this example, we assume that it decides to switch to thread B. At this point in time the call stack will look something like this:
thread_A_main()
post_event(...)
scheduler(...)
switch_threads(threadA, threadB)
switch_threads will save the CPU state on the stack, save thread A's stack pointer, and load the CPU stack pointer with the value of thread B's stack pointer. It will then load the rest of the CPU state from the stack, where the stack is now stack B. At this point, the call stack has become
thread_B_main()
wait_on_event(...)
scheduler(...)
switch_threads(threadB, threadC)
In other words, thread B has now woken up in the state it was in when it previously yielded control to thread C. When switch_threads() returns, it returns control to thread B.
These kind of manipulations of the stack pointer usually require some hand-coded assembler.
Add Interrupts
Thread B is running and a timer interrupts occurs. The call stack is now
thread_B_main()
foo() //something thread B was up to
interrupt_shell
timer_isr()
interrupt_shell is a special function. It is not called. It is preemptively invoked by the hardware. foo() did not call interrupt_shell, so when interrupt_shell returns control to foo(), it must restore the CPU state exactly. This is different from a normal function, which returns leaving the CPU state according to calling conventions. Since interrupt_shell follows different rules to those stated by the calling conventions, it too must be written in assembler.
The main job of interrupt_shell is to identify the source of the interrupt and call the appropriate interrupt service routine (ISR) which in this case is timer_isr(), then control is returned to the running thread.
Add preemptive thread switches
Suppose the timer_isr() decides that it's time for a time-slice. Thread D is to be given some CPU time
thread_B_main()
foo() //something thread B was up to
interrupt_shell
timer_isr()
scheduler()
Now, scheduler() can't call switch_threads() at this point because we are in interrupt context. However, it can be called soon after, usually as the last thing interrupt_shell does. This leaves the thread B stack saved in this state
thread_B_main()
foo() //something thread B was up to
interrupt_shell
switch_threads(threadB, threadD)
Add Deferred Service Routines
Some OSses do not allow you to do complex logic like scheduling from within ISRs. One solution is to use a deferred service routine (DSR) which runs as higher priority than threads but lower than interrupts. These are used so that while scheduler() still needs to be protected from being preempted by DSRs, ISRs can be executed without a problem. This reduces the number of places a kernel has to mask (switch off) interrupts to keep it's logic consistent.
I once ported some software from an OS that had DSRs to one that didn't. The simple solution to this was to create a "DSR thread" that ran higher priority than all other threads. The "DSR thread" simply replaces the DSR dispatcher that the other OS used.
Add traps
You may have observed in the examples I've given so far, we are calling the scheduler from both thread and interrupt contexts. There are two ways in and two ways out. It looks a bit weird but it does work. However, moving forward, we may want to isolate our thread code from our Kernel code, and we do this with traps. Here is the event posting redone with traps
thread_A_main()
post_event(...)
user_space_scheduler(...)
trap()
interrupt_shell
kernel_space_scheduler(...)
switch_threads(threadA, threadB)
A trap causes an interrupt or an interrupt-like event. On the ARM CPU they are known as "software interrupts" and this is a good description.
Now all calls to switch_threads() begin and end in interrupt context, which, incidentally usually happens in a special CPU mode. This is a step towards privilege separation.
As you can see, scheduling wasn't built in a day. You could go on:
Add a memory mapper
Add processes
Add multiple Cores
Add hyperthreading
Add virtualization
Happy reading!
Each core is separately running the kernel, and cooperates with other cores by reading / writing shared memory. One of the shared data structures maintained by the kernel is the list of tasks that are ready to run, and are just waiting for a timeslice to run in.
The kernel's process / thread scheduler runs on the core that needs to figure out what to do next. It's a distributed algorithm with no single decision-making thread.
Scheduling doesn't work by figuring out what task should run on which other CPU. It works by figuring out what this CPU should do now, based on which tasks are ready to run. This happens whenever a thread uses up its timeslice, or makes a system call that blocks. In Linux, even the kernel itself is pre-emptible, so a high-priority task can be run even in the middle of a system call that takes a lot of CPU time to handle. (e.g. checking the permissions on all the parent directories in an open("/a/b/c/d/e/f/g/h/file", ...), if they're hot in VFS cache so it doesn't block, just uses a lot of CPU time).
I'm not sure if this is done by having the directory-walking loop in (a function called by) open() "manually" call schedule() to see if the current thread should be pre-empted or not. Or maybe just that tasks waking up will have set some kind of hardware time to fire an interrupt, and the kernel in general is pre-emptible if compiled with CONFIG_PREEMPT.
There's an inter-processor interrupt mechanism to ask another core to schedule something on itself, so the above description is an over-simplification. (e.g. for Linux run_on to support RCU sync points, and TLB shootdowns when a thread on another core uses munmap). But it's true that there isn't one "master control program"; generally the kernel on each core decides what that core should be running. (By running the same schedule() function on a shared data-structure of tasks that are ready to run.)
The scheduler's decision-making is not always as simple as taking the task at the front of the queue: a good scheduler will try to avoid bouncing a thread from one core to another (because its data will be hot in the caches of the core it was last running on, if that was recent). So to avoid cache thrashing, a scheduler algorithm might choose not to run a ready task on the current core if it was just running on a different core, instead leaving it for that other core to get to later. That way a brief interrupt-handler or blocking system call wouldn't result in a CPU migration.
This is especially important in a NUMA system, where running on the "wrong" core will be slower long-term, even once the caches populate.
There are three types of general schedulers:
Job scheduler also known as the Long term scheduler.
Short term scheduler also known as the CPU scheduler.
Medium term scheduler, mostly used to swap jobs so there can be non-blocking calls. This is usually for not having too many I/O jobs or to little.
In an operating systems book it shows a nice automata of the states these schedulers go to and from. Job scheduler puts things from job queue to ready queue, the CPU scheduler takes things from ready queue to running state. The algorithm is just like any other software, it must be run on a cpu/core, it is most likely probably part of the kernel somewhere.
It doesn't make sense the scheduler can be preempted. The jobs inside the queue can be preempted when running, for I/O, etc. No the kernel does not have to schedule itself to allocate the task, it just gets cpu time without scheduling itself. And yes, most likely the data is in probably in ram, not sure if it is worth storing in the cpu cache.
Should we use a thread pool for long running threads or start our own threads? Is there some design pattern?
Unfortunately, it depends. There is no hard and fast rule saying that you should always employ thread pools.
Thread pools offer two main things:
Delegated creation/reuse of threads.
Back-pressure
IMO, it's the back-pressure property that's interesting, but often the most poorly understood. Your machine runs on a limited set of resources. If you have (say) 8 CPU cores and they are all busy working, you would like to signal that in some way that adding more work (submitting more tasks) isn't going to help, at least not in terms of latency.
This is the reason java.util.concurrent.ExecutorService implementations allow you to specify a java.util.concurrent.BlockingQueue of your choice. When this queue grows full, invoking threads will block until the thread pool has managed to complete tasks in progress.
Whether or not to have long-running threads inside the thread pool depends on what it's doing. If the thread is constantly busy (meaning it will never complete) then it will always occupy a slot in the thread pool, which is kind of pointless.
Regarding delegated creation/reuse of threads; maybe you could have two pools, one for long-running tasks and one for other tasks. Or perhaps a long-running thread pool with one single slot, this will prevent two long-running tasks from running at the same time, provided that is what you want.
As you can see, there is no single good answer. It really boils down to what you are trying to achieve and how you want to use the resources at hand.
I know what a binary semaphore is: it is a flag when is set to 1 by an ISR of an interrupt.
But what is a semaphore when we are using a pre-emptive kernel, say FreeRTOS? Is it the same as binary semaphore?
it is a flag when is set to 1 by an ISR of an interrupt.
That is neither a complete nor accurate description of a semaphore. What you have described is merely a flag. A semaphore is a synchronisation object; there are three forms provided by a typical RTOS:
Binary Semaphore
Counting Sempahore
Mutual Exclusion Semaphore (Mutex)
In the case of a binary semaphore, there are two operations give and take. A task taking a semaphore will block (i.e. suspend execution and allow other lower or equal priority threads to run threads to run) until some other thread or interrupt handler gives the semaphore. Binary semaphores are used to signal between threads and from ISRs to threads. They are often used to implement deferred interrupt handlers, so that an ISR can ve bery short, and the handler benefit from RTOS mechanisms that are not allowed in an ISR (anything that blocks or suspends execution).
Multiple threads may block on a single semaphore, but only one of those tasks will respond take the semaphore. Some RTOS have a flush operation (VxWorks for example) that puts all threads waiting on a semaphore in the ready state simultaneously - in which case they will run according to the priority scheduling scheme.
A Counting Semaphore is similar to a Binary Semaphore, except that it can be given multiple times, and tasks may take the semaphore without blocking until the count is zero.
A Mutex is used for resource locking. It is possible to use a binary semaphore for this, but a mutex provides features that make this safer. The operations on a mutex are lock and unlock. When a thread locks a mutex, and another task attempts to lock the same mutex, the second (and any subsequent) task blocks until the first task unlocks it. This can be used to prevent more than one thread accessing a resource (memory or I/O) simultaneously. A thread may lock a mutex multiple times; a count is maintained, so that it must be unlocked an equal number of times before the lock is released. This allows a thread to nest locks.
A special feature of a mutex is that if a thread with the lock is a lower priority that a task requesting the lock, then the lower priority task is boosted to the priority of the higher in order to prevent a priority inversion where a middle priority task may preempt the low priority task with the lock increasing the length of time the higher priority task must wait this rendering the scheduling non-deterministic.
The above descriptions are typical; specific RTOS implementations may differ. For example FreeRTOS distinguishes between a mutex and a recursive mutex, the latter supporting the nestability feature; while the first is marginally more efficient where nesting is not needed.
Semaphores are not just flags, or counts. They support send and wait operations. A user-space thread can wait on a semaphore without unnecessary and unwanted polling and be made ready/running 'immediately' when another thread, or an appropriately-designed driver/ISR, sends a unit.
By 'appropriately-designed driver/ISR', I mean one that can perform a send() operation and then exit via the OS scheduler whenever it needs to set a waiting thread ready/running.
Such a mechanism is vitally important on preemptive kernels because it allows them to achieve very good I/O performance without wasting time, CPU cycles and memory-bandwidth on polling. Non-preemptive systems are hopelessly slow, latency-ridden and wasteful at I/O and this is why they are essentially no longer used and why we put up with all the synchro/locking/queueing etc issues.
In a multi-threaded process,If one thread is busy on I/O will the entire process be blocked?
AFAIK, it totally depends on programmer that how they manage the threads inside the programs.
If another thread is there with no I/O, processor will never sit idle & start executing this thread. However, process in split threads such that one thread waits for the result of the other, the the entire process will be blocked.
Please comment if more information needs to be added.
Does there exist any other explaination?
If the process has only one thread, then yes.
If the process has multiple threads, then normally no if the operating system supports multithreading.
This question can also be addressed in terms of the underlying implementation of user threads. There are different models for multithreading models, in order to implement user threads they have to be mapped to a kernel thread:
Many-to-One: Many user threads to one kernel thread
One-to-One: Each user thread is assigned to a kernel thread.
Many-to-Many: Many user threads are split on different kernel threads.
In the many-to-one case, a single block-operation (system call) within the thread can block the whole process. This disadvantage is not present in the one-to-one model.
How is process-based multitasking achieved by using multi-threading in each process?
For example, consider when an operating system is running with two background process. Each process supports internally multi-threading features. Now, how does time slicing happen between and inside these processes, and how does time slicing happen between threads?
The scheduler typically works at the thread level. In simplest terms the scheduler gives each runnable thread its timeslice in turn.
So a process with two threads will get twice as much CPU time as a process with one thread.
From:
http://msdn.microsoft.com/en-us/library/ms684259(VS.85).aspx
"A multitasking operating system divides the available processor time among the processes or threads that need it. The system is designed for preemptive multitasking; it allocates a processor time slice to each thread it executes. The currently executing thread is suspended when its time slice elapses, allowing another thread to run. When the system switches from one thread to another, it saves the context of the preempted thread and restores the saved context of the next thread in the queue.
The length of the time slice depends on the operating system and the processor. Because each time slice is small (approximately 20 milliseconds), multiple threads appear to be executing at the same time. This is actually the case on multiprocessor systems, where the executable threads are distributed among the available processors. However, you must use caution when using multiple threads in an application, because system performance can decrease if there are too many threads."
Also check out This link for when to use multi-tasking
The operating system decides when and for how long each thread exectues. For Microsoft operating systems, there is no way to determine or predict which thread in which process will execute next. Each thread also has a priority that it runs at. Higher priority threads tend to get more time than lower This priority can be changed by the user or by a program. See this link for more info.
"Now, how does time slicing happen between and inside these processes, and how does time slicing happen between threads?"
That's entirely up to the operating system to decide, really. A really basic OS might not do time-slicing at all, and just let each process run through to completion on a first-come, first-serve basis.
However, most modern operating systems will use some flavor of scheduling algorithm to decide which thread gets to execute on which core and for how long, and perform the context-switching necessary to save and restore per-thread state when swapping out one thread for another.