Why must one use a mutex in addition to a semaphore when using a bounded buffer in producer consumer problem?
empty:semaphore(n)
full: semaphore(0)
mutex: semaphore(1)
"mutex" is used to lock on Buffer.
"full" is used to block consumer if buffer is empty.
"empty" is used to block producer if buffer is full.
That's why you need 3 semaphores.
You can easily google the code so I don't paste it here.
Related
I am looking into using a stream buffer in FreeRTOS to transfer CAN frames from multiple tasks to an ISR, which puts them into a CAN transmit buffer as soon as it's ready. The manual here explains that a stream buffer should only be used by one task/isr and read by one task/isr, and if not, then a critical section is required.
Can a mutex be used in place of a critical section for this scenario? Would it make more sense to use?
First, if you are sending short discrete frames, you may want to consider a message buffer instead of a stream buffer.
Yes you could use a mutex.
If sending from multiple tasks the main thing to consider is what happens when the stream buffer becomes full. If you were using a different FreeRTOS object (other than a message buffer, message buffers being built on stream buffers) then multiple tasks attempting to write to the same instance of an object that was full would all block on their attempt to write to the object, and be automatically unblocked when space in the object became available - the highest priority waiting task would be the first to be unblocked no matter the order in which the tasks entered the blocked state. However, with stream/message buffers you can only have one task blocked attempting to write to a full buffer - and if the buffer was protected by a muted - all other tasks would instead block on the mutex. That could mean that a low priority task was blocked on the stream/message buffer while a higher priority task was blocked on the mutex - a kind of priority inversion.
In the UDP/IP Stack solution example, here, there is a proposed solution for blocking on a single event queue.
What would be the go to solution for protecting the data that the pointer points to until it has been handled by the task waiting for the queue.
Say for example that the queue is filled from a ISR. The ISR should not write to *pvData if it has not been processed by the appropriate task. But since there can be several event sources the queue should probably be longer than one item. Should the struct be made:
typedef struct IP_TASK_COMMANDS
{
eIPEvent_t eEventType;
SemaphoreHandle_t data_read;
void *pvData;
} xIPStackEvent_t;
With the semaphore taken in the ISR and given in the task that processes the data when it's done with it.
If you take the UDP example - normally you would have a pool of buffers (or dynamically allocate a buffer) from which a buffer would be obtained and given to the DMA. When the DMA fills the buffer with received data a pointer to the buffer goes into the UDP stack - at which point only the UDP stack knows where the buffer is and is responsible for it. At some point the data in the buffer may get passed from the UDP stack to the application where it can be consumed. The application then returns the buffer to the pool (or frees the allocated buffer) so it is available to the DMA again. The reverse is also true - the application may allocate a buffer that is filled with data to be sent, via the UDP stack, to the Tx function where it is actually placed onto the wire - in which case it is the Tx end interrupt that returns the buffer to the pool.
So, in short, there is only one thing that has a reference to the buffer at a time, so there is no problem.
[note above where it says the application allocates or frees a buffer, that would be inside the UDP/IP stack API called by the application rather than by the application directly - this is in fact partly at least how our own TCP/IP stack is implemented.]
You don't want your ISR to block and wait for the data buffer to become available. If it's appropriate for your ISR to just skip the update and move on when the buffer is not available then perhaps a semaphore makes sense. But the ISR should not block on the semaphore.
Here's an alternative to consider. Make a memory pool containing multiple appropriately sized data buffers. The ISR allocates the next available buffer from the pool, writes the data to it and puts the pointer to it on the queue. The task reads the pointer from the queue, uses the data, and then frees the buffer back to the pool. If the ISR runs again before the task uses the data, the ISR will be allocating a fresh buffer so it won't be overwriting the previous data.
Here's another consideration. The FreeRTOS Queue passes items by copy. If the data buffer is relatively small then perhaps it makes sense to just pass the data structure rather than a pointer to the data structure. If you pass the data structure then the queue service will make a copy to provide to the task and the ISR is free to update it's original buffer.
Now that I think about it, using the Queue service copy feature may be simpler than creating your own separate memory pool. When you create the Queue and specify the queue length and item size, I believe the queue service creates a memory pool to be used by the queue.
if there is 1 producer, 1 consumer and a buffer of >1 size is insert mutex required? is remove mutex required?
if there is 1 producer, 1 consumer and a buffer of 1 size is insert mutex required? is remove mutex required?
if there is >1 producer, >1 consumer and a buffer of 1 size is insert mutex required? is remove mutex required?
Can someone explain how you get to answer these questions. I know that two threads should never read from a the buffer while its being written into, but doesn't that mean that all of the scenarios require both mutexs?
Answers from professor: first case is yes, second two are no because when buffer is nonempty that is equivalent to a full buffer. When the buffer is empty the consumer is blocked. When the buffer contains an item the producer is blocked. So mutual exclusion is guaranteed with out using mutex. Which didn't help with understanding why that's the case. Good thing more experience has been had since this post was made.
Consider the following linked list queue pop method:
Object pop() {
if(this.head != null) {
Node n = this.head;
this.head = n.next;
return n.data;
}
return null;
}
This method is not thread safe. Consider what would happen if the thread paused after executing line 3, and another thread calls pop; both threads would get the same object.
Mutexes ensure that two threads cannot access the same resource at the same time, protecting against this 'race condition'. By ensuring that only one thread can pop an element at a time, the consistency of the queue is maintained.
It is possible to implement a queue without using mutexes (ex. Java's ConcurrentLinkedList), but it is much more difficult.
I tried to find out how to ensure a mutex should be entered into by each thread (POSIX thread in Linux) averagely.
In my program, there is a global queue and it has own mutex lock. A couple of writing threads write element into queue one at a time, and a single reading thread reads out a group of elements from the queue every time. The result is that the size of queue always grows large than the limitation.
so my question is how to ensure that the mutex should be accessed by every thread averagely. Any comments will be appreciated!
I am assuming the scenario of two writer threads, one reader thread and a common buffering queue with some buffer limit.
There are couple of ways doing this.
Create the reader thread with higher priority then writer threads. So every time when the lock will be released by any of the writer thread, it will be acquired by the reader thread immediately if it is waiting in the scheduler queue along-with the second writer thread.
Use a global synchronized flag to perform the task in queue, and give a threshold for certain reading and writing conditions (say if my queue count is 10, so if the max count will be achieved, next time I will be able to schedule reader thread only with the help of flag for a certain number of times and then will release the flag to work normally). This will help restricting the queue growing larger then the limit.
Hope you understand both the points.
I wanted someone to resolve my confusion on this topic. It may sound simple, but am really confused.
In producer/consumer problem, I used 4-semaphore solution. I used a different lock for each of the critical sections.
Say,
Pseudo code of producer:
wait(slot) // counting sem
wait(mutex1) // binary sem
rear <-- rear + 1
buffer[rear] <-- item
signal (mutex1)
signal(items)
Where I use, "mutex2" as a second Mutex for my consumer, as "mutex1" in producer.
Now, my question is. If my producer and consumer is not using a buffer (rear and front) but using a stack, where only they can manipulate [top]. Do I need to use one mutex or two different locks as in my 4-semaphore, in order to ensure mutual exclusion.
Pseudo code of consumer with stack:
wait (message)
wait (mutex)
getspace <-- stack[top]
top – 1
signal (mutex)
signal (slot)
Personally, I think I need one lock for both procedures, so I make sure none of the producer and consumer access the top concurrently. But am not sure about that.
Thank you.
I'm not 100% sure that I follow your pseudo-code but I'll do my best to explain how to use semaphores to manage a stack from within the Producer-consumer process.
When you have a stack that is being accessed across multiple threads, you will need to lock it when the data is being accessed or, more specifically, when it is being pushed and popped. (This is always an underlying assumption of the Producer-consumer problem.)
We start off by defining a mutex that we will use to lock the stack.
Global Declaration of Process Semaphores
stackAccessMutex = semaphore(1) # The "(1)" is the count
# initializer for the semaphore.
Next, we will need to lock it when we are adding or removing data from it in our Consumer and Producer threads.
Producer thread
dataPushBuff #Buffer containing data to be pushed to the stack.
…dataPushBuff is assigned…
stackAccessMutex.wait()
stack.push(dataPushBuff)
stackAccessMutex.signal()
Consumer thread
dataRecvBuff = nil # Defining a variable to store the pushed
# content, accessible from only within
# the Consumer thread.
stackAccessMutex.wait()
dataRecvBuff = stack.pop()
stackAccessMutex.signal()
…Consume dataRecvBuff as needed since it's removed from the stack…
So far, everything is pretty straight forward. The Producer will lock the stack only when it needs to. The same is true for the consumer. We shouldn't need another semaphore, should we? Correct? No, wrong!
The above scenario makes one fatal assumption-- that the stack will always be initialized with data before it is popped. If the consumer thread executes before the producer thread gets a chance to pop any data, you will generate an error within your consumer thread because stack.pop() will not return anything! To fix this, we need to signal the consumer that data is available in the stack.
First, we need to define a semaphore that can be used to signal whether data in the stack exists or not.
Global Declaration of Process Semaphores, Version #2
stackAccessMutex = semaphore(1)
itemsInStack = semaphore(0)
We initialize our itemsInStack to the number of items in our stack, which is 0 (see 1).
Next, we need to implement our new semaphore into our Producer and Consumer threads. First, we need to have the Producer signal that an item has been added. Let's update the Producer now.
Producer thread, Version #2
dataPushBuff
…dataPushBuff is assigned…
stackAccessMutex.wait()
stack.push(dataPushBuff)
stackAccessMutex.signal()
itemInStack.signal() #Signal the Consumer, we have data in the stack!
#Note, this call can be placed within the
#stackAccessMutex locking block, but it doesn't
#have to be there. As a matter of convention, any
#code that can be executed outside of a lock,
#should be executed outside of the lock.
Now that we can check to see if there is data in the stack via a semaphore, let's re-write our Consumer thread.
Consumer thread, Version #2
dataRecvBuff = nil # Defining a variable to store the pushed
# content, accessible from only within
# the Consumer thread.
itemsInStack.wait()
stackAccessMutex.wait()
dataRecvBuff = stack.pop()
stackAccessMutex.signal()
…Consume dataRecvBuff as needed since it's removed from the stack…
… and that's it. As you can see, there are two semaphores and both are mandatory (see 2) because we need to lock our stack when it's accessed and we need to signal our consumer when data is available and lock it when there is nothing in the stack.
Hope that answered your question. I'll update my response if you have any specific questions.
Theoretically, when the process starts, you could
pre-initialize your stack with data. In this case, you can should
initialize your itemsInStack semaphore with the value that is
equal to the stack count. However, in the case of this example, we
are assuming that there is no data in the stack, nor none to
initialize.
It is worth mentioning that under one, specific circumstance you
can theoretically get away with using just the stackAccessMutex.
Consider the case where the stack always contains data. If the
stack is infinite, we do not need to signal our Consumer that data
has been added because there always will be data. However, in
reality an "infinite stack" doesn't exist. Even if that should be
the case within your current context, there's no overhead in adding
the safety net of the itemsInStack semaphore.
Also, it may be tempting to to throw out the itemsInStack counting
semaphore if under your current circumstance a call to
stack.pop() would not cause any error if it were to not return any
data on an empty stack.
This is plausible, but not recommended. Assuming the Consumer thread is executing the
code on a loop, the loop will continuously execute the stack consumption code while
there is no data to consume. By using the itemsInStack semaphore, you are pausing the
thread until data arrives which should save a few CPU cycles.