Understanding the Producer-Consumer Pattern in C Sharp
The Producer-Consumer pattern is one of the most common patterns in concurrent programming.
In simple English, it means this:
One part of the program creates work.
Another part of the program processes that work.
The part that creates work is called the Producer.
The part that processes work is called the Consumer.
Between them, there is usually a shared queue.
private static readonly Queue<int> _queue = new();
The producer adds items into the queue, and the consumer removes items from the queue.
A real-life example
Imagine a kitchen.
The chef prepares meals and places them on the counter.
The waiter takes meals from the counter and serves them to customers.
The chef is the Producer.
The waiter is the Consumer.
The counter is the Queue.
But there is one problem.
The counter has limited space.
If the counter is full, the chef must wait before placing more meals.
If the counter is empty, the waiter must wait before taking anything.
That is exactly what the Producer-Consumer pattern solves.
It helps two parts of a system work together safely, even when they work at different speeds.
The code example
public sealed class ProducerConsumer
{
private static readonly Queue<int> _queue = new();
private const int MaxSize = 5;
private static readonly SemaphoreSlim _itemsAvailable = new(0);
private static readonly SemaphoreSlim _spaceAvailable = new(MaxSize);
private static readonly object _lock = new();
public async Task Producer(int id, int count)
{
for (int i = 0; i < count; i++)
{
await _spaceAvailable.WaitAsync();
lock (_lock)
{
_queue.Enqueue(i);
Console.BackgroundColor = ConsoleColor.Green;
Console.WriteLine($"Producer {id} produced {i}");
Console.ResetColor();
}
_itemsAvailable.Release();
Console.WriteLine($"Producer {id} waiting for space...");
await Task.Delay(300);
}
}
public async Task Consumer(int id)
{
while (true)
{
await _itemsAvailable.WaitAsync();
int item;
lock (_lock)
{
item = _queue.Dequeue();
Console.BackgroundColor = ConsoleColor.Red;
Console.WriteLine($"Consumer {id} consumed {item}");
Console.ResetColor();
}
_spaceAvailable.Release();
Console.WriteLine($"Consumer {id} waiting for items...");
await Task.Delay(500);
}
}
public async Task Run()
{
var producer = Producer(1, 10);
var consumer = Consumer(1);
await producer;
await Task.Delay(3000);
}
}
What this code does
This example creates a small queue that can hold up to 5 items.
private const int MaxSize = 5;
The producer creates numbers and adds them into the queue.
_queue.Enqueue(i);
The consumer takes numbers from the queue.
item = _queue.Dequeue();
So the flow is:
Producer -> Queue -> Consumer
The producer does not directly give the item to the consumer.
Instead, it places the item into the queue.
The consumer then takes the item from the queue when it is ready.
This is useful because the producer and consumer do not need to work at the exact same speed.
Why the queue has a maximum size
The queue has a maximum size of 5.
private const int MaxSize = 5;
That means the producer is not allowed to keep adding items forever.
If the queue becomes full, the producer must wait.
This is important because, in real applications, unlimited queues can become dangerous. If a producer creates work faster than consumers can process it, memory usage can grow too much.
A bounded queue helps apply back pressure.
Back pressure means:
The producer is slowed down when the system cannot process work fast enough.
This keeps the system more stable.
What is SemaphoreSlim?
SemaphoreSlim is a lightweight synchronization tool in .NET.
In simple English, it is like a counter that controls how many operations are allowed to continue.
In this example, we use two semaphores:
private static readonly SemaphoreSlim _itemsAvailable = new(0);
private static readonly SemaphoreSlim _spaceAvailable = new(MaxSize);
The first one is _itemsAvailable.
private static readonly SemaphoreSlim _itemsAvailable = new(0);
It starts at 0 because, at the beginning, the queue is empty.
So the consumer must wait here:
await _itemsAvailable.WaitAsync();
This means:
Wait until at least one item is available.
The second one is _spaceAvailable.
private static readonly SemaphoreSlim _spaceAvailable = new(MaxSize);
It starts at MaxSize, which is 5.
That means, at the beginning, the queue has 5 free spaces.
So the producer waits here:
await _spaceAvailable.WaitAsync();
This means:
Wait until there is free space in the queue.
Microsoft’s documentation explains that Wait or WaitAsync enters the semaphore and decreases its count, while Release increases the count again. When the count reaches zero, new waiters must wait until something releases the semaphore. (Microsoft Learn)
How the producer works
The producer first waits for free space:
await _spaceAvailable.WaitAsync();
If there is space, it enters the lock block and adds an item to the queue:
lock (_lock)
{
_queue.Enqueue(i);
}
Then it releases _itemsAvailable:
_itemsAvailable.Release();
This tells the consumer:
One item is now available.
So the producer flow is:
Wait for space
Add item to queue
Signal that an item is available
How the consumer works
The consumer first waits for an item:
await _itemsAvailable.WaitAsync();
If an item is available, it enters the lock block and removes the item from the queue:
lock (_lock)
{
item = _queue.Dequeue();
}
Then it releases _spaceAvailable:
_spaceAvailable.Release();
This tells the producer:
One space is now free.
So the consumer flow is:
Wait for item
Remove item from queue
Signal that space is available
Why do we still need lock?
The semaphores control when the producer and consumer can continue.
But they do not protect the queue itself.
The queue is shared between the producer and the consumer:
private static readonly Queue<int> _queue = new();
Queue<T> is not thread-safe by itself. If two threads try to access it at the same time, we can get incorrect behavior.
That is why we use a lock:
private static readonly object _lock = new();
The lock statement makes sure that only one thread enters the protected block at a time. Microsoft’s C# documentation describes lock as a way to ensure that only a single thread exclusively reads or writes a shared resource. (Microsoft Learn)
So this code is protected:
lock (_lock)
{
_queue.Enqueue(i);
}
And this code is also protected:
lock (_lock)
{
item = _queue.Dequeue();
}
One important rule is that we should not use await inside a lock block. The C# documentation explicitly says that await cannot be used inside the body of a lock statement. (Microsoft Learn)
That is why this code is correct:
await _spaceAvailable.WaitAsync();
lock (_lock)
{
_queue.Enqueue(i);
}
The async waiting happens before the lock, not inside it.
Other options instead of lock
In this example, lock is the best simple option.
But there are other synchronization tools we can use.
Option 1: Monitor
In C#, lock is actually a cleaner syntax over Monitor.
This:
lock (_lock)
{
_queue.Enqueue(i);
}
is similar to this:
Monitor.Enter(_lock);
try
{
_queue.Enqueue(i);
}
finally
{
Monitor.Exit(_lock);
}
Microsoft’s documentation explains that Monitor.Enter and Monitor.Exit provide functionality equivalent to the C# lock statement, but the language syntax wraps the logic in a try/finally block so the monitor is released safely. (Microsoft Learn)
Monitor gives you more manual control.
For example, you can use Monitor.TryEnter if you do not want to wait forever:
if (Monitor.TryEnter(_lock))
{
try
{
_queue.Enqueue(i);
}
finally
{
Monitor.Exit(_lock);
}
}
Use Monitor when you need extra control, for example:
Try to enter without waiting forever
Use timeout logic
Manually control entering and exiting
For normal code, lock is usually better because it is shorter, cleaner, and less error-prone.
Option 2: Mutex
A Mutex also gives exclusive access to one thread at a time.
Example:
private static readonly Mutex _mutex = new();
_mutex.WaitOne();
try
{
_queue.Enqueue(i);
}
finally
{
_mutex.ReleaseMutex();
}
The important difference is that a Mutex can also be used across processes.
For example, if two different applications on the same machine need to access the same file, a named Mutex can help coordinate them.
private static readonly Mutex _mutex = new(false, "Global\\MyProducerConsumerMutex");
Microsoft’s documentation says that Mutex can provide exclusive access to a resource, uses more system resources than Monitor, and can synchronize threads in different processes. It also explains that named system mutexes are visible throughout the operating system and can be used to synchronize process activity. (Microsoft Learn)
Use Mutex when different processes need to coordinate access to the same resource.
For normal synchronization inside one application, prefer lock or Monitor.
Why this pattern is useful
The Producer-Consumer pattern is useful when one part of the system produces work and another part processes it.
Some real-world examples are:
Background job processing
Logging systems
File processing
Message queues
Image processing
Email sending
Report generation
Data pipelines
Server request handling
For example, imagine an application that receives uploaded files.
One part of the application accepts the files.
Another part processes them in the background.
Instead of processing everything immediately, the application can place each file into a queue.
Then one or more consumers can process the files one by one.
This keeps the application responsive.
One issue with the current code
The current consumer runs forever:
while (true)
That is okay for a learning example, but in real production code, we usually want a clean way to stop the consumer.
For example, we could use:
CancellationToken
A special final message
Channel completion
BlockingCollection.CompleteAdding()
Without a stopping mechanism, the consumer keeps waiting forever, even after the producer has finished.
Alternatives to this approach
The manual version is great for learning because it shows what happens behind the scenes.
But in real applications, .NET gives us higher-level tools.
Alternative 1: Channel<T>
For modern async code, Channel<T> is usually the best alternative.
System.Threading.Channels provides synchronization structures for passing data between producers and consumers asynchronously. Microsoft’s documentation describes channels as an implementation of the producer-consumer model, where producers asynchronously produce data and consumers asynchronously consume it through a FIFO queue. (Microsoft Learn)
Example:
var channel = Channel.CreateBounded<int>(5);
Producer:
await channel.Writer.WriteAsync(item);
Consumer:
var item = await channel.Reader.ReadAsync();
With Channel<T>, we do not need to manually combine:
Queue<T> + lock + SemaphoreSlim
The channel already handles much of that logic for us.
A bounded channel can also apply back pressure. When the channel reaches its capacity, the default behavior is that WriteAsync waits until space becomes available. (Microsoft Learn)
Use Channel<T> when you are building:
Async background workers
Job queues
Pipelines
Producer-consumer services
High-throughput async processing
For modern .NET applications, this is usually the cleanest option.
Alternative 2: BlockingCollection<T>
BlockingCollection<T> is another producer-consumer tool in .NET.
Microsoft describes BlockingCollection<T> as a thread-safe collection that implements the Producer-Consumer pattern, supports concurrent adding and taking from multiple threads, supports optional maximum capacity, and can block when the collection is empty or full. (Microsoft Learn)
Example:
var collection = new BlockingCollection<int>(boundedCapacity: 5);
Producer:
collection.Add(item);
Consumer:
var item = collection.Take();
This is simpler than manually writing semaphore logic.
However, BlockingCollection<T> is mainly blocking-based. That makes it useful for classic threaded code, but less ideal for modern async workflows.
For async code, Channel<T> is usually a better fit.
Alternative 3: ConcurrentQueue<T>
ConcurrentQueue<T> is a thread-safe FIFO queue.
It allows multiple threads to safely enqueue and dequeue items without manually using lock. Microsoft’s documentation describes it as a thread-safe first in, first out collection and states that its public and protected members are thread-safe and can be used concurrently from multiple threads. (Microsoft Learn)
Example:
private static readonly ConcurrentQueue<int> _queue = new();
Producer:
_queue.Enqueue(item);
Consumer:
if (_queue.TryDequeue(out var item))
{
Console.WriteLine(item);
}
This removes the need for a manual lock.
However, ConcurrentQueue<T> does not automatically wait when the queue is empty. It also does not automatically block producers when the queue is full.
So if you need waiting behavior, you still need another signal mechanism, or you can use Channel<T> instead.
Alternative 4: Semaphore
Semaphore is the older and heavier version compared with SemaphoreSlim.
Both are counting synchronization primitives, but Semaphore can be useful when system-wide synchronization is needed. Microsoft’s documentation explains that a Semaphore uses WaitOne, while SemaphoreSlim uses Wait or WaitAsync. (Microsoft Learn)
Use Semaphore when you need cross-process counting synchronization.
For normal async code inside one application, prefer SemaphoreSlim.
Which option should you use?
| Option | Best use case |
|---|
lock | Simple in-process protection of shared data |
Monitor | Same idea as lock, but with more manual control |
Mutex | Cross-process exclusive locking |
SemaphoreSlim | Async-friendly in-process counting synchronization |
Semaphore | Cross-process counting synchronization |
Queue<T> + lock + SemaphoreSlim | Learning how the pattern works internally |
Channel<T> | Modern async producer-consumer workflows |
BlockingCollection<T> | Classic blocking producer-consumer workflows |
ConcurrentQueue<T> | Thread-safe queue operations without waiting logic |
For this specific article, the manual approach is perfect because it teaches the mechanics.
For production code, especially in modern .NET, I would usually prefer Channel<T>.
Final summary
The Producer-Consumer pattern helps two parts of an application work together safely.
The producer creates work.
The consumer processes work.
The queue stores work temporarily.
The semaphores control when each side is allowed to continue.
The lock protects the shared queue from unsafe access.
In this example:
Producer -> Queue -> Consumer
_spaceAvailable protects the queue from becoming too full.
await _spaceAvailable.WaitAsync();
_itemsAvailable protects the consumer from reading an empty queue.
await _itemsAvailable.WaitAsync();
lock protects the queue itself.
lock (_lock)
{
_queue.Enqueue(i);
}
Monitor can be used when we need more manual control.
Monitor.Enter(_lock);
try
{
_queue.Enqueue(i);
}
finally
{
Monitor.Exit(_lock);
}
Mutex can be used when different processes need to coordinate access to the same resource.
private static readonly Mutex _mutex = new(false, "Global\\MyProducerConsumerMutex");
This is a simple but powerful pattern.
Once you understand it, tools like Channel<T>, BlockingCollection<T>, ConcurrentQueue<T>, Monitor, and Mutex become much easier to understand.
References
Microsoft documentation for SemaphoreSlim, Semaphore, lock, Monitor, Mutex, Channel<T>, BlockingCollection<T>, and ConcurrentQueue<T> was used as the technical reference for this article. (Microsoft Learn)
