Parallel Computing Simplified - Starring Gordon Ramsay
My girlfriend is currently working on a degree in Computer Science. I do my best to butt out, but sometimes when I see concepts and topics covered VERY poorly, I have to intervene. Last quarter, her classes covered topics such as parallel computing, task scheduling, and other OS-level concepts. These concepts are notoriously confusing for novice programmers, and her teachers weren't helping that situation to say the least. I spent some time creating a digestible narrative and figured it's worth sharing
Hell's Kitchen (I promise this is going somewhere)
Hell's Kitchen (a terrible reality show about cooking) is a guilty pleasure at my house. For those of you who aren't familiar with the show, each season starts with two teams of chefs, a red team and blue team. Each night, they cook in separate kitchens, and whichever team does worse, loses a member permanently (calm down, they don't kill them). Below I've crudely drawn a picture of what this looks like.
There are a few things you should know when you're cooking in Hell's kitchen.
- The "pass" is the counter where waiters bring orders to the kitchen. In Hell's Kitchen, only Gordon Ramsay can call out orders to the chefs. This means his time must be split between the red kitchen and the blue kitchen.
- Orders are done on a per-table basis. All dishes are cooked to order and must complete at the same time (otherwise the food will get cold).
The process looks like this:
- A waiter brings a table's "ticket" (order) to the pass.
- Gordon Ramsay validates the ticket (makes sure it's a valid order). He then calls out the ticket items to the respective kitchen. He DOES NOT address individual chefs, they are expected to know what they are responsible for when they hear it.
- Chefs begin cooking, while coordinating between themselves to ensure they finish their dishes at the same time.
- All dishes are brought up to the pass. Gordon Ramsay checks each dish for quality control.
- Assuming each dish is correct (remember, if one goes back, they all go back), Gordon now rings a bell which tells the waiters that an order is ready.
The Hell's Kitchen Pipeline
If you understand the last section, you already understand the hardest concepts behind parallel computing. Let's quickly define the concepts we want to learn.
Parallel - Multiple things happening in the same moment of time
Async - Something you start but don't sit and wait to be done
Concurrent - Multiple tasks being submitted without waiting for the previous task to complete (not a single file line).
Parallel
The easiest is parallel. Many things are happening in parallel in Hell's Kitchen. The chefs in the blue team are all cooking their dishes at the same time, this is parallelism. The two kitchens are both cooking orders at the same time too, that's parallelism. The waiters are taking orders while the chefs are cooking, the waiters are therefore working in parallel with the chefs. Ask the following question to determine if something is parallel:
Would I need a clone of myself to complete the task?
If the answer is yes, it's most likely parallel
Concurrent
Concurrent and parallel are the easiest for people to conflate. Concurrency means that in some period of time, multiple things are being done. The human ability to multi-task is concurrent and not parallel because you're not able to literally do "two things at once".
In Hell's Kitchen, Gordon Ramsay is working concurrently when he goes back and forth between the kitchens inspecting the food. He's not working in parallel because that would be impossible, there is only one Gordon Ramsay. This is an example where something is concurrent but not parallel because...
He can either be on the left side of the kitchen...
Or the right side...
But never both sides! When concurrency is implemented in this manner, it is know as "interleaved concurrency".
Parallelism and concurrency are closely related because parallelism is one method of achieving concurrency. If Gordon Ramsay decided to hire a new head chef that managed the blue kitchen, while he managed the red kitchen, that would be both concurrent and parallel.
Async
Asynchronous tasks are a bit tricky for people. Asynchronous simply means that you do not wait for something to complete after you start it.
In Hell's Kitchen, waiters brings tickets to the pass so meals can be cooked. They do not stand and wait for those meals to be done before going back into the dining room. Instead, they rely on a bell to know when the order is ready.
People are often confused by the relationship between parallelism/concurrency and asynchronous tasks.
Consider this, Gordon Ramsay could decide that instead of the teams cooking and waiters taking orders, he will do it all himself. He also decides to continue managing the pass and collecting tickets (he is Gordon Ramsay after all). As Gordon Ramsay is only one person, this change immediately removes all parallelism from the restaurant. But as long as Gordon Ramsay doesn't bring an order to the pass and stand and wait for it to be done (he's the only person working so that would be silly), it's asynchronous.
As a note, if Gordon were to stand and wait for himself to cook a meal, it would be synchronous
To summarize, asynchronous tasks require at least concurrency but not parallelism, although that works too.
Conclusion
Parallel - Do you need a clone?
Concurrent - Can you submit multiple tasks without waiting for the previous one to complete?
Asynchronous - Not sitting and waiting for something to complete
If this article was helpful, I have quite a few other concepts that I've integrated into the analogy and wouldn't mind creating another post.
