SMP Theory (was: Re: Interesting analysis of linux kernel threading by IBM)

[David Schwartz]

> OK, if you have an n-way SMP box, then you have n processors with
> n (local)
> caches sharing a single block of main system memory. If you then run a
> threaded program (like a renderer) with a thread per processor,
> you wind up
> with n threads all looking at a single block of shared memory - right?
>
> OK, if a thread accesses (I assume writes, reading isn't destructive, is
> it?) a memory location that another processor is "interested" in, then
> you've invalidated that processor's local cache - so it has to be flushed
> and refreshed. Have enough cross-talk between threads, and you can achieve
> the worst-case scenario where every memory access flushes the cache of
> every processor, totally defeating the purpose of the cache, and perhaps
> even adding nontrivial cache-flushing overhead.
>
> If this is indeed the case (please correct any misconceptions I have) then
> it strikes me that perhaps the hardware design of SMP is broken. That
> instead of sharing main memory, each processor should have it's own main
> memory. You connect the various main memory chunks to the
> "primary" CPU via
> some sort of very wide, very fast memory bus, and then when you spawn a
> thread, you instead do something more like a fork - copy the relevent
> process and data to the child cpu's private main memory (perhaps via some
> sort of blitter) over this bus, and then let that CPU go play in its own
> sandbox for a while.

	If you wanted completely separate memory spaces for each processor, the
current hardware will let you have it. Just separate the address space into
logical chunks and code each processor only to use its chunk. The current
hardware design lets you do exactly what you are suggesting. And if one
processor does need to acceess the memory earmarked for another, the current
hardware provides a fast way to do it.

	DS


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo en vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/