CPU swapping on Linux

[Mark Hahn]

> > My tests on one of those dual celerons show that in the extreme case of a set
> > of processes that use all the available cache Linux is still far too eager to
> > swap processes across CPU's.

this is unsurprising, since AFAIKS, the cost calculation is quite wrong.

current machines deliver around 350 MB/s of dram bandwidth [*],
which means that the machine I'm sitting at (dual celeron 450) will
take around 375 us to load or flush the cache.  but the scheduler is 
using an estimate of 25 us!

in arch/i386/kernel/smpboot.c:

	cacheflush_time = cpu_hz/1024*cachesize/5000;

which, if I'm reading it right, assumes ~5 MB/s dram bandwidth.
the calculation should be

	cacheflushInCycles = cpuHz * cachesizeBytes / bytesPerSecond;

a better calculation would be:

	static const unsigned bandwidth = 350000000;  
	cacheflush_time = cpu_hz * (cachesize*1024) / bandwidth;

or, overflow-obscurified:
	cacheflush_time = (cpu_hz>>20) * (cachesize<<10) / (bandwidth>>20)

with this fix, you should see a little more reluctance to bounce 
processes across CPUs.  in fact, the heuristic itself should probably
be applied to goodness() in general, even for uniprocessors, since 
this heuristic is currently only used in the SMP reschedule_idle.

tweaking of PROC_CHANGE_PENALTY and p->mm==this_mm weights might help too.

regards, mark hahn.

* the bandwidth figure actually ranges from around 150 MB/s for 
socket7/super7 machines, to around 500 MB/s for good high-end ia32's
(PC133/rambus, Athlon), to ~1.2 GB/s for 264 alphas *slobber*.)



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo en vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/