Interesting analysis of linux kernel threading by IBM

[Peter Rival]

Mark Hahn wrote:

> > bigger thing for Alpha than for Intel).  On our newer systems we fully expect
> > hundreds, if not thousands, of tasks.  The more commercially accepted Linux
>
> the issue is *runnable* tasks.  do your machines routinely report
> loadaverages of 1000?  if so, I'm impressed!
>

Yes, I did get that. :)  And yes, they do routinely report such load averages.  As
a matter of fact, I was just stress-testing a system and when I looked the load
average was 2002.  Granted the test I was running (AIM) is artificial (all tests
of its ilk are), it was designed as a representative measure of system
performance.  Representative in that it represents what systems actually do.

Do we here have systems with that high a load average from actual use?  No - but
we're not running any massive databases or web servers either.  Do I have hard
proof that our customers do that?  I doubt anyone would tell me if they did anyway
;)

Point is, it's possible, and it's becoming more and more probable every day.  (I
had a co-worker testing a system and he gave up at a load average of somewhere
around 16000 because he didn't want to wait any longer.)  Remember - this is the
age of server consolidation - more stuff on fewer systems.  Lots of little piddly
things on tiny boxen scattered here there and everywhere was the NT way and it has
proven to not work (on a large scale at the very least).

>
> the issue here is whether someone can come up with a maintainable
> scheduler that has the requisite performance.  since the runqueue is
> normally short, the scheduler's performance function must have a
> very small constant term.  if it's true that there are applications
> that result in long runqueues, then the performance curve needs to
> be as flat and horizontal as possible, again, without degrading the
> constant term.
>

Agreed.  That's one of the reasons that Tru64 has a per-cpu runqueue backed by a
global runqueue (I believe everyone that cares to would know that, so I think
I can say that ;).  I know such an idea is never going to make it into Linux, but
it's in the direction we should be looking.  I haven't looked at the scheduler
lately... maybe I should again...hmm...

>
> AFAIK, loopback volanomark does not resemble _any_ real application.
>

No, probably not technically.  It just means that they didn't have to configure a
big system with a whole bunch of clients and enough bandwidth to put the same type
of load on the system.  With the advent of multi-hundred MB/s plus Internet
connections and massive Intranet requirements, such bandwidth isn't impossible to
imagine.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo en vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/