sysmark@physics.utoronto.ca (Mark Bartelt) (06/05/91)
Several months ago I posted something describing problems with tools
like "du" and "find", which not only were taking much longer than one
would expect, but were also racking up enormous amounts of CPU time
(sys time, not user time) on a 4D/280. After that system was upgraded
from IRIX 3.3.1 to 3.3.2, I'd thought at first that there had been some
improvement, but it seems I was mistaken; things are still quite wretched.
In any case, here are some interesting numbers. They show the results
of doing
/bin/time find . -type f -print | wc -l
on some large hierarchies (4000-8000 files), and dividing the sys time
by the number of files found ...
DEC 5000 (Ultrix 4.1, 32mb) 0.8 milliseconds
Sun Sparc-1+ (SunOS 4.1, 16mb) 1.7 milliseconds
SGI 4D/25 (IRIX 3.3.1, 16mb) 2.5 milliseconds
VAX 750 (4.3 BSD, 5mb) 7.7 milliseconds
SGI 4D/280 (IRIX 3.3.2, 224mb) 146.6 milliseconds (!!!)
What on earth is going on in the kernel? I'd guess that it's spending
a lot of time in namei(), but *why*? Some horribly botched algorithm
for doing cache searches? (Aren't these things hashed?) Note that on
the 4D/25, things are OK, which leads me to suspect that the problem
is related somehow to the size of the name cache (how big are these by
default on a 16mb and a 224mb system, anyhow?), and that the problem is
characterized by the fact that the time required to do an unsuccessful
search of the cache, as a function of cache size, is a lot worse than
linear. Just a guess, but what else could be causing this?
As I mentioned last January (when I first discovered the problem), this
problem means that tools like "find" and "du" are essentially useless on
large hierarchies, at least on our 280. Does anyone from SGI know what's
wrong here? And is it fixed in 4.0 (or, for that matter, 3.3.3)?
Mark Bartelt 416/978-5619
Canadian Institute for mark@cita.toronto.edu
Theoretical Astrophysics mark@cita.utoronto.ca