[comp.lang.icon] sorting files larger than physical memory

talmage@luvthang.aquin.ori-cal.COM (David W. Talmage) (06/05/91)

Once upon a time, Richard Goerwitz posted some text indexing code he
called gettext().  I've been using it and trying to improve it.  One
thing I'm working on is sorting the keys in the index files so
searches for keys are faster.  This presents a problem when the keys
are too many to fit in available memory at one time.  I'm looking for
solutions to this problem.

My solution, for the time being, is to read the keys in batches that
fit in my limited memory, sorting each batch with the built-in
function sort().  Each sorted batch I write to a temporary file.  When
all batches are sorted, I merge the temporary files one record at a
time in much the same way as you'd merge piles of sorted index cards.

I welcome comments and code from anyone.

-----------------------------------------------------------------------------
David W. Talmage (talmage@luvthang.aquin.ori-cal.com)
"I need fifty dollars to make you hollar.  I get paid to run this luvthang."