mjs@behemoth.genetics.wisc.edu (Mike Schmelzer) (04/28/91)
I'm sorry, I should have said that the big sort I'm crashing on is 16M, but I'll need to handle 29M and maybe more. Thanks! -- ==== Mike Schmelzer, UW Genetix, mjs@genetics.wisc.edu, (608)263-7459. === ========= Schmelzer's Law: You can have anything you don't want. ========= ================ "How can hatred uplift a race?" - 3rd Bass ==============
mjs@behemoth.genetics.wisc.edu (Mike Schmelzer) (04/28/91)
I'm trying to sort a large file (actually a stream) that consists of 6 letters per line. I'm running a Sparcstation 1+ SunOS 4.1 w/ 24M real memory and ~75M swap space. When I try to pipe this stream into sort(1), it eventually dies with a '/: write failed, file system is full' Thinking "aha! sort needs more scratch space!", I linked /tmp to a partition with 48Meg free and ran sort again. Now here's the tricky part: It still bombs with a write error to '/'. Why?!? Does sort use something other than /tmp for scratch space? If /tmp is soft-linked somewhere else, and that runs out of room, does it get reported as a failure to write to '/' rather than '/mnt/bigpartition' ? Most importantly: How can I sort BIG files like this? I'm ready to sacrifice speed to accomplish this. (I'm ready to sacrifice *anything* to accomplish this, but that's another story.) Please mail your reply, and I will post a summary. Thanks! -- ==== Mike Schmelzer, UW Genetix, mjs@genetics.wisc.edu, (608)263-7459. === ========= Schmelzer's Law: You can have anything you don't want. ========= ================ "How can hatred uplift a race?" - 3rd Bass ==============
mjs@behemoth.genetics.wisc.edu (Mike Schmelzer) (04/28/91)
Okay, here's what was going on: sort(1) tries to use /usr/tmp (which is on /) instead of /tmp (which I had doinked around with so that it was on /mnt/bigpartition). Why sort does is is a mystery to me. Fortunately, sort takes a -T argument, which allows you to specify where to put sort's scratchfiles. So, all'z I had to do was sort -T /mnt/bigpartition/scratch, and it worked like a charm. Here endeth the lesson. -- ==== Mike Schmelzer, UW Genetix, mjs@genetics.wisc.edu, (608)263-7459. === ========= Schmelzer's Law: You can have anything you don't want. ========= ================ "How can hatred uplift a race?" - 3rd Bass ==============
gwyn@smoke.brl.mil (Doug Gwyn) (04/30/91)
In article <MJS.91Apr27192345@behemoth.genetics.wisc.edu> mjs@behemoth.genetics.wisc.edu (Mike Schmelzer) writes: >sort(1) tries to use /usr/tmp (which is on /) instead >of /tmp (which I had doinked around with so that it was >on /mnt/bigpartition). Why sort does is is a mystery to >me. Fortunately, sort takes a -T argument, which allows >you to specify where to put sort's scratchfiles. /usr/tmp is the usual default directory for temporary files, and is normally expected to have more space in it than /tmp. /tmp should be used only by critical system software, not for potentially huge user temporary files (such as are created during merge sorting). You might check whether or not your utilities honor the TMPDIR environment variable.
Nicole Delbecque & Paul Bijnens <FFAAC09@cc1.kuleuven.ac.be> (05/05/91)
In article <16010@smoke.brl.mil>, gwyn@smoke.brl.mil (Doug Gwyn) says: >/usr/tmp is the usual default directory for temporary files, and is >normally expected to have more space in it than /tmp. /tmp should >be used only by critical system software, not for potentially huge >user temporary files (such as are created during merge sorting). Relating to this: vi makes its temporary files in /tmp. With 'set dir=/usr/tmp' in EXINIT (or .exrc) I can make vi put it's temporary files there, like I want it (users edit HUGE files on our system with vi). However, if the system crashes, then in the start-up script when going from single to multi-user mode, the program '/usr/lib/expreserve -' tries to make the best of the temporary files from vi/ex in /tmp and makes them recoverable in the directory /usr/preserve. 'expreserve' does not look in /usr/tmp. There is nothing in the manual about the options to 'expreserve', e.g. what is the meaning of the single dash... (I can guess...). There isn't even anything in the manual about expreserve. Is there some way to indicate to 'expreserve' where to look for the cadavers of vi/ex-temporary files? At this moment I do: ... mount filesystems ... mv /usr/tmp/[ER]x[0-9][0-9][0-9][0-9][0-9] /tmp /usr/lib/expreserve - However, there may not always be enough place in /tmp to hold all these files. Just lucky we never crash... :-) Any better suggestion? If it matters: SYSV various flavors here. Vi "ver" says: Version 3.7 10/31/81 on SYSV2 old Version 3.9 2/9/83 on SYSV2 newer Version SVR3.1 on SYSV3 -- Polleke (Paul Bijnens) Linguistics dept., K. University Leuven, Belgium FFAAC09@cc1.kuleuven.ac.be