garvey@cmic.UUCP (Joe Garvey) (02/12/90)
I was in the middle of creating a perl script for archiving off certain files
plucked from the news, when I got the message "Arguments too long". The
directory I was in had several hundred files (I need the disk space back,
so it's time to write our own little news archiving tool). Here is a little
demo script that creates the problem. On small directories, this works great.
---------------------------------------
#!/usr/contrib/bin/perl
$dir = "/users/garvey/tmp/archive/tmp";
foreach $afile (<$dir/*>) {
print ($afile, "\n");
}
exit 0;
--------------------------------------
*** Question ***
Is there an easy way around this? There seems to be a limit built into the
<$dir/*> of about 250 files (no it's not 256). Have I done something wrong?
Why would there be a limit? I'm sure I'm not running out of memory,
and Larry says in the first page of the man-page "Unlike most Unix utilities,
perl does not arbitrarily limit the size of your data -- if you've go the
memory". What gives?
perl -v yeilds:
$Header: perly.c,v 3.0.1.3 89/12/21 20:15:41 lwall Locked $
Patch level: 8
E-mail replies preferred.
--
Joe Garvey UUCP: {apple,backbone}!versatc!mips!cmic!garvey
California Microwave Internet: garvey%cmic@mips.com
990 Almanor Ave HP Desk: garvey (cmic@mips.com) /hp1900/ux
Sunnyvale, Ca, 94086 800-831-3104 (outside CA)
408-720-6439 (let it ring) 800-824-7814 (inside CA)tchrist@convex.COM (Tom Christiansen) (02/12/90)
In article <237@cmic.UUCP> garvey%cmic@mips.com writes: >I was in the middle of creating a perl script for archiving off certain files >plucked from the news, when I got the message "Arguments too long". > >Is there an easy way around this? There seems to be a limit built into the ><$dir/*> of about 250 files (no it's not 256). Have I done something wrong? >Why would there be a limit? I'm sure I'm not running out of memory, >and Larry says in the first page of the man-page "Unlike most Unix utilities, >perl does not arbitrarily limit the size of your data -- if you've got the >memory". What gives? The problem is that perl is using a shell to glob the pattern. There is a basic limitation on the length of the command args imposed by your shell, your exec system call, or both. What this actually works out to be varies from system to system. On my system, the real number is #define NCARGS 3*NBPG /* # characters in exec arglist */ which for me is 3*4k, or 12168 bytes. Your string is already 29 bytes long, add one for another slash, and now each file that the star expands into takes another 30+length($afile) bytes. The workaround (until and if Larry builds globbing into perl without having to call the shell) is to recode using the opendir() built-ins if you have them. Instead of: $dir = "/users/garvey/tmp/archive/tmp"; foreach $afile (<$dir/*>) { print ($afile, "\n"); } use the virtually equivalent (but not snagged by NCARGS problems): opendir(DIRECT, $dir); foreach $afile (sort grep (!/^\./, readdir(DIRECT))) { print $dir, '/', $afile, "\n"; } closedir(DIRECT); This is actually more powerful than the <*.*> method, because you can do perl-style globbing with grep. >E-mail replies preferred. And duly sent. --tom -- Tom Christiansen {uunet,uiucdcs,sun}!convex!tchrist Convex Computer Corporation tchrist@convex.COM "EMACS belongs in <sys/errno.h>: Editor too big!"
gnb@bby.oz.au (Gregory N. Bond) (02/13/90)
$dir = "/users/garvey/tmp/archive/tmp";
foreach $afile (<$dir/*>) {
print ($afile, "\n");
}
Yep, an arg length problem. You can increase the number of names b
doing something like
$dir="/users/garvey/tmp/archive/tmp";
chdir($dir);
foreach $afile (<./*>) {
print $afile, "\n";
}
which will avoid a copy of $dir in each expansion and (approx!) triple
the number of directories matched in this case.
Greg.
--
Gregory Bond, Burdett Buckeridge & Young Ltd, Melbourne, Australia
Internet: gnb@melba.bby.oz.au non-MX: gnb%melba.bby.oz@uunet.uu.net
Uucp: {uunet,pyramid,ubc-cs,ukc,mcvax,prlb2,nttlab...}!munnari!melba.bby.oz!gnb