piet@cs.ruu.nl (Piet van Oostrum) (05/22/91)
I have the following program that does a grep on a directory tree. On a
large tree (like the news spool directory) it keeps growing. So I suspect a
memory leak somewhere. Of course there is a recursive subroutine in it but
after a while it should IMHO stabilize when it has processed the deepest
directory. Instead it grows to more than 3MB.
#! /local/bin/perl
die "Usage: rgrep [-iredblL] regexp filepat ...\n rgrep -h for help\n"
if $#ARGV < $[;
# Written by Piet van Oostrum <piet@cs.ruu.nl>
# This is really free software
$nextopt = 1;
$igncase = '';
$regpat = 0;
$links = 0;
$error = 0;
$skipbin = 1;
$debug = 0;
do { $regexp = shift (@ARGV); } while &checkopt ($regexp);
$icreg = $igncase;
$igncase = '';
eval 'sub grep_file {
while (<F>) {
$ln++;
if (/$regexp/o' . $icreg .') {
print "$file:$ln:$_";
print "\n" if substr($_, -1, 1) ne "\n";
}
}
}';
for (@ARGV) {
if (! &checkopt ($_)) {
if ($igncase || $regpat || /[?*[]/ || ! -e) {
if ($regpat) {
s/#/\\#/g;
$_ = "#$_#";
} else { # translate File pattern into regexp
$re = '#($|/)'; $save = $_;
while (/[[*?+()|.^$#]/) {
$re .= $`;
$c = $&;
$_ = $';
if ($c eq '*') { $c = '[^/]*'; }
elsif ($c eq '?') { $c = '[^/]'; }
elsif ($c eq '[') {
if (/.\]/) { $c = "[$`$&"; $_ = $'; }
else {
$error++;
printf stderr "Illegal filepattern %s\n", $save;
}
} else { $c = "\\$c"; }
$re .= $c;
}
$_ = "$re$_\$#$igncase";
}
print "filepat: $_\n" if $debug;
push (@filepat, $_);
}
else { push (@files, $_); print "file: $_\n" if $debug; }
}
}
exit 1 if $errors ;
if ($#filepat < $[) {
eval "sub in_pat {1;}" ;
}
else {
$subtxt = 'sub in_pat { local ($f) = @_;';
$or = "";
for (@filepat) {
$subtxt .= $or . '$f =~ m' . $_;
$or = " || ";
}
$subtxt .= ';};1';
if (! eval $subtxt) {
print $@;
exit 1;
}
}
@files = (".") if $#files < $[;
for $file (@files) {
&do_grep ($file);
}
sub do_grep {
local ($file) = @_;
local (*F, $ln, $f, $g, @dirfiles);
if (-f $file) {
if (open (F, $file)) {
if (-B F) { # binary file -- may be compressed/compacted
if (($cx1 = getc(F)) eq "\377" && (getc(F) eq "\037")) {
open (F, "uncompact < $file|");
if ($skipbin && -B F) { close (F); return; }
}
elsif ($cx1 eq "\037" && (getc(F) eq "\235")) {
open (F, "uncompress < $file|");
if ($skipbin && -B F) { close (F); return; }
}
elsif ($skipbin) {
close (F); return;
}
}
print "Reading $file\n" if $debug;
&grep_file;
} else {
print stderr "Cannot open $file\n";
}
}
elsif (-d $file) {
print "Entering $file\n" if $debug;
if (opendir (F, $file)) {
@dirfiles = readdir (F);
closedir (F);
for $f (@dirfiles) {
next if ($f eq '.' || $f eq '..');
$g = "$file/$f";
next if (-l $g && ($links < 1 || $links == 1 && -d $g));
if (-f $g && &in_pat ($g) || -d _) {
&do_grep ($g);
}
}
} else {
print stderr "Can't open $file\n";
}
}
}
sub checkopt {
local ($_) = $_[0];
if (/^-/ && $nextopt) {
$nextopt = 1;
@opt = split (/-*/,$_); shift (@opt);
for $opt (@opt) {
if ($opt eq 'i') { $igncase = 'i'; }
elsif ($opt eq 'd') { $debug = 1; }
elsif ($opt eq 'l') { $links = 1; }
elsif ($opt eq 'L') { $links = 2; }
elsif ($opt eq 'b') { $skipbin = 0; }
elsif ($opt eq 'r') { $regpat = 1; }
elsif ($opt eq 'e') { $nextopt = 0; }
elsif ($opt eq 'h' || $opt eq 'H') { & help; }
else { $error++; printf stderr "Unknown option -%s\n", $opt; }
}
return 1;
}
$nextopt = 1;
return 0;
}
sub help {
print <<'HELP'; exit 0;
Usage: rgrep [-iredblL] regexp filepat ...
regexp = perl regular expression to search
filepat ... = a list of files and directories to be searched or
file patterns to match filenames.
filepat will be interpreted as file or directory name if it exists
as such, and does not contain the metacharacters [ ] ? or *. After
the options -i and -r all filepats will be considered patterns.
rgrep will search all files in any of the directories given (and its
subdirectories) that match any of the filepats, except binary files.
Compressed files will be searched in uncompressed form.
Note: filepats may contain / contrary to find usage.
-b Don't skip binary files.
-i Ignore case, either in the regexp or in filename matching (depending
on the location). Before the regexp only applies to the regexp,
otherwise to the filepats following it.
-r The following filepats are treated as real perl regexps rather than
shell style filename patterns. In this case / is not a special
character, i.e. it is matched by . and matching is not anchored (you
must supply ^ and $ yourself). E.g. a.b matches the file /xa/by/zz.
-l Do follow symbolic links only for files (default is do not follow).
-L Do follow symbolic links for files and directories.
-e Do not interpret following argument as option. Useful if regexp or
filepat starts with a -.
-d Debugging: Give a lot of output on what happens.
-h print this message and exit.
Piet van Oostrum <piet@cs.ruu.nl>
HELP
}piet@cs.ruu.nl (Piet van Oostrum) (05/22/91)
Sorry, I forgot to mention which version etc. This is on HP-UX 7.05 on a HP9000/425 workstation. This is perl, version 4.0 $RCSfile: perl.c,v $$Revision: 4.0.1.1 $$Date: 91/04/11 17:49:05 $ Patch level: 3 -- Piet* van Oostrum, Dept of Computer Science, Utrecht University, Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. Telephone: +31 30 531806 Uucp: uunet!mcsun!ruuinf!piet Telefax: +31 30 513791 Internet: piet@cs.ruu.nl (*`Pete')
lm@slovax.Eng.Sun.COM (Larry McVoy) (05/24/91)
piet@cs.ruu.nl (Piet van Oostrum) writes:
[stuff about memory leak]
If you use the malloc that comes with perl (a slightly tweaked version
of the later BSD malloc) you will use a lot of space. That malloc does
a power of two buddy system (i.e., very fast, it's about 60 instructions
for malloc and 20 instructions for free on average on a sparc) and does
no coalescing. It typically wastes about 50%.
Try linking w/ the system malloc and see if the problem goes away.
Note: this is nothing against the BSD malloc, it has traded off space
for time and in perl's case that is a smart tradeoff.
---
Larry McVoy, Sun Microsystems (415) 336-7627 ...!sun!lm or lm@sun.com