[comp.mail.sendmail] Resolver initialization bug in sendmail.

rickert@mp.cs.niu.edu (Neil Rickert) (10/11/90)

 There is a potential problem in sendmail-5.6[45], relating to proper
initialization of the resolver library.  While my tests have been with the
IDA versions of sendmail I believe the problem to also be present in
the Berkeley released version.

 The error is that the resolver library is not properly initialized when
a freeze file is in use.  A patch is attached at the end of this message.

 Demonstrate the error as follows:

   (This assumes that the configuration file is /etc/sendmail.cf, and that
    the freeze file /etc/sendmail.fc exists).

 1.  Use the command:
	sendmail -bt -d8.8 -C/etc/sendmail.cf

     Now, in test mode, test any address which requires a resolver lookup,
     using the $[ ... $]  operations in sendmail.cf

     Notice that a great deal of debugging output is printed by the
     resolver library.

 2.  Repeat the test, this time using the freeze file
	sendmail -bt -d8.8

     With the same address, notice that the resolver debugging output does
     not appear.

Explanation:

    The '-d8.8' option should cause the debug options to be set in the
    resolver state variable, _res .  They are actually set whether a
    freeze file is used or not.  But when the freeze file is in use the
    resolver has not yet been initialized.  Thus the first call to the
    resolver library internally calls res_init(), which replaces the
    current value of _res with the default value.

    When the freeze file is not used, the resolver is called very early to
    canonicalize the internally defined value of '$w', and this call
    initializes the resolver library before the debug options are set.

Implications:

    The problem with debug is of course trivial.  If this were all that
    happened there would be no concern.  But the bug potentially effects
    any assignment to the state variable _res .

    Although I have not attempted to reproduce it, the following scenario
    demonstrates what could happen:

  Scenario:
    Imagine that the current default domain is 'foo.com'.  Assume, further,
    that there is a WILDCARD MX record for *.foo.com.

    Suppose now that sendmail uses a freeze file.  Assume mail is received
    from a UUCP neighbor, foobar.UUCP.  The destination of the mail is
    bar.BITNET.

    Because of the use of the freeze file, the resolver library is not
    consulted for the value of $w.  Because the sender address is from
    the UUCP domain, it is not processed by a $[ ... $] rule in
    the configuration file.  Likewise, because the destination is for
    the BITNET pseudo-domain, it also is not processed bt $[ ... $].
    But ruleset #0 has a specific rule for BITNET mail, something like

    R$+@$+.BITNET		$# tcp $@ CUNYVM.CUNY.EDU $: $1@$2.BITNET

    After ruleset 0 selects the 'tcp' mailer, the mail is processed by
    deliver.c.  To avoid wildcard MX problems, deliver.c carefully disables
    local domain qualification by setting a flag in _res .  Unfortunately
    the resolver has not been initialized, so this flag is internally
    reset during initialization, and getmxrr(), which is called from
    deliver.c, incorrectly matches CUNYVM.CUNY.EDU with the local
    wildcard MX record, resulting in the incorrect processing of the message.

*** main.c.orig	Fri Jul 20 16:19:20 1990
--- main.c	Wed Oct 10 14:55:48 1990
***************
*** 220,225 ****
--- 220,230 ----
  	UserEnviron[i] = NULL;
  	environ = UserEnviron;
  
+ #ifdef NAMED_BIND
+ 	/* Make sure the resolver library is initialized */
+ 	res_init();
+ #endif /* NAMED_BIND */
+ 
  # ifdef SETPROCTITLE
  	/*
  	**  Save start and extent of argv for setproctitle.

-- 
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
  Neil W. Rickert, Computer Science               <rickert@cs.niu.edu>
  Northern Illinois Univ.
  DeKalb, IL 60115.                                  +1-815-753-6940