vixie@decwrl.dec.com (Paul A Vixie) (06/19/91)
when i explode a message to a large mailing list and half the recipients are all going to people inside the same remote organization but with addresses in different subdomains such that i'm going to use the same MX for all of them, i end up sending N copies of the mail since M...F=m only works if all the S0 $@'s have the same string in them. i'm about to hack on King James Sendmail to deal with this problem, but it's going to be a pretty significant change since i'll have to search for all the MX's fairly early and save the results so that they can be part of the sort/ merge operation. does anyone have any special requests as to other things i should do while i'm in there, or does anyone have a suggestion as to how i ought (not) to implement this? i'm going to add the ability to keep an external host status database at the same time. i run a lot of parallel sendmail daemons here and they currently do not benefit from eachother's experiences as to unreachable hosts et al. (they may also be keeping their database in terms of $@ names instead of actual hostnames a la MX, which is what makes me think of this just now). cheers, -- Paul Vixie DEC Western Research Lab <vixie@pa.dec.com> <paul@vixie.sf.ca.us> Palo Alto, California, USA ...!decwrl!vixie ...!vixie!paul
Craig_Everhart@TRANSARC.COM (06/19/91)
I did a similar thing to sendmail 5.61 some years back; I called it MX-piggybacking. I can give you the implementation, but it's more limited than you wanted. It does work well, though, and helps a lot on big address lists. The version that I built assumes a preprocessor on address lists that sorts recipients by MX. Instead of doing multiple deliveries when the target domain names are identical, it does multiple deliveries when the sorted arrays of MX receiver names are identical. It keeps only a one-element cache, so it does only one-name lookahead. That is, it keeps two vectors of MX hosts: the current one (corresponding to the ``current'' host/target domain), and the single lookahead one (corresponding to the lookahead target). If the MX vectors are identical, it will piggyback the lookahead recipient onto the current list, and look ahead one more time. It treats the absence of any MX records as identical to having a single MX record to the destination domain. This was a case I missed in the initial implementation. It also randomizes MX elements with identical preferences. The only trick here is that to keep from defeating your MX-piggybacking, you want to randomize the same set of domain names in the same order on any given sendmail run, but you do want to randomize the same set of domain names in different orders on different runs. I do this by taking a snapshot of time if a static variable is still zero, and using that snapshot and the domain name in the MX (in case-folded form) to come up with a signature value, then sorting by signature value. It's tricky to get the temp-failure and persistent-failure cases right as you look ahead for MX results. Clearly, this is important, and it's what I concentrated on in my first version. In my use, the single look-ahead buffer works just great, because AMDS looks up hosts also before sending to them (so it can maybe issue rejections for host-not-found without de-formatting any ATK-format mail), and it sorts the recipients. I recently found a weakness of its method, though: it sorts only on the first named MX receiver, not all the MX receivers, so MX-piggybacking is sort of defeated for mail that goes through *.uu.net, since there is currently a lot of variation in the MX records there. (Most mail currently goes to relay1.uu.net and relay2.uu.net, but some goes to news.uu.net and/or other hosts.) Let me know if you want to see my implementation. Craig
vixie@decwrl.dec.com (Paul A Vixie) (06/19/91)
craig, if i wanted to accomplish the task you set for yourself, it sounds like your "mx piggybacking" hacks would be just the right thing. i suspect that since i'm trying to rationalize the whole deliver() function, that your diffs are not going to be directly useful. however, given that you have dealt with lots of the icky boundary conditions, i would like to go over your code to make sure i know what kind of problems i'm dealing with. can you mail it to me or post it here? paul -- Paul Vixie DEC Western Research Lab <vixie@pa.dec.com> <paul@vixie.sf.ca.us> Palo Alto, California, USA ...!decwrl!vixie ...!vixie!paul
fair@Apple.COM (Erik E. Fair) (06/24/91)
While you're at it, can you change sendmail so that it delivers all E-mail bound for the same host (or same MX) in one connection, rather than in 20 (for 20 different letters)? Erik E. Fair apple!fair fair@apple.com