[comp.protocols.tcp-ip] VERY strange behavior at USENET/BITNET gateway

jbn@glacier.STANFORD.EDU (John B. Nagle) (01/07/89)

      The following message was returned to me by mail, in response to
a USENET posting to comp.society.futures.  Somehow, a strange form
of address translation was applied to the CONTENT of the message, resulting
in an attempt to interpret the 1988 election returns as Internet
addresses.

					John Nagle

   ====
From @cunyvm.cuny.edu:MAILER-DAEMON@wisdom.bitnet Thu Jan  5 20:35:12 1989
Received: from labrea.Stanford.EDU by glacier.stanford.edu with TCP; Thu, 5 Jan 89 20:35:02 PST
Received: from cunyvm.cuny.edu by labrea.stanford.edu with TCP; Thu, 5 Jan 89 20:34:56 PST
Message-Id: <8901060434.AA15336@labrea.stanford.edu>
Received: from wisdom.bitnet by CUNYVM.CUNY.EDU (IBM VM SMTP R1.1) with BSMTP id 7800; Thu, 05 Jan 89 23:33:43 EST
From: Mail Delivery Subsystem <MAILER-DAEMON%WISDOM.BITNET@cunyvm.cuny.edu>
Date: 4 Jan 89 17:40:03 GMT
Subject: Returned mail: Unable to deliver mail
Received: from cunyvm.bitnet by wisdom.bitnet; Fri, 6 Jan 89 06:32:46 -0200
To: jbn%glacier@labrea.stanford.edu
Status: R

   ----- Transcript of session follows -----
554 "|/usr/new/fanews fa.futures futures-list"... Address too long: Bad file num
554 "|/usr/new/fanews fa.futures futures-list"... Address too long: Bad file num

   ----- Unsent message follows -----
Received: from cunyvm.bitnet by wisdom.bitnet; Fri, 6 Jan 89 06:32:46 -0200
Received: from CUNYVM by CUNYVM.BITNET (Mailer R2.01) with BSMTP id 4248; Thu,
 05 Jan 89 23:32:21 EST
Received: from multimax.encore.com by CUNYVM.CUNY.EDU (IBM VM SMTP R1.1) with
 TCP; Thu, 05 Jan 89 23:32:03 EST
Received:  by multimax.encore.com (5.59/25-eef)
    id AA03285; Wed, 4 Jan 89 14:45:29 EST
Received: from UCBVAX.Berkeley.EDU by multimax.encore.com (5.59/25-eef)
    id AA03266; Wed, 4 Jan 89 14:45:08 EST
Received: by ucbvax.Berkeley.EDU (5.61/1.33)
    id AA25695; Wed, 4 Jan 89 11:15:08 PST
Received: from USENET by ucbvax.Berkeley.EDU with netnews
    for info-futures-mail@encore.com (info-futures@encore.com)
    (contact usenet@ucbvax.Berkeley.EDU if you have questions)
Date: 4 Jan 89 17:40:03 GMT
From: jbn%glacier@wisdom.bitnet  (John B. Nagle)
Organization: Stanford University
Subject: Election results (was re: voting system)
Message-Id: <17973@glacier.STANFORD.EDU>
References: <2604@ficc.uu.net>, <292@gloom.UUCP>
Sender: info-futures-request@wisdom.bitnet
To: info-futures@encore.com


 Here is the final national vote tally for the 1988 presidential
 election:

 Candidate            Party                             Votes     %
 ==================================================,
        ===================George.Bush.Republican.48@wisdom.bitnet,
        881@wisdom.bitnet,
        011.53.37.Michael.Dukakis.Democratic.41@wisdom.bitnet,
        828@wisdom.bitnet, 350.45.67.Ron.Paul.Libertarian.431@wisdom.bitnet,
        499.0.47.Lenora.Fulani.New.Alliance.218@wisdom.bitnet,
        159.0.24.David.Duke.Populist.48@wisdom.bitnet,
        267.0.05.Eugene.McCarthy.Consumer.30@wisdom.bitnet,
        510.0.03.James.Griffin.American.Independent.27@wisdom.bitnet,
        818.0.03.Lyndon.LaRouche.National.Economic.Recovery.25@wisdom.bitnet,
        082.0.03.William.Mara.Right.to.Life.20@wisdom.bitnet,
        497.0.02.Ed.Winn.Workers.League.18@wisdom.bitnet,
        579.0.02.James.Warren.Socialist.Workers.13@wisdom.bitnet,
        338.0.01.Herbert.Lewin.Peace.and.Freedom.10@wisdom.bitnet,
        312.0.01.Earl.Dodge.Prohibition.7@wisdom.bitnet,
        984.0.01.Larry.Holmes.Workers.World.7@wisdom.bitnet,
        719.0.01.Willa.Kenoyer.Socialist.3@wisdom.bitnet,
        800.0.00.Delmar.Dennis.American.3@wisdom.bitnet,
        456.0.00.Jack.Herer.Grassroots.1@wisdom.bitnet,
        949.0.00.Louie.Youngkite.Independent.372.0.00.John.Martin.Third.World.As
        934.0.01[Statistics.quoted.from.the.Associated.Press]@wisdom.bitnet

david@ms.uky.edu (David Herron -- One of the vertebrae) (01/07/89)

I've seen things like this happen if the blank line(s) between the
header and the body weren't truly blank but had spaces in it.

Something which happens (at least with UREP) is that files coming
in from BITNET-land have lines which are supposed to be truly
empty instead contain some number of blanks.  You would think this
would be non-destructive, but the header parser in the news software
gets confused and thinks that the first part of the body is actually
part of the header.

On a related note, sending out files with truly empty lines cause
those "records" to disappear.
-- 
<-- David Herron; an MMDF guy                              <david@ms.uky.edu>
<-- ska: David le casse\*'      {rutgers,uunet}!ukma!david, david@UKMA.BITNET
<-- Now I know how Zonker felt when he graduated ...
<--          Stop!  Wait!  I didn't mean to!

IRWIN@pucc.Princeton.EDU (Irwin Tillman) (01/08/89)

In article <10836@s.ms.uky.edu>, david@ms.uky.edu (David Herron -- One of the vertebrae) writes:

>I've seen things like this happen if the blank line(s) between the
>header and the body weren't truly blank but had spaces in it.
>
>Something which happens (at least with UREP) is that files coming
>in from BITNET-land have lines which are supposed to be truly
>empty instead contain some number of blanks.  You would think this
>would be non-destructive, but the header parser in the news software
>gets confused and thinks that the first part of the body is actually
>part of the header.

Yup.  I've found that the following situation will exhibit the
problem.  (I've tried it with inews, version B 2.11, patchlevel 14.)

Feed inews an article in which the separator line contains whitespace
AND make sure that the body of the article begins with a line containing
only whitespace.  You'll find that the second line of the body will be
concatenated with the last header.

There are several ways that sites moving news from VM to UNIX can deal
with the problem.  I'm using a news batcher on the VM side written by
Thomas Habernoll;  it does the EBCDIC->ASCII conversion, and produces
truly blank lines.  Then I send the batch as a binary file.  Another way
is to change the whitespace lines to truly blank lines once they arrive
on the UNIX side.

DEDOUREK@UNB.CA (01/09/89)

> Something which happens (at least with UREP) is that files coming
> in from BITNET-land have lines which are supposed to be truly
> empty instead contain some number of blanks.  You would think this
Perhaps some of you would take a hand in convincing IBM that an
empty line should exist.  My system (MVS TSO on an IBM 3090 etc.)
refuses to store "zero length" records.  (Lines are of course
"records".)
 <empty line>
On another note, perhaps the standards should be more careful
to define things like "blank line".  My interpretation would be
"a line with zero or more blanks, but no other characters".

gc@ewok.amd.bnl.gov (Graham Campbell) (01/10/89)

	Status: RO
	
	On another note, perhaps the standards should be more careful
	to define things like "blank line".  My interpretation would be
	"a line with zero or more blanks, but no other characters".
	
Quoting from RFC822, page 5

	It is separated from the headers by a null line (i.e., a line
	with nothing preceding the CRLF)
	
You may argue that such a definition was not well enough thought out
since it does not acknowlege the MVS problem with null lines, but it 
seems like a pretty clear definition to me.

Graham

gc@EWOK.AMD.BNL.GOV (Graham Campbell) (01/10/89)

	Status: RO
	
	On another note, perhaps the standards should be more careful
	to define things like "blank line".  My interpretation would be
	"a line with zero or more blanks, but no other characters".
	
Quoting from RFC822, page 5

	It is separated from the headers by a null line (i.e., a line
	with nothing preceding the CRLF)
	
You may argue that such a definition was not well enough thought out
since it does not acknowlege the MVS problem with null lines, but it 
seems like a pretty clear definition to me.

Graham


----- End Forwarded Message -----

jbn@glacier.STANFORD.EDU (John B. Nagle) (01/12/89)

      The SMTP standard seems clear on the division between header and
body of a message.  Difficulties in some implementations in dealing with
such a standard are irrelevant.   Such problems must be dealt with in
individual implementations.  

					John Nagle

NJG@CORNELLA.CIT.CORNELL.EDU (Nick Gimbrone) (01/15/89)

>> Something which happens (at least with UREP) is that files coming
>> in from BITNET-land have lines which are supposed to be truly
>> empty instead contain some number of blanks.  You would think this
>Perhaps some of you would take a hand in convincing IBM that an
>empty line should exist.
This is not an IBM problem. The IBM software has no problem treating
the line of blanks as a "null line". In this case it sounds more like
a UREP problem. UREP is the one who is trying to do cross protocol
conversions, thus it is UREP's job to convert between "null lines" and
"blank lines". Remember, if it came via UREP from an IBM system then
it was talking BSMTP, not SMTP (and thus "null lines" are represented
as "blank lines"). In the case of IBM's VM TCP/IP package the translation
between these two forms occurs just fine (a true "null line" is send on
SMTP when recieved on the BSMTP side, etc).

jpe@romeo.cs.duke.edu (John P. Eisenmenger) (01/24/89)

In article <8901151801.AA12681@ucbvax.Berkeley.EDU> NJG@CORNELLA.CIT.CORNELL.EDU (Nick Gimbrone) writes:
>>> Something which happens (at least with UREP) is that files coming
>>> in from BITNET-land have lines which are supposed to be truly
>>> empty instead contain some number of blanks.  You would think this
>>Perhaps some of you would take a hand in convincing IBM that an
>>empty line should exist.
>This is not an IBM problem. The IBM software has no problem treating
>the line of blanks as a "null line". In this case it sounds more like
>a UREP problem.

It is a UREP problem.  If I remember correctly (though I never handled
mail with UREP -- just batch jobs, plus I'm now at a different site)
files come in using Fortran Carriage control characters.  The UREP
software doesn't handle this correctly at all, which leads to horrible
printouts and such.  I traced the problem to the routing rdr uses to con-
vert from IBM to UNIX formats.

In addition to the blanks on blank lines, you'll find that UREP REQUIRES
a 66 line page for printouts, and counts lines internally.  It keeps this
line count in order to simulate a form-feed by (you guessed it) sending
lots of blank-filled lines to output.

I rewrote the FCC stuff to (almost) match the output from fpr (I just used
\r instead of backspacing for overstrike).  This gave us correct printouts
and decreased the output file sizes.

-John
 jpe@dukee.egr.duke.edu