david@indetech.uucp (David Kuder) (08/07/89)
I've finally gotten C news up and running to the point where I can post.
I wish Henry and Geoff hadn't thought they were smarter than I was and
then proved it. Those monolithic installation scripts are hard to figure
out when they fail. Of course, the failures were always mine but still
more "makeability" and editable prototypes might make life easier if there
is ever a C2 news.
Given my track record on this stuff I may have missed it, but is there
a method for gatewaying an inbound mailing list into a news group in
C news? In B I used "recnews" to catch mail and toss it into a news
group. We don't really need to turn postings into mail since the news
group is there to archive and provide a more convient interface. It
might be simple to do but it is late enough that I can barely see
straight.
Speaking not being able to see straight. Any ideas were nearly empty
batches might be generated? We send out the occasional cunbatch script
wrapped around the magic that says 12 bit compressed and zero actual
data.
--
____*_ David A. Kuder {sun,sharkey,pacbell}!indetech!david
\ / / Independence Technologies
\/ / 42705 Lawrence Place FAX: 415 438-2034
\/ Fremont, CA 94538 Voice: 415 438-2003osm@heifetz.ann-arbor.mi.us (Owen Scott Medd) (08/08/89)
In article <1989Aug7.060124.8135@indetech.uucp> david@indetech.UUCP (David Kuder) writes: >I've finally gotten C news up and running to the point where I can post. > [...] >Speaking not being able to see straight. Any ideas were nearly empty >batches might be generated? We send out the occasional cunbatch script >wrapped around the magic that says 12 bit compressed and zero actual >data. I *think* we see this problem when a togo.[0-9] contains nothing but expired/cancelled articles. The batcher -> muncher -> sender pipeline in sendbatches doesn't allow for the fact that a togo file might not contain anything of value. In Bnews, you were just chunking through one big batch queue, so the fact that some articles were missing wasn't that big a problem. Since we have some downstream feeds that send sendme's that date from weeks ago and some other feeds that don't pick up their news for up to a week or so at a time, I tend to get those annoying "Inbound news garbled" messages. I've got a little shell script that checks the batches to make sure at least one article exists, but this is definitely a hack. Owen -- USMail: M & S Associates, 628 Brooks, Ann Arbor, MI 48103 Phone: +1 313 761-6624 FAX: +1 313 971-0804 UUCP: uunet!sharkey!heifetz!osm Internet: osm@heifetz.ann-arbor.mi.us
henry@utzoo.uucp (Henry Spencer) (08/08/89)
In article <1989Aug7.060124.8135@indetech.uucp> david@indetech.UUCP (David Kuder) writes: >Given my track record on this stuff I may have missed it, but is there >a method for gatewaying an inbound mailing list into a news group in >C news? In B I used "recnews"... If you look at the newsmail(8) manual page, you'll see our equivalents. You may also want to look at contrib/nntpmail/mail*, which contains stuff done elsewhere at U of T for such things. We didn't include anything fancy as an official part of the distribution -- note the disclaimers in contrib/README -- partly because we didn't have an opportunity to test it ourselves. One thing that can still make minor difficulties is that "inews -h" still insists on overriding some headers. Fixed in next patch. >Speaking not being able to see straight. Any ideas were nearly empty >batches might be generated? We send out the occasional cunbatch script >wrapped around the magic that says 12 bit compressed and zero actual >data. Is it causing problems? It's a normal occurrence if you have stuff expiring quickly enough that the batcher sometimes can't find any of the articles that are supposed to go into a batch. Normally this only happens when a system which does fast expiring is feeding a site which has communications problems, so some of the batching gets deferred until after a lot of stuff has expired. -- 1961-1969: 8 years of Apollo. | Henry Spencer at U of Toronto Zoology 1969-1989: 20 years of nothing.| uunet!attcan!utzoo!henry henry@zoo.toronto.edu
david@indetech.uucp (David Kuder) (08/11/89)
In article <1989Aug8.154631.2816@utzoo.uucp> henry@utzoo.uucp (Henry Spencer) writes: >In article <1989Aug7.060124.8135@indetech.uucp> I wrote: >>Given my track record on this stuff I may have missed it, but is there >>a method for gatewaying an inbound mailing list into a news group in >>C news? In B I used "recnews"... > >If you look at the newsmail(8) manual page, you'll see our equivalents. >You may also want to look at contrib/nntpmail/mail*, which contains stuff >done elsewhere at U of T for such things. We didn't include anything >fancy as an official part of the distribution -- note the disclaimers >in contrib/README -- partly because we didn't have an opportunity to test >it ourselves. I got several specific solutions from other folks. I will get around to picking one of them this weekend when I have time. As for newsmail(8) I did see those but had interpreted them as transport mechanisms and not gatewaying programs. In particular recenews and recpnews both presume that they are recieving mail that has been encoded. I overlooked the stuff in contrib because of the NNTP in the name. Silly me. I consider it with the other candidates this weekend. In article <1989Aug8.030953.13302@heifetz.ann-arbor.mi.us> osm@heifetz.ann-arbor.mi.us (Owen Scott Medd) writes: =>Speaking not being able to see straight. Any ideas were nearly empty =>batches might be generated? We send out the occasional cunbatch script =>wrapped around the magic that says 12 bit compressed and zero actual =>data. = =I *think* we see this problem when a togo.[0-9] contains nothing but =expired/cancelled articles. The batcher -> muncher -> sender pipeline =in sendbatches doesn't allow for the fact that a togo file might not =contain anything of value. Back to Henry: >Is it causing problems? It's a normal occurrence if you have stuff >expiring quickly enough that the batcher sometimes can't find any of >the articles that are supposed to go into a batch. Normally this only >happens when a system which does fast expiring is feeding a site >which has communications problems, so some of the batching gets deferred >until after a lot of stuff has expired. And quoting Owen: =Since we have some downstream feeds that send sendme's that date from =weeks ago and some other feeds that don't pick up their news for up to =a week or so at a time, I tend to get those annoying "Inbound news =garbled" messages. I've got a little shell script that checks the batches =to make sure at least one article exists, but this is definitely a hack. What he said! Those error messages that down stream sites send back consume my time. The sending of the empty batches and the error messages consume phone time even with a telebit. An empty (or short) UUCP batches chew up a space on the UUCP queue. This compounds of the problem. If a downstream site isn't picking up its UUCP batches fast enough that articles referenced by the out.going/site/togo file are no longer on my finite disk then short UUCP articles are created. Since these batches are short the queue length isn't an accurate representation of how much news is queued to go out. The feedback here could end up with a site that managed to be just slow enough guarenteeing that it never actually got any news. Now, I have a hack in hand. Given that I have sites that don't poll me often enough for my own good I have a perl program that clears news batches from their UUCP queue. A quick mod to it makes it just nuke the (nearly) empty batches. In the long run (or this weekend which ever comes first) I will figure out how to enforce the policy I want using the mechanism Henry and Geoff have supplied. My guess it that a check for empty batches in the sendbatches pipeline after the muncher runs will be liveable. To really insure full batches would require rethinking batchsplit and its interaction with sendbatches. -- ____*_ David A. Kuder {sun,sharkey,pacbell}!indetech!david \ / / Independence Technologies \/ / 42705 Lawrence Place FAX: 415 438-2034 \/ Fremont, CA 94538 Voice: 415 438-2003
henry@utzoo.uucp (Henry Spencer) (08/13/89)
In article <1989Aug10.221915.15905@indetech.uucp> david@indetech.UUCP (David Kuder) writes: >>>... In B I used "recnews"... >> >>If you look at the newsmail(8) manual page, you'll see our equivalents. > >...newsmail(8) I did see those but had interpreted them as transport >mechanisms and not gatewaying programs. In particular recenews and >recpnews both presume that they are recieving mail that has been >encoded... As did B recnews, unless my memories are much mistaken. Recpnews is meant to be fully compatible with B recnews (which, incidentally, is a *lousy* transport mechanism and not a particularly good gateway). >=>Speaking not being able to see straight. Any ideas were nearly empty >=>batches might be generated? We send out the occasional cunbatch script >=>wrapped around the magic that says 12 bit compressed and zero actual >=>data. >> >>Is it causing problems? It's a normal occurrence if you have stuff >>expiring quickly enough that the batcher sometimes can't find any of >>the articles that are supposed to go into a batch... > >... Those error messages that down stream sites send back >consume my time. The sending of the empty batches and the error messages >consume phone time even with a telebit. An empty (or short) UUCP >batches chew up a space on the UUCP queue... Mmmm, yes, you're right. A partial fix, which will appear in the next patch, is for newsrun to ignore batches which are empty after decompression. That at least eliminates the error messages from C News sites. >... Since >these batches are short the queue length isn't an accurate >representation of how much news is queued to go out... One change some people have made is to measure queue length in bytes rather than batches. It's an interesting idea that may eventually become official. >... The feedback here >could end up with a site that managed to be just slow enough >guarenteeing that it never actually got any news. Such a site is probably too slow to be getting a full feed. It is necessary to have a reasonable speed margin in hand, even in the absence of such problems; news traffic is very bursty, and a site which is only barely fast enough is going to have continual trouble. >... My guess it that a check for empty batches in the >sendbatches pipeline after the muncher runs will be liveable... Possible but awkward, given that it will mean buffering onto disk so you can check the size of the whole batch and then decide what to do. There really isn't any very graceful way out of this one; the only real fix is to smush together several of the currently-separate functions in batching, so that batch preparation and splitting can interact. -- V7 /bin/mail source: 554 lines.| Henry Spencer at U of Toronto Zoology 1989 X.400 specs: 2200+ pages. | uunet!attcan!utzoo!henry henry@zoo.toronto.edu
kdg@nirvo.uucp (Kurt Gollhardt) (08/13/89)
In article <1989Aug13.001906.27932@utzoo.uucp> henry@utzoo.uucp (Henry Spencer) writes: >In article <1989Aug10.221915.15905@indetech.uucp> david@indetech.UUCP (David Kuder) writes: >>>>... In B I used "recnews"... >>> >>>If you look at the newsmail(8) manual page, you'll see our equivalents. >> >>...newsmail(8) I did see those but had interpreted them as transport >>mechanisms and not gatewaying programs. In particular recenews and >>recpnews both presume that they are recieving mail that has been >>encoded... > >As did B recnews, unless my memories are much mistaken. Recpnews is meant >to be fully compatible with B recnews (which, incidentally, is a *lousy* >transport mechanism and not a particularly good gateway). I'm afraid you *are* much mistaken. B recnews definitely expects user-generated mail which is *not* encoded. Now, it's not a transport mechanism, so I guess it *would* be a lousy one. It does seem to be intended as a gateway, but I do agree with you that it's not a particularly good one, even though I *am* using it (previously with B news; now, after some hacking, with C news - I, too, had dismissed the contrib/nntpmail stuff as not relevant since I don't use NNTP; now that you've mentioned it, and I actually looked at it, I know better, but it *was* rather unfortunately named). (BTW, in general I'm very happy with C news. Thank you for creating it.) -- Kurt Gollhardt \ Nirvonics, Inc. -- Plainfield, NJ Kurt.Gollhardt@nirvo.uucp /\ Software Design and Consulting ...!rutgers!nirvo!Kurt.Gollhardt / \ "It's all about people; not you and me or him and her, but *us*."