[news.admin] How to eliminate the cost of over 1/5

brad@looking.UUCP (Brad Templeton) (05/06/88)

Last month there were 24 megs of binaries and sources and maps sent out on the
net, part of 120 megs of net traffic, or just over 20%.

It's actually a higher percentage, because many sites don't get the
full 120 megs, but source and binary groups get high propagation.
Note also that binaries, usually posted in ARC form, don't get nearly
the compression (even uuencoded) that text does.

Note that at my $10/K figure, which many call conservative, that's
around $240,000 spent shipping binaries and sources in one month alone!

Even at a $1 per K figure, which only assumes 40 long distance links for
the whole net, we're talking $24,000!

The important thing to remember is that binaries, sources and maps are
not urgent, time-critical stuff.

Thus I propose the creation of a "mail-net" underneath the net structure,
for binary, source and map groups.

If you have a large posting, you put it on disk in one of a number of
commonly understood formats that the folks at some central place (UUNET?)
can read.   They collect all the binaries, sources and maps for a week,
put them on another media and mail (yes, postal mail!) them out to
all subnets that want them.  We're talking 7 1 meg floppies here, or a tape.

7 megs with a telebit takes about 2.5 hours, which is about $20 at night,
so sites with telebits would still pick stuff up by phone, as long as
it was all batched at night.

The disks get mailed to the non-telebit sites, who then distribute it over
local calling and NNTP subnets with free links.  If there are 300 such
subnets, and disks get re-used, the cost is about $3/subnet, or around
$1000 per week (even cheaper with bulk mail), or $4000 per month.

That's $4000 per month, down from $240,000 per month.  

Some side notes:
	a) Even saving hundreds of thousands of dollars per month, I
	   doubt this will get done.  Hard to believe.

	b) Sending binaries on disk means that either the whole pack gets
	   there or nothing does.  No more "missing parts" and repostings.

	c) The phone transport mechanism is always there as a backup, since
	   the post office will probably lose a few of each shipment.
	
	d) This does require the personal loading of the disks into
	   one machine (any machine) in each subnet, so there is this
	   physical cost.  I'm sure there's one Xenix in almost every
	   subnet, so it should not be hard to find a compatible format.

	e) Bulk mail is slower, but the postage might only be as little as
	   30 cents per piece, plus a few dollars every month to return
	   all the disks for recycling.  You only need 200 pieces for
	   US bulk mail.

	f) Overnight service, at around $10/piece, would increase my cost
	   estimate to $12,000 per month, but provide higher reliability
	   and fast turnaround.  Still far less than $240,000.  You can
	   also probably get a deal on 300 pieces of overnight mail.

	g) One overnight shipment to Europe, and around $50, would save
	   a lot, I think.

	h) Canada and Europe would want to set up their own internal
	   distributions, quite possibly.

	i) Other high-volume, non urgent stuff could go in these packs,
	   at almost no incremental cost.  Sites that have shut off most
	   of the net could get it, if they wanted it.

The big barrier is somebody to administer this.  If the USENIX or UUNET
people can find it within their scope, it might actually fly.

Operations of BBSs might also want to use this.
-- 
Brad Templeton, Looking Glass Software Ltd. - Waterloo, Ontario 519/884-7473

rsalz@bbn.com (Rich Salz) (05/10/88)

In news.admin (<1616@looking.UUCP>), brad@looking.UUCP (Brad Templeton) writes:
>The important thing to remember is that binaries, sources and maps are
>not urgent, time-critical stuff.

If this is the criteria, then we could save 90% of the current costs.

The important thing to remember is that almost nothing that appears on
Usenet is urgent, time-critical stuff.
-- 
Please send comp.sources.unix-related mail to rsalz@uunet.uu.net.

rick@seismo.CSS.GOV (Rick Adams) (05/10/88)

uunet already sends a weekly tape (full news feed) to South Korea,
Indonesia and the FBI.

Its not that crazy an idea.

---rick

cdaf@iuvax.cs.indiana.edu (Charles Daffinger) (05/11/88)

In article <44308@beno.seismo.CSS.GOV> rick@seismo.CSS.GOV (Rick Adams) writes:
>uunet already sends a weekly tape (full news feed) to South Korea,
>Indonesia and the FBI.
                   ^^^---- do they read it, or do they study it? --or-- should
                           the fodder for the NSA line-eater really be fodder 
                           for the FBI line-eater?
>
>---rick


Just wondering,

-charles

-- 
Charles Daffinger  >Take me to the river, Drop me in the water<  (812) 339-7354
cdaf@iuvax.cs.indiana.edu             {pur-ee,rutgers,pyramid,ihnp4}!iuvax!cdaf
Home of the Whitewater mailing list:    whitewater-request@iuvax.cs.indiana.edu

ben@idsnh.UUCP (Ben Smith) (05/11/88)

Many may feel that this sounds like a lot of work.  But if they were to
consider that eventually they have to share the costs of sending non-critical
data through a pricey communications system, they might see the common
sense of using diskettes and land based mail. The above referenced article
is sensible and should be used as a first draft of a RFC.
-ben

brad@looking.UUCP (Brad Templeton) (05/11/88)

>
>The important thing to remember is that almost nothing that appears on
>Usenet is urgent, time-critical stuff.

Well, not in the sense of "Delivery by 10:30 am or you don't pay," it
isn't, but most of what takes place on usenet is discussion, which is
improved by short propagation times.

Of course, when I say "improved," I mean "is easier."  Long delays would
help stop flame wars and reduce volume, and that would be good in many
groups, but I don't think that's the goal right now.

No so with maps, sources and binaries.  If a binary is 4 days late,
nothing really gets interfered with.

For other groups, as I have noted, the delay is a two edged sword.  Many
people bitch about moderators, mostly because of the delay they cause, but
at the same time, the delay is one of the stronger cooling factors a
moderator brings to things, even if he/she doesn't censor.

I often delay stuff several weeks, but then again my group is mostly
not time critical in the discussion sense.
-- 
Brad Templeton, Looking Glass Software Ltd. - Waterloo, Ontario 519/884-7473

david@infopro.UUCP (David Fiedler) (05/11/88)

From article <8650@iuvax.cs.indiana.edu>, by cdaf@iuvax.cs.indiana.edu (Charles Daffinger):
>In article <44308@beno.seismo.CSS.GOV> rick@seismo.CSS.GOV (Rick Adams) writes:
>>uunet already sends a weekly tape (full news feed) to South Korea,
>>Indonesia and the FBI.
>                    ^^^---- do they read it, or do they study it? --or-- should
>                            the fodder for the NSA line-eater really be fodder 
>                            for the FBI line-eater?

My guess is that they want to read it, but they don't want the security
problems of a UUCP hookup. Can you imagine what would happen if some
hacker got into their system?

-- 
David Fiedler         {ames,attmail,hplabs,ihnp4,pyramid,ucdavis}!infopro!david
USMail: InfoPro Systems, PO Box 220, Rescue CA 95672      Phone: (916) 677-5870

"Trust me. I know what I'm doing."

bill@ssbn.WLK.COM (Bill Kennedy) (05/11/88)

In article <719@fig.bbn.com> rsalz@bbn.com (Rich Salz) writes:
>In news.admin (<1616@looking.UUCP>), brad@looking.UUCP (Brad Templeton) writes:
>>The important thing to remember is that binaries, sources and maps are
>>not urgent, time-critical stuff.
>
>If this is the criteria, then we could save 90% of the current costs.
>
>The important thing to remember is that almost nothing that appears on
>Usenet is urgent, time-critical stuff.

That is a pretty decent recommendation for alternate delivery methods for
the less time critical/more bulky groups.  Naturally, I'm inclined towards
Stargate because that's where all of my moderated groups come from.  It's
easy enough to do and I hear that they are going to post an article soon
that will explain what they are going to do.

There are some delivery alternatives that could be explored and they should
not be too difficult to manage.  If your uucico is smart enough to handle
the "grades", you could give a lower grade to the bulky stuff so that it would
not go out as a matter of routine.  My former news feed had my news graded
such that he would call me on demand with mail but news would only leave when
I called.  That was entirely satisfactory (I only have one connection that is
a toll free call).

If you are using batched sends, the sendbatch could be smart enough to defer
scheduling bulky stuff until lower rates were in effect, like all day Saturday.
That would be an additional benefit to the other users during "prime time"
because there would be less competition for cycles.  The change to sendbatch
is rather trivial, just toss certain groups back into the batch spool if the
time of day isn't right.

Finally, for those blessed with a higher speed modem, some technique could
be developed to let the bulky groups only go to fast connections during
"prime time".  That would be a little more difficult than only chosing a
cheaper time, but if you had sendbatch on the table, already cut open...

I think that satellite delivery makes a lot of sense for the "long haul"
trips.  I'm not sure how to inject the stuff into the satellite carrier,
but assuming that's easy and done (the current Stargate feed is off of
emory, so net bandwidth is already taken up to get it there), it could work
this way.  No site would pass a moderated group long distance unless it
wanted to.  The first backbone to get a moderated group would pass it on
to the satellite carrier and be done with it.  Major metropolitan areas
would have satellite subscribers who would make local distributions much
as they do now.  Sites out in the boondocks (like ssbn) would be responsible
for their own satellite subscriptions but would still use dial-up to get
the rest (that's how ssbn functions now).

A number of benefits would accrue from one or more of these suggestions.
The propagation delays for more timely traffic would be shorter, long
distance charges would be more reasonably spent for the smaller volume
articles.  No, I don't know what to do about high volume unmoderated
groups, ssbn's solution is to not post to them or pay to get them.  I am
ignoring that part of the problem (so I don't have to hide behind it).

-- 
Bill Kennedy  usenet      {rutgers,ihnp4!killer,cbosgd}!ssbn!bill
              internet    bill@ssbn.WLK.COM

wcs@skep2.ATT.COM (Bill.Stewart.<ho95c>) (05/11/88)

There's been some discussion in news.admin about shipping binaries via
post-office (tape or floppy) instead of online.

In article <44308@beno.seismo.CSS.GOV> rick@seismo.UUCP writes:
> uunet already sends a weekly tape (full news feed) to South Korea,
> Indonesia and the FBI.
		^^^^^^^
While the FBI has as much right to read this stuff as anyone, I wonder
if they're reading it in an "official" capacity.  I assume you're sending
them alt.drugs?

(Just because you're paranoid doesn't mean they really aren't out to get you.)
-- 
#				Thanks;
# Bill Stewart, AT&T Bell Labs 2G218, Holmdel NJ 1-201-949-0705 ihnp4!ho95c!wcs
# skep2 is a local machine I'm trying to turn into a server.  Please send
# mail to ho95c or ho95e instead.  Thanks.

brian@premise.ZONE1.COM (Brian Moran) (05/11/88)

The "Saving $228,000" sounds great.  There is a slight problem, however. With 
the current distribution method, the 240k cost is not incurred by a single 
billable entity (nor a group of entities).  The $12k replacement scheme would 
unfortunately bring these "hidden" charges to light, since the costs would have
to be recovered in a more direct fashion (net costs are hidden in phone bills
for many sites).  Also, those "free" links (unless they are a direct serial
line or something) sometimes really aren't.

I'd really like to see the traffic in binaries decrease, since to me it seems
the abuse of a good thing (the net).  What particularly bothers me is in some
groups messages like:

"Hey, parts 1 and 2 (of 14) of the latest "FrogSnax fractal DNA sequence 
modeller and disk formatting utilities" didn't make it to my site. Can someone 
repost it?"

which is then followed (ALL TOGETHER NOW, in n-part harmony) by:

"Hey, we didn't get parts 3 and 7." (ed. note: A QUANTUM LEAP IN STATISTICAL
INFERENCE FOLLOWS:) "Looks like the propogation is screwed up. Maybe we
should repost the whole thing." (keyboard spring resilience and $$ are
consumed at an alarming rate).

Perhaps if the actual transmission costs were born by the people who are 
interested in these things, there may be less wastage, and more concern
for these "bean counter-ian" tasks.

How about a scheme like this:

-	binaries and sources posted ONCE.
-	didn't get the information? well then, dial up (yes, use YOUR PHONE)
	a regional (hey, let's get cheap and just make ONE for each 
	continent) distribution center, and get it yourself with anonymous
	uucp.  Oh, ARPA/internet sites. You guys (& gals) may be charged for
	usage of what has previously been for free for a long time.  
        You may like doing uucp as well after you get your first bill 
	(1/2 :-), only ; have you read about some of the charge-back schemes
	lately?).

People will still be able to mail stuff around (thereby circumventing any
savings) but I bet the few that did this would still be less than the 
wholesale bit-shipping (say that fast a few times) going on now.

Or, ever more draconian:

-	Binaries and sources don't get posted. DESCRIPTIONS and where
	to get it gets posted.  A person would actually have to be 
	interested enough in the post to expend EFFORT to get what
	he/she wants.  This might even cut down on the "COLLECTIVE
	DISK STORAGE CONSCIOUSNESS" - how many of you have tucked 
	away that posting, thinking "Hmmm, this looks interesting"
	only to have those disk blocks sit around, untouched, until
	you were out of disk space? (Oh, those of you with unlimited
	quotas -- you either don't realize (yet) that you are doing
	this, or have an iron will). (Why do capitalized words in
	otherwise normal text remind me of ZIPPY?) CURE THAT JUNK
	BINARY HABIT! READ A BOOK INSTEAD!

Well, that seems like enough. What do I prefer? Hmmm. I think distributions
should be accomplished by encoding binaries and sources alike using 
the "Kansas City" cassette encoding method, and having a "cassette of the
month club".  Either that, or paper tape.  Can't make this stuff easy
for the "computer weenies" out there.

(okay, okay, okay, before you hit the 'r' or 'f' key - some of this stuff is
tongue in cheek, some of it (well, I thought at least) may deserve some
fleeting increase in overall brain activity (don't try this at home - people
who understand these things are TRAINED PROFESSIONALS!)).

Brian K. Moran            { ...harvard!mit-eddie,...!mirror}!premise!brian
Premise, Inc.             brian@premise.zone1.com
3 Cambridge Center
Cambridge, MA 02142       (617) 225-0422 

P.S. Of course, this should NEVER be used to more directly pass on the costs
of groups like soc.singles, alt.flame and their ilk. (ooooh doggie, I can
feel them keyboards heating up now...)
	
	

-- 
Brian K. Moran            { ...harvard!mit-eddie,...!mirror}!premise!brian
Premise, Inc.             brian@premise.zone1.com
3 Cambridge Center
Cambridge, MA 02142       (617) 225-0422 

chuq@plaid.Sun.COM (Chuq Von Rospach) (05/12/88)

>There are some delivery alternatives that could be explored and they should
>not be too difficult to manage.  If your uucico is smart enough to handle
>the "grades", you could give a lower grade to the bulky stuff so that it would
>not go out as a matter of routine.

It's funny, but a couple of years ago I outlined a design that came quite
close to this. At that time, I was told it was overkill and nobody would
ever need it....

Anyway, it should be fairly easy to implement, and gives you the advantages
of both grading news by priority and limiting the amount of data you shove
through a news feed.

Step 1 is to define a grade for every newsgroup. Arbitrarily lets call them
	A-F and Z. That should be fine enough granulairity for most folks.
	Highest priority is A, lowest priority is F, Z is for groups you
	refuse to pass along. You could, in theory set things up so you
	could define priorities on a by-feed basis instead of global.

Step 2 is to modify the batching procedure to take the batching files
	and to turn them into batches based on priority. It takes all of
	the A groups, then the B, the C, etc, etc. You could, optionally,
	give it a high-water mark for number of bytes to transfer. If it 
	hits the highwater mark before it runs out of news, it stops
	batching and leaves the leftover for the next batching cycle.

The only program that would need to be modified is sendbatch. Everything
else remains the same, and it doesn't require any kind of interface changes
so you don't require your leafs or upstream sites to change.

You can, with this plan, strictly limit the amount of data sent down a feed,
and you can do it while making sure the data you want transferred gets
transferred. Whenever a sendbatch is started, it starts with the A groups
and works its way down. If it stops in the D groups, for instance, the rest
of the messages have to wait until a batch is done with enough room to
spare. If they expire before they get transferred, well, tough. That's why
they're low priority messages.

This has the effect of throttling the net. It will only grow so far, and
once it hits that highwater, messages start disappearing based on their
(lack of) priority. For folks on a phone budget, they can finally stop being
held hostage by factors outside of their control.

This would be simple to implement, compatible, and allow admins to define 
how much net they're willing to handle, and where they prefer to have their
dollars spent. If volume is lower than that, the low priority groups can
come along for the ride, but if the net gets busy, they get put on hold
until the traffic lightens up. If it doesn't, the low priority data gets
shunted to make room for more important stuff.

I'm sure I'm going to get accused for being fascist by claiming that all
messages are not created equal, but that's life. If I only have so much
bandwidth, I'd rather spend it on things like comp than talk. As an admin,
this plan would finally give me that choice without having to actually stop
carrying the groups completely. 




Chuq Von Rospach			chuq@sun.COM		Delphi: CHUQ

	Robert A. Heinlein: 1907-1988. He will never truly die as long as we
                           read his words and speak his name. Rest in Peace.

brad@looking.UUCP (Brad Templeton) (05/12/88)

Indeed, there are commercial shareware/PD distribution organizatons that
sell PD disks from anywhere to 3.50 to $6 per disk.  It would be very
easy to convince these folks to catalog the various binaries sent here,
if they already don't, and arrange a deal for usenetters.

But they wouldn't be keen on binaries/sources for things they don't support,
since their support people wouldn't know how to answer questions on them,
and they probably wouldn't care for uucp maps, so a usenet organized
mailnet is possibly worth something.

Now a Telebit is nice because you get the cost down to about $4 per meg if
you do it at night.  You need 3 megs before a UPS overnight letter is
cheaper.

Anything less than a telebit and an overnight letter is cheaper than a meg.
With 4 megs a day, an overnight letter every two days would cost a lot less
and probably even *speed up* propagation on some things.

The smart thing to do would be to make two classes of news, namely urgent
and non-urgent.  Use the mailnet for the non-urgent, and the phones for
the urgent.

Only problem would be making sure people didn't abuse the "urgent"
distribution.  If you had to pay $5/K to post an urgent message, that would
do it, but there's no way to enforce that.

But right now, costs would drop a great deal if the binaries, sources,
map, talk and some other groups went to this form of distribution.

It's an irony that Stargate, which has the same costs no matter how many
people pick it up, has to be restricted due to broadcast rules.
-- 
Brad Templeton, Looking Glass Software Ltd. - Waterloo, Ontario 519/884-7473

wnp@dcs.UUCP (Wolf N. Paul) (05/12/88)

In article <17@idsnh.UUCP> ben@.UUCP (Ben Smith) writes:
>
>Many may feel that this sounds like a lot of work.  But if they were to
>consider that eventually they have to share the costs of sending non-critical
>data through a pricey communications system, they might see the common
>sense of using diskettes and land based mail. The above referenced article
>is sensible and should be used as a first draft of a RFC.
>-ben

The poster of the referenced article himself included the statement "It
probably won't work."

Why?

Currently, in many companies and organizations the cost of USENET transmissions
is assumed as a matter of course to be a part of the phone bill; no special
approval or budget category is reqired. I'm not sure that the management in
many of these organizations even knows what they are paying for.

If you reduce net traffic by putting voluminous groups onto distributable media,
these companies' financial officers will rejoice in the lower phone bill,
but fail to see why they should approve purchase requisitions or check requests
for the acquisition of source code which may or may not be relevant to the
company's purpose.

It's a lot easier to spend thousands of dollars on an already-approved item 
such as phone bills, than to spend a tenth of that amount on an unapproved,
unbudgeted item like public domain sources distributions.


-- 
Wolf N. Paul * 3387 Sam Rayburn Run * Carrollton TX 75007 * (214) 306-9101
UUCP:     ihnp4!killer!dcs!wnp                 ESL: 62832882
INTERNET: wnp@DESEES.DAS.NET or wnp@dcs.UUCP   TLX: 910-280-0585 EES PLANO UD

friedl@vsi.UUCP (Stephen J. Friedl) (05/13/88)

In article <52918@sun.uucp>, chuq@plaid.Sun.COM (Chuq Von Rospach) writes:
< Step 1 is to define a grade for every newsgroup. Arbitrarily lets call them
< 	A-F and Z. That should be fine enough granulairity for most folks.
< 	Highest priority is A, lowest priority is F, Z is for groups you
< 	refuse to pass along.
< [...]
< I'm sure I'm going to get accused for being fascist by claiming that
< all messages are not created equal, but that's life.

How about defining grades on a per-message basis, so "certain
net individuals" get graded as well ;-) ?

-- 
Steve Friedl    V-Systems, Inc. (714) 545-6442    3B2-kind-of-guy
friedl@vsi.com    {backbones}!vsi.com!friedl   attmail!vsi!friedl

henry@utzoo.uucp (Henry Spencer) (05/13/88)

> ... They collect all the binaries, sources and maps for a week,
> put them on another media and mail (yes, postal mail!) them out to
> all subnets that want them...

One minor negative aspect that's worth mentioning is that mailing tapes,
unlike queueing uucp transfers, requires significant amounts of human time.
(The AT&T Software Toolchest, for example, won't mail you a tape no matter
how much you beg and plead, last I heard.)  If this is to be done it would
pretty well have to be done by UUNET or the equivalent, so that the tape
mailers can get paid.  This also means that the old problem emerges:  such
a service has to be explicitly bought, rather than just hidden in the
phone bills like a lot of Usenet traffic.  Not impossible -- UUNET service
has to be explicitly bought, and a lot of people have managed that -- but
it limits the audience.
-- 
NASA is to spaceflight as            |  Henry Spencer @ U of Toronto Zoology
the Post Office is to mail.          | {ihnp4,decvax,uunet!mnetor}!utzoo!henry

sl@van-bc.UUCP (pri=-10 Stuart Lynne) (05/14/88)

In article <719@fig.bbn.com> rsalz@bbn.com (Rich Salz) writes:
>In news.admin (<1616@looking.UUCP>), brad@looking.UUCP (Brad Templeton) writes:
>>The important thing to remember is that binaries, sources and maps are
>>not urgent, time-critical stuff.

>If this is the criteria, then we could save 90% of the current costs.

>The important thing to remember is that almost nothing that appears on
>Usenet is urgent, time-critical stuff.

I would go so far as to say that almost nothing that appears on 
Usenet is urgent, time-critical stuff -- with the *exception* of the map
data. 

If we assume for a moment that accurate map data would generate accurate
mailing paths (I said assume, it might happen someday) then it is important 
to ensure that the data is distributed on a timely basis as widely as
possible.  Usenet does a fairly good job of doing so right now, at probably
not too high a cost. 

In other words, it isn't to improbable to assume that the increased cost of
running the map data over Usenet could be more than compensated for by 
the reduced costs of routing mail of shorter or correct routes.




-- 
{ihnp4!alberta!ubc-vision,uunet}!van-bc!Stuart.Lynne Vancouver,BC,604-937-7532

brad@looking.UUCP (Brad Templeton) (05/16/88)

It is not important that map data be distributed urgently.  If it takes
a few more days for monthly maps to come out, that isn't going to
cause trouble.

What is important is that they be distributed reliably (no missing parts)
and cheaply (because they're huge).

Any urgency can easily be handled by sending out just updates that
occur during the delay slower transmission methods would cause.

Physical mail with a backup of either couriers or phone transmissions
would be reliable and cheap.
-- 
Brad Templeton, Looking Glass Software Ltd. - Waterloo, Ontario 519/884-7473

mangler@cit-vax.Caltech.Edu (Don Speck) (05/16/88)

[ in reference to Chuq's prioritized batching scheme ]

I had to throttle one of my outgoing newsfeeds.  First in, first out.
A backlog accumulates during the week, and clears on the weekends.
The throttle will top off the uucp queue for that site to a certain
level and hold the rest until something gets sent.  With a backlog
that can reach thousands of articles (and no batching for that site)
it was *essential* to keep the backlog out of the uucp queue.
Grading is not enough.	So, I can attest to the value of a throttle
(and of weekend phone rates).

[ in reference to binaries, etc ]

One of the distinguishing features of binaries and other machine-readable
postings is their large size.  I would guess that the larger the posting,
the less likely that it will be read by a human, especially a busy human.

As I recall, only about 2% of postings exceed 16K bytes, yet they account
for half of the total bytes.

I propose that batches be sorted in shortest-first order.  This will hold
the problem postings for the weekend, and, if done widely, will encourage
more succinct postings (and speed the propagation of cancel messages).
The st_mtime of the article file would be a good tie-breaker.

(I wonder if it would help the compression ratio.  I already sort batches
by article filename in pursuit of higher compression).

Don Speck   speck@vlsi.caltech.edu  {amdahl,ames!elroy}!cit-vax!speck

heiby@mcdchg.UUCP (Ron Heiby) (05/17/88)

Don Speck (mangler@cit-vax.Caltech.Edu) writes:
> I had to throttle one of my outgoing newsfeeds.  First in, first out.
> A backlog accumulates during the week, and clears on the weekends.

I have a similar throttle on my outgoing feeds.

> I propose that batches be sorted in shortest-first order.  This will hold
> the problem postings for the weekend, and, if done widely, will encourage
> more succinct postings (and speed the propagation of cancel messages).
> The st_mtime of the article file would be a good tie-breaker.
> 
> (I wonder if it would help the compression ratio.  I already sort batches
> by article filename in pursuit of higher compression).

Don has an interesting idea.  My guess is that it would help the compression
ratio a great deal on short messages, where the header takes up a larger
percentage of the total being compressed.  It would also encourage the other
good things Don mentions.  Unfortunately, it would also mean that we would
lose all semblance of discussion order.  If person A posted a 150 line article
and person B posted a 10 line rebuttle, the rebuttle would appear on systems
before the original article.  (Often, it would not be much sooner, but it
would be at least a little earlier.)  The other problem I see with the idea
is that moving all the big stuff on the weekends means that it's happening
when there is no one (or almost no one) around to keep an eye on the system.
-- 
Ron Heiby, heiby@mcdchg.UUCP	Moderator: comp.newprod & comp.unix
"I believe in the Tooth Fairy."  "I believe in Santa Claus."
	"I believe in the future of the Space Program."

david@ms.uky.edu (David Herron -- One of the vertebrae) (05/17/88)

In article <7869@mcdchg.UUCP> heiby@mcdchg.UUCP (Ron Heiby) writes:
>Don has an interesting idea.  
>....  Unfortunately, it would also mean that we would
>lose all semblance of discussion order.  If person A posted a 150 line article
>and person B posted a 10 line rebuttle, the rebuttle would appear on systems
>before the original article.  

That already happens quite a bit.  We don't lose here.

>....  The other problem I see with the idea
>is that moving all the big stuff on the weekends means that it's happening
>when there is no one (or almost no one) around to keep an eye on the system.

You mean you don't catch up on your news reading on the weekends ??
-- 
<---- David Herron -- The E-Mail guy            <david@ms.uky.edu>
<---- or:                {rutgers,uunet,cbosgd}!ukma!david, david@UKMA.BITNET
<---- 
<---- Goodbye RAH.

rsweeney@dasys1.UUCP (Robert Sweeney) (05/20/88)

In article <9304@g.ms.uky.edu> david@ms.uky.edu (David Herron -- One of the vertebrae) writes:
>In article <7869@mcdchg.UUCP> heiby@mcdchg.UUCP (Ron Heiby) writes:
>>Don has an interesting idea.  
>>....  Unfortunately, it would also mean that we would
>>lose all semblance of discussion order.  If person A posted a 150 line article
>>and person B posted a 10 line rebuttle, the rebuttle would appear on systems
>>before the original article.  
>That already happens quite a bit.  We don't lose here.

It happens quite a bit now, but it would be much worse if the proposed change
is made.  Sorting articles by length rather than simply sending them in more-
or-less chronological order would practically guarantee that articles would
arrive out of sequence.

Netnews conversations are already difficult enough to follow.  I like the idea
of having some way to force news volume down (although currently it's not
a problem at dasys1, since we're practically dedicated to news), but I
don't think sorting by length is an appropriate answer. 

-- 
Robert Sweeney              {sun!hoptoad,cmcl2!phri}!dasys1!rsweeney
Big Electric Cat Public Access Unix (212) 879-9031 - System Operator
You do it because you're drunk, you're numb, and you just don't care.

heiby@mcdchg.UUCP (Ron Heiby) (05/21/88)

David Herron -- One of the vertebrae (david@ms.uky.edu) writes:
|In article <7869@mcdchg.UUCP> heiby@mcdchg.UUCP (Ron Heiby) writes:
|>Don has an interesting idea.  
|>....  Unfortunately, it would also mean that we would lose all semblance
|> of discussion order.
| 
| That already happens quite a bit.  We don't lose here.
Maybe it happens quite a bit where you are, but I don't see it all that
often.  Making this change would cause it to occur *MUCH* more frequently,
to the point where it would be a lose.

| You mean you don't catch up on your news reading on the weekends ??
My wife doesn't let me.  :-)
-- 
Ron Heiby, heiby@mcdchg.UUCP	Moderator: comp.newprod & comp.unix
"I believe in the Tooth Fairy."  "I believe in Santa Claus."
	"I believe in the future of the Space Program."

kent@happym.UUCP (Kent Forschmiedt) (05/27/88)

In article <7869@mcdchg.UUCP> heiby@mcdchg.UUCP (Ron Heiby) writes:
>Don Speck (mangler@cit-vax.Caltech.Edu) writes:
>> I propose that batches be sorted in shortest-first order.  This will hold
>> the problem postings for the weekend, and, if done widely, will encourage
>> more succinct postings (and speed the propagation of cancel messages).
>
>good things Don mentions.  Unfortunately, it would also mean that we would
>lose all semblance of discussion order.  If person A posted a 150 line article
>and person B posted a 10 line rebuttle, the rebuttle would appear on systems
>before the original article.  (Often, it would not be much sooner, but it

This is already happening on my site.  I see a lot of replies before
the article that they reply to.  I don't know if this is because of
size-driven priority schemes, or just the general randomness of the net.

I get news from several directions, and there are some notefiles sites
in sme of the directions.  I don't know whether they tend to scramble
article order.
-- 
--
	Kent Forschmiedt -- kent@happym.UUCP, tikal!camco!happym!kent
	Happy Man Corporation  206-282-9598