spaf@cs.purdue.EDU (Gene Spafford) (10/24/88)
I recently made a presentation on the Usenet and NNTP to the IETF
(Internet Engineering Task Force), and I pulled together a bunch of
stats about the Usenet. To my knowledge, no one has ever done this
before, so I thought I'd also publish some of them here for your
amusement.
The following numbers are derived from old postings and data from the
following people: Henry Spencer, Steve Bellovin, Mark Horton, Rick
Adams, Brian Reid, and me.
Some observations:
1) Growth in sites. The Usenet has been growing by an approximate
doubling (or better) each year. (see below)
2) Growth in volume. The number of articles posted to the net has
approximately doubled each year. The sum total of article sizes has
not been growing as quickly as the number of articles. That is, the
average article SIZE has decreased over time, but the article COUNT has
increased. (see below)
3) Well over 1 million articles have been posted to the Usenet since
its origination in 1979.
4) If you make some very conservative assumptions about the cost of
operation of Usenet, you get some astonishing numbers. Assume that
each of the estimated 11000 current Usenet sites spends approximately
$10 per day on Usenet -- communicattions charges, cpu time and disk
time. Further assume that, on the average, each of the 303,000 (est.)
Usenet readers spends 20 minutes per day reading/posting news, at an
average hourly wage of $15 per hour per person (if they were working).
Then, the total cost of Usenet at its current size is $593,125,000 per
year! Even if those numbers are off by a factor of 10 (doubtful),
those numbers are staggering!
5) Latest figures show that 97% of all articles reach the
well-connected sites within 72 hours. Effectively, this means that
almost every site has a delay of at most 6 days before seeing a
posting, and most see articles within 3 days. Approximately 82% of all
posted articles are available to the well-connected sites within 24
hours of posting (largely thanks to NNTP).
Some possible conclusions that can be drawn from this:
I) Volume has been increasing due to the addition of new sites and new
posters. The trend with people who have been on the net for a while
seems to be to post less and post shorter articles. The increase in
the number of newsgroups does not correlate well with the increase in
volume (although it may correlate well with postings going into the
wrong groups due to namespace pollution).
II) At the current rate of growth, Usenet will pass its 2nd millionth
message sometime in 1990. By the end of that year, message traffic
would be approximately 8000 messages per day, exchanged by 50,000
Usenet sites. I cannot conceive of that happening (although 2 years
ago I could not conceive of over 10K sites & 4Mb per day traffic,
either!); I suspect something will happen to break the network
up before then -- either due to internal pressure, or external
forces concerned about costs and traffic. We already see this
happening with alternate distributions and the surge in mailing
lists.
III) I don't want to even try to speculate what will happen once
hypermedia Usenet becomes available....
Some year-by-year figures:
1979: 3 sites, 2 articles per day
1980: 15 sites, 10 articles per day
1981: Usenet described in Usenix conference -- sites invited to join.
Notesfile system comes on-line and joins Usenet.
This explains jump in sites, although postings remain low
because groups are mostly technical & Unix-oriented and
few "novice" users use the groups.
1981: about 150 sites, 20 articles per day
1982: about 400 sites, 35 articles per day
1982: Did 4.1 or 4.2 BSD come out around here? That would explain
the sudden jump in postings, I believe.
1983: over 600 sites, 120 articles per day
1984: over 900 sites, 225 articles per day
1985: over 1300 sites, 375 articles per day, 1Mb+ per day
1986: about 2500 sites, 500 articles per day, 2Mb+ per day
1986: NNTP introduced. MLZ compression in news 2.10.
1987: about 5500 sites, 1000 articles per day, 2.4Mb+ per day
10/1/1988: almost 11,000 sites, 1800 articles per day, 4Mb per day
--
Gene Spafford
NSF/Purdue/U of Florida Software Engineering Research Center,
Dept. of Computer Sciences, Purdue University, W. Lafayette IN 47907-2004
Internet: spaf@cs.purdue.edu uucp: ...!{decwrl,gatech,ucbvax}!purdue!spafbrad@looking.UUCP (Brad Templeton) (10/25/88)
The reason that the net has been able to grow at the rate it has can be found by examining similar jumps in technology. When the net started, articles were sent unbatched, uncompressed and over 1200 bps modems. There were some sites on the net using 300 bps modems. Over time the following factors have come into play: A) 14,000 bps modems - 11.6 factor improvement B) Data compression - 2.2 factor improvement C) Batching - 1.2 factor improvement D) Long Distance Rate reductions - 1.3 factor improvement? Total improvment: over 40 times! Thus from late 1982 (100K/day) to today (4 megabytes/day) USENET has actually not grown in terms of the cost to handle a link! That's pretty astounding. If you add in other things, like PC-Persuit, X.25 links, Internet links, Bitnet links, StarGate and, in the future, ISDN we see an actual reduction! Of course while the cost/link has stayed the same, the number of links has increased, thus increasing the total cost of the net. However, the number of non-free links has not grown anywhere near the way the number of sites has. Most new sites are on free links -- things like SUNS, personal workstations and at-home Unix boxes. The one great cost that has gone up is the human time spent reading waste of time postings. (Like this one?) -- Brad Templeton, Looking Glass Software Ltd. -- Waterloo, Ontario 519/884-7473
tr@wind.bellcore.com (tom reingold) (10/25/88)
On the subject of "Some interesting news stats", spaf@arthur.cs.purdue.edu (Gene Spafford) gives some very interesting statistics indeed! $ [...] $ Further assume that, on the average, each of the 303,000 (est.) $ Usenet readers spends 20 minutes per day reading/posting news, at an $ average hourly wage of $15 per hour per person (if they were working). $ Then, the total cost of Usenet at its current size is $593,125,000 per $ year! Even if those numbers are off by a factor of 10 (doubtful), $ those numbers are staggering! $ [...] This one, however, does not worry me. Let the employers worry about lost productivity on an individual basis. (Not that I think Gene is asking us to worry.) You could make the same argument about coffee breaks and how they are a threat to our GNP, etc., etc. If all of us newsaholics weren't "wasting" our time and our bosses' time reading news, we would probably be wasting it in other ways, so it's questionable as to whether or not there is some kind of real loss. Tom Reingold PAPERNET: |INTERNET: tr@bellcore.bellcore.com Bell Communications Research |UUCP-NET: bellcore!tr 445 South St room 2L350 |SOUNDNET: (201) 829-4622 [work], Morristown, NJ 07960-1910 | (201) 287-2345 [home]
smb@ulysses.homer.nj.att.com (Steven Bellovin[jsw]) (10/26/88)
In article <2206@looking.UUCP>, brad@looking.UUCP (Brad Templeton) writes: ... > When the net started, articles were sent unbatched, uncompressed and > over 1200 bps modems. There were some sites on the net using 300 bps > modems. In fact, when the net started 300 baud was the norm; very few sites had 1200 bps, since Vadic wasn't licensing their stuff very much, AT&T was just starting to license theirs, and if you wanted a ``genuine Bell'' unit you had to rent it, at ~$40/mo..... --Steve Bellovin Serious use of 1200 bps for netnews took at least a year, maybe even longer.
david@ms.uky.edu (David Herron -- One of the vertebrae) (10/26/88)
I dunno about the rest of you ...
But I value the information I get from the news much more than I
value what I 'learn' in all those freebie magazines I get inundated
with. Shoot, I simply stopped reading a couple of them months
ago! And I was so happy when Information Week sent me a letter
today saying that if I didn't fill in the enclosed card and beg
them for a continuance of my subscription, why they'd shut me off!
I chuckled all the way to the trash can, let me tell you :-)
(Some of those freebies are good, and I see a different sort of
information that I see here, but still ...)
Fortunately my employers feel the same as I do...
Make sure that Usenet makes a contribution to the work environment.
It'll defend itself.
How much is stuff like compress worth to you? Compress, just to pick
on one good contribution to EVERYONE, was written with the idea
of lowering phone costs. But the code has surfaced in so many
places it's not funny. And it's saved countless amounts of disk
space on computers everywhere. What are the savings to the world
simply because compress exists?
But it's not just compress. There's so many things we've been able
to do here at UK because we knew something we'd learned from the net.
Like the broadcast storm we were having on our ethernet which I
finally diagnosed with the help of one of Charles Hedricks articles.
It goes on and on.
In my view the money (resources) we spend on Usenet (disk space,
my time to administer it, and varying amounts of staff time for
reading it) is worth every penny.
--
<-- David Herron; an MMDF guy <david@ms.uky.edu>
<-- ska: David le casse\*' {rutgers,uunet}!ukma!david, david@UKMA.BITNET
<--
<-- Controlled anarchy -- the essence of the net.isolated@alix.UUCP (20 James D. Corder) (10/26/88)
In article <11250@bellcore.bellcore.com>, tr@wind.bellcore.com (tom reingold) writes:
=> On the subject of "Some interesting news stats", spaf@arthur.cs.purdue.edu
=> (Gene Spafford) gives some very interesting statistics indeed!
=>
=> $ [...]
=> # 303,000 (est.) Usenet readers spends 20 minutes per day reading/posting
=> $ news, at an average wage of $15 per hour per person (if they were working).
=> $ Then, the total cost of Usenet at its current size is $593,125,000 per
=> $ year! Even if those numbers are off by a factor of 10 (doubtful), those
=> $ numbers are staggering!
=> $ [...]
=>
=> Let the employers worry > about lost productivity on an individual basis.
=> (Not that I think Gene is asking us to worry.) You could make the same
=> argument about coffee breaks and how they are a threat to our GNP, etc.,
=> If all of us newsaholics weren't "wasting" our time and our > bosses' time
=> reading news, we would probably be wasting it in other > ways, so it's
=> questionable as to whether or not there is some kind of real loss.
=>
=> Tom Reingold
Did anyone stop to think that maybe the company time spent reading
the news is a form of relaxation that might inspire the creative mind, thereby
increasing productivity? :-)
Thank you,
James D. Corder
P.O. Box 27473
Columbus, Ohio 43227
...osu-cis!alix!corderhenry@utzoo.uucp (Henry Spencer) (10/26/88)
In article <10767@ulysses.homer.nj.att.com> smb@ulysses.homer.nj.att.com (Steven Bellovin[jsw]) writes: >> When the net started, articles were sent unbatched, uncompressed and >> over 1200 bps modems.... > >In fact, when the net started 300 baud was the norm; very few sites had >1200 bps... In fact, it wasn't unheard-of for early news connections to be 300 baud MANUALLY DIALED. Boy, were we glad to get our first 1200-baud autodialing modem... -- The dream *IS* alive... | Henry Spencer at U of Toronto Zoology but not at NASA. |uunet!attcan!utzoo!henry henry@zoo.toronto.edu
baer@Jessica.stanford.edu (Paul Baer) (11/06/88)
In article <2206@looking.UUCP> brad@looking.UUCP (Brad Templeton) writes: >The reason that the net has been able to grow at the rate it has can >waste of time postings. (Like this one?) >-- Amen! --paul baer Anything I say can and will be held against me in a court of law, I think. So don't tell anyone what I said.