dcox@ssd.kodak.com (Don Cox (253-7121)) (11/15/90)
I am running NNTP 1.5.10 with CNews on a Sun 4/280 (SunOS4.0.3).
Currently I have about 100 people using this machine to read netnews.
I haven't notice any performance degration yet, but since this
machine is also a fileserver for 10 diskless clients, I am concerned
at what point I may start to suffer because of the nntp hits.
Does nntp cause much of a performance hit on the system the clients are
getting the news from? What would be a ballpark figure for the max
number of clients you would want to service before considering down-
loading the news to a remote site?
Thanks.
--
Don Cox
Phone (716) 253-7121 KMX (716) 253-7998
INTERNET dcox@ssd.kodak.comstealth@caen.engin.umich.edu (Mike Pelletier) (11/17/90)
In article <1990Nov15.155532.3384@ssd.kodak.com> dcox@ssd.kodak.com (Don Cox (253-7121)) writes: >Does nntp cause much of a performance hit on the system the clients are >getting the news from? What would be a ballpark figure for the max >number of clients you would want to service before considering down- >loading the news to a remote site? I'm running C-news/NNTP on a DECstation 3100, and with fifty-five connections at any given time, there's barely over one load point, and that's with about 10 news transmitters running and an AFS file server going as well. The CPU time is minimal -- think about it: the process accesses and transmits only that which is needed, and the vast majority of time in an NNTP client connection is spent with the person reading the text that was sent, with the program doing nothing. -- Mike Pelletier - Usenet News Admin & Programmer "Wind, waves, etc. are breakdowns in the face of the commitment to getting from here to there. But they are the conditions for sailing -- not something to be gotten rid of, but something to be danced with."
brtmac@maverick.ksu.ksu.edu (Brett McCoy) (11/17/90)
In <1990Nov16.220048.22474@engin.umich.edu> stealth@caen.engin.umich.edu (Mike Pelletier) writes: >In article <1990Nov15.155532.3384@ssd.kodak.com> dcox@ssd.kodak.com (Don Cox (253-7121)) writes: >>Does nntp cause much of a performance hit on the system the clients are >>getting the news from? What would be a ballpark figure for the max >>number of clients you would want to service before considering down- >>loading the news to a remote site? >I'm running C-news/NNTP on a DECstation 3100, and with fifty-five connections >at any given time, there's barely over one load point, and that's with >about 10 news transmitters running and an AFS file server going as well. >The CPU time is minimal -- think about it: the process accesses and >transmits only that which is needed, and the vast majority of time >in an NNTP client connection is spent with the person reading the text >that was sent, with the program doing nothing. Something most people forget about here is the memory involved. Each nntp connection requires that an nntpd process be running. If the client is even moderately busy this involves several hundred K worth of memory being devoted to the nntpd, which is several hundred K lost to any other process trying to run. Also, people reading news using nfs cause the nfs cache to get filled with news articles instead of swap space or system or user files. I agree that the amount of user cpu time used by the nntpd processes is minimal, but when you add up all of the other resources used, especially memory, which still seems to be a valuable commodity even with 16M, news readers can impose a serious burden on a system. I know this for a fact because the machine that I use is also the news host. It not only processes all of the news, but it does the nfs serving and nntp serving for all of the people who read news on campus, and this causes a serious decrease in overall performance for the machine during busy times. -- When an eel bites your leg, and the pain makes you beg, that's a moray! If I spent as much time on my classes as I do reading news, I'd graduate. Brett McCoy Computing and Telecommunications Activities brtmac@maverick.ksu.ksu.edu Kansas State University
mjr@hussar.dco.dec.com (Marcus J. Ranum) (11/18/90)
Brett McCoy writes: >Something most people forget about here is the memory involved. Each nntp >connection requires that an nntpd process be running. If the client is even >moderately busy this involves several hundred K worth of memory being devoted >to the nntpd, which is several hundred K lost to any other process trying to >run. I assume you don't waste any of your memory with abominations like X-window, then. :) Hmmm... if your system is groaning from too many read-only nntp connections from rrn, I suppose you could switch to reading articles via NFS. I doubt you'd save much, though. mjr. -- "When choosing between two evils, give preference to the council of your tummy over that of your testes. The history of mankind is full of disasters that could have been averted by a good meal, followed by a nap on the couch." -Me, as explained to me by my wife's cat Strummer.
brtmac@maverick.ksu.ksu.edu (Brett McCoy) (11/19/90)
In <1990Nov18.035402.11348@decuac.dec.com> mjr@hussar.dco.dec.com (Marcus J. Ranum) writes: >Brett McCoy writes: >>Something most people forget about here is the memory involved. Each nntp >>connection requires that an nntpd process be running. If the client is even >>moderately busy this involves several hundred K worth of memory being devoted >>to the nntpd, which is several hundred K lost to any other process trying to >>run. > I assume you don't waste any of your memory with abominations like >X-window, then. :) > Hmmm... if your system is groaning from too many read-only nntp >connections from rrn, I suppose you could switch to reading articles via >NFS. I doubt you'd save much, though. I'm not saying that people shouldn't run nntpd, but the question was raised about how much of an impact it puts on system performance. I took it as meaning what will be the difference between a system running without any nntpd's and a system running with nntpd's. CPU wise there is very little difference, but when you take everything else into account, there is a performance impact. The difference between a system running X and one that isn't is also pretty large, but I put up with it. :-) Most of the news reading done here is through NFS. It is generally faster and since nfsd's are always running and the disk gets acessed one way or the other, I feel that using nfs to read news is better than using nntp. Less resources in the way of memory, context switching, etc. being used. -- When an eel bites your leg, and the pain makes you beg, that's a moray! If I spent as much time on my classes as I do reading news, I'd graduate. Brett McCoy Computing and Telecommunications Activities brtmac@maverick.ksu.ksu.edu Kansas State University
kordash@mitre.org (John Kordash) (11/19/90)
In article <1990Nov18.173733.21964@maverick.ksu.ksu.edu> brtmac@maverick.ksu.ksu.edu (Brett McCoy) writes:
Most of the news reading done here is through NFS. It is generally faster
and since nfsd's are always running and the disk gets acessed one way or the
other, I feel that using nfs to read news is better than using nntp. Less
resources in the way of memory, context switching, etc. being used.
This topic came up a couple of weeks (or has it been more than a month now?)
ago. However, to be best of my recollection, there wasn't and real good answer
to the NFS vs. NNTP question.
Does anyone have any hard facts/numbers to support either claim?
What about the UDP(NFS)/TCP(NNTP) issue?
Comments?? (or answers :-)
-John
--
-------------------------------------------------------------------------------
John L. Kordash kordash@linus.mitre.org
The Mitre Corporation or
Burlington Rd, Bedford MA 01748 ...!linus!kordash
(617)-271-2016
-------------------------------------------------------------------------------stealth@caen.engin.umich.edu (Mike Pelletier) (11/22/90)
In article <1990Nov17.054512.13632@maverick.ksu.ksu.edu> brtmac@maverick.ksu.ksu.edu (Brett McCoy) writes: > >Something most people forget about here is the memory involved. Each nntp >connection requires that an nntpd process be running. If the client is even >moderately busy this involves several hundred K worth of memory being devoted >to the nntpd, which is several hundred K lost to any other process trying to >run. Compile your nntpd with the "-n" option (on a DEC at least) to make the text portion read-only and shared among all users executing the file. That way, you will only have one copy of the executable sitting in memory at any given time, no matter how many users are connecting. Might also want to consider setting the sticky bit... Also, when you're talking about your experience with your NFS/NNTP/etc... server machine, you're not commenting on the performance load imposed by nntp connections, but rather on NFS+NNTP+whatever else you have going. -- Mike Pelletier - Usenet News Admin & Programmer "Wind, waves, etc. are breakdowns in the face of the commitment to getting from here to there. But they are the conditions for sailing -- not something to be gotten rid of, but something to be danced with."
crs@convex.cl.msu.edu (Charles Severance (System Manager)) (11/22/90)
This is indeed a tough question. I keep waiting for the answer to come out from someone who really knows based on some real testing. We use a SPARC/1 to serve 600-800 remote news readers per day. This machine is dedicated to news and no-one reads news on this machine at all. My first thought was that NFS was the way to go to because NFS is better at transferring large amounts of data and it would save memory on my machine. About 6 months later we had a major news reading crisis due to NFS. These are the dangers which we ran into which caused our problem: - With NFS you are at the mercy of the system administrator on the remote end as to the timeouts, etc. - While NFS is efficient at transferring large amounts of data, the act of searching directories and opening files has some overhead. News reading does a lot of opens and directory searches. - When NFS begins losing packets, it does not gracefully back off and assume that beating on a poor network does not improve its ability to respond quickly. - When NFS fails to deliver a packet after some number of furious retries the message that the user sees from the news reader is often very cryptic. Programmers doing reads and writes may or may not check every error code. Often the user sees a message like "SERVER Timeout: bad lstat" as their reader crashes. This points the blame to the server rather than a poor network or ill-configured local machine. As a contrast, NNTP uses TCP. - Most TCP implementations have an excellent backoff when network congestion rears its ugly head. TCP was designed to work across 9.6 lines taking 10 hops to get across the country. - The overhead of file opens and directory searches under NNTP is only in the host and with caching may not require a disk hit at all. - When news readers are developed which use NNTP, the programers generally include adequate error checking and user friendly messsages when networks fail. Using NFS, we had a major crisis using NFS as our primary mechanism for news access across a long campus backbone. Here is the story. There was a network problem which lost packts once in a while. The remote system administrators had the NFS timeout way too low and the remote servers didn't wait long enough before re-trying. This caused things to get much worse because most of the requests were actually getting to the server but the responses were getting lost. This made the server very busy accomplishing nothing. Eventually we fixed the problem by solving all of the problems: 1) bad NFS timeouts, 2) network problems, and 3) added memory to the server to eliminate paging completely. We did these things all at about the smae time so I can't assign a percentage of blame to each cause but I do know that all of the problems had some part in the situation. I also know that NNTP would have handled the network problem MUCH more gracefully. (Becaause no-one ever complained about a sluggish telnet during the crisis period). Now when people ask me whether to use NFS or not, I have to hem and haw. Perhaps if pushed, I might make the folowing recommendations: - Certainly use NFS if the systems are on one very reliable network which is never congested. I truly think that more simultaneous users can be supported by NFS than NNTP. Keep the retry timeouts very generous (5 sec or so) to give some backoff if network congestion occurs and packets start getting dropped. This also prevents retry requests and the response from the previous request from crossing on the network. - I would not use NFS if there is an IP router between the server and the remote news reading machine. The answer might also be a hybrid. Use NNTP to span "long" multi-hop distances and NFS for short distances. I still do not know how many simultaneous NNTP sessions are practical even with -n on the load command. I still want to hear some other experiences from the rest of the folks out there. Charles Severance Michigan State University 301 Computer Center East Lansing, MI 48824 internet: crs@convex.cl.msu.edu phone: (517) 353-2984 fax: (517) 353-9847 -- Charles Severance internet: crs@convex.cl.msu.edu Michigan State University phone: (517) 353-2984 301 Computer Center fax: (517) 353-9847 East Lansing, MI 48824 bitnet: 20095CRS@MSU