merce@iguana.uucp (Jim Mercer) (05/15/91)
[ please note that i am under the buzz of a couple beers (a lot for me) and i really probably shouldn't be posting right now, but what the heck. ] [ see end of article for pleas for help ] [ speaking as jim@lsuc.on.ca, for reasons illustrated below ] [ lsuc.on.ca is a "central" mail hub in toronto, all connections are via dial-up, no TCP, no SLIP, just uucp ] it seems that every 6 weeks or so, some bonehead on one of our ~75 mail connections, decides to BITFTP something huge. first it was a 60 Meg VMS utility (attributed to ignorance). next it was a 12 Meg VMS uucp suite (same bonehead, attributed to lack of respect of the 'net) [ BTW: the bonehead in this case, when contacted via voice, told me "if you can handle the volume, get out of the gateway business", to which i (wished) i said "fuck you and the mutant OS you live in" ] last night it was another bonehead ordering gcc source and gas source (16 Meg) (attributed to ignorance). [ the mail processing on this bopped the load average up to 17.00+, as well as "discovering" some bad blocks on the drive which the system want's to use for inode tables. do i need this? ] i've about had my fill of this. effective now, we will be developing scanners to trash BITFTP and listserve type requests flowing via lsuc. that is requests and responses. (any hints on keywords would be appreciated) lsuc's mail/uucp system over flowed last night, resulting in unknown quantities of news and mail being dropped on the floor. coincidentally, our news partition ran out of inodes at the same time and the entire disk seems trashed. i spent an hour and a half (11:30pm - 1:00am) last night, doing damage control from at home (in kitchener). i still managed to get up and catch my bus into toronto. today (Tuesday), i spent no less than 3 hours trying to put the disk back together so we could get news and mail back up. i had to leave before the job was done (pre-natal class at 7:00pm in kitchener) and i had to leave instructions on how to shut down news before i left. when i got home, i called in and news was shut down (newsrunning off). i'm not sure what kind of news loss we'll have (major for sure) and i don't know if mail is totally functional. (the file systems were messed up) i am really peaved. [ ok jim, calm down (previous 6 lines of expletives deleted) ] Education! Education! Education! Education! Education! Education! Education! Education! we can put in place all the filters we want, but the only way to resolve this issue of file transfer by email is Education. Henry Spencer defined email at a user group meeting as (paraphrase from (currently fuzzy) memory) "text entered by hand, ie. not machine generated". file transfer by email would be fine if all of the hubs had infinite disk space for spooling stuff up to dial-up sites, but we don't. i might also note that in all 3 cases of abuse, the requested items were more than likely available locally. and if not, were of interest to the local community (ie. someone at UofToronto could have ftp'd it). please tell your users to post to the local *.general groups to see if it is local. .... how much of a net.lobby do we have to do to get pucc.princeton.edu to shut down BITFTP? can we at least get them to limit responses to systems that can be verified as being on BITNET (as i assume the system was intended)? or maybe, get them to limit responses to "official" internet sites? grrrrrrrrrr!!!!!!!!! this really pisses me off. HELP: i am looking for the following "tools": - Cnews spacefor that checks remaining inodes as well as free blocks - efficient rmail frontend which will "act" on key phrases in the To: and From_ headers - how to mark bad blocks on a 3B2/500 (SysV 3.2.1) please reply to jim@lsuc.on.ca (as i have tried to set the Reply-To: header) thanx -- [ Jim Mercer work: jim@lsuc.on.ca home: merce@iguana.uucp +1 519 570-3467 ] [ "Anarchists Unite!" - seen spray painted on a wall ]
carlo@electro.com (Carlo Sgro) (05/15/91)
In article <1991May15.042146.29800@iguana.uucp> jim@lsuc.on.ca (Jim Mercer) writes: >it seems that every 6 weeks or so, some bonehead on one of our ~75 mail >connections, decides to BITFTP something huge. My sympathies. We are a smaller site with a few connections and plenty of disk space but no modem power (2400 baud max. speed). We have had so many problems with people using BITFTP (and other large file transfer) tying up our lines (and our main feeds' lines) that we have had to drastically restrict mail through our site. Luckily, we have had cooperation from those connected to us (and those downstream from them). >effective now, we will be developing scanners to trash BITFTP and listserve >type requests flowing via lsuc. I'm sure that I and many others would be interested in this, but ... >we can put in place all the filters we want, but the only way to resolve this >issue of file transfer by email is Education. > ... >i might also note that in all 3 cases of abuse, the requested items were more >than likely available locally. and if not, were of interest to the local >community (ie. someone at UofToronto could have ftp'd it). Damn right! There are people out there who believe that having a modem and a UUCP connection means that they have god-given rights to do whatever they want. There are many, many more that don't believe anything much but just fail to think before they act. We at Electrohome might seem like a large company with money to spend on Telebits and disks and the such. However, there are private systems who have better setups than we do. The bean-counters don't know anything about UUCP connectivity. We're probably lucky for that. However, it also means that a Telebit is something that I've tried to get for almost 3 years. We depend on the good grace of the large sites to which we connect. We simply don't have the resources and can't risk losing the good grace of our neighbours by transferring large reams of BITFTP stuff that could be more easily obtained by using a bit of resourcefulness. >please tell your users to post to the local *.general groups to see if >it is local. Would it be a desirable thing to set up local groups specifically for this sort of thing? I would think that it would be easier for a neophyte leaf admin to find out about kw.software (as an example) than to find out about BITFTP. -- Carlo Sgro Not a card-carrying member of the watmath!watcgl!electro!carlo Laurie Bower Singers Fan Club. carlo@electro.com System Administrator, Electrohome, Ltd., Kitchener, ON, (519)744-7111x7210
tower@buitc.bu.edu (Leonard (Len) H. Tower Jr.) (05/16/91)
You might talk to the folks who run the BITFTP gateway, and see if they could slow down the rate at which they mail a large request. 50k an hour? That requires them to have a lot of spooling space, but would limit the harm done small systems who forward mail themselves. Brian Reid's mail-based server (in use at a lot of Unix sites) does something like this on a per-address basis. You might also see if you can configure your mailer to bounch (or bit-bucket (if you want to be rude)) messages larger then a certain size (64k is traditional between UUCP hosts, though I've seen limits as small as 25k from some of the oversea gateway machines). I personally think any mail-based server that distributes any large packages (source, data, et al) is doing a dis-service to the Matrix (the world wide net as defined by John Quarterman). It causes sites who are willing to enlarge the community by passing small quantities of human generated mail along to stop cooperating due to resource use much larger then they can handle. This makes the net a smaller and less useful community for all of us. thanx -len
lyndon@cs.athabascau.ca (Lyndon Nerenberg) (05/16/91)
[ Sorry, but NN trashed the original group list. I didn't think ont.general was a very good place to post to from Alberta, so I faked it ... ] tower@buitc.bu.edu (Leonard (Len) H. Tower Jr.) writes: >You might talk to the folks who run the BITFTP gateway, and see if >they could slow down the rate at which they mail a large request. What we should really do is pressure the BITFTP sites to only honour requests from machines that are in the BITNET RSCS node tables. This would restrict the traffic to the NJE leased lines, which are usage insensitive as far as charging goes. This doesn't solve the problem of BITFTP requests plugging up the RSCS queues at the BITNET hub sites, but that's not what we're discussing :-) There are enough machines out there providing anonymous UUCP access to software archives that use of BITFTP from UUCP sites is no longer justifiable. If you want the software, you can bloody well pay your own phone line charges to pick it up. Maybe you didn't want that software so badly after all ... As an example of making the user pay, about a year ago I decided to stop carrying the comp.binaries groups here at AU. There were a number of reasons for doing this: potential viruses and cost in modem time and disk space being two of them. When we informed the user community of the change there was no end of howling and bitching from the PC users who were used to getting all this "free software" from the net. We pointed out that it was not "free," as the university was paying line charges (both leased lines and LD dialup) to transfer these files, most of which were of little or no use to the operation of the university. In essence, we said that if they *really* *wanted* the software, they could dial up any number of BBS's (from home on their dime) and pick it up that way. After a few weeks the complaints stopped coming in. From what I've heard (or not heard), not providing the binary groups has not resulted in the end of the world as these people know it, so I assume that they either don't need the code that badly, or they have made alternate arrangements. Yes, a few of them starting poking the various mail archive servers, however the traffic flow was not substantial (very few people even knew about them), and has trickled off to almost nothing. My conclusion is that they got tired of having to take a proactive stance to get the stuff and lost interest in it when we stopped handing it to them on a platter. What you describe is an unfortunate side effect of running a large mail relay site. I've run a few of them in my time and I've dealt with this problem before. I find the best way to deal with it is to make it very clear to the system administrators of any site you provide e-mail connectivity to just what sort of traffic you are willing to forward. Let them know that any violation of those guidelines could (and probably will) result in the termination of their feed. Place the responsibility upon them to police their own user community. In nearly every case, they will do so. If you see things happening that you don't like, contact the system administrator at the site that's responsible, and let them know what's happening right away! It is very important to keep the lines of communication open. If the person at the other end feels like they are being dumped on arbitrarily, they will be much less sympathetic towards *your* problem. (And it *is* your problem - you set yourself up in the business of providing e-mail service in the first place.) The bottom line is, be fair to others - they probably don't know they screwed up. Respect the other site administrators - they probably don't like these things any more than you do, and will do their best to deal with it as soon as they are made aware of the problem. Don't change the rules in the middle of the game - make it *very* *clear* at the *outset* what sorts of traffic and volumes you are willing to accept, and make the continued provision of the feed contingent on their following those guidelines. How you deal with abuses is up to you. I prefer a solution that doesn't penalize users who did not contribute to the problem. However you accomplish that is going to be very site dependent. I wish you luck ... [ BTW - most of our problems went away when we got onto the Internet. I now run all my non-local UUCP connections over TCP. Keep pushing for that ONET link :-) ] -- Lyndon Nerenberg VE6BBM / Computing Services / Athabasca University atha!cs.athabascau.ca!lyndon || lyndon@cs.athabascau.ca Packet: ve6bbm@ve6bbm.ab.can.noam The only thing open about OSF is their mouth. --Chuck Musciano
moraes@cs.toronto.edu (Mark Moraes) (05/18/91)
stanley@phoenix.com (John Stanley) writes: > No! This is a valuable service to the entire UUCP community. Well, at >least it is here. Ironic. It was intended for the Bitnet community (who play by different rules and have size grading on files, er, virtual card decks). I agree that it would be a valuable service for uunet/uupsi/your-favourite-internet-uucp-service-provider to make a mail based archive service available for their directly connected customers. (I thought uunet already did this) Take that up with them. (I just hope they implement it to cope with several hundred people simultaneously requesting the X.V11R5 distribution :-) Most complaints about mail based archive servers (MBAS) come from the people who provide free uucp connections to their neighbours to improve mail connectivity or because they're nice folk who think a global network is A Good Thing. Usually these people two or three hops upstream from the eventual destination of the MBAS mail. The difficulty is both disk space and modem time tied up by large file xfers. UUCP isn't packet switched, alas. Not much flow control either. Picture all those files from the MBAS piling up behind a Telebit which is the 100Kbps -> 10Kbps chokepoint! At present, we keep logs on MBAS traffic and send "please don't route this mail through us" messages to people who are recipients of a lot of such traffic. It seems to be working fairly well, except for the occasional glitch (the mail Jim was complaining about passed through us too!) and the human cost in monitoring the logs. An option we're considering is to modify our mailer to downgrade messages greater than M Kbytes and remove downgraded messages if they're older than N hours. (may as well learn from Bitnet :-) More human cost, but then, one of utai's postmasters is interested in exploring the frontiers of mailer science! Mark. -- "Coping with Mail Based Archive Servers" -- coming soon to a thesis near you.
andy@mks.com (Andy Toy) (05/18/91)
In article <1991May15.135732.9749@electro.com> carlo@electro.UUCP (Carlo Sgro) writes: >Would it be a desirable thing to set up local groups specifically for this >sort of thing? I would think that it would be easier for a neophyte leaf >admin to find out about kw.software (as an example) than to find out about >BITFTP. There have been occasional postings in ont.archives, can.usrgroup, and {kw,ont,can}.uucp newsgroups asking for software locally. It may be desirable to have local newsgroups with more descriptive and consistent names. Otherwise, use the existing local *.uucp and *.archives newgroups. -- Andy Toy, Department of Computing Services, Extension 31, second floor annex
glenn@gla-aux.uucp (Glenn Austin) (05/24/91)
In article <91May18.002523edt.1028@smoke.cs.toronto.edu>, moraes@cs.toronto.edu (Mark Moraes) writes: > stanley@phoenix.com (John Stanley) writes: > At present, we keep logs on MBAS traffic and send "please don't route > this mail through us" messages to people who are recipients of a lot of > such traffic. It seems to be working fairly well, except for the > occasional glitch (the mail Jim was complaining about passed through us > too!) and the human cost in monitoring the logs. > > An option we're considering is to modify our mailer to downgrade messages > greater than M Kbytes and remove downgraded messages if they're older > than N hours. (may as well learn from Bitnet :-) More human cost, but > then, one of utai's postmasters is interested in exploring the frontiers > of mailer science! I seem to recall limitations of number of files and/or size as well over Bitnet. I really don't care WHEN I get the file, I just would like to GET the thing! =============================================================================== | Glenn L. Austin | "Turn too soon, run out of room. | | Macintosh Wizard and | Turn too late, much better fate." | | Auto Racing Driver | -- Jim Russell Racing School Instructors | |-----------------------------------------------------------------------------| | Usenet: glenn@gla-aux.uucp | CI$: 76354,1434 | | GENie: G.AUSTIN3 | AOnline: GAustin | ===============================================================================