john@wa3wbu.UUCP (John Gayman) (02/08/88)
I'm currently attempting to have two systems feed me News. Is this advisable and if so, how can I set up my "sys" file so that all the news I get from each site doesn't get forwarded to the other site ? I put the "L" flag on the one feed but it looks like I never saw any of the News that came in last night. Any ideas ??? John -- John Gayman, WA3WBU | UUCP: uunet!wa3wbu!john 1869 Valley Rd. | ARPA: wa3wbu!john@uunet.UU.NET Marysville, PA 17053 | Packet: WA3WBU @ AK3P
paul@vixie.UUCP (Paul Vixie Esq) (02/10/88)
In article <484@wa3wbu.UUCP> john@wa3wbu.UUCP (John Gayman) writes: > > I'm currently attempting to have two systems feed me News. Is this >advisable and if so, how can I set up my "sys" file so that all the news >I get from each site doesn't get forwarded to the other site ? I put >the "L" flag on the one feed but it looks like I never saw any of the >News that came in last night. Any ideas ??? Unless you have several modems and a system powerful enough to take in two feeds at the same time, you should look into ihave/sendme protocol on one of the two systems. This is actually pretty easy to set up; I think you just use the :I: flag, and the batch file gets written in a different format. When you 'sendbatch', it sends a list of article-ID's to the remote system, and it sends back 'sendme' messages (in a batch) for the ones it hasn't got already. If the neighbor uses :I: for you as well, it'll be bidirectional. This imposes an extra delay on getting news, so you shouldn't use it on all your feeds -- all but one is how I run it at UB.Com. It is imperfect -- you can often get an article through another source between the time you send a 'sendme' and the time the article is actually sent by that feed. So you still get duplicates. Just not as many. The :Lx: flag is also very important -- and you need it on both sites, since what it means is "only send articles to this site which originated within x hops of my site." (I could be one off here.) :L0: means only the articles posted locally get sent to that machine, :L1: means your local articles and all articles local to your immediate netnews neighbors, etc. If you put this on one feed's sys line and not another's, you will be sending the :L:-less site all the articles you get from any other site. -- Paul A Vixie Esq paul%vixie@uunet.uu.net {uunet,ptsfa,hoptoad}!vixie!paul San Francisco, (415) 647-7023
jbuck@epimass.EPI.COM (Joe Buck) (02/11/88)
This article was originally in news.software.b, but it's really a news administration issue, so I'm directing followups there. In article <484@wa3wbu.UUCP> john@wa3wbu.UUCP (John Gayman) writes: > > I'm currently attempting to have two systems feed me News. Is this >advisable and if so, how can I set up my "sys" file so that all the news >I get from each site doesn't get forwarded to the other site ? I put >the "L" flag on the one feed but it looks like I never saw any of the >News that came in last night. Any ideas ??? There are many advantages to doing this. But I wouldn't advise it unless you want to consider your feeds as being two-way. That is: nei-a <----> wa3wbu <----> nei-b Remember that news will never transmit articles to a site if that site name already appears in the Path. Because of this, this system provides redundancy at low cost. If nei-b has problems, you'll get all your news from nei-a. If both are operating, and you contact your neighbors frequently, there won't be a whole lot of duplication, because a given article will arrive from nei-a and you'll pass it on to nei-b before nei-b attempts to send it to you. You'll also be supplying extra redundancy for your two neighbors in case one of their feeds goes down, so you're doing them a service. To set this up, write your sys file as if you were passing both nei-a and nei-b a full feed. If you have only these two neighbors, the total volume of outgoing articles will be equivalent to one full feed, and the total number of incoming articles will probably be around 1.5 feeds, less if you contact your neighbors frequently. Quite a few news nodes in this area are set up that way -- chains of sites with two-way feeds, with leaf nodes hanging off the sides of the chains here and there. It definitely increases reliability. -- - Joe Buck {uunet,ucbvax,sun,<smart-site>}!epimass.epi.com!jbuck Old Internet mailers: jbuck%epimass.epi.com@uunet.uu.net
paul@vixie.UUCP (Paul Vixie Esq) (02/12/88)
In article <802@vixie.UUCP> I write:
# [...] you should look into ihave/sendme protocol
# on one of the two systems. This is actually pretty easy to set up; I
# think you just use the :I: flag, and the batch file gets written in a
# different format. When you 'sendbatch', it sends a list of article-ID's
# to the remote system, [...]
Ooops. That's "sendbatch -i"...
--
Paul A Vixie Esq
paul%vixie@uunet.uu.net
{uunet,ptsfa,hoptoad}!vixie!paul
San Francisco, (415) 647-7023mikel@codas.att.com (Mikel Manitius) (02/12/88)
In article <484@wa3wbu.UUCP> john@wa3wbu.UUCP (John Gayman) writes: > > I'm currently attempting to have two systems feed me News. Is this > advisable and if so, how can I set up my "sys" file so that all the news > I get from each site doesn't get forwarded to the other site ? I put > the "L" flag on the one feed but it looks like I never saw any of the > News that came in last night. Any ideas ??? If it's advisable depends upon your resources. You should keep in mind that although the news system will not send an article to your neighbor if he has already seen it, it is possible, and highly probable you and your neighbors will have a high duplicate rate. a ------ b -- x \ / \ / \ / c If (b) posts something, he will send it to (a, c, x), (a) will send to (c). If (c) gets it from (b) first, it will send to (a). However if (x) posts, it will send to (b), who will send to (a, c) which will in turn feed each other, causing dups. Duplicates can be avoided using ihave/sendme. In ihave/sendme each site will first send the other a list the article IDs of what it has. The receiving site will then request what it doesn't have. This is good, however it causes time delay between reception of messages, and duplicates may still occur, because (c) may receive an article from (b) after it already requested it from (a). An ihave/sendme sys entry, using batching looks like this: foovax:[newsgroups]:FI: With a sendbatch cron entry of: sendbatch -i -c foovax It also requires "to.foovax" to exist with the proper distribution. Another possibility is for (c) to send only articles to (a) that have not been to (b). This can signifigantly reduce duplicates in a small network, but the topology of USENET is too large for a site such as for example "codas" which has many feeds, to set this up effectivley. A sys line for (c), not to send things to (a) that have been through through (b) could be: a/b:[newsgroups]:F: The "a" is the site you're feeding, everything after the slash, up to the colon is a list of sites that should not have seen the message if it is to be sent. This could be "a/b,x:", to include (b) and (x). Finally, the "L" type in the third field. A simple "L" sais to feed only things that were posted locally. An "Ln" where "n" is a number between 1 and 9, sais to feed only things that were posted at a maximum of "n" hops away. It doesn't approach the above issues. Other possibilities exist, the above are most appropriate for what you are trying to do. -- Mikel Manitius @ AT&T mikel@codas.att.com
james@bigtex.uu.net (James Van Artsdalen) (02/15/88)
I have been experimenting with "delayed" ihave/sendme messages recently to use with backup news feeds. The impetus for this was a nearly month-long disruption in the news feed for bigtex due to downtime at an upstream site. bigtex has a number of PC Pursuit links used for mail around the country. Because of connection difficulties they aren't generally useful for carrying news. However, it is possible to do so, and desirable if there is a major feed disruption. The goal is to have the backup PC Pursuit feed kick in without human intervention. The regular ihave/sendme isn't quite what is desired, because it gets the ihave message to the remote system too quickly. The "regular" path to the remote system may be a half dozen hops or more, and the direct PC Pursuit call will often beat the regular route (for messages generated nearby). The result is that the remote system gets the ihave before getting the article, and generates the sendme even though the article is likely to arrive soon via the regular route. For example, if I post a message on bigtex in the evening, it is probable that the ihave would reach remote-sys before the article itself did via the normal path. What I have done is slightly modify the sendbatch script in the following manner: if test -n "$DOIHAVE" -a -s /usr/spool/batch/$rmt.ihave then mv /usr/spool/batch/$rmt.ihave /usr/spool/batch/$rmt.$$ ( echo /usr/lib/news/inews -t \"cmsg ihave $DOIHAVE\" \ -n to.$rmt.ctl '<<END_IHAVES' cat /usr/spool/batch/$rmt.$$ echo END_IHAVES ) | at now + 3 days 2>/dev/null rm -f /usr/spool/batch/$rmt.$$ else This queues up an "at" job to run three days hence. Unfortunately this doesn't work as-is on BSD systems: I don't know what the BSD equivalent of "at now + 3 days" is. I stuff the article-ids into the batch job for lack of a better place to put them. The result is that the article I posted on bigtex now has at least three days to get to remote-sys before the ihave gets there: remote-sys is very unlikely to see the ihave first unless something is wrong with the news paths, in which case the article *should* be sent via bigtex!remote-sys. Assuming that bigtex expires news no quicker than seven days, the news should still be available if remote-sys sends a sendme. Of course, the prospect of a 10K/day mail link zooming to 2meg/day+ when this kicks in must not be forgotten by anyone allocating dialtone... If bigtex and remote-sys exchange ihaves in this manner, ihaves are normally sent but almost never acknowledged with sendmes, because the news arrived at each system within three days via conventional means. But if remote-sys's feed breaks for some reason, the articles that were lost will be automatically sent by bigtex for the duration of the outage. Once the feed to remote-sys is fixed, remote-sys stops issuing sendmes, and both systems automatically revert back to normal feed paths. I had not had a chance to thoroughly test this yet, but hope to soon. I would be interested in hearing from anyone doing this now. -- James R. Van Artsdalen ...!uunet!utastro!bigtex!james "Live Free or Die" Home: 512-346-2444 Work: 328-0282; 110 Wild Basin Rd. Ste #230, Austin TX 78746