gary@dgcad.SV.DG.COM (Gary Bridgewater) (03/14/90)
In article <Mar.11.12.07.46.1990.10122@pilot.njin.net> limonce@pilot.njin.net (Tom Limoncelli) writes: >Compressed news would work really well if it was handled by the file >system, and could be transparent to the applications. > >Method #1: Dynamically uncompress a file when it's opened. Use a LRU >algorithm to decide when to re-compress it. Of course, if you grep >through your entire partition you will find the entire disk >uncompressed. This has the basis for a fairly big win. Doing one file at a time is not space effective since a large percentage of the news files are within the N/C size of compress ( a previous posting the last time this issue came up made that clear ). What is needed is an NFS style filesystem spoofer that pretends to be /usr/spool/news externally. Internally, it keeps larger compressed chunks in a big file (or group of files) indexed in some clever way (hash article IDs spring to mind). In essence - it becomes a news database server. This is similar to the concept of SQL servers. The intrepid developer would, of course, add TCP support (could be RPC based but that leaves out a lot of systems) so the thing was naturally distributable, but not necessarily so. All existing software continues to work, to the extent that it is needed. However, expire would be built in (news comes in, old news goes out) as would the history function (you already hashed the article id in order to store it). This would use a fair number of cycles on the server but I really wonder if it would use more than inews/expire use already? It would be fighting the file system optimizer, in some sense, but it might end up beating up the disks less, in the long run. -- Gary Bridgewater, Data General Corporation, Sunnyvale California gary@sv.dg.com or {amdahl,aeras,amdcad}!dgcad!gary The impossible we understand right away - the obvious takes a little longer.