eric@snark.UUCP (Eric S. Raymond) (09/23/88)
In article <207@cvbnet2.uucp>, aperez@cvbnet2.UUCP (Arturo Perez Ext.) writes: > What I'm curious about is the fact that I've never heard of any record > access libraries for Unix. I know that I've written simpleminded record > access applications. I'm sure other people have as well. Is there anyone > actually selling record access libraries for the Unix community? If not > why isn't anyone doing it? Shortest answer: because it's not worth doing. Medium-length answer: because, for the most common case of sequentially- accessed fixed-length records, 'record access' under UNIX is adequately handled from C by fread(3)/fwrite(3) of a struct buffer. The less common cases are traditionally handled by ASCII encoding with a \n-terminated line structure, the one kind of variable-length 'record' for which there is strong support under UNIX. Long answer: well, what does a record-access library *do*, anyhow, that byte- stream access doesn't give you? There are basically, two possible answers: 1. Encourage code simplification by provision of 'higher-level' primitives. 2. Access speed or space optimization. Both of these turn out to be mostly illusion. UNIX's byte-stream primitives are sort of ultimately simple, certainly a lot simpler than the thicket of RMS entry points. The text-file support in stdio is pretty clean and basic too (which is why it's become a de-facto standard for C implementations even on non-UNIX systems). A possibility of significant speed or space optimization over stdio or simple fread/fwrite only enters if you're talking about access to variable-length records that can't reasonably be represented in ASCII. Such applications tend to a) require indexed access, in which case you're already talking database and the 'record library' likely has to be custom to handle housekeeping info, or b) be sufficiently speed or space-critical that no savvy programmer is really going to like someone else's packaged library for access. And don't forget that at some level the record-access has to be doing byte-stream I/O *anyway*. The only way it can win is by buffering. And wouldn't you rather tune your own buffer sizes? Record-access libraries sound like a decent idea, but in practice they tend to introduce a lot of interface complexity (which makes your code ugly) and premature optimization (which actually hurts performance). VMS's RMS could stand as a perfect example of both these lossages. -- Eric S. Raymond (the mad mastermind of TMN-Netnews) UUCP: ...!{uunet,att,rutgers}!snark!eric = eric@snark.UUCP Post: 22 S. Warren Avenue, Malvern, PA 19355 Phone: (215)-296-5718
bzs@xenna (Barry Shein) (09/25/88)
Eric, I disagree and think you dismiss the value of record access libraries too glibly. There's more to it than fixed and blocked record formats. Consider various associative schemes like ISAM or B-TREE management of records by keys (dbm is similar to this under Unix.) These can be very critical to manageable performance in certain areas, particularly the management of large numbers of records where lookup is only by a very few keys and quick insertion of new key/record pairs is needed. Your typical example is a customer data base where lookup/insertion keys is based on some customer code. Particular implementation strategies are a whole other kettle of fish. -Barry Shein, ||Encore||
eric@snark.UUCP (Eric S. Raymond) (09/27/88)
In article <3717@encore.uucp>, bzs@xenna (Barry Shein) writes: > Eric, I disagree and think you dismiss the value of record access > libraries too glibly. There's more to it than fixed and blocked record > formats. > > Consider various associative schemes like ISAM or B-TREE management... I agree that libraries like UNIX dbm are a Good Thing -- but then ISAM, B-TREE management, and other associative retrieval schemes are not within the scope of the original question. Let's not muddy the waters by confusing 'record access' in the RMS "fixed and blocked record" sense with more general database access techniques. -- Eric S. Raymond (the mad mastermind of TMN-Netnews) UUCP: ...!{uunet,att,rutgers}!snark!eric = eric@snark.UUCP Post: 22 S. Warren Avenue, Malvern, PA 19355 Phone: (215)-296-5718
oz@yunexus.UUCP (Ozan Yigit) (09/30/88)
[disclaimer: I am an ex-VMS-hack, used RMS extensively in the past, like its power, puke at the sight of its interface, and here, I am speaking of facilities, rather than a messy filesystem format to support multiple file formats] In article <e1Lt8#2CJhqZ=eric@snark.UUCP> Eric S. Raymond writes: > ...other associative retrieval schemes are not within the scope >of the original question. Let's not muddy the waters by confusing 'record >access' in the RMS "fixed and blocked record" sense with more general > database access techniques. They happen to be an integral part of RMS, and extremely well done in functionality, leaving the "goriness" of the interface aside. [Reportedly, even the designer of RMS admitted to its ugliness :-)] If I had a "library" half as powerful as some of RMS's facilities, I would be damn happy... It is a shame that best un*x can come with is ndbm (and that is, if you are lucky :-) Why do you think andrew@alice (Bell Labs) was asking about something that can handle gigantic databases ?? Aren't you curious why more and more programs across variants of un*xes rely on dbm/ndbm type databases ?? It must be RMSicide !! RMS Record access you say ?? this concept is especially revolting for our purist(!) byte-stream-with-a-nl (usually 512 bytes or shorter for idiotic programs :-) perspective. Alas, we merrily keep faking that our byte stream is in fact a collection of average size lines (records for less religious :-) and rush to our soapboxes at the suggestion that something as ugly as RMS can actually be useful. (*because* of its support of "more general database access tecniques".) You know, there is no such thing as "a little bit of purity". On the other hand, crap is crap. [Sounds of shuffling a plane ticket to alaska, to escape flames...] oz -- Reflections are | Usenet: ...!utzoo!yunexus!oz images of tarnished aspirations. | ...uunet!mnetor!yunexus!oz RACTER | Bitnet: oz@[yulibra|yuyetti] [an Artifically Insane program.] | Phonet: +1 416 736-5257x3976
ccdn@levels.sait.edu.au (DAVID NEWALL) (10/05/88)
In article <864@yunexus.UUCP>, oz@yunexus.UUCP (Ozan Yigit) writes: > RMS Record access you say ?? this concept is especially revolting for our > purist(!) byte-stream-with-a-nl (usually 512 bytes or shorter for idiotic > programs :-) perspective. Alas, we merrily keep faking that our byte > stream is in fact a collection of average size lines (records for less > religious :-) and rush to our soapboxes at the suggestion that something > as ugly as RMS can actually be useful. (*because* of its support of "more > general database access tecniques".) I know little about VMS; not nothing, because the Institute (where I work) uses VMS as it's priniciple operating system. On the other hand, before I joined the Institute, `it' knew little about Unix; Unix is a new thing here. Something that has surprised me a few times, is how often RMS seems to get in the way of what you are doing. For example, there was a tar tape written on some Unix machine, from which some files were to be extracted. Easy to do if you have pd-tar running on VMS, but we didn't. (This actually happened before I started working for the Institute, and so my understanding of the problem is vaugue at best). The thought was that the tar file could be extracted from the tape and kermitted to the Unix box. (We had just got the Unix machines, you must understand, and hadn't yet got sophisticated things like tape drives or ethernet cards). I don't quite understand why that didn't work, but I recall people mumbling about "RMS" and "File Type" or "Record Format". Or then there was the time I suggested backing up our Unix machines across ethernet, onto the VAX tape drive. The answer then was similarly something about "FILES-11", "too difficult". Please excuse my quite obvious ignorance of VMS; I use it very little. This is not intended as a criticism of VMS. I am just surprised at the apparent stumbling blocks one can encounter using RMS. The power provided seems to have cost in terms of flexibility. David Newall Phone: +61 8 343 3160 Unix Systems Programmer Fax: +61 8 349 6939 Academic Computing Service E-mail: ccdn@levels.sait.oz.au SA Institute of Technology Post: The Levels, South Australia, 5095