forrest@sybase.com (09/21/89)
I used to work at Britton Lee. Now I work at Sybase. At Britton Lee
query languages were parsed in the clients and parse trees were sent
to the server. Here at Sybase we send the query language to the
server to be parsed.
I'd like to hear peoples comments about the strengths and weaknesses
of both approaches. I'll start.
Assuming the client runs on a machine that supports sharable code,
the parser can be shared among all the clients. If the client runs
on a machine that doesn't support shareable code then this is a big
disadvantage since each executable and executing image must contain
the parser. Obviously, having a server do the parsing result in only
one copy of the parser even if the platform it runs on doesn't support
shareable code.
It could be argued that parsing is not data serving. If you accept
this opinion then you probably don't think that a server should
do parsing.
Parse trees might result in less network traffic than actual query text.
Detecting syntax errors in the client can prevent sending a
syntactically incorrect query over a potentially slow network.
It would take fewer resources to find out that a query is syntactically
incorrect.
I'm sure there are many other concerns that I'm missing.
Jon Forrest
forrest@sybase.com
{pacbell,sun,{uunet,ucbvax}!mtxinu}!sybase!forrest
415-596-3422jeffl@sybase.Sybase.COM (Jeff Lichtman) (09/21/89)
> At Britton Lee > query languages were parsed in the clients and parse trees were sent > to the server. Here at Sybase we send the query language to the > server to be parsed. I believe the latter approach is better. The world is moving toward open, standardized interfaces. There is already an ANSI SQL standard, and a subcommittee of ANSI X3H2 is working on remote database access protocols. Eventually, a client will be able to run queries on anyone's server without having to know much about the server. If sending parse trees across the network becomes the standard, then the format of the parse trees and the protocol for sending them will have to be part of the standard. RDA stuff is hard enough without introducing this problem. > Detecting syntax errors in the client can prevent sending a > syntactically incorrect query over a potentially slow network. > It would take fewer resources to find out that a query is syntactically > incorrect. This is only important when users are doing ad hoc queries. When most of the queries are embedded in the application, or are pre-compiled, there is next to no possibility of getting a syntax error. --- Jeff Lichtman at Sybase {mtxinu,pacbell}!sybase!jeffl -or- jeffl@sybase.com "Saints should always be judged guilty until they are proved innocent..."
jas@postgres.uucp (James Shankland) (09/21/89)
In article <6155@sybase.sybase.com> forrest@sybase.com writes: >I'd like to hear peoples [sic] comments about the strengths and weaknesses >of [parsing a query in the client & sending a parse tree to the server, >vs. sending query language to the server & letting the server parse]. Not sure it makes an awful lot of difference -- parsing a query is pretty cheap, compared to all the other work that has to be done to compile it. ("Parsing" here means just the context-free stuff; the semantic checking pretty much has to be done in the server, which has the system catalogs.) As you note, parse trees may be more compact than query language (read "SQL"); and syntax errors can be caught in the client. On the other hand, passing readable stuff back and forth may make for easier debugging. Also, given that SQL has become the relational *lingua franca*, sending SQL to the server may make interoperability easier. As a matter of commercial practice, I'd say sending SQL to the server is the way to go. jas
klint@usdtsg.Dayton.NCR.COM (W. E. Klint) (09/21/89)
In article <6155@sybase.sybase.com> forrest@sybase.com writes: >...At Britton Lee query languages were parsed in the clients and parse >trees were sent to the server. Here at Sybase we send the query >language to the server to be parsed. > >I'd like to hear peoples comments about the strengths and weaknesses >of both approaches. > The following is an article I posted for NCR distribution. I post it here in reply to your request, and in hope that it contributes to what I consider an essential question of future database/query architecture. ****************************************************************************** The following is ...[text deleted}...intended to generate discussion on client/server database architectures. Perhaps this discussion can precipitate, at the very least, clarification of a perceived problem, perhaps acknowledgement of a problem, and, at best, architectural changes. SQL has become, or is becomming, the de-facto standard relational database query language. The pure implementation of SQL in a client/server architecture seems rediculously simple; the client passes an SQL command to the server where it is parsed and executed and the solution set then is returned to the client. In actuality, it appears that almost all proprietary database vendors have seen fit to address the most economically expedient short term solution; convert SQL commands to their native database language commands, at the client, send them to the server, where, if the server runs their proprietary DBMS, everything is copacetic. However, if the server happens to run another proprietary DBMS, an interface must be built to convert vendor A's proprietary calls to vendor B's proprietary calls, or, since all vendors support SQL, convert the proprietary calls back into SQL where they can easily be converted to vendor B's proprietary DBMS calls. I know that off-loading processing to the client is one of the benefits of the client/server architecture. But the above described implementations seem to double the amount of processing required for data retrieval at *both* the client and the server! If ...[text deleted]... vendors were to incorporate a "pass-through" facility where, if selected, one could choose not to utilize that particular vendor's call library, but merely pass the ASCII SQL text through to the communications interface for transmittal to the server, things would be greatly simplified. At the server (whatever the DBMS), that SQL could be converted to database calls and a solution set derived and passed back to the client. This would provide for truly *open* systems. An ASCII SQL command could come from a batch program, an interactive terminal or a client application from any "foreign" client. Any server that recognizes ASCII SQL could then use whatever means necessary to satisfy that particular request. This would allow vendors to implement "back-end" strategies to best satisfy customer (client) demands.... ****************************************************************************** The checking of SYNTAX at the client should be considered a MUST. Its not worth network overhead if the query is syntactically incorrect. However, if we are to achieve truly OPEN DISTRIBUTED database systems, I feel we must let the servers interpret the query, generate the appropriate library calls (for their specific DBMS), compile a solution set, and return this solution set to the client. Please note that these are personal opinions and do not necessarily reflect those of NCR. -- William E. Klint USDPG, Information Systems & Services, Systems Support NCR Corporation, SDC-5 | Bill.Klint@Dayton.NCR.COM 1700 S. Patterson | klint%usdtsg.Dayton.NCR.COM@RELAY.CS.NET Dayton, OH 45373 (513) 445-3483 | klint%usdtsg.Dayton.NCR.COM@uunet.uu.net
hargrove@harlie.sgi.com (Mark Hargrove) (09/22/89)
In article <6155@sybase.sybase.com> forrest@sybase.com writes: >Parse trees might result in less network traffic than actual query text. But the standards committee necessary to decide upon the X.nnn Standard Relational Parse Tree Format will generate sufficient extra network traffic arguing over the issue to make up for any possible savings over, say, a 10 year period :-) It doesnt' really seem like anyone's arguing this issue: consensus is for pushing SQL (or CL/1, or whatever) down the pipe in native format; let the server figure it out. The Hard Problem here is getting vendors to agree on what "standard" SQL means. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Mark Hargrove Silicon Graphics, Inc. email: hargrove@harlie.corp.sgi.com 2011 N.Shoreline Drive voice: 415-962-3642 Mt.View, CA 94039
mao@eden (Mike Olson) (09/22/89)
> From: forrest@sybase.com (Jon Forrest) > I used to work at Britton Lee. Now I work at Sybase. At Britton Lee > query languages were parsed in the clients and parse trees were sent > to the server. Here at Sybase we send the query language to the > server to be parsed. with the release of britton lee's server/8000 (known to old-timers like jon as the omega), this won't be true. or qualify that: the full omega release, including new host software, will send sql strings to the server for parsing. i'm not sure where the 8000 is in the release cycle right now. among the problems with this approach: users can't write their own query languages as easily, though this probably doesn't concern anyone outside the research world. also, parameterized strings (that is, substituting arguments into the string you send) is harder, since the host needs to know enough about syntax to figure out when to send arguments. you can substitute "front end" and "back end" for "host" and "server", here, if that's what floats your boat. probably the worst thing about the decision, though, is that it sounds the death knell of idl, sharebase's quel-like query language, and further propagates sql. given that i'm now working on the postgres research project, heir to ingres, you can imagine where my sympathies lie. jon's points regarding network traffic, sharability, and early error detection are all good. also an issue: when compiling a C program that contains embedded database queries, the data server must be up in order to guarantee that there are no syntax errors in the program. for the record: britton lee is now called sharebase. i no longer work at sharebase, and do not presume to speak for the company or the people there. as an observer, i believe that the decision to move parsing to the server is a mistake. it doesn't add any new functionality, and it has required substantial effort to put in place. mike olson postgres research group mao@postgres.Berkeley.EDU
jkrueger@daitc.daitc.mil (Jon Krueger) (09/22/89)
forrest@sybase.com writes: >Assuming the client runs on a machine that supports sharable code, >the parser can be shared among all the clients. True, as long as all clients execute on the same machine. This is not the direction that most database solutions are heading. >Parse trees might result in less network traffic than actual query text. Both are probably small compared to network overhead. Database applications that spend more time communicating than computing represent (a) poorly partitioned problems, (b) lack of power in the data model, (c) bad database design or implementation, (d) poor development tools, or (e) inherently large data types, e.g. images, with respect to i/o bandwidth, or (f) one or more of the above. The choice of form to send queries to the server has some effect, but it's probably small. >Detecting syntax errors in the client can prevent sending a >syntactically incorrect query over a potentially slow network. >It would take fewer resources to find out that a query is syntactically >incorrect. Fewer bottlenecked resources, perhaps. But isn't that what the validations, parsing, and generation logic in the front end tool is for? In other words, isn't this a solved problem, at least for well behaved applications? One advantage to using the query language is that it's a well understood form. You can build in correctness checks that are not specific to a given engine, parsing strategy; you can even build tools to check correctness for arbitrary operations on given schema. This property is less accessible if you use a different intermediate form for internal communication. -- Jon -- Jonathan Krueger jkrueger@daitc.daitc.mil uunet!dgis!jkrueger Isn't it interesting that the first thing you do with your color bitmapped window system on a network is emulate an ASR33?
jkrueger@daitc.daitc.mil (Jon Krueger) (09/22/89)
jas@postgres.uucp (James Shankland) writes: >In article <6155@sybase.sybase.com> forrest@sybase.com writes: >("Parsing" here means just the context-free stuff; the semantic checking >pretty much has to be done in the server, which has the system catalogs.) One of the nice things about keeping the data dictionary in ordinary tables is that the application can use them to perform the same checks. This can help distribute the load by trapping bad queries before they're sent to the server. But the server still has to check anyway for all the queries that are sent it. -- Jon -- Jonathan Krueger jkrueger@daitc.daitc.mil uunet!dgis!jkrueger Isn't it interesting that the first thing you do with your color bitmapped window system on a network is emulate an ASR33?
tim@binky.sybase.com (Tim Wood) (09/25/89)
In article <632@daitc.daitc.mil> jkrueger@daitc.daitc.mil (Jon Krueger) writes: > >>In article <6155@sybase.sybase.com> forrest@sybase.com writes: >>("Parsing" here means just the context-free stuff; the semantic checking >>pretty much has to be done in the server, which has the system catalogs.) > >One of the nice things about keeping the data dictionary in ordinary >tables is that the application can use them to perform the same checks. >This can help distribute the load by trapping bad queries before they're >sent to the server. But the server still has to check anyway for all >the queries that are sent it. > >-- Jon Yes, but in order for the application to test against the system tables, it has to send more query language text to the server! Granted, that text will be predefined and parameterized, so presumably correct. Still, a lot more work will be done (compared to the same functions performed within the server) to avoid the cost of sending the bad query than to just send it and let the server signal any error! Users expect friendly error handling from the server, regardless of their applications; that's one of the inherent advantages and requirements of server designs. -TW Sybase, Inc. / 6475 Christie Ave. / Emeryville, CA / 94608 415-596-3500 tim@sybase.com {pacbell,pyramid,sun,{uunet,ucbvax}!mtxinu}!sybase!tim Voluntary disclaimer: This message is solely my personal opinion. It is not a representation of Sybase, Inc. OK.
dlw@odi.com (Dan Weinreb) (09/25/89)
In article <6167@sybase.sybase.com> jeffl@sybase.Sybase.COM (Jeff Lichtman) writes: > At Britton Lee > query languages were parsed in the clients and parse trees were sent > to the server. Here at Sybase we send the query language to the > server to be parsed. I believe the latter approach is better. The world is moving toward open, standardized interfaces. There is already an ANSI SQL standard, and a subcommittee of ANSI X3H2 is working on remote database access protocols. Eventually, a client will be able to run queries on anyone's server without having to know much about the server. There's one aspect of this standardization that I've never understood. The ANSI SQL definition provides a standard for queries to relational databases. However, there doesn't seem to be anything that provides a standard describing the format of the data returned by the server to the client. For example, what delimits one value from the next, or one tuple from the next? Are floating point numbers sent in an ASCII representation, or binary using IEEE format, or what? Is there currently any standardization between/among more than one vendor in this area? Is this something being addressed by the aforementioned subcommittee of X3H2? If so, how's the subcommittee's work progressing? Thank you. Dan Weinreb Object Design, Inc. dlw@odi.com
philf@xymox.metaphor.com (Phil Fernandez) (09/25/89)
In article <631@daitc.daitc.mil> jkrueger@daitc.daitc.mil (Jon Krueger) writes: >forrest@sybase.com writes: >... >>Detecting syntax errors in the client can prevent sending a >>syntactically incorrect query over a potentially slow network. >>It would take fewer resources to find out that a query is syntactically >>incorrect. > >Fewer bottlenecked resources, perhaps. But isn't that what the >validations, parsing, and generation logic in the front end tool >is for? In other words, isn't this a solved problem, at least >for well behaved applications? >... Exactly. Definitely a solved problem. At Metaphor, our whole business depends on building sophisticated (but easy-to-use) graphical interfaces to relational data bases. All of our various query applications generate SQL, but no end-user ever sees SQL. And, none of our query applications generate malformed SQL, so no real need for SQL syntax-checking on the client. pmf (These opinions are mine only; Metaphor pays me for real opinions) +-----------------------------+----------------------------------------------+ | Phil Fernandez | philf@metaphor.com | | | ...!{apple|decwrl}!metaphor!philf | | Metaphor Computer Systems |"Does the body rule the mind, or does the mind| | Mountain View, CA | rule the body? I dunno..." - Morrissey | +-----------------------------+----------------------------------------------+
philf@xymox.metaphor.com (Phil Fernandez) (09/25/89)
In article <1989Sep24.215650.15732@odi.com> dlw@odi.com writes: >In article <6167@sybase.sybase.com> jeffl@sybase.Sybase.COM (Jeff Lichtman) writes: > > > At Britton Lee > > query languages were parsed in the clients and parse trees were sent > > to the server. Here at Sybase we send the query language to the > > server to be parsed. > > I believe the latter approach is better. The world is moving toward > open, standardized interfaces. There is already an ANSI SQL standard, > and a subcommittee of ANSI X3H2 is working on remote database access > protocols. Eventually, a client will be able to run queries on anyone's > server without having to know much about the server. > >There's one aspect of this standardization that I've never understood. >The ANSI SQL definition provides a standard for queries to relational >databases. However, there doesn't seem to be anything that provides a >standard describing the format of the data returned by the server to >the client. For example, what delimits one value from the next, or >one tuple from the next? Are floating point numbers sent in an ASCII >representation, or binary using IEEE format, or what? > >Is there currently any standardization between/among more than one >vendor in this area? Good question! I'd also be interested in information about international standardization efforts in this area. At Metaphor, we have an internal standard for return data from relational data base servers. In our architecture, we implement data base gateways to each vendor's data base whose primary function is to convert that vendor's proprietary return format to our standard: Metaphor Table Transfer Format (MTTF). MTTF deals with encoding formats, numeric representations, byte ordering, etc. Currently, MTTF supports Sharebase, Oracle, Sybase, Ingres, Teradata, DB2, and SQL/DS. Clearly, this is an area that could well take advantage of standardization. (These are purely my own opinions; Metaphor pays me for real opinions) pmf +-----------------------------+----------------------------------------------+ | Phil Fernandez | philf@metaphor.com | | | ...!{apple|decwrl}!metaphor!philf | | Metaphor Computer Systems |"Does the body rule the mind, or does the mind| | Mountain View, CA | rule the body? I dunno..." - Morrissey | +-----------------------------+----------------------------------------------+
jkrueger@daitc.daitc.mil (Jon Krueger) (09/26/89)
tim@binky.sybase.com (Tim Wood) writes: >Yes, but in order for the application to test against the system tables, >it has to send more query language text to the server! Could be more, could be less. It's the setup cost traded off against fewer transactions during the session. >...a lot more work will be done (compared to the same functions >performed within the server) to avoid the cost of sending the bad query >than just to send it and let the server signal any error! Why not give an example? I'll try to show how the cost could be higher either way, depending on workload. >Users expect friendly error handling from the server, regardless >of their applications; that's one of the inherent advantages and >requirements of server designs. Users talk to a user agent, not a server. Unless you're talking X Windows, of course :-) Then again, Sybase glossies have been known to use the terms "user server" and "data server"; could that be the cause of confusion here? Who's the client, anyway? :-) -- Jon -- Jonathan Krueger jkrueger@daitc.daitc.mil uunet!dgis!jkrueger Isn't it interesting that the first thing you do with your color bitmapped window system on a network is emulate an ASR33?
news@rtech.rtech.com (USENET News System) (09/26/89)
In article <1989Sep24.215650.15732@odi.com> dlw@odi.com writes: > open, standardized interfaces. There is already an ANSI SQL standard, > and a subcommittee of ANSI X3H2 is working on remote database access >There's one aspect of this standardization that I've never understood. >The ANSI SQL definition provides a standard for queries to relational >databases. However, there doesn't seem to be anything that provides a >standard describing the format of the data returned by the server to >the client. For example, what delimits one value from the next, or >one tuple from the next? Are floating point numbers sent in an ASCII >representation, or binary using IEEE format, or what? There is a format specified in the RDA standard, but it's probably not how you expect it to be defined: it doesnt specify how the data is returned to you, but how the data is transmitted on the wire. I believe that the format of the returned tuple data is embodied in the SQLDA structure. SQL experts? The format of the RDA messages comprising the dialogue between client and server are defined in terms of their abstract syntax notation (ASN.1) as embodied in the X.409 standard. It is the responsibility of the underlying communications mechanism to convert the data in the messages between ASN.1 and a format suitable for the architecture of the local machine. For instance, a floating point number on MACHINE X represented in IEEE format and needs to be sent to a VAX. The RDA user supplies the value in the IEEE format, the underlying communications software (particularly, the Presentation Layer, in the case) converts it to ASN.1/X.409 syntax (i.e. representation) and puts it out on the wire. The receiving VAX converts it from ASN.1 to VAX F- or G-floating (or whatever) format for use by the application. >Is there currently any standardization between/among more than one >vendor in this area? It depends what you mean. In order for an implementation to be able to interoperate the messages have to be in a specific format/order on the wire; this is usually what is meant by 'RDA complient'. This is the requirement for (I hate to use this word) all the communications standards. Think of it as a variation on the Turing Test: if from my end of the wire, I can't tell the difference between your implementation and a know complient implementation then your implementation is complient. If you use the protocol state tables described in the spec, the you're probably more likely to interoperate than if you use some huge if tests; if you implement your stuff in some semblance of the ISO 7-layer model, you'll have an easier chance of building it than a monolithic program :-> Either way, though, you should still be able to interoperate. >Is this something being addressed by the aforementioned subcommittee >of X3H2? If so, how's the subcommittee's work progressing? Yes, it is being addressed. RDA proposes to use ASN.1/X.409 and it's highly unlikely to change. There are some things which RDA needs that X.409 doesn't do (I believe "arrays of records" are missing) but, IMHO, the committee is tackling much larger fish (is there a special smiley for mixed metaphors? :->) - bob +-------------------------+------------------------------+--------------------+ ! Bob Pasker ! Relational Technology ! ! ! pasker@rtech.com ! 1080 Marina Villiage Parkway ! INGRES ! ! (415) 748-2434 ! Alameda, California 94501 ! ! +-------------------------+------------------------------+--------------------+
jkrueger@daitc.daitc.mil (Jon Krueger) (09/26/89)
dlw@odi.com (Dan Weinreb) writes: >There's one aspect of this standardization that I've never understood. >The ANSI SQL definition provides a standard for queries to relational >databases. However, there doesn't seem to be anything that provides a >standard describing the format of the data returned by the server to >the client. For example, what delimits one value from the next, or >one tuple from the next? Are floating point numbers sent in an ASCII >representation, or binary using IEEE format, or what? Excellent point. And it gets worse. How does the client find out how to parse and generate vendor X's date and money data types? His abstract data types of the future? How does the server flag nulls? How does the server return errors and exceptions, e.g. permission denied, integrity violation, domain violation, deadlock, end of table, end of transaction (committed), disk full, disk read error, network disconnect? How does the client tell the server it's ready for the next row (or group of them)? How does the client tell the server to break out of a retrieve loop? How does the client tell the server to terminate whatever it's doing ("guess I didn't want to know that bad")? How does the client start and end sessions (connect and disconnect)? And that's not even implementing merely desirable features like the server telling the client how many rows were affected, or what a specific integrity violation was, or the client telling the server limits for query processing ("expensive query trap"). And it's assuming that providing clean, secure lines will be done by something else, and in a transparent manner. Nasty, eh? Still think that Intergalactic Dataspeak will solve all your problems :-) ? Yes, it's a start. It's kind of like getting everyone to agree on how to build cars and roads so that any car can drive on any road, but lacking a standard for maps: somewhat fewer cars will arrive at their destination than set off on the trip. -- Jon -- Jonathan Krueger jkrueger@daitc.daitc.mil uunet!dgis!jkrueger Isn't it interesting that the first thing you do with your color bitmapped window system on a network is emulate an ASR33?
miket@blia.BLI.COM (Mike Tossy) (09/26/89)
> > > At Britton Lee (ShareBase) > > query languages were parsed in the clients and parse trees were sent > > to the server. Here at Sybase we send the query language to the > > server to be parsed. > (Note the use of past tense. This is changing for exactly the reasons outlined below.) > > ... The world is moving toward > open, standardized interfaces. There is already an ANSI SQL standard, > and a subcommittee of ANSI X3H2 is working on remote database access > protocols. Eventually, a client will be able to run queries on anyone's > server without having to know much about the server. > > (For completeness let me offer these comments): One argument in favor of parsing (or at least scanning) on the client: Imagine a network with hundreds of clients talking to a few servers. The clients represent a very valuable source of MIPS - MIPS which are fairly difficult to apply to the core of the RDBMS problem, but which are easy to apply to the scanning/parsing problem. In this scenerio a server which used server based scanning/parsing would need to be bigger than a server that used client based scanning/parsing. Another argument in favor of client based scanning/parsing: Precompiler code probably needs to be scanned on the client anyway; so using a server based scanning/parsing system results in double scanning. Personnally I think a good case can be made for either side; but the new standards are going toward scanning/parsing on the server and therefore it is the existance of a standard, not the merits of the technical argument that will decide where to do the parsing. (I am reminded of the time C.J. Date was a guest at our user's group and one customer asked if SQL or QUEL was the better langauge (we offer both). Date responded "QUEL, but it doesn't matter; SQL is the standard"). Final note: parsing on the client does NOT mean you can use a dumb terminal connected directly to a server. You still need "smarts" on the client end. Mike Tossy ShareBase Coropration miket@blia.bli.com 14600 Wichester Blvd (408) 378-7575 ext2200 Los Gatos, CA 95030 (Formerly: Britton Lee, Inc.) The preceeding might or might not be close to the opinion of ShareBase Corp; if you think I bothered to clear it with anybody other than myself you're crazy.
billc@rtech.UUCP (Bill Coffin) (09/26/89)
From article <17466@pasteur.Berkeley.EDU>, by mao@eden (Mike Olson): >> From: forrest@sybase.com (Jon Forrest) > Hi Jon, Hi Mike. For those of you that just joined us, I also worked at Britton-Lee (er, ah, "Sharebase"). I now work at RTI. Jon, I don't think you can avoid a front-end parse if you support host-language embeddings. You still have to poke around for host-language variables and other information at compile time. Personally, I think RTI (oh yes, and Sybase too) have the right idea in sending text: * Interoperability is simpler, and will get simpler yet as vendors continue to converge on a standard SQL. * If you have a parse-tree interface, you don't know who might have written the parser -- you might get some strange stuff. There were lot's of eccentric parser that talked to BLI machines, and BLI was not very robust in the face of parse-tree berserkness. * When BLI started up they expected VARs to write these parsers. VARs decided this was too hard, and thunderously ignored BLI. BLI had to write a parser and port it everywhere. If you want VARs and others to get involved, it behooves you to support a non-befuddling interface. * Mike argues that taking away the parse-tree interface makes it impossible to implement new languages. I don't believe this -- BLI's parse-trees are semantically QUEL (IDL) oriented, so it was just about impossible to do a different language. I implemented Britton-Lee's first version of SQL, translating SQL into these parse trees. It was a dreadful hack that couldn't adequately model SQL. (And when BLI went into nosedive mode, yours-truly's head was amongst those that rolled.) I don't think it's possible to do an interface that is independent of the query-language and also non-procedural. billc@rtech.uucp ( or, if you must, {sun,pyramid,mtxinu,amdahl}!rtech!billc ) <<standard disclaimer>> <<gratuitous witticism>> <<your ad here>>
tim@binky.sybase.com (Tim Wood) (09/27/89)
In article <637@daitc.daitc.mil> jkrueger@daitc.daitc.mil (Jon Krueger) writes: >tim@binky.sybase.com (Tim Wood) writes: > >>Yes, but in order for the application to test against the system tables, >>it has to send more query language text to the server! > >Could be more, could be less. It's the setup cost traded off against >fewer transactions during the session. Let's agree on the scenario. For server checking, a batch of SQL (procedure definition or ad-hoc) is sent straight to the server. Each batch must be validated on its own, independent of previously-run batches. The server parses the batch and validates all name, datatype and permissions usages (this is the bulk of the lookups to the system tables.) That validation entails no communication to the client (unless there are errors). That validation need only be done once for as long as that batch executes. If the batch is a procedure, the validation need not done again unless the database schema on which the procedure depends is changed. I'm not clear on what you mean by "setup cost". The cost of the validations is amortized over the life of the procedure. The more you use the procedure, the more insignificant the validation cost becomes. The only runtime validations are (optimized) schema and permission checks. For ad-hoc (single-use) queries, client checking loses if it costs more than server checking, especially when the validation cost is significant compared to execution cost. For client checking, the validations have to be done in SQL text. Each "validation" query must still be validated in the server, so you've added in the number of network messages needed to validate plus the number of server validations of those messages to the batch validation cost. Those added costs don't exist for the server-checking case. What's more, the coupling between the validation and the compiled procedure here is much looser, since a bundle of separate SQL queries must somehow be associated with the procedure for the client case, whereas validation is just an internal step in procedure processing for the server case. > >>...a lot more work will be done (compared to the same functions >>performed within the server) to avoid the cost of sending the bad query >>than just to send it and let the server signal any error! > >Why not give an example? I'll try to show how the cost could be >higher either way, depending on workload. The above explanation hopefully suffices (in lieu of designing a sample schema, giving a trace of network messages and estimated milliseconds per operation, which the question does not justify.) > >>Users expect friendly error handling from the server, regardless >>of their applications; that's one of the inherent advantages and >>requirements of server designs. > >Users talk to a user agent, not a server. Unless you're talking X >Windows, of course :-) Then again, Sybase glossies have been known to >use the terms "user server" and "data server"; could that be the cause >of confusion here? Who's the client, anyway? :-) True, I glossed over a level here. Users expect friendly error handling from (all) their applications. Applications WRITERS expect complete, descriptive and accurate error handling from the server, so they can handle the errors properly in the application. The "client" is the process running the user application. I haven't read the "user/data server" lierature, so can't respond to that. -TW Sybase, Inc. / 6475 Christie Ave. / Emeryville, CA / 94608 415-596-3500 tim@sybase.com {pacbell,pyramid,sun,{uunet,ucbvax}!mtxinu}!sybase!tim Voluntary disclaimer: This message is solely my personal opinion. It is not a representation of Sybase, Inc. OK.
philf@xymox.metaphor.com (Phil Fernandez) (09/27/89)
In article <3699@rtech.rtech.com> billc@rtech.UUCP (Bill Coffin) writes: > * When BLI started up they expected VARs to write these parsers. VARs > decided this was too hard, and thunderously ignored BLI. BLI had to > write a parser and port it everywhere. If you want VARs and others to > get involved, it behooves you to support a non-befuddling interface. Wait a minute. Metaphor was perfectly successful in writing a SQL parser for BLI. No one at Metaphor decided it was too hard; no one even thought it particularly hard at all. It was just another piece of hte project... Now, that was 5 years ago; in this day-and-age, we'd prefer not to have to provide this piece of the puzzle again. SQL parsing on the server is much more convenient... (This opinions are mine; Metaphor pays me for real opinions) +-----------------------------+----------------------------------------------+ | Phil Fernandez | philf@metaphor.com | | | ...!{apple|decwrl}!metaphor!philf | | Metaphor Computer Systems |"Does the body rule the mind, or does the mind| | Mountain View, CA | rule the body? I dunno..." - Morrissey | +-----------------------------+----------------------------------------------+
jkrueger@daitc.daitc.mil (Jon Krueger) (09/27/89)
OK, I'll provide an example. The database is movies, the schema are film(fname,fdate,director,company,country,language) actor(aname,bdate,sex,height) cast(fname,aname,role,charname) This will allow us to put all sorts of delightful queries. For instance, has the average height of leading men decreased over the last twenty years? What actors have directed their own movies? But the users of this database are more likely to want to know: what films has Fred Astaire played in? In which of them did he appear with Ginger Rogers? And so on. >Let's agree on the scenario. For server checking, a batch of SQL >(procedure definition or ad-hoc) is sent straight to the server. >Each batch must be validated on its own, independent of previously-run >batches. OK. By "batch" I assume you mean a "bunch"; that is, you intend to support interactive applications? Please supply some sample queris. Then we'll run through the whole scenario. -- Jon -- Jonathan Krueger jkrueger@daitc.daitc.mil uunet!dgis!jkrueger Isn't it interesting that the first thing you do with your color bitmapped window system on a network is emulate an ASR33?
news@rtech.rtech.com (USENET News System) (09/28/89)
In article <9463@blia.BLI.COM> miket@blia.BLI.COM (Mike Tossy) writes: >Final note: parsing on the client does NOT mean you can use a dumb terminal >connected directly to a server. You still need "smarts" on the client end. Uh, nothing precludes having 'smart' front-ends which can trap lexical errors, if not some semantic errors, and dumb terminals which cant. Scanning/parsing could be a run-time negotiable parameter determined between the client and server. If the client is dumb, then the server has to run the 'front-end' smarts for him on the back-end. A logical extension (in my (warped, networking) mind) is to have a pipeline: queries get done in stages: entry, scanning, parsing, execution, etc. There's no reason why these couldnt be done in heterogeneous machines. So lets say you want to run a (text based) RDA front-end against a parse-tree back-end. Stick a parse-server in the middle which takes text in one side and emits a parse tree out the back. Have a parse tree front-end? Point him directly at the back-end. See, isn't networking easy? - bob +-------------------------+------------------------------+--------------------+ ! Bob Pasker ! Relational Technology ! ! ! pasker@rtech.com ! 1080 Marina Villiage Parkway ! INGRES/Net ! ! (415) 748-2434 ! Alameda, California 94501 ! ! +-------------------------+------------------------------+--------------------+
jeffl@sybase.Sybase.COM (Jeff Lichtman) (09/28/89)
Here's how it is: in order for the front end to do validation of column
names, table names, datatypes, etc., it has to send queries to the server.
For example:
select mycolumn from mytable
On Sybase, one might use the following queries to validate the above query:
select id from sysobjects where name = 'mytable'
select colid from syscolumns where id = :id and name = 'mycolumn'
Now, suppose the first query (the one we're trying to validate) is wrong
(for example, either the table or column doesn't exist). We can save the
cost of sending the query to the server by validating it in the front end,
but only at the cost of running the other two queries. This is actually
worse than just sending the first query and seeing whether it will run.
After all, the server has to do the two validation queries even for a valid
query (it needs the table and column ids). This can be done more efficiently
in the server by using hard-wired access method calls.
I assume, by the way, that we're not talking about validating the user's
query just so we can send the unmodified text of that query to the server.
That would be a terrible way of doing it. In that case, the server would
still have to validate the query once the front end decided it was safe
to send, so in the best case, the validations would be costly, and in the
worst case, they would have to be done twice. We're still talking about
sending parse trees across the network, aren't we?
---
Jeff Lichtman at Sybase
{mtxinu,pacbell}!sybase!jeffl -or- jeffl@sybase.com
"Saints should always be judged guilty until they are proved innocent..."tim@binky.sybase.com (Tim Wood) (09/29/89)
In article <641@daitc.daitc.mil> jkrueger@daitc.daitc.mil (Jon Krueger) writes: >OK, I'll provide an example... > [ example film library schema ] >This will allow us to put all sorts of delightful queries.... > >>Let's agree on the scenario. For server checking, a batch of SQL >>(procedure definition or ad-hoc) is sent straight to the server... > >OK. By "batch" I assume you mean a "bunch"; that is, you intend to >support interactive applications? Yes. Note that the front-end may application may be a conventional "batch" application, or interactive. The server is not aware of which, however it assumes interactive response time needs. >Please supply some sample queries. >Then we'll run through the whole scenario. No, thank you. I just don't find this problem interesting enough. -TW Sybase, Inc. / 6475 Christie Ave. / Emeryville, CA / 94608 415-596-3500 tim@sybase.com {pacbell,pyramid,sun,{uunet,ucbvax}!mtxinu}!sybase!tim Voluntary disclaimer: This message is solely my personal opinion. It is not a representation of Sybase, Inc. OK.
news@rtech.rtech.com (USENET News System) (09/29/89)
In article <3695@rtech.rtech.com> pasker@rtech.com (Bob Pasker) writes: >Yes, it is being addressed. RDA proposes to use ASN.1/X.409 Well, Bob, you are wrong. After checking this out, it appears that RDA will not use ASN.1/X.409, but their homegrown (committee-grown?) encoding which shares some things with X.409. - bob +-------------------------+------------------------------+--------------------+ ! Bob Pasker ! Relational Technology ! ! ! pasker@rtech.com ! 1080 Marina Villiage Parkway ! INGRES/Net ! ! (415) 748-2434 ! Alameda, California 94501 ! ! +-------------------------+------------------------------+--------------------+
billc@rtech.UUCP (Bill Coffin) (09/29/89)
From article <811@metaphor.Metaphor.COM>, by philf@xymox.metaphor.com (Phil Fernandez): >> write a parser and port it everywhere. If you want VARs and others to >> get involved, it behooves you to support a non-befuddling interface. > > Wait a minute. Metaphor was perfectly successful in writing a SQL > parser for BLI. No one at Metaphor decided it was too hard; ... Yes, and Metaphor did a great job. Several other customers wrote frontends. Boeing did one even before BLI did. However, the argument stands. Most potential VARs were not willing to enter into that kind of development investment to gamble on a new company offering a worthy but controversial technology. BLI started with no front-end parsers. With a few exceptions, VARs were overwhelmingly uninterested. BLI then wrote a few parsers for demo purposes to try to show potential VARs how easy it was. It soon became obvious that, again with a few notable exceptions, no-one was biting, so BLI cleaned up some of the demo stuff and offered it as a product set. Eventually, BLI had a portable (but minimal) frontend product. However, this was an afterthought -- the original philosophy of the company founders was that the BLI product was some sort of smart disk drive. billc@rtech.uucp ( or, if you must, {sun,pyramid,mtxinu,amdahl}!rtech!billc ) <<standard disclaimer>> <<gratuitous witticism>> <<your ad here>>
marc@infmx.UUCP (Marc Kenig) (09/29/89)
In article <638@daitc.daitc.mil> jkrueger@daitc.daitc.mil (Jon Krueger) writes: >dlw@odi.com (Dan Weinreb) writes: >>There's one aspect of this standardization that I've never understood. >>The ANSI SQL definition provides a standard for queries to relational >>databases. However, there doesn't seem to be anything that provides a >>standard describing the format of the data returned by the server to >Excellent point. And it gets worse. How does the client find out how >to parse and generate vendor X's date and money data types? His >abstract data types of the future? How does the server flag nulls? > >Nasty, eh? Still think that Intergalactic Dataspeak will solve all >-- >Jonathan Krueger jkrueger@daitc.daitc.mil uunet!dgis!jkrueger Nasty, yes. Here at Informix, we've had to address all of these problems and more with our Informix-4GL for Rdb/VMS product development. In point of fact, you can't just "simply parse" on the client. You have to do almost as much work as the server will do in order to do a more than cursory (read: useless) syntax scan of the query. There is the "date format" problem, for example, which illustrates this point: SELECT EMPLOYEE_ID, " Retires Next Month" FROM EMPLOYEES WHERE "01-OCT-1924" > (SELECT DOB FROM PERSONAL ...) Here, the client would have to change the date format to whatever the server expects. But without a complete parse, and schema info, how could an automata know that the sub-query returns a date and that the character literal should be interpreted as a date and diddled??? I agree: it's too easy to assume that pre-parsing is just a syntax game. BTW, getting back to the original posting, this is how DEC conceived DSRI (the Digital Standard Relational Interface). They define a low- level parse tree language called BLR (Binary Language Represetation). Embedded SQL, RDO (their proprietary query language), and SQL modules are turned into BLR at compile time, or dynamically at run-time. The back-end engine (Rdb) is blind to the actual higher level representation of the query. Others can write interfaces or DSRI compliant database engines, though as with BLI, few do. Lastly, a read-only ANSI SQL-I client-server interface would leave users hungry for more real fast. Marc Kenig "I like VMS better" ...uunet!pyramid!infmx!marc Beeeeeeeep* This has been a test of MY OPINION broadcasting system. Usenet machines in your area, in voluntary contribution with my node, have participated in propogating my uninformed drivel. Had there been an actual emergency, you would have been instructed to tune to MY EMPLOYERS OPINIONS for instructions and factual information.
clh@tacitus.tfic.bc.ca (Chris Hermansen) (09/29/89)
In article <6333@sybase.sybase.com> jeffl@sybase.Sybase.COM (Jeff Lichtman) writes: > >I assume, by the way, that we're not talking about validating the user's >query just so we can send the unmodified text of that query to the server. >That would be a terrible way of doing it. In that case, the server would >still have to validate the query once the front end decided it was safe >to send, so in the best case, the validations would be costly, and in the >worst case, they would have to be done twice. We're still talking about >sending parse trees across the network, aren't we? First, a disclaimer: I'm just a poor schmuck user of databases, not a high- powered designer... In your argument above about `pre-validating' queries at the front end, the server will have to re-validate things anyway, UNLESS the server locks `mytable' in some sense to prevent alterations, etc. etc. coming down another wire while the original process is ruminating over the results. For example, suppose dbclient1 sends some queries to the system catalogs on dbserver, for the purposes of verifying the semantics of a specific chunk of sql dealing with `mytable'. When the response comes back, dbclient1 is busy with another task that results in a delay of, say, two seconds before the (client) validated sql is sent back to dbserver. Meanwhile, dbclient2 sends a request to dbserver to, say, change the name of column `mycolumn' in table `mytable' to `foo'. Next, dbserver receives a (client) validated request to select a bunch of rows displaying `mycolumn' from `mytable'. What happens? I claim that `mytable' cannot be locked, since dbserver doesn't know that a request will be coming from dbclient1 for `mytable'. All it knows is that dbclient1 wants some info from the system catalogs. It seems to me that the logical conclusion here is that anything more than syntactic validation at the server is a waste of time, at least if there are any possibilities of table modification between the validation and request for information phases. Please see disclaimer above... Chris Hermansen Timberline Forest Inventory Consultants Voice: 1 604 733 0731 302 - 958 West 8th Avenue FAX: 1 604 733 0634 Vancouver B.C. CANADA uunet!ubc-cs!van-bc!tacitus!clh V5Z 1E5 or Chris_Hermansen@mtsg.ubc.ca
bsa@telotech.UUCP (Brandon S. Allbery) (09/30/89)
In article <3715@rtech.rtech.com>, news@rtech (USENET News System) writes: +--------------- | In article <9463@blia.BLI.COM> miket@blia.BLI.COM (Mike Tossy) writes: | >Final note: parsing on the client does NOT mean you can use a dumb terminal | >connected directly to a server. You still need "smarts" on the client end. | | Uh, nothing precludes having 'smart' front-ends which can trap lexical | errors, if not some semantic errors, and dumb terminals which cant. +--------------- I think the issue is more along the lines of how information OTHER than the SQL statement itself is communicated between the front-end and the back-end. The back end is going to require input and output bind specifications to be sent to it, and will return output binding structures -- not to mention a (binary!) SQLDA structure -- as a result of the SQL statement's execution. ++Brandon -- -=> Brandon S. Allbery @ telotech, inc. (I do not speak for telotech.) <=- Any comp.sources.misc postings sent to this address will be DISCARDED -- use allbery@uunet.UU.NET instead. My boss doesn't pay me to moderate newsgroups. ** allbery@NCoast.ORG ** uunet!hal.cwru.edu!ncoast!{allbery,telotech!bsa} **
news@rtech.rtech.com (USENET News System) (10/03/89)
In article <1989Sep29.172452.2619@telotech.uucp> bsa@telotech.UUCP (Brandon S. Allbery) writes: >In article <3715@rtech.rtech.com>, news@rtech (USENET News System) writes: ( In article <9463@blia.BLI.COM> miket@blia.BLI.COM (Mike Tossy) writes: [ Final note: parsing on the client does NOT mean you can use a dumb terminal [ connected directly to a server. You still need "smarts" on the client end. ( Uh, nothing precludes having 'smart' front-ends which can trap lexical ( errors, if not some semantic errors, and dumb terminals which cant. >I think the issue is more along the lines of how information OTHER than the >SQL statement itself is communicated between the front-end and the back-end. >The back end is going to require input and output bind specifications to be >sent to it, and will return output binding structures -- not to mention a >(binary!) SQLDA structure -- as a result of the SQL statement's execution. I believe What you're saying is that the BE needs to accept binary input and produce binary output. Something has got to transform it to and from text so that users can SEE it (reminds me of the guy who could read paper tape!) It can be something like the INGRES terminal monitor ("TM") which takes SQL from from the user and prints nicely formatted tables on his screen. Behind TM has to be the smarts to convert the text into something the back-end wants to see and vice-versa. If the back-end wants a parse tree, then the query can get parsed by the client or some intermidiary (like the "parser server" which I described in my last post.) - bob +-------------------------+------------------------------+--------------------+ ! Bob Pasker ! Relational Technology ! ! ! pasker@rtech.com ! 1080 Marina Villiage Parkway ! INGRES/Net ! ! <use this address> ! Alameda, California 94501 ! ! ! <replies will fail> ! (415) 748-2434 ! ! +-------------------------+------------------------------+--------------------+
miket@blia.BLI.COM (Mike Tossy) (10/05/89)
In article <3715@rtech.rtech.com>, news@rtech.rtech.com (USENET News System) writes: > In article <9463@blia.BLI.COM> miket@blia.BLI.COM (Mike Tossy) writes: > >Final note: parsing on the client does NOT mean you can use a dumb terminal > >connected directly to a server. You still need "smarts" on the client end. > > Uh, nothing precludes having 'smart' front-ends which can trap lexical > errors, if not some semantic errors, and dumb terminals which cant. > Scanning/parsing could be a run-time negotiable parameter determined > between the client and server. If the client is dumb, then the server > has to run the 'front-end' smarts for him on the back-end. > (Did I miss the joke?) This is not called client/server this is called traditional software rdbms. It is possible but it is undesirable for all the usual performance reasons that caused companies to go to client/ server architectures. Your "server" would also need to run programming interfaces and 4GLs etc, etc. Looks like a time shareing system to me. If you need to connect dumb terminals connect them to a little UNIX box and have it talk to the server. --miket Mike Tossy ShareBase Coropration miket@blia.bli.com 14600 Wichester Blvd (408) 378-7575 ext2200 Los Gatos, CA 95030 (Formerly: Britton Lee, Inc.) The preceeding might be (probably is - in fact) close to the opinion of ShareBase Corp; but if you think I bothered to clear it with anybody other than myself you're crazy.
ajk@goanna.oz (Dr Alan Kent) (10/08/89)
I thought I would add my 2 cents worth to this discussion.
We here at RMIT (Royal Melbourne Institute of Technology) have a
research project (Titan+) which involves building a database system
which supports complex objects. I personally dont think it really
classifies as object oriented, but it does support nested tables and
good text support. We plan to also include structured or tuple types
and links (like refs in LauRel). The system is very definitely
experimental, although the good bits are slowly to be merged with
a commerical database system produced locally. We intend to use the
database system as a basis for a document retrieval package.
Currently we construct expression trees and send them down the communication
channel. As we saw it (when we made the decision), the advantages were
(1) Emedded query language commands only needed to be parsed once when
the application program was compiled (though some checking still needs
to be done to make sure the schema had not changed).
(2) There would be lower communication overheads (very debatable!)
(3) Query trees for complex dynamic conditions would be easier to construct.
Consider a program dynamically forming a query such as
SELECT *
FROM PARTS
WHERE cond1 AND (cond2 OR cond3) OR cond4
The problem with creating this text dynamically is knowing when you
need the string "AND", "OR", parenthises etc. This is much easier
with expression trees (or so we have found in practice).
(4) Sending strings or binary data is easy as you dont have to be careful
of quote characters in strings (e.g. consider 'John's car').
The current programming interface consists of a large library of small
functions for building expression trees. For a quick and dirty embedded
query language preprocessor, embedded queries were mapped to expressions
which returned expression tree pointers. To actually insert constants
into the tree involves calling another function to replace identifier
nodes with constant value nodes. (As a side benefit, identifier nodes
can also be replaced with any other subtree allowing conditions to
be constructed dynamically and inserted into the where clause).
For example
qry = $ SELECT * FROM PARTS WHERE COLOR = thecolor $;
qry = tqlsubst(qry, "thecolor", color);
Anyway, after using this approach for a while now, we have found the
following:
(1) When using sockets or pipes etc., the size of the message is negligable.
The cost of doing a system call or context switch outweighs other costs.
(2) Dynamically formed conditions are fairly rare, but have proved
*extremely* useful on occasion (especially when dealing with text).
(3) You have to learn a lot to construct expression trees manually in a
program and is a great source of bugs.
(4) People faint when they see the number of functions provided to build
expression trees (but the range of functions do make life easier).
(5) Debugging is easier for users when then generate text queries rather
than expression trees. Even the simple embedded query language above
was a god-send after building trees manually.
(6) We now have the parser as function in the library that given a string
returns and expression tree.
(7) Users can see the query generated by say a forms package which can
help them learn a query language.
(8) We do not use standard SQL as it is not powerful enough to handle the
data types we support. We now have in fact two query languages to
access a database - Standard SQL (which can only access some of the
tables) and our own special query language. They both produce the
same expression tree structure. If the query language was built
into the kernel, we would have had to turn SQL into our query language
at the textual level (which would be harder, but certainly not
impossible).
One reason we have not gone the approach of most embedded SQL systems
of using actual variable names embedded in the query language
(e.g. SELECT color INTO :variable FROM PARTS) is that while
this works fine for integers and strings, the more complex data
structures we support (such as nested tables) do not map easily
to programming language data types. (Ideas anyone?)
A QUESTION:
What is the best way to send user entered constants in a query language
when those constants may contain the string delimiter characters?
Having a function to turn a string into a string constant is my best
solution so far (ie. a function given "John's Car" returns "'John\'s Car'")
IN SUMMARY, we have found the expression tree interface to be very flexible,
but dangerous and difficult to learn to use. You can always hide these
details from users if you want to. We however are not a pure relational
database and so have different types of problems. Also being only an
experimental system, we have not had much user feedback.
PS: We are still using the expression tree interface for the time being.
Hope this was of some interest to someone.
Dr Alan Kent
Dept. of Comp. Sci, RMIT, ACSnet: ajk@goanna.cs.rmit.oz.au
GPO BOX 2476 V, ARPA: ajk%goanna.cs.rmit.oz.au@uunet.uu.net
Melbourne, 3001 CSNET: ajk%goanna.cs.rmit.oz.au@australia
AUSTRALIA UUCP: ...!uunet!goanna.cs.rmit.oz.au!ajk