oysteing@idt.unit.no (Oystein Groevlen) (06/01/91)
A few months ago I got a demonstration of the 'revolutionary', according to an ad from FoxPro, Rushmore Technology. I am interesting in hearing some comments from people that have had some experience with it. From what I could see, the query performance was much better than what I have experienced with ORACLE/INGRES, but we only got an limited demonstration. My impression was that the technology is based on some technique where indexes occupy much less space than usual. Thus, it is possible to index all fields of a table and keep all indexes in memory. This way queries can be processed with a minimum of disk access. Is this your impression, too? I seem to remember reading on this group that FoxPro needs a lot of memory. What happens if all indexes does not fit into memory? Is there a significant decrease in performance? The table used for the demonstration consisted of only character fields. Does an index of an integer field take much more space than an index of a character filed? As you probably know, you can usually compress text much more than numbers (Huffman code, Tries etc.) We only got a demonstration of selection from one table. How is the performance on joins? Do you think the Rushmore Technology is really revolutionary? As memory gets cheaper, it will maybe be possible to store large amounts of data in memory for faster query processing. This will significantly increase the query performance. Do you think this will cause great changes in the DBMS-market? Thanks for any response. Please post to the net. If you prefer email I will summarize to the net. -- Oystein Groevlen Division of Computer Systems and Telematics The Norwegian Institute of Technology The University of Trondheim Email: oysteing@idt.unit.no
onder@ISI.EDU (Bruce Onder) (06/06/91)
In article <1991Jun1.165211.27480@ugle.unit.no> oysteing@idt.unit.no (Oystein Groevlen) writes: >A few months ago I got a demonstration of the 'revolutionary', >according to an ad from FoxPro, Rushmore Technology. I am interesting >in hearing some comments from people that have had some experience >with it. > >From what I could see, the query performance was much better than what >I have experienced with ORACLE/INGRES, but we only got an limited >demonstration. I attended a demo of FoxPro 2.0. TRushmore was very fast, but we weren't comparing it to other things. From personal experience, I know it's faster than FoxPro 1.02, R:Base, and dBase x.0. >My impression was that the technology is based on some technique where >indexes occupy much less space than usual. Thus, it is possible to >index all fields of a table and keep all indexes in memory. This way >queries can be processed with a minimum of disk access. Is this your >impression, too? Yes. >I seem to remember reading on this group that FoxPro needs a lot of >memory. What happens if all indexes does not fit into memory? Is there >a significant decrease in performance? Dunno. >The table used for the demonstration consisted of only character >fields. Does an index of an integer field take much more space than an >index of a character filed? As you probably know, you can usually >compress text much more than numbers (Huffman code, Tries etc.) You are right, although the difference is not very noticeable on a human-user level. >We only got a demonstration of selection from one table. How is the >performance on joins? Dunno. >Do you think the Rushmore Technology is really revolutionary? I can't tell you, because I don't know what they're doing, specifically. I was impressed with the speed, though. >As memory gets cheaper, it will maybe be possible to store large >amounts of data in memory for faster query processing. This will >significantly increase the query performance. Do you think this will >cause great changes in the DBMS-market? Sure! There will be less and less performance hit on larger and larger files. >Thanks for any response. Please post to the net. If you prefer email I >will summarize to the net. Okay. Sorry I can't tell you more, besides: Impressed, and liking it. Bruce -- Bruce W. Onder onder@isi.edu (He's not your everyday-type prankster!) I'm Ice-T: Original Gangster (O.G.: Original Gangster)
mao@eden.Berkeley.EDU (Mike Olson) (06/06/91)
in <1991Jun1.165211.27480@ugle.unit.no>, oysteing@idt.unit.no (Oystein Groevlen) asks about something called "rushmore technology" from foxpro. i'm interested in finding out what this is, exactly. can anyone from foxpro clue us in? mike olson postgres research group uc berkeley mao@postgres.berkeley.edu
maurit@nrtc.nrtc.northrop.com (Mark Aurit <maurit>) (06/07/91)
>In article <1991Jun1.165211.27480@ugle.unit.no> oysteing@idt.unit.no (Oystein Groevlen) writes: > >>My impression was that the technology is based on some technique where >>indexes occupy much less space than usual. Thus, it is possible to >>index all fields of a table and keep all indexes in memory. This way >>queries can be processed with a minimum of disk access. Is this your >>impression, too? I seem to remember that Rushmore is a combination of things, essentially a new index structure that attempts to resolve a search strictly within the index and not bothering with pointers to the database. Therefore (and again, this is from memory - I think there was an article in DBMS I read about it) Rushmore may be of limited use in SEEK situations. Its speed is seen in COUNTs, etc. >>I seem to remember reading on this group that FoxPro needs a lot of >>memory. A user I know has the full foxpro lan version (not the runtime) running with a disk cache on - Ive come in under pcANYWHERE and executed the runtime version of R&R report writer. Very fast performance both in program execution and disk retrieval. Im not concerned with memory, I think it manages pretty well. BTW this is foxpro 1.x > >You are right, although the difference is not very noticeable on a >human-user level. You're right here. Users take it for granted - and they should. Only we appreciate it - and maybe we shouldnt! > >>Do you think the Rushmore Technology is really revolutionary? One companies "revolutionary" is another companies "marketing hype". IMO its just a faster method to use indexes (and as noted above that may be only under certain circumstances). SO how could that be "revolutionary"; "evolutionary" may be more correct. THis whole "Rushmore Technology" thing is a bit much. Mark
nolan@helios.unl.edu (Michael Nolan) (06/07/91)
The demo of FoxPro 2.0 that was being shown at COMDEX in Atlanta was very impressive, and FoxPro 2.0 was named 'Best PC Product' at COMDEX. Apparently, the "Rushmore" name comes from the fact that the designer of the new indexing method (or whatever it is) was watching "North by Northwest" the evening he came up with the idea. Fox Software is investingating patenting the process. Michael Nolan nolan@helios.unl.edu
dhartung@chinet.chi.il.us (Dan Hartung) (06/07/91)
oysteing@idt.unit.no writes: >A few months ago I got a demonstration of the 'revolutionary', >according to an ad from FoxPro, Rushmore Technology. I am interesting >in hearing some comments from people that have had some experience >with it. Certainly. As a FoxPro developer I have some insight into this subject. :-) >From what I could see, the query performance was much better than what >I have experienced with ORACLE/INGRES, but we only got an limited >demonstration. They are claiming that it beats DB2 on an AS/400 operating on an identical one-million-record database. I think this was on a 386 machine; have to check my papers. > >My impression was that the technology is based on some technique where >indexes occupy much less space than usual. Thus, it is possible to >index all fields of a table and keep all indexes in memory. This way >queries can be processed with a minimum of disk access. Is this your >impression, too? Compact indexes are one of the featurs of the new FoxPro. In and of themselves, however, they don't constitute the RushMore technology. What they are doing is using indexes with a number of commands that previously did not use the indexes: xBase LOCATE and COUNT commands, for instance. Thus it is helpful to have as many indexes open as possible; and the compact index format helps this be efficient. >I seem to remember reading on this group that FoxPro needs a lot of >memory. What happens if all indexes does not fit into memory? Is there >a significant decrease in performance? FoxPro operates impressively well for an xBase product on even a 640K XT. When you give it extra memory, though, it takes off. It makes automatic used of expanded memory; Fox 2 will use extended automatically, but now you have to use a (well-behaved) EMS emulator if you only have extended. Fortunately RAM is cheap. Fox 2 will have a much more granular overlay and an LRU algorithm for swapping program segments to disk, something like VROOOOM from Borland. >The table used for the demonstration consisted of only character >fields. Does an index of an integer field take much more space than an >index of a character filed? As you probably know, you can usually >compress text much more than numbers (Huffman code, Tries etc.) xBase is not very good with number fields. In general numbers are best stored as a character representation anyway. >We only got a demonstration of selection from one table. How is the >performance on joins? The SQL-mode performance is supposed to be very good, due to the compact indexes and Rushmore. I don't know how it matches up against Paradox, say, but it surely beats dBase IV's clunky implementation. Most amazing, FoxSQL allows full user-defined function use in SQL. Third-party vendors will be implementing connectivity links to SQL servers, and Fox is working on Fox Server for '92. > >Do you think the Rushmore Technology is really revolutionary? Depends. It is certainly a big step forward for xBase, but I don't know if it is really something new to other RDBMS platforms. What seems to be the case is that the combination of raw power now available cheaply in the PC world, combined with a superb RDBMS (FoxPro) is going to result in a new generation of powerful databases. > >As memory gets cheaper, it will maybe be possible to store large >amounts of data in memory for faster query processing. This will >significantly increase the query performance. Do you think this will >cause great changes in the DBMS-market? I don't think it will really change things, since the memory-for-query- processing is more or less need-driven. That is, the need for this kind of hardware optimization is there already, the only question is how fast can the software to implement it get there? > >Oystein Groevlen >Division of Computer Systems and Telematics >The Norwegian Institute of Technology The University of Trondheim >Email: oysteing@idt.unit.no How is Norway? I was in Oslo in '86 and really enjoyed my visit....... -- Daniel A. Hartung | "What's the difference anyway, between being dhartung@chinet.chi.il.us | safe and being rad, the joke's on us, we've Birch Grove Software | all been had." -- John Wesley Harding
dhartung@chinet.chi.il.us (Dan Hartung) (06/07/91)
By the way: If anyone knows of any FoxPro jobs out there, especially in Chicagoland, drop me an email. I have a year's worth of FP 1.x work under my belt, for a local client as well as developing my own vertical- market app, and I know Fox inside out. I am an active "guru" on FoxForum and have an article coming out in one of the Fox newsletters. Shameless, aren't I? :-) -- Daniel A. Hartung | "What's the difference anyway, between being dhartung@chinet.chi.il.us | safe and being rad, the joke's on us, we've Birch Grove Software | all been had." -- John Wesley Harding
tarjeij@ulrik.uio.no (Tarjei Jensen) (06/07/91)
Is it the speed associated with finding out whether a record can be found or not? If this is it then it might just a variation of the technique used by certain spellcheckers to see if a word is in the dictionary (a special hashing technique). I think the unix spell program is using such a technique. If I remember correctly it uses one bit per index (word) entry. Greetings from Norway, -- // Tarjei T. Jensen - if it ain't broken, fix it anyway! // tarjeij@ulrik.uio.no || +47 87 21138 // Working, but not speaking for the Norwegian National Library.
shevett@dworkin.UUCP ( Sysop) (06/09/91)
dhartung@chinet.chi.il.us (Dan Hartung) writes: >oysteing@idt.unit.no writes: >They are claiming that it beats DB2 on an AS/400 operating on an identical >one-million-record database. I think this was on a 386 machine; have to >check my papers. Don't have any paperwork here either, but it's probably on a '386 with EMS implemented properly. That's what makes FoxPro happiest. >>My impression was that the technology is based on some technique where >>indexes occupy much less space than usual. >Compact indexes are one of the featurs of the new FoxPro. In and of >themselves, however, they don't constitute the RushMore technology. >What they are doing is using indexes with a number of commands that >previously did not use the indexes: xBase LOCATE and COUNT commands, >for instance. This is an interesting point. In xBase products, a Locate function would do a non-indexed SEEK of a file, but without using any indexes. The advantage was it had some form of REPEAT function, which would search for the next occurrance of the expression. Since 2.0 will support LOCATE's via Indexes, does that mean the REPEAT function (or whatever it's called) will work correctly? If so, it could be a great boon in simplifying group searches. >>I seem to remember reading on this group that FoxPro needs a lot of >>memory. What happens if all indexes does not fit into memory? Is there >>a significant decrease in performance? >FoxPro operates impressively well for an xBase product on even a 640K XT. >When you give it extra memory, though, it takes off. It makes automatic >used of expanded memory; CAREFUL HERE! The biggest fault in FoxPro 1.02 is that it does *NOT* make complete use of Expanded memory. It will only cache indexes and dbf info in the expanded RAM, and no memory variables or program code will go there. Therefore, you may have 16meg of expanded memory, but it doesn't help you one whit if you program needs 200k of memory variables (which one of our system's does). If you have a program that needs to FOXSWAP out to another app (which we do), it's very difficult running FoxPro, even with an EMS manager installed (we use QEMM, which lets us use FOXQ, but it's still a tight squeeze) >Fox 2 will use extended automatically, but now >you have to use a (well-behaved) EMS emulator if you only have extended. Really? Extended memory automagically detected and used? Could you confirm this? >>The table used for the demonstration consisted of only character >>fields. Does an index of an integer field take much more space than an >>index of a character filed? >xBase is not very good with number fields. In general numbers are best >stored as a character representation anyway. EXCUSE ME? I'm confused here. Why would you use character representations of numeric fields? Seems pretty dopey to do a VAL conversion whenever retrieving from the database. Dave Shevett - shevett@dworkin.amber.mccc.edu - Lawrenceville, NJ
dhartung@chinet.chi.il.us (Dan Hartung) (06/10/91)
tarjeij@ulrik.uio.no (Tarjei Jensen) writes: >Is it the speed associated with finding out whether a record can be found or >not? No. Since SEEKs already operate virtually instantaneously, what this does is apply index usage to commands that previously would have done a physical record-by-record search. For instance, if you know the first word of a company name is "Smith" of "Smith & Jones" you can just SEEK &string where string="Smith" and up comes your record. On the other hand if you only knew "Jones" you had to do LOCATE FOR &jones$comp.name which would take quite a while. Under RushMore it's as fast as SEEK. If you have complex queries, every field that has an index available will be optimizable. No index, no optimization. -- Daniel A. Hartung | "What's the difference anyway, between being dhartung@chinet.chi.il.us | safe and being rad, the joke's on us, we've Birch Grove Software | all been had." -- John Wesley Harding -----------FoxPro Programmer Looking For Work--------------
dhartung@chinet.chi.il.us (Dan Hartung) (06/10/91)
shevett@dworkin.UUCP ( Sysop) writes: >dhartung@chinet.chi.il.us (Dan Hartung) writes: > >>FoxPro operates impressively well for an xBase product on even a 640K XT. >>When you give it extra memory, though, it takes off. It makes automatic >>used of expanded memory; > >CAREFUL HERE! The biggest fault in FoxPro 1.02 is that it does *NOT* make >complete use of Expanded memory. It will only cache indexes and dbf info >in the expanded RAM, and no memory variables or program code will go there. >Therefore, you may have 16meg of expanded memory, but it doesn't help you >one whit if you program needs 200k of memory variables (which one of our >system's does). If you have a program that needs to FOXSWAP out to another >app (which we do), it's very difficult running FoxPro, even with an EMS >manager installed (we use QEMM, which lets us use FOXQ, but it's still a >tight squeeze) This is true. I guess I was just happy to get the extra performance when I put in an EMS board that I didn't pay attention to what it DIDN'T do! Still, if the expanded memory is there and available, Fox will just take it without asking (you can turn this off of course!). >>Fox 2 will use extended automatically, but now >>you have to use a (well-behaved) EMS emulator if you only have extended. > >Really? Extended memory automagically detected and used? Could you >confirm this? Yes. Included with the package is an "Extended Mode" FoxPro for 386s and 486s, and I believe this is connected w/ the extended memory usage. It is confirmed that Fox will use XMS just as it will EMS, though the only caveat is that I'm not sure you can do this on a 286, since you can't run the "EM". >>xBase is not very good with number fields. In general numbers are best >>stored as a character representation anyway. > >EXCUSE ME? I'm confused here. Why would you use character representations >of numeric fields? Seems pretty dopey to do a VAL conversion whenever >retrieving from the database. Well, I'm told by "experts" that it's "safer", whatever that means. For one thing, indexes should be on STR(number), and use the full syntax STR(number,1,length) or the index will eventually break, even on Fox. And the character field lets you store a "n/a" value or "unknown" as opposed to 0. Also, this can affect the index order for things that look like numbers, but really shouldn't be handled that way -- e.g. zip codes. 53545-1234 will index after 53546 if you're not careful. I guess I should have said that you should consider character fields the default, and only use numeric when necessary. -- Daniel A. Hartung | "What's the difference anyway, between being dhartung@chinet.chi.il.us | safe and being rad, the joke's on us, we've Birch Grove Software | all been had." -- John Wesley Harding -----------FoxPro Programmer Looking For Work--------------
oysteing@idt.unit.no (Oystein Groevlen) (06/17/91)
In article <1991Jun09.215527.10286@chinet.chi.il.us>, dhartung@chinet.chi.il.us (Dan Hartung) writes: |> No. Since SEEKs already operate virtually instantaneously, what this does |> is apply index usage to commands that previously would have done a physical |> record-by-record search. For instance, if you know the first word of a |> company name is "Smith" of "Smith & Jones" you can just SEEK &string where |> string="Smith" and up comes your record. On the other hand if you only |> knew "Jones" you had to do LOCATE FOR &jones$comp.name which would take |> quite a while. Under RushMore it's as fast as SEEK. In the demonstration we got of FoxPro 2.0, matching the beginning of a field was fast, but matching patterns in the middle or end of a field was slow. Are you saying that also such matches could be executed fast in FoxPro? For example, will searching for the string "th & J" somewhere in a field go fast too? If that is the case, Rushmore is really revolutionary. -- Oystein Groevlen Division of Computer Systems and Telematics The Norwegian Institute of Technology The University of Trondheim Email: oysteing@idt.unit.no
onder@ISI.EDU (Bruce Onder) (06/19/91)
In article <1991Jun17.145014.15679@ugle.unit.no> oysteing@idt.unit.no (Oystein Groevlen) writes: >In the demonstration we got of FoxPro 2.0, matching the beginning of a >field was fast, but matching patterns in the middle or end of a field >was slow. > >Are you saying that also such matches could be executed fast in >FoxPro? For example, will searching for the string "th & J" somewhere >in a field go fast too? If that is the case, Rushmore is really >revolutionary. There is such a beast as partial optimization. Not as fast as the normal search, but still better than being unoptimized. -- Bruce W. Onder onder@isi.edu (He's not your everyday-type prankster!) I'm Ice-T: Original Gangster (O.G.: Original Gangster)