tgd@tesla.cs.orst.edu (Tom Dietterich) (09/30/90)
Frequently I use large files of S-expressions as a database for moving information from one lisp program to another. I find that using (read) to read these expressions back into lisp is extremely slow. If, however, I set up the file so that it can be (load)-ed instead, it comes in much much faster. Have other people observed this? Has anyone got an explanation? -- Thomas G. Dietterich Department of Computer Science Dearborn Hall, 303 Oregon State University
michaelg@Neon.Stanford.EDU (Michael Greenwald) (10/01/90)
tgd@tesla.cs.orst.edu (Tom Dietterich) writes: >Frequently I use large files of S-expressions as a database for moving >information from one lisp program to another. I find that using >(read) to read these expressions back into lisp is extremely slow. >If, however, I set up the file so that it can be (load)-ed instead, it >comes in much much faster. Have other people observed this? Has >anyone got an explanation? If you use (read) it must parse the file character by character, tokenize it, (albeit usually by a relatively efficient FSM). If you use (load ...) then you are presumably loading already parsed data structures. On many implementations an array of numbers, for example, will usually LOAD much faster than it can be READ. Perhaps I'm misunderstanding you. Perhaps you are comparing two ascii files whose only difference is that the latter one has forms like (SETQ A '( ... s-expressions ...)) instead of plain s-expressions? If so, then both have to be (READ), so it would be hard for someone to explain why LOAD is so much faster than READ without knowing more details about the particular files you are talking about. >-- >Thomas G. Dietterich >Department of Computer Science >Dearborn Hall, 303 >Oregon State University
tgd@tesla.CS.ORST.EDU (Tom Dietterich) (10/03/90)
In article <1990Oct1.123907.24074@Neon.Stanford.EDU> michaelg@Neon.Stanford.EDU (Michael Greenwald) writes: >Perhaps I'm misunderstanding you. Perhaps you are comparing two ascii >files whose only difference is that the latter one has forms like > (SETQ A '( ... s-expressions ...)) >instead of plain s-expressions? > >If so, then both have to be (READ), so it would be hard for someone to >explain why LOAD is so much faster than READ without knowing more >details about the particular files you are talking about. > Yes, you are misunderstanding me. I am comparing calling READ on a file containing a big s-expression versus calling LOAD on a nearly identical file of the form (setq foo s-expression) It doesn't matter much what is in the s-expression, in my experience. >>Thomas G. Dietterich >>Department of Computer Science >>Dearborn Hall, 303 >>Oregon State University -- Thomas G. Dietterich Department of Computer Science Dearborn Hall, 303 Oregon State University
kanamori@Neon.Stanford.EDU (Atsushi Kanamori) (10/03/90)
In article <20644@orstcs.CS.ORST.EDU> tgd@tesla.cs.orst.edu (Tom Dietterich) writes: >Frequently I use large files of S-expressions as a database for moving >information from one lisp program to another. I find that using >(read) to read these expressions back into lisp is extremely slow. >If, however, I set up the file so that it can be (load)-ed instead, it >comes in much much faster. Have other people observed this? Has >anyone got an explanation? > Gee, you got me. Perhaps you might check up on the following theory? 1. Whoever wrote your Lisp system hard-coded the parser into his "load" command without thinking about the fact that the parser is needed separately. 2. Later, he realizes that Lisp also requires a "read" interface to the parser. 3. Groans and slaps his head. 4. Rather than rewrite the parser, codes up "read" as follows: a. Copy characters from input stream to a /tmp file. b. Do quick scan to find s-expression boundaries and edit /tmp file to change SEXPR into (setq tmpvar 'SEXPR) c. Calls "load" to read in /tmp file. d. Returns value of tmpvar. o o | (a big smiley for above paragraph) \ | / \_____/ P.S.: Maybe your "load" command is secretly caching a preprocessed version of your datafiles?