krulwich@ils.nwu.edu (Bruce Krulwich) (08/30/90)
Hi. I've been toying with an EMACS-LISP package to maintain simple records of information (phone lists, grades, etc). Ignoring all the issues of user interface, I'm concerned about GNU EMACS's ability to react well to reasonably large amounts of information stored in CONS-cells and strings as opposed to editor buffers. (I'm sure there are those who'd say I should be using a buffer instead of lists, but lists seem better for the functions I want.) My question is: Is the EMACS-LISP heap from which CONS cells and strings are given memory handled differently than the memory in which editor buffers are stored, and is the former worse for storing information than the latter? I ask this because I can imagine GNU EMACS written such that the regular heap (for CONS cells and strings) is memory-resident but the buffers have a scheme of storing in temporary files when not active. In this case storing non-trivial amounts of information in the heap will have a detrimental effect on the whole system, while storing information in buffers would not. Thanks for any info anyone can give me. Bruce Krulwich krulwich@ils.nwu.edu Institute for the Learning Sciences
jbw@bucsf.bu.edu (Joe Wells) (09/03/90)
In article <1537@anaxagoras.ils.nwu.edu> krulwich@ils.nwu.edu (Bruce Krulwich) writes:
Ignoring all the issues of user interface, I'm concerned about GNU
EMACS's ability to react well to reasonably large amounts of
information stored in CONS-cells and strings as opposed to editor
buffers. My question is:
Is the EMACS-LISP heap from which CONS cells and strings are given
memory handled differently than the memory in which editor buffers are
stored, and is the former worse for storing information than the
latter?
With GNU Emacs 18, memory for both types of storage is allocated using
malloc. Thus, they both use the same "type" of memory. Using a buffer
for storing a large database may allow much better virtual memory
performance if you access the buffer in a non-expensive manner. Expensive
types of access to a buffer include lots of random-access modifications
and, to a lesser degree, any other random access to the buffer.
Buffer modifications can be expensive because any modification requires
moving the "gap" in the buffer to the point of modification. If you make
a change at the beginning of the buffer, and then one at the end, every
byte in the buffer will be moved (probably from one page of memory to the
next). Normal editing usually doesn't require moving the gap very far or
very often. Thus, buffer operations are very cheap for editing.
Of course there are also some obvious reasons for not using lists either.
Using lists, it is possible for each cons cell to be located in a
different virtual memory page. If you are accessing a large list, you
will quickly exhaust your active set and start forcing a lot of paging.
Also, garbage collections are invoked depending on the frequency of
consing. If modifying your database requires a lot of consing this can be
a problem. Since Emacs doesn't do any compaction when it garbage collects
(except for strings), over time things become fragmentedm, virtual memory
behavior becomes horrible, and thus garbage collection takes longer and
longer and longer ... (Of course, you could avoid this by using the
(often considered dangerous) setcar and setcdr functions.)
I ask this because I can imagine GNU EMACS written such that the
regular heap (for CONS cells and strings) is memory-resident but the
buffers have a scheme of storing in temporary files when not active.
GNU Emacs relies entirely upon the operating system's support of virtual
memory to keep only active things in memory. (That is, No, Emacs doesn't
do that.)
Separate, unrelated piece of advice:
Emacs provides hash tables, except that they're called "obarray"s. You
can use the "intern" function to hash a string key into a Lisp symbol.
You can supply your own symbol table so that this new Lisp symbol isn't
visible to the rest of Emacs Lisp. Simply store your database entry on
the symbol.
--
Joe Wells <jbw@bu.edu>