mwebb@maths.tcd.ie (Mark Webb) (12/10/88)
I'm looking for information about tokenisation of data in a database where space is at a premium. Isn't it always! I have a data file (about 200k) mainly consisting of names and addresses which needs to be stored (and retreived) more efficiently. (e.g. street names could be stored as a single token) The tokenisation process (analysis, storage etc) need not necessarily be fast, but the de-tokenisation process must be rapid (probably by means of a `look-up' table of some sort). If anyone could send me some references or better still some code (C, ASM, Pascal), I would be most grateful. -- -Mark Webb mwebb@maths.tcd.ie