[comp.misc] Compression algorithm wanted

jjoshua@remus.rutgers.edu (J. Joshua) (12/05/90)

I am looking for a compresion algorithm that has a good trade off
between speed and compression % leaning more towards speed.  I would
like the algorith to work in as real-time as possible.  Any
suggestions?  Any code?


Thanks,
JOn.
-- 
 ________
|        |      This Messsage             A Service By:
|        |     Closed Captioned                 Jon Joshua
|        |  For the Hearing Impaired            jjoshua@remus.rutgers.edu
`----. .-'
     |/                 #include <whittyComment.h>   

jtc@van-bc.wimsey.bc.ca (J.T. Conklin) (12/05/90)

In article <Dec.4.19.32.13.1990.16026@remus.rutgers.edu> jjoshua@remus.rutgers.edu (J. Joshua) writes:
>I am looking for a compresion algorithm that has a good trade off
>between speed and compression % leaning more towards speed.  I would
>like the algorith to work in as real-time as possible.  Any
>suggestions?  Any code?

This is a classic example of an incomplete specification.  To
satisifactorily answer this question, the nature of the data to be
compressed, acceptable time/space tradeoffs, etc. is unknown.

It would be irresponsible for anyone to suggest a compression
algorithm when there is nothing to go on.

	--jtc


-- 
J.T. Conklin	Toolsmith, Language Lawyer
		UniFax Communications Inc.
		...!{uunet,ubc-cs}!van-bc!jtc, jtc@wimsey.bc.ca

jjoshua@remus.rutgers.edu (J. Joshua) (12/05/90)

xIn article <774@van-bc.wimsey.bc.ca> jtc@van-bc.wimsey.bc.ca (J.T. Conklin) writes:

> It would be irresponsible for anyone to suggest a compression
> algorithm when there is nothing to go on.
> 
> 	--jtc

if it help you to know...  i want to compress a byte stream as it goes
from a hard disk to a floppy disk.  ie a backup program.  this will
run on a macintosh and i would like to acheive a good compression
ratio without too much cost in time.

now can you suggest an algorithm?

JOn.
-- 
 ________
|        |      This Messsage             A Service By:
|        |     Closed Captioned                 Jon Joshua
|        |  For the Hearing Impaired            jjoshua@remus.rutgers.edu
`----. .-'
     |/                 #include <whittyComment.h>   

jtc@van-bc.wimsey.bc.ca (J.T. Conklin) (12/06/90)

In article <Dec.5.00.30.55.1990.3826@remus.rutgers.edu> jjoshua@remus.rutgers.edu (J. Joshua) writes:
>if it help you to know...  i want to compress a byte stream as it goes
>from a hard disk to a floppy disk.  ie a backup program.  this will
>run on a macintosh and i would like to acheive a good compression
>ratio without too much cost in time.
>
>now can you suggest an algorithm?

I usually recommend against compressing backups, as it is almost
impossible to recover anything if even a single bit is changed.  For
your application, it will be imporatant to design in error recovery.

So you probably don't want to compress all of the files together in
one stream, but rather compress each file individually.  This way, a
damaged archive will only effect the one file where the damage occurs.
Storing files individually has the added advantage that each file can
be compressed using a different encoding method, as no one method is
approprate for every different file type.

You might want to consider adding error detection and correction codes
to the compressed data.  This will expand the data a bit, but when
combined with compression it should be a net gain.


I'd start by supporting two compression methods: no compression at
all, and LZW encoding.  LZW is a good compression method, especially
for text and uncompressed bilevel graphic images.  You want a "no
compression" compression methods for the files that exibit "negative
compression".  i.e. they get bigger when compressed.


For a sample LZW implementation, look at the compress source code.  It
should be availiable on your machine, as compress is used by the news
software to compress news batches.

	--jtc

-- 
J.T. Conklin	Toolsmith, Language Lawyer
		UniFax Communications Inc.
		...!{uunet,ubc-cs}!van-bc!jtc, jtc@wimsey.bc.ca

lalibert@bcarh188.bnr.ca (Luc Laliberte) (12/07/90)

I will agree that LZW is an excellent compression algorithm.  The
original article by Welch is in the June 1984 issue of IEEE Computer.
However, the algorithm was later patented by Unisys (#4,558,302)
because he did the work on company time.  Check recent issues of
Dr. Dobbs Journal for a discussion of this topic.

Eric Ball