[comp.sys.ibm.pc.programmer] How dows LZEXE work ???

widman@cs.utexas.edu (Lawrence Widman) (07/10/90)

I've got a question...  I downloaded the program LZEXE from Compuserve, and
aside from having to use my sketchily recalled knowledge of High School 
French classes to interpret the instructions and warnings, it's a great
program!!! It shrinks my executable for one program from 230K with 
symbol table to 55k, and from 120K with no symbol table to the same 55k.
It even shrinks Turbo C ver 2.0 from 290K or so to about 160K.

It doesn't work right with EXE files for extended mode things like
Turbo Debugger 386 or Instant C, but that's to be expected.

What I want to know is:  How does it work?  (Aside from stripping
debugging garbage).  Turbo C, lzexe'd, works fine and, though I don't
have time to run benchmarks, faster.

		Thanks,
			Avi Freedman
			_using account of widman@cs.utexas.edu_

baalke@mars.jpl.nasa.gov (Ron Baalke) (07/10/90)

In article <9857@cs.utexas.edu> widman@cs.utexas.edu (Lawrence Widman) writes:
>I've got a question...  I downloaded the program LZEXE from Compuserve, and
        .
        .
        .
>It even shrinks Turbo C ver 2.0 from 290K or so to about 160K.

I've also tried LZEXE on Turbo C 2.0.  However, once you do this the
TCINST no longer works, because it is apparently expects the configuration
data to be in a specific spot in TC.EXE.  So, if you like the way TC is 
configured, then go ahead and compress it with LZEXE (and keep a backup
copy somewhere).  Otherwise, don't use LZEXE on Turbo C 2.0.

imp@dancer.Solbourne.COM (Warner Losh) (07/10/90)

In article <4263@jato.Jpl.Nasa.Gov> baalke@mars.UUCP (Ron Baalke) writes:
>In article <9857@cs.utexas.edu> widman@cs.utexas.edu (Lawrence Widman) writes:
>>I've got a question...  I downloaded the program LZEXE from Compuserve, and
>        .
>        .
>        .
>>It even shrinks Turbo C ver 2.0 from 290K or so to about 160K.
>
>I've also tried LZEXE on Turbo C 2.0.  However, once you do this the
>TCINST no longer works, because it is apparently expects the configuration
>data to be in a specific spot in TC.EXE.  So, if you like the way TC is 
>configured, then go ahead and compress it with LZEXE (and keep a backup
>copy somewhere).  Otherwise, don't use LZEXE on Turbo C 2.0.

The reason for this is that the data doesn't exist in the same format
as it did before you lzexeed the file.  What lzexe does is to encode
the file using the Lempel-Zev compression algorithms (more or less)
and then placing a loader that knows how to expand the results at the
front of the file.  It also does relocation after it expands the file,
but I don't know[*] if MS-DOS does the expansion or not.  Nor do I
know if the LZ expander stays in memory or not after the program has
been exploded.  I have never seen the source to this beast.

TCINST works by reading in the data area of TC.EXE and poking new
values into the variables that control whatever it is that you are
changing.  When there is no data area that is recognizable (due to
compression), TCINST can't work.

One last thing, LZEXE does not work by "removing all that debugging
garbage".  Under DOS there is not standard debugging format, and most
programs that customers see don't have that info anyway.  However,
LZEXE proves useful on those programs none the less.

One last warning about LZEXE, it seems that most virus detection
programs can't find a virus inside an LZEXEed program.

Warner

[*] If you feel the need to flame my lack of knowledge, or my
uncertainty, please send me email.  There is enough noise in this
group already.
--
Warner Losh		imp@Solbourne.COM
Boycott Lotus.		#include <std/disclaimer>

roy@cs.umn.edu (Roy M. Silvernail) (07/10/90)

imp@dancer.Solbourne.COM (Warner Losh) writes:

> One last warning about LZEXE, it seems that most virus detection
> programs can't find a virus inside an LZEXEed program.

The current version of MacAfee's SCANV will detect a LZEXE'd file,
expand it and scan the result.

--
    Roy M. Silvernail   |   "It won't work... I have an  | Opinions found
    now available at:   |   exceptionally large mind."   | herein are mine,
 cybrspc!roy@cs.umn.edu | --Marvin, the paranoid android | but you can rent
(cyberspace... be here!)|                                | them.

nelson@sun.soe.clarkson.edu (Russ Nelson) (07/10/90)

In article <1990Jul9.232906.8904@Solbourne.COM> imp@dancer.Solbourne.COM (Warner Losh) writes:

   The reason for this is that the data doesn't exist in the same format
   as it did before you lzexeed the file.  What lzexe does is to encode
   the file using the Lempel-Zev compression algorithms (more or less)
   and then placing a loader that knows how to expand the results at the
   front of the file.  It also does relocation after it expands the file,
   but I don't know[*] if MS-DOS does the expansion or not.  Nor do I
   know if the LZ expander stays in memory or not after the program has
   been exploded.  I have never seen the source to this beast.

MS-DOS just loads the whole thing.  The loader moves the file and
itself up in memory, and then expands the file back down.  Then it
runs through the relocation table, which is has rewritten in a more
compact form.  It sets up registers just like MS-DOS does, and jumps
to the program.


--
--russ (nelson@clutx [.bitnet | .clarkson.edu])  Russ.Nelson@$315.268.6667
In Communism's central planning, citizens are told "you will make widgets".
In Capitalism's advertising, citizens are told "you will buy widgets".

frisk@rhi.hi.is (Fridrik Skulason) (07/10/90)

In article <1990Jul9.232906.8904@Solbourne.COM> imp@dancer.Solbourne.COM (Warner Losh) writes:
>One last warning about LZEXE, it seems that most virus detection
>programs can't find a virus inside an LZEXEed program.

Well, as far as I know, there are two programs able to scan LZEXE-packed
files.  One is the SCAN program from McAfee, but the other is F-PROT 1.11
which I wrote.

I think no other virus scanning program has implemented LZEXE-scanning yet.

-- 
Fridrik Skulason      University of Iceland  |       
Technical Editor of the Virus Bulletin (UK)  |  Reserved for future expansion
E-Mail: frisk@rhi.hi.is    Fax: 354-1-28801  |   

ts@uwasa.fi (Timo Salmi LASK) (07/10/90)

In article <1835@krafla.rhi.hi.is> frisk@rhi.hi.is (Fridrik Skulason) writes:
>In article <1990Jul9.232906.8904@Solbourne.COM> imp@dancer.Solbourne.COM (Warner Losh) writes:
>>One last warning about LZEXE, it seems that most virus detection
>>programs can't find a virus inside an LZEXEed program.
>
>Well, as far as I know, there are two programs able to scan LZEXE-packed
>files.  One is the SCAN program from McAfee, but the other is F-PROT 1.11
>which I wrote.
>I think no other virus scanning program has implemented LZEXE-scanning yet.

One obvious, but less sophisticated way is to decompress the lzexe'd
program first, and then scan it.  This process can even be automated
using a batch file.  There is one in /pc/ts/tsbat21.arc to show the
principle.  Available by anonymous ftp from chyde.uwasa.fi, Vaasa,
Finland.  (Note that I'm not saying that direct lzexe scanning
weren't preferable (it is the better alternative), but just giving
information.)

Oh yes, and since someone is probably going to ask how to decompress
an lzexe'd file, there is a utility called /pc/pd2/unlzexe5.zip
available at the same site. 

...................................................................
Prof. Timo Salmi        (Moderating at anon. ftp site 128.214.12.3)
School of Business Studies, University of Vaasa, SF-65101, Finland
Internet: ts@chyde.uwasa.fi Funet: gado::salmi Bitnet: salmi@finfun

doerschu@rex.cs.tulane.edu (Dave Doerschuk) (07/11/90)

In article <1990Jul9.232906.8904@Solbourne.COM> imp@dancer.Solbourne.COM (Warner Losh) writes:
> [...]
>One last warning about LZEXE, it seems that most virus detection
>programs can't find a virus inside an LZEXEed program.
>
>Warner

Warner is quite right here, I merely want to add a rider to mention that
John McAfee's "SCAN" program is not only one of the better virus scanners
out there, but currently supports (as of version 63) checking LZEXE'ed
programs for viruses.  SCAN is currently at version 64 (I think) and
is distributed as "SCANV64.ZIP" at simtel20 and your local simtel mirror
site. :-)

Dave
doerschu@rex.cs.tulane.edu