fadden@cory.Berkeley.EDU (Andy McFadden) (08/11/89)
Whoops! I managed to insert the same BinSCII file in twice, and omitted the
last one. Since that cut the docs in half, I'm posting them here (in un-
packed form, since pack + BinSCII isn't much savings on a 12K file).
NOTES:
I encountered a glitch when EXtracting more than one file at a time
from an archive. Fumigating in progress.
--
fadden@cory.berkeley.edu (Andy McFadden)
...!ucbvax!cory!fadden
*****
Remarks about NuARC NuFX archive utility (formerly NuView)
Version 2.0 July 1989
First and foremost:
This program is Freeware (NuView was 100% public domain). You may distribute
it as widely as you like, but you may not sell it. My liability for this
is limited to the purchase price (in other words, zero), so caveat emptor.
I will not be held responsible for destroyed archive files or other damage
associated with the use of this program. So there.
Source code for this program will be made available through e-mail on the
Internet and Usenet once the UNIX version is complete. My goal is to have a
NuFX program which will work on virtually any system with a C compiler. I
could use help someday with some MS-DOS modifications (wildcard expansion,
subdir creation, memory management, etc), but that won't happen until v2.1 is
read to go (read all about the exciting details herein).
This is an early release, but everything *should* work as stated. I wouldn't
try using it on bizarre stuff (like GS/OS extended files).
Full documentation is in the works. Until then...
***** Revision history:
vers 2.0 - Conversion to an (incomplete) shell-based NuFX archive utility.
The LAMESEEK option has been removed, and some minor fixes have been
made. New features: eX)tract and A)dd (uncompressed only), D)elete,
filename-only output format, CRC calculation/verification.
[ versions earlier than 2.0 were distributed as "NuView" ]
vers 1.2 - Fixed some minor bugs, and added some new functions (Malloc(), etc)
to make it work under APW C. Now uses %ld instead of %d to print
four byte values. Made some minor alterations to output format, and
generally cleaned up the source code.
vers 1.1 - Did major rewrite of the way data is read in. Used to just read
into a buffer and pick parts out of it by applying a struct directly;
now read into a special buffer and copy individual entries into a
struct. Cleaned up code a bit. Used fseek instead of xx getc() calls.
Added code to hold the position in file of all threads. Fixed bug
with non-(void) sprintf.
vers 1.0 - Initial release. Works great, but only under UNIX on a Sun 3/50.
***** Program description:
NuARC is intended to be a NuFX archive utility similar to ARC for the IBM PC
and tar and ar under UNIX. It can list archives in different formats, add
files, delete files, extract files, and perform other miscellaneous functions.
To get a list of available options, type "nuarc" without any parameters.
In its present state, it can produce a ProDOS 8 ShrinkIt-like listing, a brief
filename-only listing (especially useful for archives containing long
pathnames), or a detailed breakdown of every known facet of the archive. It
can add, extract, or delete a series of files with a single command.
Adding files allows wildcards and (someday) subdirectories. I used the APW
shell wildcard commands, which return full pathnames. If the pathname is
in the current directory (e.g. you typed "foo/bar/filename" at the command
line), the file will be added with all of the paths after the current prefix
(so "foo/bar/filename" will appear in the archive). If the pathname is not in
the current directory in a subdirectory of the current directory, then only
the filename will be added (so "../zip/bang/filename" would appear in the
archive as "filename").
When extracting files, NuARC compares the command arguments with the first
few characters of the archived pathname. This allows you to extract sub-
directories without extra commands (so typing "nuarc x archive foo" will
extract "foozle", "food/recipies", and "food/stock"). Directories will be
created if necessary, but NuARC will abort if an element of the path exists
and is not a directory. Someday you will be asked if you want to overwrite
an existing file (it does), and there will be an "extract all" option.
Deleteing files works the same way as extracting; it will delete whatever
pattern matches. A new archive is created, and the files that are not going
to be deleted are placed in the new archive (so essentially it isn't deleteing
anything; it just isn't copying the archives that you name). Note that you
must have enough disk space for the new archive! If the operation fails,
however, the old archive will be left intact. Someday I will allow prompting.
When adding files to an archive, there are three different ways to do it.
Option A (add) just appends files to the end. This is relatively fast, and
doesn't require any disk space beyond the size of the compressed files. At
present, it does *not* check to see if a similarly named file already exists.
Options U (update) and F (freshen) make a new copy of the archive, storing the
most recent copy of a file (it compares dates with the archived file and the
disk file). This means you must have enough disk space for a new archive to
be created before the old one is deleted. The difference between update and
freshen is that freshen does not add new files; it just updates the old ones.
***** The down side:
When ProDOS 8 ShrinkIt packs a file, it always uses records with one thread
(of type data_thread). I have tried to make the code flexible, but since I
can't generate proper archives with multiple threads and resource forks until
GS/OS ShrinkIt is available, I can't honestly say that it will work when
presented with such an archive.
The big problem this program has is speed. Under APW, written in APW C, it
takes about three days to do anything. What I envision is people using
ShrinkIt at home on their Apple //s, but NuARC on UNIX systems. This will
facilitate binary uploads of large compressed files, which can then be quickly
unpacked on the destination system (which will likely have a much greater
computing power).
Part of the problem has been resolved by using straight open() calls instead
of fopen() for large blocks. While buffered I/O is nice when you are
performing several small consecutive reads (like NuRead does), it isn't so
great for reading 32K chunks. A quick test on a huge ProDOS file required 40
seconds for fread(), but only 13 for read(). So, the A)dd and E)xtract
routines use the "raw" I/O routines (which use unsigned int lengths, allowing
a 64K read/write buffer), and are almost as fast as ProDOS 8 ShrinkIt (and can
be much faster if you have a special GS/OS driver).
Another (unavoidable) problem is that the user interface isn't nearly as nice
as the one in ShrinkIt. While shell-based programs have their advantages
(especially if you're running in a shell...), it's hard to compete with the
ease of use of the "point and shoot" method. Of course, it's also a lot more
portable...
I have tried to make this easy to extend for new advances. Adding new output
formats only requires extending an enumerated type and inserting a new
if-then-else clause, and interfacing existing code to new extensions should
be fairly simple.
***** System dependencies:
When compiling this on a Sun 3/50, I noticed something unfortunate: the byte
ordering for the //gs and the Sun are backward. The header file "nuread.h"
currently contains a definition of "HILO"; when this is defined, the code
generated while reverse the byte ordering automatically. You should clear
this if you start getting really weird results from valid archives (or if
you are sure that your machine works in LO->HI byte order).
When the program is started, it will do a quick byte ordering check. If your
values for onebyt and twobyt are not properly set, it may come to the wrong
conclusion (although it will look for such things also).
There are definitions for one byte, two byte, and four byte variable types;
my compiler uses char, short, and long. If these are different for your
compiler, be sure to change the typedefs. These appear in "nuread.h".
If this is being run under BSD UNIX, you should define BSD43 in "nuread.h".
This will enable certain UNIX functions (like time routines), and disable
others (like text translation). The program will *probably* work under any
UNIX system with this defined, but I'm not absolutely positive. I will try
not to use any BSD-specific features.
***** TO DO:
Roughly in order of importance:
- Add ShrinkIt compress/uncompress routines. This will require more than
a little cooperation on the part of Andy Nicholas, since I'm not up on my
dynamic LZW (especially not his implementation thereof).
- Add Update/Freshen, implement Move, and improve the file integrity check.
- Improve UNIX version. Need to handle system clock, file date stamp, and
subdirectory expansion.
- Add Binary II file handling.
- Add minor things, like "Y/n/q" prompts when files already exist, generate
real temporary names (currently only "nuarc.tmp" is returned), do "none
selected" stuff (especially in D)elete).
- Cope somehow with APW segmentation.
The above is my goal for version 2.1 (basically, make everything on the menu
function as advertised). Future enhancements include:
- Improve GS/OS handling. Need to handle resource forks and file system IDs.
Would probably help if I had a GS/OS reference...
- Add disk support.
- Add to thread methods (insert ASCII messages, etc).
- Add different compression methods (lharc, UNIX compress).
***** APW Notes:
Version 1.2 used a custom malloc() routine; version 2.0 uses the APW C
version. The only major APW-specific routines are an APW perror() and
ToolErrChk() (both in nuetc.c). However, APW-specific code appears in various
places, primarily in nuadd.c and nuext.c (which must cope with file types,
creation dates, subdirectory expansion, etc.)
***** Bugs / glitches:
UNIX seek uses longs, which are usually four bytes. Signed. If an archive
is larger than 2 gigabytes, there may be a problem (cough).
Pathnames longer than the #defined maximum (1024 bytes) will cause
unpredictable results (most likely bad ones, though). Pathnames with a null
('\0') in them can cause problems when modifying archives, and should
generally be avoided.
A maximum of 255 files may be added/deleted/whatever at at time. Expanded
subdirectories count. This is an arbitrary number and is easy to change if
someone can convice me that you'd need to archive more than 255 files at a time.
There is a maximum of 65535 threads per record.
The ProDOS filetype names may or may not be the offical Apple versions. Feel
free to tell me what's wrong.
Error output is informative but ugly.
Version 2.0 is not complete (obviously).
***** Final notes:
I said I wasn't going to write a file extractor. Okay, I changed my mind.
But before it unpacks files, I'm going to need some help with the compression
routines...
If nothing else, this will allow you to check the output of sciibin (a UNIX
BinSCII extractor) before you actually download the files.
In the works (once this gets finished): the NuView CDA. View NuFX archives
from any application (I'll try to include both ProDOS 8 and GS/OS support).
***** Who I Be:
I'm presently a Junior in Electrical Engineering and Computer Science (EECS)
at the University of California, Berkeley.
fadden@cory.berkeley.edu (Andy McFadden)
...!ucbvax!cory!fadden
No relation to the author of ShrinkIt and NuFX, Andy Nicholas. This was and
continues to be my idea; throw all kudos and blame in my general direction.
All code herein is mine, developed from the NuFX Documentation Final Revision
Three (2/3/89), except as noted below.
The Binary II unpack routines and the C unsqueeze code were written by Marcel
J.E. Mol (based on usq2/sq3 by Don Elton). They will be included (in a future
release) with permission from the author.
If you have suggestions, comments, or excess cash, you can send mail to:
Andy McFadden
1474 Saskatchewan Dr.
Sunnyvale, CA 94087
If you find this program especially useful, send money to the guy who came up
with NuFX and ShrinkIt (this program is my contribution):
Paper Bag Productions CSNET: shrinkit@moravian.edu
c/o Andy Nicholas InterNET: shrinkit%moravian.edu@relay.cs.net
Box 435
Moravian College
Bethlehem, PA 18018 AppleLink PE: ShrinkIt
*****
END OF NUARC V2.00 DOCS