fadden@cory.Berkeley.EDU (Andy McFadden) (08/14/89)
I won't be reading comp.sys.apple for another week or so, so PLEASE, if you
have any comments or suggestions that you think I should hear, SEND MAIL.
This is the documentation for NuARC v2.1. I am currently in the process of
making the actual program fit the documentation... At any rate, I'm still
open to suggestions on how the program should run. If you have any interest
in the UNIX version of NuARC, PLEASE read this and tell me if you thing
something should be done differently.
--
fadden@cory.berkeley.edu (Andy McFadden)
...!ucbvax!cory!fadden
***** START
NuFile eXchange (NuFX) Archive Utility
NuARC v2.1 Documentation
By Andy McFadden
Slight disclaimer: I'm writing this documentation before version 2.1
is complete. Since I may release a pre-2.1 UNIX version of NuARC,
this documentation may reference features not present in the program
that came with it.
This document will be distributed as an AppleWorks (8-bit) WP document
with the //gs version, and as a standard text file with the UNIX shar
distribution.
Overview
NuARC is a shell-based NuFX archive utility, based loosely on ARC for
the IBM PC and UNIX ar. It allows you to perform certain operations
on the same archives used by ShrinkIt, including view archive
contents, add to archive, extract from archive, and delete from
archive. In addition, it will list and unpack files from Binary II
archives.
This program is primarily targeted at users of other computer systems,
from IBM PC compatibles to Sun workstations to DEC Vaxen. It will
also be of use to people who use a GS/OS shell on the //gs, such as
APW (although it is significantly slower than ShrinkIt). NuARC may
require up to 256K of free RAM to function properly.
Background
An archive is a collection of one or more files combined into a single
file. This allows groups of related files to be stored under a single
filename, reducing directory clutter, and makes it possible to
transfer groups of files without the use of a batch transfer utility.
To reduce the space required to store the archives and the time it
takes to transfer archives, most popular archiving programs
automatically compress files as they are stored. Two popular
compression methods are Huffman (variable-size codes are used, with
smaller codes assigned to bytes that appear frequently) and
Lempel-Ziv-Welch (LZW; finds common substrings in the file).
Popular archiving programs include "BLU" (Apple II), "ShrinkIt" (Apple
II), "ARC"/"PKARC" (MS-DOS), "zoo" (Amiga), "StuffIt" (Macintosh),
"tar" (UNIX), and "ar" (UNIX). NuARC works with archives that
ShrinkIt produces (NuFX).
New Features/Bug Fixes
+ suboptions are allowed, including verbose output while performing
archive operations.
+ extract works properly with multiple arguments.
+ automatically determines the machine's byte ordering.
How to Use NuARC
Usage: nuarc option[suboption] archive [filespec1] [filespec2] [...]
"option" is a single character, as specified below.
"suboption" is one or more characters, which modify the performance of
the option (see each specific option).
"archive" is the name of a NuFX archive. It doesn't need to exist for
the add, create, and move options.
Examples:
$ nuarc av foo.shk file1 subdir/file2
Adds "file1" and "subdir/file2" to "foo.shk", listing each file as it
is archived.
$ nuarc d foo.shk work/
Deletes all files in "foo.shk" that start with "work/"; this would
likely be used to delete all files archived from the directory "work".
The various options are:
*** Add/Append files:
nuarc A[V] archive files
Quickly appends the specified files to the end of the archive. This
option does not scan the archive to see if the files already exist,
and does not create a new archive file. This option corresponds to
A)dd files on the ShrinkIt menu.
If you add files from the current directory or a subdirectory of the
current directory, then the entire partial pathname is stored (ex:
"nuarc av fubar.shk foo/bar/filename" will store "foo/bar/filename" in
the archive "fubar.shk.") If you add files from directory that is not
a subdirectory of the current directory, only the filename will be
stored (ex: "nuarc av fubar.shk ../zip/bang/filename" will store
"filename" in the archive).
Wildcards are allowed, and subdirectories are automatically expanded.
The V suboption displays the filenames as they are added.
*** Binary II Operations
nuarc B[V] archive
nuarc B[X] archive archived-files
The V suboption allows you to view Binary II archives. The X
suboption extracts the named files from the archive, unsqueezing them
if necessary.
*** Create Archive
nuarc C[V] archive files
This is identical to the A option, but the "creating archive" message
is suppressed.
*** Delete from Archive
nuarc D[V] archive archived-files
Deletes the named files from the archive. NuARC scans the archive,
marking all records that start with the characters in the file
specification (case-independent). If all files are marked for
deletion, then the archive file itself is deleted. Otherwise, a new
archive is created, the unmarked records are transferred, and the old
archive is deleted. An error during this process will leave the
original archive unmodified.
Note that this does not require an exact match; "nuarc d fubar.shk
foo" will delete "foo", "Foozle", "food/recipies" and "food/stock."
The V suboption prints the list of marked records.
*** Freshen Archive
nuarc F[V] archive files
Updates files in the archive, but doesn't add any new files. Creates
a new archive file, and either transfers the old record or adds the
file depending on which is more recent. Only exact filename matches
are allowed (case-independent), including partial paths.
Wildcards are allowed, and subdirectories are automatically expanded.
The V suboption displays the filenames as they are added.
*** Integrity Check
nuarc I[V] archive
Verifies that the archive is intact. Does not modify the archive in
any way, or produce output other than to the standard output. The V
suboption prints a list of CRCs for each record (this is different
from those listed by the Z option, which are only for the record
headers, threads, and filename).
*** Move Files to Archive
nuarc M[V] archive files
This is identical to the A option, but the files are deleted after
they are added to the archive. Note that subdirectories are NOT
deleted.
*** Table of Contents
nuarc T archive
This lists only the filenames of the archived files. Not only does
this make it easier to view the archive contents (the ShrinkIt output
format is about 20 characters wide; this is 80), but the output is
suitable for transmission via a pipe to other utilities.
*** Update Archive
nuarc U[V] archive files
Updates files in the archive, keeping the archived file or the file
listed on the command line, whichever is most recent (or exists).
Creates a new archive file, and either transfers the old record or
adds the file. Only exact filename matches are allowed
(case-independent), including partial pathnames.
Wildcards are allowed, and subdirectories are automatically expanded.
The V suboption displays the filenames as they are added.
*** Verbose Archive Listing
nuarc V archive
Lists the archive contents in a format indentical to that used by the
ProDOS 8 ShrinkIt L)ist archive contents option.
*** Author Info
nuarc w
Gives a brief listing of contributors, and how to contact me.
*** Extract from Archive
nuarc X[VT] archive [archived-files]
nuarc E[VT] archive [archived-files]
The X and E options are synonymous. Extract the archived-files from
the archive. If the file already exists, you are asked if you want to
overwrite it. If part of a partial pathname does not exist, the
subdirectory will be created. Omitting the "archived-files"
specification will cause the entire archive to be unpacked.
When files are archived, a pathname separator is stored (e.g., "/" for
ProDOS and UNIX, "\" for MS-DOS, ":" for Mac HFS). During extraction,
the pathnames are broken down into component file names, converted to
names legal under the current operating system, and then recombined
using the pathname separator for the current OS. This facilitates
extraction of files archived under any OS, but can lead to filename
conflicts that didn't exist when the files were added.
Note that this does not require an exact match; "nuarc d fubar.shk
foo" will delete "FOO", "Foozle", "food/recipies" and "food/stock."
This makes it possible to extract entire subdirectories at a time.
The V suboption prints the list of marked records.
Revision history
NuARC v2.1 (Sep 1989):
- added suboption processing.
- replaced buffered I/O (fopen(), fread(), etc) on files with
low-level read/write routines.
- added automatic byte-order determination.
- wrote this documentation.
NuARC v2.0 (Aug 1989):
- added archive manipulation routines (EXtract, Add and Delete for
uncompressed archives).
- added filename-only output format.
- added CRC verification.
- added byte order and data element size checks.
- the LAMESEEK option has been removed.
NuView v1.2 (July 1989):
- major overhaul of all source code to make it work under APW C.
- new //gs-specific routines added.
- minor alterations to output format.
NuView v1.1 (June 1989):
- major rewrite of the way archives are read (had problems with
machines requiring word alignment).
- added thread file position storage to internal archive structure.
- fixed non-(void) sprintf() bug.
NuView v1.0 (May 1989):
- initial release.
- works only on a Sun 3/50 running BSD UNIX.
Limitations
This program has been tested with the output of ProDOS 8 ShrinkIt.
When ProDOS 8 ShrinkIt packs a file, it always uses records with one
thread (of type data_thread). I have tried to make the code flexible,
but since I can't generate proper archives with multiple threads and
resource forks until GS/OS ShrinkIt is available, I can't honestly say
that it will work when presented with such an archive.
The big problem this program has is speed. Since it is meant to be
portable first and efficient second, it won't run as fast as something
like ShrinkIt (written entirely in 65C02 assembly). What I envision
is people using ShrinkIt at home on their Apple //s, but NuARC on UNIX
systems or other microcomputers. This will facilitate transfers of
large compressed files, which can then be quickly unpacked on the
destination system (which will likely have greater computing power).
Note: because of the flexibility of NuFX archives, it is not
inconceivable that they will become popular on other machines.
System Dependencies
When compiling this on a Sun 3/50, I noticed something unfortunate:
the byte ordering for the //gs (65816) and the Sun (68020) are
backward. The present version of NuARC will determine the byte
ordering and treat data appropriately (although this may fail if the
data size definitions are wrong).
There are definitions for one byte, two byte, and four byte variable
types; my compiler uses char, short, and long. If these are different
for your compiler, be sure to change the typedefs in "nuread.h". If
you don't have 8-bit bytes, though, it may not work (most machines
do).
Notes on UNIX implementation:
If this is being run under UNIX, you should #define UNIX in
"nuread.h". This will enable certain UNIX functions (like system()
calls and time routines), and disable others (like file types and text
translation). If this is being run under BSD UNIX, you should also
#define BSD43 in "nuread.h"; certain small differences were
unavoidable (strrchr() vs rindex()).
UNIX subdirectory expansion has not been implemented yet.
Notes on //gs implementation:
The //gs implementation is somewhat lacking. GS/OS support is lacking
(extended files are not handled, different file systems are not
recognized, and mixed-case ProDOS filenames are stored as upper case),
and disk support is a long way off.
APW shell-specific code has been avoided where possible. Places where
I couldn't easily work around it include wildcard expansion and
ERROR(). It was deliberately used in several places to allow the user
to STOP() processing with Apple-period.
Notes on MS-DOS implementation:
Needs help. Anybody wishing to write and debug routines for reading
the current time, getting/setting file creation/modification dates,
expanding command line arguments, and expanding subdirectories is
welcome to do so. Write it, send me the context diffs, and I'll give
you a share of the profits ($0.00).
Bugs / Glitches
UNIX seek uses longs, which are usually four bytes. Signed. If an
archive is larger than 2 gigabytes, there may be a problem (cough).
Pathnames longer than the #defined maximum (1024 bytes) will not be
processed. Pathnames with a null ('\0') in them should generally be
avoided.
A maximum of 255 files may be added/deleted/whatever at at time.
Expanded subdirectories count. This is an arbitrary number and is
easy to change if someone can convice me that you'd need to archive
more than 255 files at a time.
The same file may be added/extracted/whatever more than once if the
user enters multiple file selectors on the command line. This means
that U)pdate could insert the same file twice, etc.
There is a maximum of 65535 threads per record.
The ProDOS filetype names may or may not be the offical Apple
versions. Filetype support for other operating systems is presently
nil.
Error output is informative but ugly (mostly debugging-type messages).
In the Works
Roughly in order of importance:
- Add ShrinkIt LZW compress/uncompress routines.
- Add Update/Freshen, implement Move, and improve the file integrity
check.
- Improve UNIX version. Need to handle system clock, file date stamp,
and subdirectory expansion.
- Add Binary II file handling.
- Add minor things, like "Y/n/q" prompts when files already exist,
generate real temporary names (currently only "nuarc.tmp" is
returned), do "none selected" stuff (especially in D)elete).
- Cope somehow with APW segmentation.
The above is my goal for version 2.1 (basically, make everything on
the menu function as advertised). Future enhancements include:
- Improve GS/OS handling. Need to handle resource forks and file
system IDs. Would probably help if I had a GS/OS reference...
- Add disk support.
- Add to thread methods (insert ASCII messages, etc).
- Add different compression methods (lharc, UNIX compress).
Closing Notes
I said I wasn't going to write a file extractor. Okay, I changed my
mind. But before it unpacks files, I'm going to need some help with
the compression routines...
If nothing else, this will allow you to check the output of sciibin (a
UNIX BinSCII extractor) before you actually download the files.
In the works (once this gets finished): the NuView CDA. View NuFX
archives from any application (I'll try to include both ProDOS 8 and
GS/OS support).
Author Info
I'm presently a Junior in Electrical Engineering and Computer Science
(EECS) at the University of California, Berkeley.
fadden@cory.berkeley.edu (Andy McFadden)
...!ucbvax!cory!fadden
No relation to the author of ShrinkIt and NuFX, Andy Nicholas. This
was and continues to be my idea; throw all kudos and blame in my
general direction. All code herein is mine, developed from the NuFX
Documentation Final Revision Three (2/3/89), except as noted below.
Dynamic LZW used in ShrinkIt courtesy Kent Dickey.
The Binary II unpack routines and the C unsqueeze code were written by
Marcel J.E. Mol (based on usq2/sq3 by Don Elton).
If you have suggestions or comments, you can send US mail to:
Andy McFadden
1474 Saskatchewan Dr.
Sunnyvale, CA 94087
If you find this program especially useful, send money to the guy who
came up with NuFX and ShrinkIt (this program is my contribution):
Paper Bag Productions CSNET: shrinkit@moravian.edu
c/o Andy Nicholas InterNET: shrinkit%moravian.edu@relay.cs.net
Box 435 AppleLink PE: ShrinkIt
Moravian College
Bethlehem, PA 18018
Legal Stuff
Disclaimers, borrowed from Dave Whitney and Andy Nicholas:
This program is FREEWARE; both source and binaries may be distributed
freely, but not sold. If you wish to include NuARC in the
distribution of a commercial product, you will need to get my
permission beforehand.
Copyright (C) 1989 by Andy McFadden. All rights reserved.
I (Andy McFadden) MAKE NO WARRANTY ON THIS MANUAL OR SOFTWARE, EITHER
EXPRESS OR IMPLIED, WITH RESPECT TO QUALITY, MANUAL'S ACCURACY,
MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
IN NO EVENT WILL I BE HELD RESPONSIBLE FOR DIRECT, INDIRECT, SPECIAL,
INCIDENTAL, OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OF THE
SOFTWARE OR INACCURACY IN THE MANUAL.
In other words, I have tested the program to the best of my ability,
and I find that as distributed by me, it is safe for general use. It
isn't necessarily bug-free, and as a result, loss of data, however
unlikely, is entirely possible. Use at your own risk, but also for
your own enjoyment.
***** END