jeff@tc.fluke.COM (Jeff Stearns) (11/10/88)
Apple has defined a "standard" way of storing Macintosh files on primitive
UNIX-like computers which don't support the concept of resource forks.
It's called AppleDouble, and it cleaves the Macintosh file into two parts
for storage on UNIX. The data fork is stored byte-for-byte in one file,
and the resource fork is stored in another. (The resource fork file is
named "%file".) The %file is always 256 bytes larger than the resource fork
of the Macintosh file istelf.
There's a second standard called AppleSingle, which stores the two forks
in one UNIX file, it apparently includes some header information necessary
to separate the two forks later.
Now the question: What's the exact format of those extra 256 bytes
of an AppleDouble '%' resource file?
And what is the format of an AppleSingle file?
For example:
Imagine a Microsoft Word file named "My Word File". The data fork contains
25088 bytes; the resource fork contains 0 bytes.
If stored on a UNIX filesystem in AppleDouble format, it would be split
into two files; the 25088 bytes of the data fork would be stored in
"My Word File". No surprises there.
Now the 0-byte resource fork is stored into a file named "%My Word File",
but this UNIX file contains 256 bytes of magic stuff. Some bytes are
obvious strings, but there are plenty of inscruitable numbers. What do
those bytes mean? For your amusement, here's a real-life sample:
0000 0 5 16 7 0 1 0 0 4d 61 63 69 6e 74 6f 73 | ........Macintos
0010 68 20 20 20 20 20 20 20 0 3 0 0 0 9 0 0 | h ........
0020 0 80 0 0 0 40 0 0 0 7 0 0 0 c0 0 0 | .....@..........
0030 0 10 0 0 0 2 0 0 1 0 0 0 0 0 0 0 | ................
0040 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
0050 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
0060 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
0070 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
0080 57 44 42 4e 4d 53 57 44 1 0 ff ff 0 7d 0 0 | WDBNMSWD.....}..
0090 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
00a0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
00b0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
00c0 eb 6 65 26 eb b 67 ec 80 0 0 0 0 0 0 0 | ..e&..g.........
00d0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
00e0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
00f0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | ................
--
Jeff Stearns John Fluke Mfg. Co, Inc. (206) 356-5064
jeff@tc.fluke.COM {uw-beaver,microsoft,sun}!fluke!jeff
PS - Calling all users of the Vitalink TransLAN IV Ethernet bridge! Please
drop me a line.zuhn@umn-cs.CS.UMN.EDU (Dave Zuhn) (11/11/88)
In article <5918@fluke.COM> jeff@tc.fluke.COM (Jeff Stearns) writes: >Apple has defined a "standard" way of storing Macintosh files on primitive >UNIX-like computers which don't support the concept of resource forks. > >It's called AppleDouble, and it cleaves the Macintosh file into two parts >for storage on UNIX. The data fork is stored byte-for-byte in one file, >and the resource fork is stored in another. (The resource fork file is >named "%file".) The %file is always 256 bytes larger than the resource fork >of the Macintosh file istelf. > >There's a second standard called AppleSingle, which stores the two forks >in one UNIX file, it apparently includes some header information necessary >to separate the two forks later. > > Now the question: What's the exact format of those extra 256 bytes > of an AppleDouble '%' resource file? > And what is the format of an AppleSingle file? This was saved from the nets some months ago: ------- Cut Here ------------ Below is a document produced by Apple and put forth as a proposed standard way of representing Macintosh and ProDOS (as well as other file systems') files on foreign file systems. What appears below is really still a draft, since the official version will be printed by Apple and distributed by the Apple Programmers and Developers Association. I'm sending it out now on the net to give it perhaps a wider distribution, and to give all of you some advance notice. Some of you may recall that I asked for suggestions some time ago on this topic. I received numerous valuable responses, and a lively discussion took place at Apple for about four months. The result is the document below. Don't be alarmed if your favorite feature is not included, or you disagree with part of the specification. We fully expect that this may evolve over time. Having said that, I must put forth this disclaimer: I am not the author of this document; I merely chaired the meetings that produced it. Furthermore, I am no longer involved with the project. Any technical input should be forwarded through Apple Technical Support. Thanks for all your support in the past. I hope this specification proves useful to you. --------------------------------------------------------------------------- Formats for Apple files in foreign file systems (DRAFT) Apple Computer, Inc. 14 January, 1988 Apple Computer, Inc., is putting forth two standards for representing files on foreign file systems, with the goal of preserving all attributes of the fileUs home file system on file systems that do not support the same attributes. Experience seemed to indicate that a single format would be inadequate to cover all cases. Two closely related formats, however, can serve most needs. Although the primary impetus for developing these formats is storing Macintosh files on file systems that do not support the notion of two forks, the proposed formats are general enough that they can be used to represent a file from any file system on any other file system. The two formats are called AppleSingle and AppleDouble. In AppleSingle format, all contents and attributes of a file are kept in a single file on the foreign file system. For example, both forks of a Macintosh file, the Finder information, any associated icons, and so on are arranged in a single file with a simple structure. This format is intended to be used primarily as a storage format, i.e., for cases in which the Macintosh file must be stored on a foreign file system and later reconstructed into a true Macintosh file. AppleDouble format is more appropriate for applications in which the users of the foreign file system might want to modify the contents of the file. Since most Macintosh applications keep the file data in the data fork, AppleDouble format saves the contents of the data fork in one file. All other file attributes are kept in a separate file. Specifically, these formats do not rule out the possibility of building applications that can access and modify AppleSingle-format files. Nor do they preclude using AppleDouble format for simple storage of Macintosh files. We merely present them as alternatives. The only assumption these formats make is that each file system on which these file formats will be supported allows the creation of a simple file: an uninterpreted stream of bytes. AppleSingle and AppleDouble formats are not directly related to the AppleTalk Filing Protocol (AFP). True, one of their possible uses is as the storage format for non-Macintosh-based AFP servers, but that was not the primary motivation behind their development. If you are building an AFP server (or an NFS or other server that supports Macintosh) then you may wish to use one of the formats as your internal file storage format. However, the choice is yours. It really doesn't matter what format is used within your application, as long as, externally, the files appear as they are supposed to. There are at least two basic reasons for using AppleSingle or AppleDouble formats: 1. As a standard for transferring files between host computers (inter- host). For example, Macintosh files could be easily and completely shipped among heterogeneous systems if they all understand one of these common formats. Any existing e-mail system or file transfer utility could be used without modification. 2. As a standard for operating on foreign files within a single host (intra- host). In the near future, we expect to see, for example, UNIX applications that can build and manipulate Macintosh resource forks (perhaps a cross-development system). If a set of users of a host computer wishes to write Macintosh-aware applications, they need to agree on a common storage format, such as AppleSingle and AppleDouble. The following discussion uses these terms: home file system means the file system for which the fileUs contents were created. For example, a UNIX application could create an AppleSingle file that holds a resource and data fork in which is contained a MacWrite-formatted document. The home file system for such a file is Macintosh, because the file is intended to be compatible with a Macintosh application. In most cases, where a file is created and used on the same file system, the home file system is that system. foreign file system means the other file system which will store or process the file. An AppleSingle or AppleDouble file is usually a representation of a fileUs contents on the foreign file system. AppleSingle format An AppleSingle file contains a header followed by data. The header consists of several fixed fields and a list of entry descriptors, each pointing to an entry. Apple defines these standard entries: Data Fork, Resource Fork, Real Name (name in the home file system), Comment, Icon, and File Info. Each entry is optional and may or may not appear in the file. Header: Magic Number (4 bytes) Version Number (4 bytes) Home File System (16 bytes ASCII encoded) Number of entries (2 bytes) For each entry: Entry ID (4 bytes) Offset (4 bytes) Length (4 bytes) The "Magic Number" field is modeled after the feature in UNIX. It is intended to be used in whatever way the foreign file system distinguishes a file as AppleSingle format. The Magic Number for AppleSingle format is $00051600 or 0x00051600. The "Version Number" field denotes the version of AppleSingle format in case the format evolves (more fields may be added to the header). The version described here is version $00010000 or 0x00010000. The "Home File System" is a fixed-length, 16-byte ASCII string not preceded by a length byte but possibly padded with blanks. Apple has defined these values: Macintosh: 'Macintosh' or $4D616369 $6E746F73 $68202020... ProDOS: 'ProDOS' or $50726F44 $4F532020 $20202020... MS-DOS: 'MS-DOS' or $4D532D44 $4F532020 $20202020... Unix: 'Unix' or $556E6978 $20202020 $20202020... VAX/VMS 'VAX VMS' or $56415820 $564D5320 $20202020... Apple welcomes suggestions for other file systems that should be included in this list. The "Number of entries" field tells how many different entries are included in the file. It is an unsigned 16-bit number, and may be zero. If it is non-zero, then that number of entry descriptors immediately follows this field. For each entry, the entry descriptor indicates just what the entry is, where the entry is located in the file, and how big the entry is. Apple has defined a set of Entry IDs and their values: Data Fork 1 (standard Macintosh data fork) Resource Fork 2 (standard Macintosh resource fork) Real Name 3 (the fileUs name in the home file system) Comment 4 (standard Macintosh comment) Icon, B&W 5 (standard Macintosh black-and-white icon) Icon, Color 6 (reserved for Macintosh color icon) File Info 7 (file information - attributes, etc.) Finder Info 9 (standard Macintosh Finder Info) Apple reserves the range of Entry IDs from 0 to $7FFFFFFF for future use. The rest of the range is available for other systems to define their own entries. Apple will not arbitrate the use of the rest of the range. Icon entries will probably not appear in most files since they are typically stored as a bundle in the application file's resource fork. The File Info entry is different for each home file system. For Macintosh HFS, the entry is 16 bytes long and consists of three long integer dates (Create Date, Last Mod Date, and Last Backup Date) and a long integer containing 32 boolean flags. Using the bit numbering scheme where bit 0 is the least significant bit and 31 is the most significant, bit 0 of the Macintosh Finder Info entry is the Locked bit; bit 1 is the Protected bit. Formats for MS-DOS, Unix, and ProDOS are shown below. Macintosh File Info MS-DOS File Info ProDOS File Info -------------------- -------------------- -------------------- + + + + + + + Create Date + +Modification Date + + Create Date/Time + + + + + + + -------------------- -------------------- -------------------- + + + + + + +Modification Date + + Attributes + + Mod Date/Time + + + + + + + -------------------- -------------------- -------------------- + + + Access + + Last Backup Date + -------------------- + + Unix File Info + File Type + -------------------- -------------------- -------------------- + + + + + + + Attributes + + Create Date/Time + + Aux Type + + + + + + + -------------------- -------------------- -------------------- + + +Last Use Date/Time+ Macintosh Attributes + + (Each small box is -------------------- -------------------- two bytes; large + 0 0 0 0 0 0 0 0 + + + boxes are four. + 0 0 0 0 0 0 0 0 + +Last Mod Date/Time+ This looks much + 0 0 0 0 0 0 0 0 + + + better on a Mac.) + 0 0 0 0 0 0 P L + -------------------- -------------------- P = Protected L = Locked The Finder Info field consists of 16 bytes of Finder Info followed by 16 bytes of extended Finder Info (the fields ioFlFndrInfo followed by ioFlXFndrInfo, as returned by the Macintosh PBGetCatInfo call described in Inside Macintosh, page IV-155). The internal structures (subfields) of ioFlFndrInfo and ioFlXFndrInfo are described in Inside Macintosh, page IV-183. Newly created files have zeroes in all Finder Info subfields. If you are creating an AppleSingle or AppleDouble file whose Home File System is Macintosh, you may zero any subfield whose value is unknown (indeed, most subfields are undefined if the file does not reside on a valid HFS volume), but you may want to set the fdType and fdCreator subfields. The actual data representing the entry must be in a single contiguous block. It is pointed to by the offset field, which is an unsigned 32-bit number indicating the byte offset from the start of the file to the start of the entry. The entry length is also an unsigned 32-bit number representing the length in bytes. The length may be zero. After some number of entry descriptors, the actual entry data appears. The entries could appear in any order, but since the data fork is the entry that is most commonly extended, Apple strongly recommends that the data fork entry always be kept last in the file to facilitate its extension. Apple also recommends that those entries that will most often need to be read, such as Finder Info, Real Name, and Dates, be kept as close as possible to the header to maximize the probability that a read of the first block or two of the file will retrieve these entries. It is possible to have holes in the file (unused space between entries). To find where the holes are, you must take the list of entry descriptors and sort them into increasing offset order. If the offset field of an entry is greater that the offset plus length of the previous entry, then a hole exists between the entries. You can make use of such holes; for example, if a file's comment is 10 bytes long, you could create a hole of 190 bytes after the comment field to easily allow for the comment to later expand to its maximum length of 200 bytes. Because an AppleSingle file may contain holes, you must find each entry by getting its offset from its entry descriptor, not by assuming that it begins after the previous entry. Byte ordering in the file header fields will follow 68000 and 68020 conventions. AppleDouble format AppleDouble format is the same as AppleSingle format, except that the data fork is kept in a separate foreign file. The file containing the data fork is called the AppleDouble Data File, and the other file is called the AppleDouble Header File. The AppleDouble Data File consists of just the standard Macintosh data fork, with no extra header at all. The AppleDouble Header File has exactly the same format as the AppleSingle file, except that it does not contain a data fork entry. The Magic Number of an AppleDouble Header File differs from that of an AppleSingle file so an application can tell whether or not it needs to look elsewhere for the data fork. The Magic Number for AppleDouble format is $00051607 or 0x00051607. The entries in the Header File could appear in any order, but since the resource fork (in this case) is the entry that is most commonly extended, Apple strongly recommends that the resource fork entry always be kept last in the file. The data fork is easily extended, because it resides by itself in the AppleDouble Data File. If it is possible on the foreign file system, one could create a new type of entry that "pointed" to the AppleDouble Data File to make it easy to find. Filename conventions AppleSingle format specifically does not include an algorithm for generating the AppleSingle filename from the fileUs real name. The foreign file systems of interest differ quite a bit in filename syntax, and the fileUs real name can be kept as an entry within the AppleSingle file. The same is generally true for AppleDouble Data File names. However, Apple is proposing a standard for deriving the AppleDouble Data File and AppleDouble Header File names from the fileUs real name. Because filename syntax differs in the various file systems, the proposed standard varies by file system: Unix: To generate the AppleDouble Data File name, use character substitution to replace any illegal characters with an underscore (_). Since different Unix systems have different requirements on maximum file name length, do not explicitly truncate the name to a specific length. Instead allow the truncation to be done by the Unix functions create(), open(), etc. To generate the AppleDouble Header File name, A/UX will prefix a single percent sign (%) to the AppleDouble Data File name. If necessary truncate the last character to keep the filename within the legal length range. Other Unix systems may prefix a directory name (AppleDouble/) to the AppleDouble Data File name to create the name of the AppleDouble Header File. In this scheme, all AppleDouble Header Files corresponding to AppleDouble Data files are kept together in a single subdirectory. ProDOS: To generate the AppleDouble Data File name, use character substitution or deletion to remove illegal characters, and use truncation if necessary to reduce the length of the name to two characters less than the maximum filename length. To generate the AppleDouble Header File name, prefix the AppleDouble Data File name with the characters uppercase-R period (R.). MS-DOS: To generate the AppleDouble Data File name, use character substitution or deletion to remove illegal characters, and use truncation if necessary to reduce the length of the name to eight characters. Then add the MS- DOS extension that is most appropriate to the file (e.g., '.TXT' for a pure text file). To generate the AppleDouble Header File name, add the extension '.ADF' (for AppleDouble File) to the eight-character filename. AppleDouble name derivations will be defined for all other file systems of interest. This will allow applications running on the foreign file system (and human users as well) to see easily which files are AppleDouble pairs. Knowledgeable users, if they know the derivation, could rename or move the files so as to preserve the connection between the two. However, there is no guaranteed way to prevent one file of the pair from being inconsistently renamed, moved, or deleted. --------------------- Cut Here ------------------ Hope this helped....I knew there was some reason I saved this file. David D Zuhn // Computer Science System Consultant // University Of Minnesota zuhn@umn-cs.cs.umn.edu zuhn@umn-cs.UUCP ...rutgers!umn-cs!zuhn I speak for myself only, I'm only 1322581 to the University.