cloos@acsu.buffalo.edu (James H. Cloos) (11/15/90)
Yup, you guessed it; I spent the last few hours pouring over hex dumps of libraries, both created from directories by usrlib, and some from the bbs. Anyway, this is what I found out: (NB, each of these fields is individually reversed in memory; unlike the dir posting, I'll write these in readable format.) A library looks like this: #02b40 LIB header #nnnnn size of the library, incl this field. #nn the length of the lib's name, in chars (If the lib is unnamed, this is #00 and the next two fields are missing) asciiascii the name of the library #nn the length of the lib's name, again (Now back to always there mode :) #nnn the LIB's id number #nnnnn offset to hash table \ #nnnnn offset to message arry | offsets are from the start of #nnnnn offset to link table | the offset field #nnnnn ofset to config object / the data follows. Named xlibs are stored in the format: #000 or #8 000=ok in alg expressions, 8= not ok in algebraics #nnn lib's id number from above #nnn the xlib's member number the actual object, starting w/ a std type field. (You'll find that lib #002 (the usrlang commands) is stored this way also.) Unnamed xlib's are stored w/o the #000nnnmmm or #8nnnmmm header, so you just start w/ the type field. The config object is what is evaluated after each system halt, and is usually a program. To borrow from the usrlib doc file, make sure this DOES NOT LEAVE ANYTHING ON THE STACK OR DROP ANYTHING THAT WAS THERE. My assumption is that only a ram clear will fix the 48 if there is a lib installed that has a bad config obj, as the post-system-halt routines will probably crash each time you try one. The link table is a hex string (a la a binary integer, but with a max wordsize of 2^5 - 6 nybbles) (proof of that is left as an exersise for the reader). This hex string is 5 nybbles long per XLIB in the library, and each group of 5 nyb's forms an offset from the table to the beginning of the type field of each xlib. These offsets are signed, so if this table is after the actual XLIB's, the offsets will be the complement (as per the NEG key on a binary integer--I'm too tired to figure out if this is 1's or 2's ;) of the actual distance from the field to the XLIB. These are the only signed offsets I'm aware of so far, though maybe all offsets are, check your local saturn guru for details. ;) The message array is a simple array of strings. (Which have the format: #029e8 type of ARRY #nnnnn size of array, incl this field #02a2c type of data in general, #2a2c is string type #nnnnn number of dimentions #nnnnn number of elements in 1st dimention ... #nnnnn '' '' '' '' nth '' data Where data is sum(i=1,no_of_dims,no_of_elem_in_this_dim) elements of the type specified. In the case of strings, each of these looks like a 5 nybble size field (in nyb's incl itself) plus the ascii chars in the string.) This leaves the hash table. This is also a hex string. At the end of this hex string are 5 nyb pointers for each named XLIB which point to that XLIB's name. These names immediately precede the pointers. These offsets, though pointing backwards, are positive (or unsigned?), ie., #00020 would point 32 nybbles back from the start of itself. The names these point to are in the familiar global name format, ie a 2 nybble size field (in char's, excluding itself) followed by that many bytes of ascii (each of course in reverse nybble order). This is followed by a 3 nybble field of that XLIB's member number. Before this is a 5 nyb field that is the length, in nybbles, of itself, & each of the XLIB names (inlc that name's length field and number field). Before all this, in each of the lib's I've looked at, is 85 nybbles of mostly 0's. They seem to be 16 fields of 5 nybbles each (resonable thought) and there would seem to be on the order of 3 or so of these fields that contain any non zero data, this is usually only 2 nybbles. I presume they are some sort of offset, but to where (or from where for that matter) I've no idea. The CONFIG, MESSAGE, LINK, HASH, and each individual XLIB can be, so far as I can tell, in any order, so long as each is itself contiguous ;^) I would presume, however, that there is some performance penalty if they are all mixed around, as minor as that might be (though if you use a XLIB in such a library in a type loop, these penalties could amount to a noticable delay). (My compiler will use a standard order which is the easiest for me to put together. ;) I better leave now, before I go over 900 words; I've already passed 100 lines. Enjoy this info. & like the post on dir's, if you don't understand it, ask & someone (maybe even me) will try to explain it better. This *is* fun. Trust me. Really. Would *I* lie to you? Seriously. As much as this looks sarcastic, it isn't. I promise. BTW, the library: "04B20E10000066600000000000000000000577E" (in ASC format) is nothing but an unnamed lib, lib# = 666h = 1638d, w/o anything in it. ASC\-> will work on the string, resulting in : `Library 1638: ` on the stack (w/o the `'s of course). I'm not going to try installing it, as I don't feel like a mem lost just now. Could someone out there who has a handy archive try this and report on it? L8r, ffolks. -JimC -- James H. Cloos, Jr. Phone: +1 716 673-1250 cloos@ACSU.Buffalo.EDU Snail: PersonalZipCode: 14048-0772, USA cloos@ub.UUCP Quote: <> THIS INFORMATION COMES WITHOUT ANY FORM OF WARRENTY, NOT EVEN IMPLIED ONES.
David_Michael_Kaffine@cup.portal.com (11/16/90)
Please bear with me - I've rarely posted my own articles before, and I'm using an unfamiliar system. In a recent article (see - I don't know how to do followups correctly yet), Jim Cloos described the structure of the Library object on the HP-48. He left a couple of question marks in the description, which I will try to elaborate here. The unknown fields were all a part of the hash table. Here's my current understanding of the hash table structure: #02a4e Prolog for binary integer #length Length of binary from this field to end #ptr1 5-nibble offset (from start of this field) to first 1 character name (#00000 if none) #ptr2 Pointer to first 2 character name ... #ptr16 Pointer to first 16 character name #decompile 5-nibble offset (from start of this field) to #xlib0 below. This tells the 48 where the pointers are for it to decompile a name from an xlib object 1-char-name-data This is as Jim described - each name is 2-char-name-data entered here, in no particular order, except 3-char-name-data that names of the same size are grouped 4-char-name-data together. Each name is a two nibble size, 5-char-name-data the ascii data, then the XLIB number. ... 16-char-name-data #xlib0 5 nibble offset, which as Jim mentioned, is positive to point backwards (unlike most of the other offsets), This points to the size nibbles of the name for XLIB object #0 ... #xlibn Pointer for the last named XLIB object's name. That's it for the hash table. One other thing that I think Jim left out - the library object has 4 CRC nibbles appended to it. These CRC nibbles are calculated using the Kermit CRC algorithm, but the CRC is calculated starting with the length field, not the #02b40 field, and includes everything up to (but, for obvious reasons, not including 8-) the CRC itself. The length field of the library includes the 4 CRC nibbles. WARNING: Be careful when playing around with library objects with incorrect CRCs. I did this several months ago, and started to notice that I had less memory than expected. It turns out that if you install a library with a bad CRC in port 0, at the next system halt, the bad CRC is detected, and the bad library object is 'pushed' aside so that all the good objects in port 0 are contiguous. But, the bad library still takes up memory!! I ended up doing a lot of peeking and poking to reclaim my memory again. I didn't try it, so I'm not even sure a memory clear would work. This warning applies to the simple library that Jim posted, since it doesn't have a CRC field (I'm not talking about the CRC needed for the ASC-> program - the library object itself has an embedded CRC). I don't know for sure since I'm not risking any memory lost right now - oh, what the hell, I just tried it, and sure enough, the library doesn't install correctly. Instead, when the bad CRC is detected, you get the message Invalid Card Data, and the behavior I just described. One final note - I don't believe there would be any performance penalty for ordering the various components of a Library object at random - they are always going to be accessed via pointers. Again, I apologize if the preceding post is all messy. It will take me some time to get used to this new system (I'm using a $#@!% line editor right now). But, I'll never learn it if I don't try it. Please direct replies to the address above, or Dave.Kaffine@MicroProd.NCR.com
David_Michael_Kaffine@cup.portal.com (11/16/90)
(I'm still trying to get used to posting, bear with my mistakes) Just one more note on library objects. I made a small mistake in my previous post. When you install a library with an incorrect CRC, it does not get moved so that all of the 'good' objects in that port are contiguous, as I stated before. Instead, the object stays still, and the prolog (in this case, #02b40 for a library) is overwritten with 0's. This 00000 is also what's used to mark the end of objects in a port (i.e Port 0, at least, is just a sequence of RPL objects, terminated by 00000). Thus, the bad object is now hidden, ALL objects that were after it in port memory are hidden, and you're suddenly missing a fair chunk of memory. The only reasonable way of getting the memory back is to archive your user data, do an ON-A-F, and restore your user data. I apologize for the mistake - my memory on this was a few months old, and wasn't refreshed until I installed Jim Cloos' sample library and saw the same things over again. To summarize: If you store the library object from Jim Cloos' article into port 0 and press ON-C (or turn calc. off and on), you will get the message "Invalid Card Data". The library will no longer show up in the port 0 listing. Anything that was previously in the port 0 listing will no longer show up. All of the objects that have disappeared are still taking up memory. A memory-clear will reclaim the lost memory (not to mention your user variables!). Dave.Kaffine@MicroProd.NCR.com or David_Michael_Kaffine@cup.portal.com
akcs.dnickel@hpcvbbs.UUCP (Derek Scott Nickel) (11/17/90)
Jim,
Requarding those unknown nibbles. You'll notice that the name-table part
of the hash-table is sorted by command name length, and then
alphabetically. These 5 nibble fields are offsets into the name table
for a specific name length. I.e., the first slot contains the offset to
one character names (or zero if no one character names), the second slot
contains the offset to the first two character name, the third is an
offset the first three character name... and so forth.
Thanks for the info on the unsigned pointers at the end of the hash
(pointing back to the individual names). That had ne stumped. I knew
they had to be related to the commands themselfs, but never guessed that
they were backward pointing.
Derek S. Nickel