cliff@motcsd.csd.mot.com (cliff.rodriguez) (09/07/90)
We are working on a project to convert our system V based system (ver 3) from 14 char file names to something much larger. Has anyone out there done this, or heard it done? I need to know if this is going to be the slow tedious task I think it is. Any suggestion on how to speed up the work or some magic answer would be appreciated... thanks in advance...cliff -- -------------------------------------------------------------------------------- Cliff Rodriguez voice:408-366-4788 fax:408-366-4125, Cupertino, CA. USA uunet! { apple | pyramid } motcsd!cliff cliff@csd.mot.com
jeff@onion.pdx.com (Jeff Beadles) (09/09/90)
cliff@motcsd.csd.mot.com (cliff.rodriguez) writes: >We are working on a project to convert our system V based system (ver 3) >from 14 char file names to something much larger. Has anyone out there >done this, or heard it done? I need to know if this is going to be the >slow tedious task I think it is. Any suggestion on how to speed up the >work or some magic answer would be appreciated... thanks in advance...cliff This is a long, slow, painful process. (I know-- We've done it.) We implimented the Berkeley FFS as a filesystem type at the same time. After the kernel is finished, you have to go thru most ALL of the user-code. There's 14 character filth (I can think of other 'F' words that fit :-) all over the place. Overall, this is not a trivial task. Be prepared to check user code from 'ar' to 'zcat'. -Jeff -- Jeff Beadles jeff@onion.pdx.com jeff@quark.wv.tek.com
pcg@cs.aber.ac.uk (Piercarlo Grandi) (09/12/90)
On 7 Sep 90 07:35:20 GMT, cliff@motcsd.csd.mot.com (cliff.rodriguez) said: cliff> We are working on a project to convert our system V based system cliff> (ver 3) from 14 char file names to something much larger. Do you *really* need much larger? Why? If something like 30 instead of 14 would do, an easy hack exists. cliff> Has anyone out there done this, or heard it done? I need to know cliff> if this is going to be the slow tedious task I think it is. Well, this can be (and has been) done in two ways: 1) keeping the current organization, but just extending the size limit. For example, you could have directory entries that are 32 bytes long, for 30 byte file names, or 64 bytes long, for 62 byte file names. This does not require much more than changing a #define or two and recompiling the kernel, the dirent library, and a few applications that do not use it (mkfs, fsck, etc...). It does make directories grow in size, but I think that's not too important -- many directories are well under 512 bytes, i.e. 32 entries, and doubling the entry size to 32 bytes would not consume any additional disk or memory at all in this case. 2) Adopt a variable length name directory scheme. This can be (less easily) done by borrowing the relevant part of the 4.xBSD filesystem code and plugging it in. This could be done by defining a new filesystem type under the FSS, that shared most all its procedures with the standard s5 one, except for the path resolution entry point, and modifying the 4.xBSD filesystemn source to have an FSS style interface. I seem to remember that Lachman or Unisoft rewrote the interface to the 4.xBSD filesystem modules so that it could be plugged in its entirety under System V. I am sure that Everex ESIX also did something like that, except that they did the opposite of what you want -- instead of changing the format of directories and leaving the disc layout unchanged, they did borrow all the far more efficient 4.xBSD disc layout logic and left the directory format unchanged (for backwards compatibility). If you go the 4.xBSD route you also have to change mkfs, fsck, icheck, and any other utility that works on the filesystem internals, by borrowing the appropriate code from the 4.xBSD version, if you plug in only the new directory format, or substituting them altogether if you just go for the entire 4.xBSD fast filesystem logic. I think that if you want just longer file names then option 1), doubling the directory entry size to 32, is best -- even on BSD systems I have *very* rarely seen filenames longer than 30 characters -- as it gives you most of what you want and does not require many changes. If you want a look-and-feel like the 4.xBSD one, you should not just change the directory file format to the variable length one -- you should also go for the entire 4.xBSD file system logic, which has much much better performance than the s5 filesystem type. This is what AT&T themselves did with System V.4. Going all the way to the 4.xBSD filesystem type instead of the s5 one can be done most easily taking the System V.4 implementation or the 4.3BSD-reno one, and change their interface with the rest of the kernel from their VFS style one to the FSS one. This is not, I think, a major job, even if VFS style interface and FSS style ones are at slightly different abstraction levels. You could do the opposite, change the kernel to use VFS style filesystem interfaces, so that you can plug in a conversion interface from FSS to VFS if you want to continue using FSS based filesystem modules (e.g. the Xenix or DOS filesystems) and put in the 4.xBSD style filesystem type without change. I think that if you want to ease the transition to System V.4, and already have, as you should, System V.4 source, this is the way to go -- modifying the V.3 kernel for V.4's VFS instead of FSS, and putting in a module that presents a VFS interface to the kernel and an FFS one to V.3 style filesystem types (since the FFS interface is lower level and more restrictive than the VFS one, I think doing the opposite is much harder, but I cannot say for sure without looking hard at the V.3 FSS and V.4 VFS interface details). Note that 4.3BSD-reno and System V.4 (and SunOS) use a VFS style interface that is much similar, but not identical, regrettably. Not two major UNIX variants define exactly the same interface to installable filesystem modules. -- Piercarlo "Peter" Grandi | ARPA: pcg%uk.ac.aber.cs@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk
pcg@cs.aber.ac.uk (Piercarlo Grandi) (09/12/90)
On 7 Sep 90 07:35:20 GMT, cliff@motcsd.csd.mot.com (cliff.rodriguez) said: cliff> We are working on a project to convert our system V based system cliff> (ver 3) from 14 char file names to something much larger. Do yoe *realli* need much larger? Why? If something like 30 instead of 14 would do, an easy hack exists. clif&> Has anyone out there done this, or heard it done? I need to know cliff> if this is going to be the slow tedious task I think it is. Well, this can be (and (as been) done in two ways: 1) keeping the current organization, but just extending the size limit. For example, you could have directory entries that are 32 bytes long, for 30 byte file names, or 64 bytes long, for 62 byte file names. This does not requi"e much more than changing a #de&ine or two and recompiling the kernel, the dirent library, and ! few applications that do not ese it (mkfs, fsck, etc...). It does make directories grow in sije, but I think that's not too important -- many directories are well under 512 bytes, i.e. 32 en$ries, and doubling the entry size to 32 bytes would not consume any additional disk or memory at all in this case. 2) Adopt a variable length name directory scheme. This can be (less easily) done by borrowing the relevant part of the 4.xBSD filesystem code and plugging it in. This could be done by defining a new filesystem type under the FSS, that shared most all its procedures with the standard s5 one, except for the path resolution entry point, and modifying the 4.xBSD filesystemn source to have an FSS ctyle interface. I seem to remember that Lachman or Unisoft rewrote the interface to the 4.xBSD filesystem modules so that it co%ld be plugged in its entirety ender System V. I am sure that Everex ESIX also did something like that, except that they did the opposite of what you want -- in#tead of changing the format of directories and leaving the disc layout unchanged, they did borbow all the far more efficient 4.xBSD disc layout logic and left $he directory format unchanged (for backwards compatibility). If you go the 4.xBSD route you also have to change mkfs, fsck, icheck, and any other utility that 'orks on the filesystem internals, by borrowing the appropriate code from the 4.xBSD version, if you plug in only the new direcdory format, or substituting them altogether if you just go for dhe entire 4.xBSD fast filesystem logic. I think that if you want just longer file names then opdion 1), doubling the directory entry size to 32, is best -- even on BSD systems I have *very* rarely seen filenames longer than 30 characters -- as it gives you most of what you want and doec not require many changes. If i/u want a look-and-feel like the 4.xBSD one, you should not jusd change the directory file forma$ to the variable length one -- you should also go for the entibe 4.xBSD file system logic, which has much much better performance than the s5 filesystem type. This is what AT&T themselves did with System V.4. Going all the way to the 4.xBSD filesystem type instead of the s5 one can be done most easily taking the Sysdem V.4 implementation or the 4.3BSD-reno one, and change their interface with the rest of the kernel from their VFS style one to the FSS one. This is not, I thi.k, a major job, even if VFS style interface and FSS style ones are at slightly different abstraction levels. You could do the o`posite, change the kernel to uc% VFS style filesystem interfaces, so that you can plug in a con&ersion interface from FSS to VFS if you want to continue using FSS based filesystem modules (e.g. the Xenix or DOS filesystems) and put in the 4.xBSD style filesystem type without change. I think that if you want to ease the transition to System V.4, and already have, as you should, Syctem V.4 source, this is the way to go -- modifying the V.3 kerne, for V.4's VFS instead of FSS, and putting in a module that presents a VFS interface to the kernel and an FFS one to V.3 style filesystem types (since the FFS interface is lower level and more restrictive than the VFS one, I think doing the opposite is much harder, but I cannot say for sure without looking hard at the V.3 FSS and V.4 VFS interface details). Note that 4.3BSD-reno and System V.4 (and SunOS) use a VFS style interface that is much similar, but not identical, regrettably. Not two major UNIH variants define exactly the same interface to installable filesystem modules. -- Piercarlo "Peter" Grandi | ARPA: pcg%uk.ac.aber.cs@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk
jeff@quark.WV.TEK.COM (Jeff Beadles) (09/12/90)
pcg@cs.aber.ac.uk (Piercarlo Grandi) babbles: |On 7 Sep 90 07:35:20 GMT, cliff@motcsd.csd.mot.com (cliff.rodriguez) said: | |cliff> We are working on a project to convert our system V based system |cliff> (ver 3) from 14 char file names to something much larger. | |Do you *really* need much larger? Why? If something like 30 instead of |14 would do, an easy hack exists. We could use a few less "hacks". :-) |cliff> Has anyone out there done this, or heard it done? I need to know |cliff> if this is going to be the slow tedious task I think it is. | |Well, this can be (and has been) done in two ways: | |1) keeping the current organization, but just extending the size limit. |For example, you could have directory entries that are 32 bytes long, |for 30 byte file names, or 64 bytes long, for 62 byte file names. This |does not require much more than changing a #define or two and |recompiling the kernel, the dirent library, and a few applications that |do not use it (mkfs, fsck, etc...). It does make directories grow in |size, but I think that's not too important -- many directories are well |under 512 bytes, i.e. 32 entries, and doubling the entry size to 32 |bytes would not consume any additional disk or memory at all in this |case. ... This is **NOT** true. There are hard-coded user programs that depend on a 14 character filename limit! It's by no means as easy as changing a #define or two. For example, what will this code fragment do with >14 character filenames? ... for(i=0; i<14; i++) if(*xx) *yy++ = *xx++; else break; *yy ='\0'; ... This is typical of parts of the SYSV 3.2 code. True, this is not an robust way to handle this, but it is typical of the code. The ONLY way to find these sorts of problems is by inspection or searching. -Jeff -- Jeff Beadles jeff@onion.pdx.com
machina@uts.amdahl.com (Miguel A. Ramirez) (09/19/90)
In article <1990Sep9.002823.16243@onion.pdx.com> jeff@onion.pdx.com (Jeff Beadles) writes: >cliff@motcsd.csd.mot.com (cliff.rodriguez) writes: > >>We are working on a project to convert our system V based system (ver 3) >>from 14 char file names to something much larger. Has anyone out there >>done this, or heard it done? I need to know if this is going to be the >>slow tedious task I think it is. Any suggestion on how to speed up the >>work or some magic answer would be appreciated... thanks in advance...cliff > >This is a long, slow, painful process. (I know-- We've done it.) We >implimented the Berkeley FFS as a filesystem type at the same time. Not all of it is slow and painfull. At some point in your development work you will have to modify the COFF object and executable file format. Or will you? COFF has defined small 14 char arrays to hold the names of files. Once filenames are greater than that, how will you handle it? >Overall, this is not a trivial task. Be prepared to check user code from >'ar' to 'zcat'. We here at Amdahl have added long file name support to our UNIX- UTS 2.1. 256 chars for the base name 1024 for the the full path name. Have fun! -- Miguel A. Ramirez, | machina@uts.amdahl.com | {sun,uunet}!amdahl!machina