pdccs@unix.cis.pitt.edu (Patrick D. Champion) (03/05/90)
I am pretty much of a novice with OS/2 still. So, my question is: I have just finished reading the OS/2 column in the March edition of Byte. I had thought that HPFS was supposed to have a 30 to 400 percent improvement in throughput over DOS FAT. Yet, the writer often had HPFS comming out slower than DOS FAT. In two cases HPFS came out faster. One case was negligibly faster, the other (random read/write) was 35 percent faster. What's going on here? Why is HPFS performing so slowly? Patrick Champion
olender@cs.colostate.edu (Kurt Olender) (03/05/90)
I take it you are referring to the article by Mark Minasi "OS/2 Notebook: To HPFS or not to HPFS". Read the other article in that issue on OS/2, "OS/2 1.2: A zaftig system" by Martin Heller. He reports significant speed increases with HPFS, anywhere from 30 to 400% faster, although he had to do some tuning, increasing buffer space, etc. I can't say which one is right. I haven't gotten my copy of v1.2 yet. Since Minasi states his were "quick and dirty" benchmarks, it is possible that he ran his tests on HPFS as it comes by default out of the box. Heller reported that the buffer was far too small in that case. That might explain things.
gordonl@microsoft.UUCP (Gordon LETWIN) (03/08/90)
In article <22705@unix.cis.pitt.edu>, pdccs@unix.cis.pitt.edu (Patrick D. Champion) writes: > > I have just finished reading the OS/2 column in the March edition of Byte. > I had thought that HPFS was supposed to have a 30 to 400 percent improvement > in throughput over DOS FAT. Yet, the writer often had HPFS comming out slower > than DOS FAT. In two cases HPFS came out faster. One case was negligibly > faster, the other (random read/write) was 35 percent faster. > What's going on here? Why is HPFS performing so slowly? There are several factors that effect HPFS performance. First and foremost is the buffer size. HPFS is designed to use a larger buffer pool then the FAT file system since buffer RAM is much more available when we're not constrained to a 640K address space. HPFS buffers are 2K each; this improves performance when buffering data and directories, but it also means a poorer buffer hit ratio compared to FAT which has 512 byte buffers. For a "good sized" HPFS buffer pool - say 256K, the difference is minor, but for the default buffer pool of (32K? 64K?) HPFS is impacted some. In other words, the default buffer size for HPFS is smaller than optimal. Secondly, HPFS is an installable file system whereas the FAT file system is built into OS/2. HPFS is much faster than FAT, but it takes longer for a DOSREAD call to get to the HPFS code then it does to the FAT code. As a result, the more calls you make for smaller and smaller requests, the more that HPFS's speed is masked by the OS/2 overhead. Issuing "FindNext" calls for a single name rather than several names, issuing reads for 10 bytes, etc., - these things won't be any faster with HPFS. Naturally, future releases of OS/2 will have their IFS performance improved to aleviate this. Finally, and most importantly, HPFS gains a lot of it's performance for "small, short" operations via it's lazy write mechanism. The author of the byte article may have failed to install the lazy write handler, thus hamstringing HPFS. In that same BYTE issue, towards the front of the magazine, there's a quickie review of OS/2 1.2; The author explicitly mentions setting his buffer size higher and reports major speed improvements for HPFS. I ran the benchmarks myself when HPFS was being developed and - at least, if properly configured - HPFS has significantly improved performance in most areas. But as I've said, OS/2 overhead masks improvements for small and quick operations. Another factor to consider are the reduced limitations of HPFS. HPFS is "this much" faster than FAT to open a file in a directory with 10 files, but HPFS is many many times faster opening a file in a directory with 1000 or 10000 files. Ditto random access to large files - HPFS can be orders of magnititude faster. Ditto heavy I/O systems that want to dedicate 2 megs of buffers to the disk. Sure, you say that nobody does these things. Thats right, they don't, because until now they couldn't. HPFS gives you the capability to do these things which - for some programs and some users - can be very handy. Humans will rarely want to manage 1000 files in a directory, but my mail manager program, for example, finds it convenient to have 10000 files named "1" to "10000". When he's told to display a message he finds its name in a data base file and tells the system to open file "5267". An important goal for HPFS was to remove the performance-mediated restrictions on file, directory, and disk sizes. Gordon Letwin Microsoft related to it's monitary cost, not the amount of precious "640K" space that it uses up. each::
gordonl@microsoft.UUCP (Gordon LETWIN) (03/08/90)
One more point to make about the relative performance of HPFS or any other file system: remember that it's the speed of the *filesystem* which is described as 30% to 400% faster, not especially the speed of the application. This is a pretty obvious point, but people often overlook it. Obviously, a real life application spends a lot of time doing stuff besides I/O. Even when it appears to be "I/O bound" it's probably doing something to process or prepare the data that it's reading or writing. Even using a copy program may introduce additional extra overhead - perhaps the program transfers the data within it's memory buffers, etc. When we benchmarked file systems we used special programs that just did file system operations. There were significant gains with real programs like RBASE and C-compilers, etc., but not 400%! As I recall, the "30%" number comes from comparing HPFS and FAT simple file operations like read and write of moderate sized transfers. It's hard to be much better than FAT here for moderate sized sequential I/O because both systems just write the data out to the disk in a contiguous hunk, there's not much to be improved. HPFS works hard to keep files contiguous and is very successful at it. FAT tends to keep files contiguous when there is only one file being written at one time because of the FAT chain. Also, the large allocation unit that FAT needs for large disks is a waste of space, but it's a win on performance because it guarantees local contiguity. HPFS allocates on a sector basis but still maintains a high degree of contiguity even when multiple files are being simultaneously written. The "400%" number comes from stuff like creating and deleting files - like the time it takes to create (say) three interpass files for a compiler or something. HPFS does particularly well at this, and FAT does particularly poorly, thence a 4 to 1 performance difference. None of our claimed results compare stuff like the relative performance of creating the 10,001th file because of course FAT would be massacred. There's no point in generating outrageous but silly numbers to throw around; we just say that it's "impractical" for FAT but can be done with undiminished performance with HPFS. Gordon Letwin Microsoft
mlegge@datlog.co.uk ( Martyn Legge ) (03/09/90)
Has anyone done benchmarks of "popular" applications or Simple OS/2 copy commands of HPFS on OS/2 Version 1.2? It seems that the "regular" HPFS (i.e NOT HPFS386) is almost twice as slow as FAT for the initial WRITE of files created by existing programs. Subsequent re-writes of these files are faster than FAT. Just the first time you create and write the file, HPFS is slower than FAT. I am looking for independent benchmarks to examine the truth of this. --- Martyn Legge
feustel@well.sf.ca.us (David Alan Feustel) (03/11/90)
A major problem that I have with switching to HPFS is that if OS/2
can't boot then I can't access the files within HPFS. I would gladly
switch to HPFS if I could boot OS/2 from a floppy and access the HPFS
files. Alternatively, a dos driver that could operate on HPFS would do
the trick. Is either option going to become available?
Also, I just upgraded to the Jan 90 version of 1.2. Is it reasonable
to expect 1.1 hard disk drivers to work with 1.2? My 1.1 Bernoulli box
device driver appears not to, but I may have configured it improperly.
--
Phone: (home) 219-482-9631
E-mail: feustel@well.sf.ca.us {ucbvax,apple,hplabs,pacbell}!well!feustel
USMAIL: Dave Feustel, 1930 Curdes Ave, Fort Wayne, IN 46805jack@csccat.UUCP (Jack Hudler) (03/13/90)
In article <16603@well.sf.ca.us> feustel@well.sf.ca.us (David Alan Feustel) writes: > >A major problem that I have with switching to HPFS is that if OS/2 >can't boot then I can't access the files within HPFS. I would gladly >switch to HPFS if I could boot OS/2 from a floppy and access the HPFS >files. Alternatively, a dos driver that could operate on HPFS would do >the trick. Is either option going to become available? I have no problems booting OS/2 from a floppy and reading the files on my HPFS system. If you have problems then the file system need's to be fsck'ed oops.. I mean chkdsk run on it, just place the disk 2 of the OS/2 installation in the drive and run chkdsk /f. Now you will be able to read it. As for the DOS driver... I suppose.. stranger things have happened. -- Jack Computer Support Corportion Dallas,Texas Hudler UUCP: {texsun,texbell}!csccat!jack