[comp.protocols.nfs] Speeding up PC-NFS

joes@islsun3.mse.lehigh.edu (Joe Sieczkowski) (08/24/89)

I recently moved my PC-NFS server from a fairly loaded 8Meg 3/260 to
an unloaded 32Meg 4/280 (new).  I expected a large performance
increase; however over-all performance on a mounted drive only
increased by a couple percent.  This test was done on a PS2/50 with
a 3c523 board.  

Is there anything I can do to increase the performance?  The 4/280
is intended to be a dedicated server for several applications.

I would appreciate any help.

Joe

geoff@hinode.East.Sun.COM (Geoff Arnold @ Sun BOS - R.H. coast near the top) (08/24/89)

In article <JOES.89Aug23230444@islsun3.mse.lehigh.edu> joes@islsun3.mse.lehigh.edu (Joe Sieczkowski) writes:
::
::I recently moved my PC-NFS server from a fairly loaded 8Meg 3/260 to
::an unloaded 32Meg 4/280 (new).  I expected a large performance
::increase; however over-all performance on a mounted drive only
::increased by a couple percent.  This test was done on a PS2/50 with
::a 3c523 board.  
::
::Is there anything I can do to increase the performance?  The 4/280
::is intended to be a dedicated server for several applications.
::
::I would appreciate any help.
::
::Joe

I assume you're running 3.0.1 with the improved cache code. You are?
Then the gating factor is the write access to the disk on the
server. The best help here is the new Legato accelerator board.
See comp.newprod or Unix Review for info. (Bob: d'you want to
post anything here about it?)


Geoff Arnold,                              Internet: geoff@East.Sun.COM
PCDS Group, Sun Microsystems Inc.
---------------------------------------------------------------------------
Recording of the week: ALMOST ACOUSTIC by the Jerry Garcia Acoustic Band.

blyon@legato (Bob Lyon) (09/08/89)

In article 756@east.East.Sun.COM, Geoff Arnold says:

] In article <JOES.89Aug23230444@islsun3.mse.lehigh.edu> joes@islsun3.mse.lehigh.edu (Joe Sieczkowski) writes:
] ::
] ::I recently moved my PC-NFS server from a fairly loaded 8Meg 3/260 to
] ::an unloaded 32Meg 4/280 (new).  I expected a large performance
] ::increase; however over-all performance on a mounted drive only
] ::increased by a couple percent.  This test was done on a PS2/50 with
] ::a 3c523 board.  
] ::
] ::Is there anything I can do to increase the performance?  The 4/280
] ::is intended to be a dedicated server for several applications.
] ::
] ::I would appreciate any help.
] ::
] ::Joe
] 
] I assume you're running 3.0.1 with the improved cache code. You are?
] Then the gating factor is the write access to the disk on the
] server. The best help here is the new Legato accelerator board.
] See comp.newprod or Unix Review for info. (Bob: d'you want to
] post anything here about it?)
] 

Yes.  But first let me apologize for a late response.  I hope that
this is not too promotional...

The Legato accelerator board, Prestoserve (tm, patent pending) is a
hardware and software product that installs into any Sun VMEbus system
that runs SunOS 4.0 or later.  The VME hardware is mostly battery-backed,
non-volatile, low power SRAM.

How it works:
The Prestoserve software is a UNIX driver that interposes itself between 
SunOS and the real device drivers.  It intercepts synchronous write requests
and caches them in the non-volatile memory.  The Prestoserve driver uses a
modified LRU algorithm to eventually, asynchronously flush its non-volatile
cache to the actual disks.  Because the major functionality is delivered
via a driver, it installs easily into object only distributions.  The
driver's interposition guarantees that Prestoserve works properly with
any (and every) disk and controller installed today.

Why it works:
The NFS protocol is stateless; NFS servers are *not* stateless, but to
behave properly in the eventuality of crashes, NFS servers must commit 
state changes to non-volatile storage (normally a disk) before
acknowledging state-change requests like write, create, remove, etc.
One write request will turn into two synchronous devices writes - one to
put the client's data on the disk, the other to update the file's inode
information.  If the file is large, then a third synchronous device
write is required to update the file's indirect block.

By copying what the UNIX kernel believes to be synchronous writes into
NV RAM, Prestoserve gives back the UNIX file system performance that 
NFS took away.  This is accomplished by:

	- write elimination:  In any given interval only a small number of
	files are being modified.  Writes of the inodes or indirect blocks
	inevitably "hit" a cached entry in the NV RAM.  In practice, this
	eliminates 50% - 70% of all physical disk writes on the NFS server.
	(Although NFS modification operations are less than 15% of the NFS 
	operation mix, they cause close to 70% of all IO on the server.)

	- IO staging:  Writes that eventually goes to the actual disks
	are written asynchronously, allowing the disk drivers to schedule
	those writes in a manner that takes advantage of disk arm locations,
	rotational delays, and other pending IO requests.

	 - Lower perceived latency:  Since writes occur at memory speeds, NFS
	clients see write speeds equal to NFS read speeds.

Is it real?
Legato currently runs 10 diskless 3/50's, 4 diskless 3/80's, and two floppy
PC-NFS (Release 3.0) machines off of one "presto-ized" 3/160 (Release 4.0.3)
server with a Xylogics 451 controlling two 850 megabyte SMD Sabres and a
Ciprico 3500 SCSI host adapter controlling two 580 megabyte SCSI Wren V's.
Performance is great!  We believe that the *CPU* will saturate at around
20 clients.

A PC-NFS-based beta site (I cannot disclose their name) measured a 
600% (!) performance increase after installing Prestoserve into their
Sun 4/110 server.  600% is a bit unbelievable.  However, we expect 
dramatic improvements to PC-NFS networks because PC applications
typically write small, uncached blocks (e.g. 512 bytes).
This means that for the PC-NFS client to write 8 Kbytes, the disk must
perform 32 writes (or 48 writes to a large files) without Prestoserve.
With Prestoserve the same task requires only 2 (or 3) asynchronous
writes to occur.

In conclusion, I am not surprised that Joe Sieczkowski is dismayed that his
upgraded server only delivers 2% more performance.  His upgrade
addressed the instruction execution latency (which is not a big problem
on Sun3-class servers); it did nothing to address the IO latency (measured
in many milliseconds) which usually is the NFS server bottleneck.

More information can be obtained from:

Legato Systems, Inc.
260 Sheridan Ave.
Palo Alto,  CA 94306
415-325-2200

or email to

prestoserve-request@Legato.COM, or sun!legato!prestoserve-request.

Bob Lyon

geoff@hinode.East.Sun.COM (Geoff Arnold @ Sun BOS - R.H. coast near the top) (09/08/89)

In article <952@legato.LEGATO.COM> blyon@Legato.COM (Bob Lyon) writes:
>[Lots of neat stuff about Prestoserve.]
>A PC-NFS-based beta site (I cannot disclose their name) measured a 
>600% (!) performance increase after installing Prestoserve into their
>Sun 4/110 server.  600% is a bit unbelievable.  However, we expect 
>dramatic improvements to PC-NFS networks because PC applications
>typically write small, uncached blocks (e.g. 512 bytes).
>This means that for the PC-NFS client to write 8 Kbytes, the disk must
>perform 32 writes (or 48 writes to a large files) without Prestoserve.
>With Prestoserve the same task requires only 2 (or 3) asynchronous
>writes to occur.

Just a small clarification. PC-NFS will write min(tsize,8192) byte
blocks if requested by the application. Most apps still think in
diskette terms, however, and so relatively few take advantage of this.
We do cache writes, but the buffer cache is relatively tiny (because
everyone's beating on us about size). So, if you measure things by
	COPY C:BIGFILE G:
you won't see much of a win; with dBase or Lotus in the other hand.....

Geoff Arnold,                              Internet: geoff@East.Sun.COM
PCDS Group, Sun Microsystems Inc.
---------------------------------------------------------------------------
Most bizarre title of the week: "LAN Standards - Do you Need Them?" (in BYTE)

beame@maccs.dcss.mcmaster.ca (Carl Beame) (09/09/89)

>
>A PC-NFS-based beta site (I cannot disclose their name) measured a 
>600% (!) performance increase after installing Prestoserve into their
>Sun 4/110 server.  600% is a bit unbelievable.  However, we expect 
>dramatic improvements to PC-NFS networks because PC applications
>typically write small, uncached blocks (e.g. 512 bytes).
>This means that for the PC-NFS client to write 8 Kbytes, the disk must
>perform 32 writes (or 48 writes to a large files) without Prestoserve.
>With Prestoserve the same task requires only 2 (or 3) asynchronous
>writes to occur.
>
>More information can be obtained from:
>
>Legato Systems, Inc.
>260 Sheridan Ave.
>Palo Alto,  CA 94306
>415-325-2200
>

I can believe it. We did some benchmarks of our PC based NFS (BWNFS) on
a Model 80 with a WD8003E card . We ran test5 from the conectathon
( 1Mbyte read and write tests). The DOS transfer rates were 101 KBytes/sec
write and 916 Kbytes/sec read. I would presume that the Prestorserve board
would make writes almost as fast as reads, and we would get an 800% improvement!
We use 8192 bytes for both read and write.

- Carl Beame
  Beame@McMaster.CA

boyter@bimbo.UUCP (Maj Brian Boyter) (09/09/89)

In article <952@legato.LEGATO.COM>, blyon@legato (Bob Lyon) writes:
>
> A PC-NFS-based beta site measured a 
> 600% performance increase after installing Prestoserve into their
> Sun 4/110 server.  600% is a bit unbelievable.  However, we expect
> dramatic improvements to PC-NFS networks

Bob....

In your article you say you expect "dramatic improvements to PC-NFS networks"...
Why doesn't Prestoserve speed-up NFS networks in general (i.e. what about
NFS networks where the clients are other Sun workstations, not PCs)????
 
Brian

-- 
---------------------------------------------------------------
   Maj. Brian A Boyter
   US Army Foreign Science & Technology Center
   Charlottesville, Va 22901                         __
   off: (804)980-7362                              (    )
   home:     973-9440                             {      }
   boyter%bimbo.uucp@virginia.acc.virginia.edu     (    )
                                                     ||
   Nuke'm 'till they glow                            ||
   ...Then shoot'm in the dark              ________<  >_______

mojo@legato (Joseph Moran) (09/15/89)

In article <436@bimbo.UUCP> boyter@bimbo.UUCP (Maj Brian Boyter) writes:
>Bob....
>
>In your article you say you expect "dramatic improvements to PC-NFS networks"...
>Why doesn't Prestoserve speed-up NFS networks in general (i.e. what about
>NFS networks where the clients are other Sun workstations, not PCs)????

Bob is away from his desk for this week, so I'll respond to your
question.

Prestoserve most definitely does "speed-up NFS networks in general".
Bob's point was that it can be even *more* dramatic when used with
PC-NFS clients using small transfer sizes.

For example, a beta site measured a 600% performance increase after
installing on Prestoserve on a Sun serving PC-NFS clients.  Our
simulations show a 50% to 300% improvement using Prestoserve with an
NFS operations mix that was taken from a software development mix for a
server with about a 50/50 mix of diskless and diskful clients running
SunOS 4.0 (34% lookup, 22% read, 15% write, 13% getattr, 2% create).
Your mileage will vary based on operations mix and server configuration
and load level.  Prestoserve shows the most dramatic improvements on
heavily loaded servers with file systems that are accessed for both
reading and writing.

Joseph Moran
Legato Systems Inc.
260 Sheridan Avenue
Palo Alto, CA  94306
(415) 329-7886
mojo@legato.com or {sun,uunet}!legato!mojo