shoshana@pdi.UUCP (Shoshana Abrass) (07/04/90)
People here are confused about the SGI's filesystem, and I'm hoping
someone out there can clear things up. As I understand it, the old
AT&T filesystem was subject to serious fragmentation, ie, the longer
a file had been around (assuming it grew gradually) the more
fragmented it became (its blocks were all over the disk) and the
slower its access time was. Over time, then entire operating system
could be seen to gradually slow down. The only cure for this was to
back the whole thing up on to tape, remake the filesystem and
restore.
Along came the Berkeley filesystem, with cylinder groups and other
niceties. With the Berkeley fs, files were completely rewritten
to disk when a size increase would have fragmented them - in other
words, the old blocks were deallocated and new, contiguous blocks
(in the cylinder group sense) were allocated. This system sustained
performance over time, but slowed noticeably when the disk became
more than 90% full.
So here are my questions:
1) Assuming the above is correct, which scheme does SGI use?
2) In the AT&T system, if your use pattern was to create a lot of
files - then remove them all - then make a bunch more - would you
still see a gradual slowdown in performance? ie, if you didn't
have files that increased in size over time?
3) Can aging slow the Berkeley fs and, if so, how can it be fixed?
As you might guess, some of our PI's are apparently slower than
other PI's with identical hardware, and we're seeking a reason. Can
anyone think of age problems other than fragmentation?
Thanks for any help - I'll summarize significant replies that don't
get posted.
-shoshana
Shoshana Abrass
pdi!shoshana@sgi.com
--------------- Disclaimer necessitated by mailpath: ----------------
I don't work for sgi, I just work downstream.
---------------------------------------------------------------------kipp@warp.esd.sgi.com (Kipp Hickman) (07/05/90)
In article <9007031822.AA44957835@pdi.UUCP>, shoshana@pdi.UUCP (Shoshana Abrass) writes: |> 1) Assuming the above is correct, which scheme does SGI use? SGI wrote the EFS - "Extent File System". It supports a notion of cylinder groups, and unlike the bsd filesystem there is no small block/large block bogosity. The EFS tends not to fragment, but I personally don't know why. |> 2) In the AT&T system, if your use pattern was to create a lot of |> files - then remove them all - then make a bunch more - would you |> still see a gradual slowdown in performance? ie, if you didn't |> have files that increased in size over time? The EFS should not have this pattern. Disk space is allocated via a bitmap, so the operation you describe should have no apparent slow down. Also, the inode space is also managed as a bitmap, so it will not change ordering either. |> 3) Can aging slow the Berkeley fs and, if so, how can it be fixed? The EFS will slow down over time, as fragmentation increases. If you are *really* experiencing this (you can write some simple i/o tests to determine this), then back your disk up somewhere (*I* would use the network - its more reliable) and clean your filesystems and reload your software. A "good" way to do this is to save all of your files on the disk, then reload the installation software doing a "clean" in the process. After thats done, reload your software from wherever. Please note that you should be very careful when doing this so that you don't lose any of your data (like your passwd file, etc.). Unless a filesystem has an automatic method for eliminating fragmentation, all filesystems will degrade over time. Each of the filesystems - AT&T, BSD, and EFS, have different properties which trigger their fragmentation, but they all do it. The way that I personally operate is to do a clean install every time a new release comes out. I have a set of scripts that I run after the install that repair the system - put back my password file, etc. You will need to personalize this, but here is my copy of the script. Also, this technique relys on having two disks - the first one I "donate" to the software release, the second I use for my personal data. --CUT-- #! /bin/sh # show whats happening set -x umask 22 # install basic configuration files cp /root/d/fixup/etc/X0.hosts /root/etc cp /root/d/fixup/etc/exports /root/etc cp /root/d/fixup/etc/fstab /root/etc cp /root/d/fixup/etc/hosts /root/etc cp /root/d/fixup/etc/passwd /root/etc cp /root/d/fixup/etc/sys_id /root/etc cp /root/d/fixup/etc/config/automount.options /root/etc/config # enable yp echo on > /root/etc/config/yp echo esd.sgi.com > /root/usr/etc/yp/ypdomain # make mail work mkdir /root/usr/lib/uucp touch /root/usr/lib/uucp/Systems cat >> /root/usr/lib/aliases << EOF root:postmaster postmaster:kipp EOF chroot /root /bin/sh << EOF mkdir /tmp cp /d/fixup/GETmail /usr/sbin domainname esd.sgi.com /usr/sbin/GETmail EOF # fixup tty types & services ed /root/etc/ttytype << EOF /ttyd1/s/?v50am/vt100/ w e /root/etc/services g/^#sgi-/s/#// w q EOF # get back disk working and on line if test ! -b /root/dev/d; then rm -f /root/dev/d ln /root/dev/dsk/dks0d2s7 /root/dev/d fi if test ! -c /root/dev/rd; then rm -f /root/dev/rd ln /root/dev/rdsk/dks0d2s7 /root/dev/rd fi # setup network mounts mkdir /root/n #for i in bonnie clyde flake newmedia ozone there; do # mkdir /root/n/$i #done #mkdir /root/n/bonnie/jake /root/n/bonnie/usr /root/n/flake/usr #mkdir /root/n/clyde/aspen ln -s /n/bonnie/jake /root/jake #ln -s /n/clyde/aspen /root/aspen #ln -s .. /root/n/warp rm -rf /root/usr/local /root/usr/games /root/usr/catman/db ln -s /n/warp/d/usr/local /root/usr/local ln -s /n/warp/d/usr/games /root/usr/games ln -s /n/warp/d/usr/catman/db /root/usr/catman/db # create remote printers mkdir /root/tmp chroot /root /usr/sbin/mknetpr PostScript newmedia rio2 rm -rf /root/tmp # ksh cp /root/d/fixup/bin/ksh /root/bin # links for using -me and -ms in psroff #ln -s /n/there/usr/local/lib/tmac/tmac.s /root/usr/lib/tmac #ln -s /n/there/usr/local/lib/tmac/tmac.e /root/usr/lib/tmac #ln -s /n/there/usr/local/lib/ms /root/usr/lib #ln -s /n/there/usr/local/lib/me /root/usr/lib chroot /root /etc/chkconfig automount on # Stuff for me CRONTAB=/root/d/kipp/.crontab if test -f $CRONTAB; then cp $CRONTAB /root/usr/spool/cron/crontabs/kipp fi # Install other packages chroot /root /usr/local/bin/install.ui xrn chroot /root /usr/local/bin/install.ui C++2.0 chroot /root /usr/local/bin/install.move move