[comp.unix.aix] file fragmentation

siva@bally.Bally.COM (Siva Chelliah/50000) (02/27/91)

I have few questions :
   1 ) Is there a way to find out the  fragmentation of a file system 
       (like the Norton utility for DOS)?
   
   3) How to re-organize the file system ?


Thanks.

Siva

woan@nowhere (Ronald S Woan) (02/27/91)

In article <476@bally.Bally.COM> siva@bally.Bally.COM (Siva Chelliah/50000) writes:
>I have few questions :
>   1 ) Is there a way to find out the  fragmentation of a file system 
>       (like the Norton utility for DOS)?

I don't think so... Outside of some old UNIX/386 implementations, I
don't really remember being able to do this with any UNIX file system
(UFS perhaps? Certainly not BSD).

>   3) How to re-organize the file system ?

What exactly do you mean by reorganize?




--
+-----All Views Expressed Are My Own And Are Not Necessarily Shared By------+
+------------------------------My Employer----------------------------------+
+ Ronald S. Woan       woan@peyote.cactus.org or woan%austin@iinus1.ibm.com +
+ other email addresses             Prodigy: XTCR74A Compuserve: 73530,2537 +

neal@mnopltd.UUCP (02/28/91)

->In article <476@bally.Bally.COM> siva@bally.Bally.COM (Siva Chelliah/50000) writes:
->>I have few questions :
->>   1 ) Is there a way to find out the  fragmentation of a file system 
->>       (like the Norton utility for DOS)?
->
->I don't think so... Outside of some old UNIX/386 implementations, I
->don't really remember being able to do this with any UNIX file system
->(UFS perhaps? Certainly not BSD).

Look for a program called fsanalyze.  It does a brute force traversal of your 
file system and reports on total fragmentation and worst files.   You may 
have to wrestle it some to compile on aix, but otherwise the concept is sound.

->
->>   3) How to re-organize the file system ?
->
->What exactly do you mean by reorganize?
->

There is no free lunch.   Simplest approach:
	- back up the junk which you added and is worst fragmented to tape;
	- delete the above;
	- do an fsck with whatever option rebuilds the free list;
	- restore backup.

------------------------------------------------------------------------------
Neal Rhodes                       MNOP Ltd                     (404)- 972-5430
President                Lilburn (atlanta) GA 30247             Fax:  978-4741
                             emory!mnopltd!neal 
                         gatech!emory!mnopltd!neal 
------------------------------------------------------------------------------

duty@ariel.ucs.unimelb.edu.au (Duty Programmer) (02/28/91)

In article <3592@d75.UUCP> woan@peyote.cactus.org writes:
>In article <476@bally.Bally.COM> siva@bally.Bally.COM (Siva Chelliah/50000) writes:
>>I have few questions :
>>   1 ) Is there a way to find out the  fragmentation of a file system 
>>       (like the Norton utility for DOS)?
>
>I don't think so... Outside of some old UNIX/386 implementations, I
>don't really remember being able to do this with any UNIX file system
>(UFS perhaps? Certainly not BSD).
>

About two years ago (from memory) a program called 'fsanalyze' was posted
to the net. The program follows the inodes for a file and thereby determines
its fragmentation. I think it was posted to comp.sources.unix, so you
should check your local archives. If you get stuck, I think I have the code
buried on a tape somewhere.

>>   3) How to re-organize the file system ?
>

I presume you mean de-fragmentation. The trick is to sort the (inode) free list
by using 'fsck -s' (you will need to be root), which rebuilds the free list
and thereby makes it ordered, and then copy the fragmented file(s), then
delete the old one(s).

rab
--
Richard Brown                     | E-mail: rab@tauon.ph.unimelb.edu.au
School of Physics                 | Phone : +61 3 344 5081
University of Melbourne           | Fax   : +61 3 347 4783
Parkville Victoria AUSTRALIA 3052 | Telex : AA35185

jcburt@ipsun.larc.nasa.gov (John Burton) (03/01/91)

In article <3592@d75.UUCP> woan@peyote.cactus.org writes:
>In article <476@bally.Bally.COM> siva@bally.Bally.COM (Siva Chelliah/50000) writes:
>>I have few questions :
>>   1 ) Is there a way to find out the  fragmentation of a file system 
>>       (like the Norton utility for DOS)?
>
>I don't think so... Outside of some old UNIX/386 implementations, I
>don't really remember being able to do this with any UNIX file system
>(UFS perhaps? Certainly not BSD).
>
>>   3) How to re-organize the file system ?
>
>What exactly do you mean by reorganize?
>
>
>
>
>--
>+-----All Views Expressed Are My Own And Are Not Necessarily Shared By------+
>+------------------------------My Employer----------------------------------+
>+ Ronald S. Woan       woan@peyote.cactus.org or woan%austin@iinus1.ibm.com +
>+ other email addresses             Prodigy: XTCR74A Compuserve: 73530,2537 +

I recall reading an announcement (in BYTE ???) about a version of
NORTON utilities for the UNIX environment...there were some differences
relative to the DOS version, so I'm not sure if the UNIX version has anything
for disk/file fragmentation...

John Burton
(jcburt@gatsibm.larc.nasa.gov)

mike (03/02/91)

In an article, ipsun.larc.nasa.gov!jcburt (John Burton) writes:
>In article <3592@d75.UUCP> woan@peyote.cactus.org writes:
>I recall reading an announcement (in BYTE ???) about a version of
>NORTON utilities for the UNIX environment...there were some differences
>relative to the DOS version, so I'm not sure if the UNIX version has anything
>for disk/file fragmentation...

Oh lord.  Please, if you have a UNIX machine, spare it the agony of having
Norton stuffed into it's bowels.

Simply use fsanalyze, which works quite swimmingly. 

-- 
Michael Stefanik, MGI Inc., Los Angeles| Opinions stated are not even my own.
Title of the week: Systems Engineer    | UUCP: ...!uunet!bria!mike
-------------------------------------------------------------------------------
Remember folks: If you can't flame MS-DOS, then what _can_ you flame?

michi@ptcburp.ptcbu.oz.au (Michael Henning) (03/04/91)

In article <476@bally.Bally.COM> siva@bally.Bally.COM (Siva Chelliah/50000) writes:
>I have few questions :
>   1 ) Is there a way to find out the  fragmentation of a file system 
>       (like the Norton utility for DOS)?

Check out Stallion Technologies' Crocodile product. It does file system
analysis and compaction.

						Michi.
-- 
      -m------- Michael Henning			+61 75 950255
    ---mmm----- Pyramid Technology		+61 75 522475 FAX
  -----mmmmm--- Research Park, Bond University	michi@ptcburp.ptcbu.oz.au
-------mmmmmmm- Gold Coast, Q 4229, AUSTRALIA	uunet!munnari!ptcburp.oz!michi

greywolf@unisoft.UUCP (The Grey Wolf) (03/05/91)

<476@bally.Bally.COM> by siva@bally.Bally.COM (Siva Chelliah/50000)
# 
# I have few questions :
#    1 ) Is there a way to find out the  fragmentation of a file system 
#        (like the Norton utility for DOS)?

What kind of filesystem -- Berkeley's Fast File System or System V's
Usel^H^H^HNIX filesystem?

If it's Berkeley, you can just run fsck -n on it.
If it's System V, you can just forget it.

#    
#    3) How to re-organize the file system ?

What happened to "2) "?
How do you mean "re-organize"?

If you mean "put it back so it is not so fragmented,", that means you
have to dump the filesystem (back it up somehow), clean the filesystem
(via mkfs(8) or newfs(8) (newfs is easier) and restore your data.

If you mean "move things around to be in a better place", that's an
environmental study project, from an electronic point of view (you need
to investigate what kind of impact it will have on your working environment).

# 
# 
# Thanks.
# 
# Siva
-- 
# The days of the computer priesthood are not over.
# May they never be.
# If it sounds selfish, consider how most companies stay in business.

rmtodd@servalan.uucp (Richard Todd) (03/06/91)

greywolf@unisoft.UUCP (The Grey Wolf) writes:
><476@bally.Bally.COM> by siva@bally.Bally.COM (Siva Chelliah/50000)

># I have few questions :
>#    1 ) Is there a way to find out the  fragmentation of a file system 
>#        (like the Norton utility for DOS)?

>What kind of filesystem -- Berkeley's Fast File System or System V's
>Usel^H^H^HNIX filesystem?

>If it's Berkeley, you can just run fsck -n on it.
>If it's System V, you can just forget it.

Actually, if it's a System V Slow Filesystem :-), there is a program called
"fsanalyse" lurking in one of the source group archives which can be used to
report the fragmentation state of your filesystem.  

As for Berkeley Fast File System, I'm not sure if "fsck -n" will report what
the guy's after.  There seems to be two things denoted by the term 
"fragmentation": 1. the presence of files that are in non-contiguous blocks
on the disk, necessitating lots of seeks in order to read the entire file, and
2. the fraction of the total file space that's allocated from "fragments" 
(in the BSD FFS sense, i.e. portions of a whole block allocated for the small
end-portions of files) instead of from whole blocks.  I think the fragmentation
that fsck -n reports is the second type, not the first; certainly the second
type should be easier to compute, given that fsck presumably already has all
the info of how many fragments, etc. are allocated in core already; figuring 
out how the blocks belonging to each file are scattered about the disk and
giving an estimate of (type 1) fragmentation is a more difficult problem.  
Alas, the amount of "type 1" fragmentation (scattering of files across the 
disks) is the figure you're interested in if you're wondering how much
disk performance you're losing, and I don't know offhand of a package that
computes this....

However, all this may be moot for the original poster's purposes.  I
note that this thread has been crossposted to comp.unix.aix, so I assume 
the original poster is using an AIX box.  On the RISC/6000 flavor of AIX (I
don't know about the others) they don't use *either* the BSD or SysV types
of filesystem layout; they use a filesystem layout all their own.  Good luck
in finding utilities that will spit out statistics about (RISC) AIX 
filesystems; you'll need it....
--
Richard Todd	rmtodd@uokmax.ecn.uoknor.edu  rmtodd@chinet.chi.il.us
	rmtodd@servalan.uucp
Motorola Skates On Intel's Head!

lws@comm.wang.com (Lyle Seaman) (03/06/91)

woan@nowhere (Ronald S Woan) writes:

>In article <476@bally.Bally.COM> siva@bally.Bally.COM (Siva Chelliah/50000) writes:
>>I have few questions :
>>   1 ) Is there a way to find out the  fragmentation of a file system 
>>       (like the Norton utility for DOS)?

>I don't think so... Outside of some old UNIX/386 implementations, I
>don't really remember being able to do this with any UNIX file system
>(UFS perhaps? Certainly not BSD).

SunOS and RISC/os both report fragmentation from fsck.  I don't know 
about vanilla BSD.

>>   3) How to re-organize the file system ?

There was a program floating around the net called "packdisk" which did
this for XENIX.  Shouldn't be too much trouble to make it work for other
SysV Unices.  The general solution is: backup your filesystem, make
a new filsystem, restore the backup.

-- 
Lyle                  Wang           lws@capybara.comm.wang.com
508 967 2322     Lowell, MA, USA     

jsalter@ibmpa.awdpa.ibm.com (03/07/91)

In article <1991Mar6.000156.27641@servalan.uucp> rmtodd@servalan.uucp (Richard Todd) writes:
>On the RISC/6000 flavor of AIX (I
>don't know about the others) they don't use *either* the BSD or SysV types
>of filesystem layout; they use a filesystem layout all their own.

Yep, it's the JFS (Journalling File System) which is supposed to provide a
lot better error handling than either of the others, as well as work with
the LVM (Logical Volume Manager).

>Good luck
>in finding utilities that will spit out statistics about (RISC) AIX 
>filesystems; you'll need it....

lsvg - list volume groups and statistics
lspv - list physical volumes and statistics
lslv - list logical volumes and statistics (this is what you probably want)
df   - lists disk and filesystem information

All shipped with the system; what's so tough about this?

>Richard Todd	rmtodd@uokmax.ecn.uoknor.edu  rmtodd@chinet.chi.il.us
>	rmtodd@servalan.uucp

jim/jsalter  IBM PSP, Palo Alto  T465/(415)855-4427  VNET: JSALTER at AUSVMQ
Internet: jsalter@slo.awdpa.ibm.com         UUCP: ..!uunet!ibmsupt!jsalter 
  PS/2 it, or DIE!  :-)  The ramblings above have nothing to do with Big Blue.

jfh@rpp386.cactus.org (John F Haugh II) (03/07/91)

In article <1991Mar6.211120.25337@ibmpa.awdpa.ibm.com> jsalter@slo.awdpa.ibm.com (Jim Salter) writes:
>>Good luck
>>in finding utilities that will spit out statistics about (RISC) AIX 
>>filesystems; you'll need it....
>
>lsvg - list volume groups and statistics
>lspv - list physical volumes and statistics
>lslv - list logical volumes and statistics (this is what you probably want)
>df   - lists disk and filesystem information
>
>All shipped with the system; what's so tough about this?

The kinds of utilities which are being referred to are far more useful
than than the 4 you list above.  The amount of knowlege and experience
in doping around with the USG and BSD filesystems is quite a bit more
widespread than the amount of experience doping around the JFS.

For example, I have a utility which will tell me how fragmented the
average file is, how fragmented the entire partition is, what are the
top X fragmented files (for some value of X), the amount of head
movement for (in terms of block separation, I think) between blocks
in the files, and so on.  I also have a utility which will compress
my filesystem while it is still mounted.
-- 
John F. Haugh II        | Distribution to  | UUCP: ...!cs.utexas.edu!rpp386!jfh
Ma Bell: (512) 832-8832 | GEnie PROHIBITED :-) |  Domain: jfh@rpp386.cactus.org
"I've never written a device driver, but I have written a device driver manual"
                -- Robert Hartman, IDE Corp.

mberkley@active.uvic.ca (Mike Berkley) (03/12/91)

LS> On 6 Mar 91 13:41:13 GMT, lws@comm.wang.com (Lyle Seaman) said:

>I have few questions :
>   1 ) Is there a way to find out the  fragmentation of a file system 
>       (like the Norton utility for DOS)?

LS> SunOS and RISC/os both report fragmentation from fsck.  I don't know 
LS> about vanilla BSD.

No.  The fragmentation reported by fsck refers to how many files
(or tail ends of large files) are small enough to be placed in
block fragments.  If the file system block size is 8k, then using
full blocks for 1k files is very wasteful.  Instead the filesystem
places 8 1k files in one 8k block.

Mike Berkley
University of Victoria
mberkley@sirius@UVic.CA