[comp.unix.questions] files recovery after rm?

dolf@idca.tds.PHILIPS.nl (Dolf Grunbauer) (11/09/89)

In article <2778@uceng.UC.EDU> schriste@uceng.UC.EDU (Steven V. Christensen) writes:
>jlee@sysmgr.cs.uh.edu writes:
>>Are there any programs in the unix world for recovering files right after
>>they are removed by rm? 
>It would seem to me that this would be difficult, since when you remove
>a file, you free up the space for the system (or another process) to
>write over it. 
>On a PC you can recover files easier because you have 100% control of
>the processing. Who knows under Unix what other processes want to
>write to disk....

Apart from that, the PC (read: MS-DOS) does not remove the file, but marks
it as removed, and it does not free the associated disk blocks yet.
If a new file is created or an old one extended, MS-DOS searches first
for never used disk blocks, and if none found it starts reusing freed blocks.

Unix on the other hand uses a FIFO principle for free blocks. I.e. when a
file is 'rm'ed its blocks are put in (the front of) the free block list.
Anyone who needed a new free block will now get one of these most recently
freed blocks, thus destroying the removed file. (Assuming zero link count :-)

Also, MS-DOS links the used blocks in the FAT, which must be used to chain
the old blocks back to a complete file. Note that when a utility (like Norton)
says that the file could not be completely recovered, it tells you that at
least some blocks are reused, as it could not completely walk through this
old block list again.
Unix stores its block administration in a completely different way (using
I-nodes) and even uses normal blocks to store block numbers (for inderict
blocks). These I-nodes also use a FIFO mechanism, so the file administration
is also lost :-(
-- 
Dolf Grunbauer          Tel: +31 55 433233  Internet dolf@idca.tds.philips.nl
Philips Telecommunication and Data Systems  UUCP ....!mcvax!philapd!dolf
Dept. SSP, P.O. Box 245, 7300 AE Apeldoorn, The Netherlands
           --> Holland is only 1/6 of the Netherlands <--

ben@nsf1.mth.msu.edu (Ben Lotto) (11/09/89)

>>>>> On 9 Nov 89 01:08:17 GMT, yahoo@unix.cis.pitt.edu (Kenneth L Moore) said:

Kenneth> I use the following aliases in my .tshrc and .cshrc

Kenneth>     alias rm 'mv -f \!:* /tmp' alias unrm 'mv /tmp/\!:* .'

Kenneth> Note that \!:* allows for the use of variable names.

By the way, the \!:* is not sound for the unrm command.  If you tried
'unrm a b c', it would execute 'mv /tmp/a b c .'.  Perhaps some sort of
for loop would work?

Here's what I use.  It's similar to the above.  In my .login file:

	setenv	RMDIR .RM$$
	mkdir $HOME/$RMDIR
	chmod 700 $HOME/$RMDIR

In .cshrc:

	alias	rm	'mv \!* $HOME/$RMDIR'
	alias	unrm	'mv $HOME/$RMDIR/\!$ .'

And in .logout:

	find /home/brain/ben/.RM* -prune \
	-atime +1 -exec /bin/rm -rf {} \; >>& $HOME/LOGOUT.log &

The advantages of this over the other method: every login shell gets its
own removal directory, the removal directories are private, there is no
clash between users if many users are using the same methods.

The disadvantages: disk space gets charged to you.

Question 1: what are the other advantages and disadvantages of these
approaches?

Question 2: is there any difference between \!* and \!:*?


--

-B. A. Lotto  (ben@nsf1.mth.msu.edu)
Department of Mathematics/Michigan State University/East Lansing, MI  48824

cpcahil@virtech.uucp (Conor P. Cahill) (11/10/89)

In article <20530@unix.cis.pitt.edu>, yahoo@unix.cis.pitt.edu (Kenneth L Moore) writes:
> I use the following aliases in my .tshrc and .cshrc
> 
>     alias rm 'mv -f \!:* /tmp'
>     alias unrm 'mv /tmp/\!:* .'
> 
> Note that \!:* allows for the use of variable names.

Also note that multiple people using the same aliases will eventually run
into problems when 

	1. /tmp runs out of space.

	2. two people rm a file with the same name.

	3. the same person rm's a file with the same name from two places.

Also, since /tmp is frequently used by lots of programs, leaving files in
those directories for a long period of time will result in the size of the 
directory growing and can have a significant performance impact on the entire
system.

> Note further that tmp is writable by everyone. 

in a sense.  Current systems use a sticky bit for a directory which means
that everybody has write permission in the directory unless it clashes with
a pre-existing file in that directory.  Then only root or the owner of the 
file has write permission on the file.  This means that two people using 
your alias with files that have the same base name will cause the second 
users rm to fail.


-- 
+-----------------------------------------------------------------------+
| Conor P. Cahill     uunet!virtech!cpcahil      	703-430-9247	!
| Virtual Technologies Inc.,    P. O. Box 876,   Sterling, VA 22170     |
+-----------------------------------------------------------------------+

tale@pawl.rpi.edu (David C Lawrence) (11/10/89)

In <20551@unix.cis.pitt.edu> yahoo@unix.cis.pitt.edu (Kenneth L Moore) writes:
Ken> I am assuming that you use a mask of 700 like I do so that people
Ken> can't overwrite your files.

This gets funnier and funnier all the time.  Why would you a) assume
anything of the sort, b) set your own umask to 700 and c) think that a
file with 066 (---rw-rw-) permission can't be overwritten by other
people?
-- 
 (setq mail '("tale@pawl.rpi.edu" "tale@ai.mit.edu" "tale@rpitsmts.bitnet"))

msimpson@Teknowledge.COM (Mike Simpson) (11/10/89)

In article <1989Nov10.012648.8942@rpi.edu> tale@pawl.rpi.edu (David C Lawrence) writes:
>In <20551@unix.cis.pitt.edu> yahoo@unix.cis.pitt.edu (Kenneth L Moore) writes:
>Ken> I am assuming that you use a mask of 700 like I do so that people
>Ken> can't overwrite your files.
>
>This gets funnier and funnier all the time.  Why would you a) assume
>anything of the sort, b) set your own umask to 700 and c) think that a
>file with 066 (---rw-rw-) permission can't be overwritten by other
>people?

He was confusing the terms "umask" and "permission mode" (referring
to "chmod").  Be kind.

jik@athena.mit.edu (Jonathan I. Kamens) (11/10/89)

In article <20551@unix.cis.pitt.edu> yahoo@unix.cis.pitt.edu (Kenneth L Moore)
writes:
>Nope. You just aren't allowed to rm. So then you unalias rm and 
>really remove the file. Then source .tshrc and start over. Or use
>"rm" to defeat the aliasing.

  Either "\rm filename" or "/bin/rm filename" will delete a file using
the real rm, even when rm is aliased to something.  Unalias'ing rm,
then removing the file using "rm filename", then sourcing .tshrc (I
thought it was .tcshrc?) all over again is killing a fly with a
sledge-hammer.

Jonathan Kamens			              USnail:
MIT Project Athena				11 Ashford Terrace
jik@Athena.MIT.EDU				Allston, MA  02134
Office: 617-253-8495			      Home: 617-782-0710

jik@athena.mit.edu (Jonathan I. Kamens) (11/10/89)

In article <1989Nov10.012648.8942@rpi.edu> tale@pawl.rpi.edu (David C
Lawrence) writes:
>In <20551@unix.cis.pitt.edu> yahoo@unix.cis.pitt.edu (Kenneth L Moore) writes:
>Ken> I am assuming that you use a mask of 700 like I do so that people
>Ken> can't overwrite your files.
>This gets funnier and funnier all the time.  Why would you a) assume
>anything of the sort, b) set your own umask to 700 and c) think that a
>file with 066 (---rw-rw-) permission can't be overwritten by other
>people?

  He didn't say "umask," he said "mask."  Obviously, taking off the
'u' reverses the meaning of the numbers :-).

Jonathan Kamens			              USnail:
MIT Project Athena				11 Ashford Terrace
jik@Athena.MIT.EDU				Allston, MA  02134
Office: 617-253-8495			      Home: 617-782-0710

jik@athena.mit.edu (Jonathan I. Kamens) (11/13/89)

Mr. Moore,

  I don't usually criticize people on the net, but I find it necessary
to do so in this case.  You've been grated on my nerves for just long
enough now that I find it necessary to respond.

  You have a lot of nerve talking pedantically to someone who
obviously is more informed than you are.

  You have shown to the net, and to comp.unix.questions in particular,
ignorance about several relatively basic subjects in the past couple
of weeks.  That would not be so bad if you were *asking* questions,
but you are *answering* them, and giving wrong answers in the process.

  When people do correct you, you have taken the position that you
must know more than they do, even when you obviously do not.  Your
attempt to dismiss your .signature garbage as a minor problem, when
the rest of the net continues to tell you that is not, is one good
example of this.


In article <20559@unix.cis.pitt.edu> yahoo@unix.cis.pitt.edu (Kenneth L Moore)
writes:
>You are confusing umask with the actual mask. I set my mask to 111 000 000
>by using a umask of 077 in my .cshrc. 

  He is confusing nothing.  Mask and umask are identical terms in this
situation.  Do you even understand why it is *called* a mask?  If you
did, you would not have posted this message, because the name
*implies* the meaning.

  It is called a mask because the following process happens whenever a
file is created:

1. Somewhere in the process of creating the file, the open() function
   is called.  This function takes as one of its arguments the
   desired access modes for the file.

2. The requested access modes are MASKED by the process' UMASK value.
   That is, a BITwise AND is performed, and anything left over is the
   permissions that actually get assigned to the newly created file.

>Here is how it works:

  Gee, thanks for the lesson.

>The first bit in the first word (pertaining to my privileges) is logically
>anded with 1 giving me read privileges. The second bit is anded with
>a 1 giving me write privileges. The third bit is anded with 1 to give me
>execute privileges.

  Actually, the whole umask is masked over the requested permissions
when a file is created using a bitwise AND, not a logical AND.

  I'm not sure what you're saying here -- are you saying that your
umask is used to determine whether you are allowed to read a file that
is already created, by AND'ing your mask with the permissions on the
file?  That's completely wrong.  Umask only applies during file
creation.

>If you and a zero with something it becomes a zero. That is what happens
>in the second two fields. This means that others have no read, write or
>execute privileges.

  Gee, thanks for telling us how AND works.

>If you need more info, please email as this is a very basic question.

  Yes, one which you obviously have gotten wrong.

  As has already been pointed out to you, setting your umask to 077
will cause all files created by you to have permissions ---rw-rw- or
---rwxrwx.  Meaning that you can't read them or execute them.  I doubt
that's really what's happening, unless you do very strange things with
your account.

Jonathan Kamens			              USnail:
MIT Project Athena				11 Ashford Terrace
jik@Athena.MIT.EDU				Allston, MA  02134
Office: 617-253-8495			      Home: 617-782-0710

jik@athena.mit.edu (Jonathan I. Kamens) (11/13/89)

(Sigh.  Insert obligatory flame about people who get worked up enough
to post a flame-like correction, and get the facts wrong in their
correction.)

ken@cs.rochester.edu has been kind enough to point out to me in E-mail
that my attempt to spell out how umask works fell somewhat short of my
intuitive understanding of it.  That is, I can set a umask to do what
I want with no problem, but I left out a step when I tried to describe
what it does :-).

The umask does not represent the access rights that are *allowed*, it
represents the access rights that are *removed* be default.
Therefore, the request access modes are compared in a bitwise AND with
the one's complement of the umask, not with the umask itself.

A 077 umask would therefore always result in files with permissions of
rwx------ or rw-------.

The general gist of my correction still applies.  This just proves
that *I* should be a bit more careful about what I post, especially
when I am correcting someone else....

Jonathan Kamens			              USnail:
MIT Project Athena				11 Ashford Terrace
jik@Athena.MIT.EDU				Allston, MA  02134
Office: 617-253-8495			      Home: 617-782-0710

ok@cs.mu.oz.au (Richard O'Keefe) (11/13/89)

In article <15844@bloom-beacon.MIT.EDU>,
jik@athena.mit.edu (Jonathan I. Kamens) flames at Moore:

>   As has already been pointed out to you, setting your umask to 077
> will cause all files created by you to have permissions ---rw-rw- or
> ---rwxrwx.  Meaning that you can't read them or execute them.

The thing about umask is that it is the COMPLEMENT of a mask, the bits
which are ON in the umask are the permissions which are to be DENIED.
So if you create a file with mode 0<u><g><o> when your umask is 0<0><7><7>
the <0> leaves the <u> permissions alone and the <7>s clear the <g> and
<o> permissions, making the result 0<u><0><0>.  For example, if I do
	creat("foo", 0666)
when my umask is     0077 what I get is 0600, or rw-------.  I'm afraid
Sro Kamens has it backwards.

chittamu@umvlsi.ecs.umass.edu (Satish Kumar .C) (11/14/89)

In article <15844@bloom-beacon.MIT.EDU> jik@athena.mit.edu (Jonathan I. Kamens) writes:
>Mr. Moore,
.
.
.
>  As has already been pointed out to you, setting your umask to 077
>will cause all files created by you to have permissions ---rw-rw- or
>---rwxrwx.  Meaning that you can't read them or execute them.  I doubt
>that's really what's happening, unless you do very strange things with
>your account.

A minor correction. The low order 9 bits of the umask value are used to
clear the corresponding bits in the mode pattern used to create a file.
So a umask value of 077 would make your file unreadable, unwritable and
unexecutable by either group or others, i.e. your mode will be
-rwx------ or some subset thereof.



-- 
	-Satish.
	chittamu@cs.umass.edu
--
The Theory of Objectivity: E = mc++

rsalz@bbn.com (Rich Salz) (11/14/89)

I believe the "best" solution to this problem is to check out the delete/
undelete programs used at the MIT Project Athena.  Contact your nearest
comp.sources.unix archive site.

It is not a good idea to alias "rm" because you get into bad habits when
you use different systems.
	/r$
-- 
Please send comp.sources.unix-related mail to rsalz@uunet.uu.net.
Use a domain-based address or give alternate paths, or you may lose out.

mikulska@odin.ucsd.edu (Margaret Mikulska) (11/14/89)

In article <20551@unix.cis.pitt.edu> yahoo@unix.cis.pitt.edu (Kenneth L Moore) writes:
>In article cpcahil@virtech.uucp (Conor P. Cahill) writes:
>
>==>In article, yahoo@unix.cis.pitt.edu (Kenneth L Moore) writes:
>
>==>Also, since /tmp is frequently used by lots of programs, leaving files in
>==>those directories for a long period of time will result in the size of the 
>==>directory growing and can have a significant 
>==>performance impact on the entire system.

The main impact is not that much performance, but 'file system full' and
'no space on device', especially when /tmp is not on a separate disk
partition. You can overwrite some interesting files this way. Some UNIX 
boxes simply crash when that happens.

>This should be addressed by the systems people. I would think that a
>program to clean /tmp on an as needed basis would be appropriate. How
>about that systems type guys. What's the poop on /tmp? Is it
>automatically cleaned?
>
>I have logged in a week after writing to /tmp and found files still
>there.

'The systems people' addressed that already. /tmp gets cleaned during 
booting. Otherwise, it depends what the sys. adm. puts in 
/usr/lib/crontab (or /var/spool/cron/crontabs/root in SunOS) - it's 
fairly common, but not standard, to put an entry there to clean /tmp.  
No special program is necessary (that would be an overkill); just a 
short line. See "man cron" or "man crontab".

Margaret Mikulska

mem@inls1.ucsd.edu
ucsd!inls1!mem

cpcahil@virtech.uucp (Conor P. Cahill) (11/15/89)

In article <7444@sdcsvax.UCSD.Edu>, mikulska@odin.ucsd.edu (Margaret Mikulska) writes:
> In article <20551@unix.cis.pitt.edu> yahoo@unix.cis.pitt.edu (Kenneth L Moore) writes:
> >In article cpcahil@virtech.uucp (Conor P. Cahill) writes:
> >
> >==>Also, since /tmp is frequently used by lots of programs, leaving files in
> >==>those directories for a long period of time will result in the size of the 
> >==>directory growing and can have a significant 
> >==>performance impact on the entire system.
> 
> The main impact is not that much performance, but 'file system full' and
> 'no space on device', especially when /tmp is not on a separate disk

Try putting 3 or 5 thousand files on /tmp and see the overall impact 
it has on the system (performance wise).  You will probably be quite
suprised.  Each time a program tries to create a file on /tmp the kernel
must read the entire directory until it finds the first free slott and 
as time goes on, the long term files (like those placed by an unfriendly rm
alias) will tend to take up the slots at the start of the directory file, so
the kernel will tend to have to read the entire directory EVERY time someone
opens, create, or removes a file in that directory.

Since /tmp is accessed by lots of programs (like some shells, vi, cc, ar, etc)
this can have a tremendous performance inpact on your system.

What you say about space is correct and is also a major consideration.

-- 
+-----------------------------------------------------------------------+
| Conor P. Cahill     uunet!virtech!cpcahil      	703-430-9247	!
| Virtual Technologies Inc.,    P. O. Box 876,   Sterling, VA 22170     |
+-----------------------------------------------------------------------+

bink@aplcen.apl.jhu.edu (9704) (11/17/89)

This was so amusing, I posted it at work.

In article <15844@bloom-beacon.MIT.EDU> Jonathan Kamens flames:
>   You have a lot of nerve talking pedantically to someone who
> obviously is more informed than you are.
> 
>   You have shown to the net, and to comp.unix.questions in particular,
> ignorance about several relatively basic subjects in the past couple
> of weeks.  That would not be so bad if you were *asking* questions,
> but you are *answering* them, and giving wrong answers in the process.
> 
>   When people do correct you, you have taken the position that you
> must know more than they do, even when you obviously do not.  [...]
> 
> In article <20559@unix.cis.pitt.edu> yahoo@unix.cis.pitt.edu (Kenneth L Moore)
> writes:
> >You are confusing umask with the actual mask. I set my mask to 111 000 000
> >by using a umask of 077 in my .cshrc. 	[well put]
> 
>   He is confusing nothing.  Mask and umask are identical terms in this
> situation.  Do you even understand why it is *called* a mask?  [...]

See flame above about posting wrong answers to "relatively basic subjects".
 
> 1. Somewhere in the process of creating the file, the open() function [...]

It's *called* creat() (usually), and it's a system call.

> >The first bit in the first word (pertaining to my privileges) is logically
> >anded with 1 giving me read privileges. The second bit is anded with [...]
> 
>   Actually, the whole umask is masked over the requested permissions
> when a file is created using a bitwise AND, not a logical AND.  [...]

*Actually*, there's not a whole heck of a lot of difference when you're
talking about one bit (as Mr. Moore was).  K&R (excuse the bitwise AND)
didn't coin the word "logical".

>   Yes, one which you obviously have gotten wrong.	[oh?]
> 
>   As has already been pointed out to you, setting your umask to 077
> will cause all files created by you to have permissions ---rw-rw- or

Anyone who's ever used umask can see this is wrong.  Contrary to your
followup, if you could show me just *one* place where Mr. Moore was wrong
and you weren't, I'd be surprised.  I think you owe this man an apology.
























					-- Greg Ubben
					   bink@aplcen.apl.jhu.edu
					   ...!uunet!mimsy!aplcen!bink

gwyn@smoke.BRL.MIL (Doug Gwyn) (11/18/89)

In article <3914@aplcen.apl.jhu.edu> bink@aplcen.apl.jhu.edu (Greg Ubben) writes:
>See flame above about posting wrong answers to "relatively basic subjects".
>> 1. Somewhere in the process of creating the file, the open() function [...]
>It's *called* creat() (usually), and it's a system call.

Hm, should we flame this flame of a flame, or should we just point out
that Greg is behind the times?

amoss@batata.Huji.AC.IL (Amos Shapira) (11/19/89)

>jlee@sysmgr.cs.uh.edu writes:

>Are there any programs in the unix world for recovering files right after
>they are removed by rm?
>Joseph Lee.                             Internet: jlee@cs.uh.edu

There is a package called undel for UNIX BSD posted on comp.sources.unix
some while ago (29 Mar '89). Here's the data of the article and the intro
from it's first part (out of 6 parts):
----------
Submitted-by: Jonathan I. Kamens <jik@PIT-MANAGER.MIT.EDU>
Posting-number: Volume 18, Issue 73
Archive-name: undel/part01

[  This will need a bit of work to run under non-BSD.  It's also
   probably going to be the LAST safe-rm/unrm/whatever posting
   I approve, except for patches.  --r$  ]

This archives contains the sources for five programs (well, actually,
four, since one is a symbolic link):
	delete
	undelete
	expunge
	purge
	lsdel

The programs were written at Project Athena in response to the problem
which many novice Unix users have with accidentally deleting files
they didn't want to delete (Hell, I've done it :-).

I am already aware of quite a few improvements I could make in the
code (I am fairly certain that there are better regexp matching
libraries *and* better directory searching libraries I could be
using.), and I will be making them as soon as I am done with my
current project.  I would appreciate hearing any suggestions you may
have and/or accepting any patches you wish to submit :-)

----------

Hope that helps,
---
- Amos Shapira
Israel Air Force

amoss@batata.bitnet
amoss@batata.huji.ac.il

kenmoore@unix.cis.pitt.edu (Kenneth L Moore) (11/20/89)

In article <3914@aplcen.apl.jhu.edu> bink@aplcen.apl.jhu.edu (Greg Ubben) writes:

==>*Actually*, there's not a whole heck of a lot of difference when you're
==>talking about one bit (as Mr. Moore was).  

==>					-- Greg Ubben
==>					   bink@aplcen.apl.jhu.edu
==>					   ...!uunet!mimsy!aplcen!bink

MISTER Moore?

I'm used to: "Hey You!"