[comp.sys.atari.st] BackupST a must have!!!

Bill_von_Hagen@TRANSARC.COM (11/09/90)

This is an enthusiastic recommendation for the new (freeware) backup
program called BackupST and its front end Bfront.  A notice about the
availability of these programs was posted a few days ago by Edwin
Kremer (edwin@praxis.cs.ruu.nl).  The program's author is Fred
Appelman (fred@cv.ruu.nl).  I have nothing to do with the software,
aside from being an ecstatic user.

As far as I`m concerned, these programs do all the right things -
they're fast, easy to use, and have some really whizzy features that
I've not found in commercial backup software (of which I own a few
different packages).  I find them much easier to use and more useful
than other program such as "turtle" and "vault".  (These are just my
opinions, but I'm happy with them.)  Anyone who finds the BSD UNIX
"restore" program's interactive mode intuitive and useful will want
this program.

I got the package from the site mentioned by Edwin (archive.cs.ruu.nl
[131.211.80.5]), but I plan to forward a copy to
atari.archive.umich.edu.  Could someone please tell me how to do this.
It's in your best interests!!!!

Here's an excerpt from Edwin's description of the program:

> BackupST treats a set of floppy's like a stream of data: no floppy disk
> space is wasted by creating subdirectories etc. Compare it to the UNIX
> "tar" utility if you wish. One of the best things of BackupST is its
> powerful restore mechanism (yes, you really might want to restore some
> files occasionally :-]). I like the interactive restore best: people
> familiar with the interactive-restore-mode of the UNIX "restore" command
> will encounter no problems at all using this facility.
>
> BackupST
> --------
>       this is the low-level program that does all the dirty work; this
>       is typically something that you use from within your favourite
>       shell (command line interpreter). Yes, is has __many__ options ;-)
>
> Bfront
> ------
>       this is the GEM frontend for BackupST: once you've seen it, you'll
>       never find yourself going back to using the low-level program...

If someone will explain how to upoload to atari.archive, and where to
put things, I will upload this today.

Once again, I have nothing to do with the program, other than being
able to recognize (this time, at least) truly excellent software when
I see it.

   Bill
   (wvh@transarc.com)

boyd@fsucs.UUCP (Mickey Boyd) (11/12/90)

In article <EbCeZYT0BwwuJF1FAS@transarc.com>, Bill_von_Hagen@TRANSARC.COM writes:
>I got the package from the site mentioned by Edwin (archive.cs.ruu.nl
>[131.211.80.5]), but I plan to forward a copy to
>atari.archive.umich.edu.  Could someone please tell me how to do this.
>It's in your best interests!!!!
>

I have already done so.  I agree that this is a great package!  I also uploaded
the following:  
               antibomb - replaces bombs with a dialog detailing the error.
               stevie39 - new(er) version of this vi clone.  No bugs found yet!

I got these both from the site metioned above.  They have some really neat 
stuff!
-- 
             Mickey R. Boyd          |  "It's amazing how much growing up 
          FSU Computer Science       |      resembles being too tired."
        Technical Support Group      |
      email:  boyd@fsucs.cs.fsu.edu  |                  - Heinlein 

bill@mwca.UUCP (Bill Sheppard) (11/13/90)

In article <EbCeZYT0BwwuJF1FAS@transarc.com> Bill_von_Hagen@TRANSARC.COM writes:
>This is an enthusiastic recommendation for the new (freeware) backup
>program called BackupST and its front end Bfront...

>   Bill
>   (wvh@transarc.com)

Can someone who is using this post some data transfer times (hopefully in
megabytes/minute, though I realize accuracy will be poor if the program
doesn't calculate this for you)...

I have Beckmeyer's backup software, and it isn't bad, but there aren't many
frills (incremental backups must have fresh disks, partial restores are
difficult, no index of files is kept in text-readable format). Does
BackupST do any of these?
-- 
################################################################################
#  Bill Sheppard  --  bills@microware.com  --  {uunet,sun}!mcrware!mwca!bill   #
#  Microware Systems Corporation  ---  OS-9: Seven generations beyond __/_!!   #
#######Opinions expressed are my own, though you'd be wise to adopt them!#######

fischer-michael@cs.yale.edu (Michael Fischer) (11/13/90)

In article <1739@mwca.UUCP> bill@mwca.UUCP (Bill Sheppard) writes:
>I have Beckmeyer's backup software, and it isn't bad, but there aren't many
>frills (incremental backups must have fresh disks, partial restores are
>difficult, no index of files is kept in text-readable format). Does
>BackupST do any of these?

Try the Vault.  It does all of these things.

==================================================
| Michael Fischer <fischer-michael@cs.yale.edu>  |
==================================================

-- 
==================================================
| Michael Fischer <fischer-michael@cs.yale.edu>  |
==================================================

klute@heike.informatik.uni-dortmund.de (Rainer Klute) (11/14/90)

In article <639@fsucs.UUCP>, boyd@fsucs.UUCP (Mickey Boyd) writes:
|>I agree that this is a great package!

It really is. Yesterday I tried it out to fully backup my Megafile44. There
were about 25 MB which took 35 diskettes.

Here a suggestion for future development to increase enthusiasm: A cute
compression feature would be nice! Perhaps I would have needed only 20
diskettes.

-- 
  Dipl.-Inform. Rainer Klute      klute@irb.informatik.uni-dortmund.de
  Univ. Dortmund, IRB             klute@unido.uucp, klute@unido.bitnet
  Postfach 500500         |)|/    Tel.: +49 231 755-4663
D-4600 Dortmund 50        |\|\    Fax : +49 231 755-2386

fred@prisma.cv.ruu.nl (Fred Appelman) (11/15/90)

In <2792@laura.UUCP> klute@heike.informatik.uni-dortmund.de (Rainer Klute) writes:

>Here a suggestion for future development to increase enthusiasm: A cute
>compression feature would be nice! Perhaps I would have needed only 20
>diskettes.
>

As the author of the program I would like to comment on this. I am really
looking for a suitable compression algorithm. I want an algorithm which
is fast as has acceptable compression ratios. 

If anybody on the net has such a piece of software please send it to me.

I had a complaint about the software not running on a Mega ST with a 19
inch screen. I looked in the software and found a bug. If allocate 
32K of memory for the screen toggle option in Bfront. This is not enough
for a 19 inch screen. Does anybody know the correct method of determining
the proper amount of memory to be allocated?

	Fred

--
Fred J.R. Appelman, 3D Computer Vision, Utrecht University
AZU, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands.
Telephone: +31-30-506710 Fax: +31-30-513399
e-mail: fred@cv.ruu.nl or appelman@cs.unc.edu
-- 
Fred J.R. Appelman, 3D Computer Vision, Utrecht University
AZU, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands.
Telephone: +31-30-506710 Fax: +31-30-513399
e-mail: fred@cv.ruu.nl or appelman@cs.unc.edu

bill@mwca.UUCP (Bill Sheppard) (11/16/90)

In article <27248@cs.yale.edu> fischer-michael@cs.yale.edu (Michael Fischer) writes:
>In article <1739@mwca.UUCP> bill@mwca.UUCP (Bill Sheppard) writes:
>>I have Beckmeyer's backup software, and it isn't bad, but there aren't many
>>frills (incremental backups must have fresh disks, partial restores are
>>difficult, no index of files is kept in text-readable format). Does
>>BackupST do any of these?
>
>Try the Vault.  It does all of these things.

>| Michael Fischer <fischer-michael@cs.yale.edu>  |

I've used the Vault, and I agree, it does these things nicely. My main
complaint with the Vault is that it uses TOS format to backup the files,
so speed suffers. Yes, I know of the relative advantages and disadvantages
of TOS format versus direct track writes in proprietary format...I'm
ultimately looking for a backup program that works as well and fast as
PC-Tools, though I understand that the ST's drive controller chip doesn't
easily allow for format+write in the same pass, nor for automatic detection
of floppy inserted...
-- 
################################################################################
#  Bill Sheppard  --  bills@microware.com  --  {uunet,sun}!mcrware!mwca!bill   #
#  Microware Systems Corporation  ---  OS-9: Seven generations beyond __/_!!   #
#######Opinions expressed are my own, though you'd be wise to adopt them!#######

fischer-michael@cs.yale.edu (Michael Fischer) (11/16/90)

In article <1744@mwca.UUCP> bill@mwca.UUCP (Bill Sheppard) writes:
>I've used the Vault, and I agree, it does these things nicely. My main
>complaint with the Vault is that it uses TOS format to backup the files,
>so speed suffers.

Speed may suffer in the Vault for a variety of reasons, but the use of
TOS format isn't generally a major one.  The Vault uses a
sophisticated caching scheme so that the floppy disk is written out in
a few large multi-track blocks.  Directories are created in memory and
written to disk only once.  The FAT is written only at the end.  Data
sectors are written in large blocks.  The effect is similar to that of
Turtle where an image of an entire disk is created in memory and then
dumped to floppy, but unlike Turtle, the Vault can work with less
memory than the size of a floppy. 

The only place where TOS format really hurts is with really small
files, for every file and every directory takes at least one cluster =
two sectors on disk.

Where the Vault spends a lot of time is in getting the files off the
hard disk.  It just uses straight Gemdos calls for that task, and TOS
doesn't allow for simultaneous I/O to the hard disk and the floppy.

One way to speed up floppy disk writes is to turn off write verify.
If you value your data, this is a bad idea, and if you don't, then why
are you bothering with a backup anyway?  Of course, sophisticated data
compression using error correcting codes might make this unnecessary
and allow for faster backups, but getting significantly greater speed
than the Vault provides isn't so easy. 

Try copying a large folder from hard disk to floppy using the desktop
and then try it again using the Vault.  You will see that the Vault
is much faster.

-- 
==================================================
| Michael Fischer <fischer-michael@cs.yale.edu>  |
==================================================

klute@heike.informatik.uni-dortmund.de (Rainer Klute) (11/19/90)

In article <fred.658682337@prisma>, fred@prisma.cv.ruu.nl (Fred Appelman)
writes:
|> As the author of the program I would like to comment on this. I am
|> really
|> looking for a suitable compression algorithm. I want an algorithm which
|> is fast as has acceptable compression ratios. 

I suggest the compression scheme implemented in Arc or Zoo. Advantages:

- sources are freely available,

- the compression factor is really good (somewhere in the 40-50% order),

- fast, at least in the long run.

Let me comment on the last statement as one might well argue about that. I
do backups in the following way: First I do a full backup, then I do a
differential backup once a week. When I am about to run out of backup disks
I have to do another full backup.

Now, doing a full backup takes a lot of time so it would be nice if it
could be done one as seldom as possible. On the other hand during one week
between two backups generally only few files change so the time for a
differential backup may well be neglected (well, almost). With a very good
compression scheme the backup disk pool can be exploited much better and
the time consuming full backup is not neccessary that often.

Bottom line: The time to write data to disk is only one side of the coin.

-- 
  Dipl.-Inform. Rainer Klute      klute@irb.informatik.uni-dortmund.de
  Univ. Dortmund, IRB             klute@unido.uucp, klute@unido.bitnet
  Postfach 500500         |)|/    Tel.: +49 231 755-4663
D-4600 Dortmund 50        |\|\    Fax : +49 231 755-2386

Henry_Burdett_Messenger@cup.portal.com (11/20/90)

Compression should never, ever be used in backups. The idea of a
backup is to enhance data security by providing alternate copies of
data to be used the event of a failure or accidental deletion. 

Compression makes your backups less secure by increasing the
significance of each bit in the backup saveset. (e.g. if you lose one
bit in a compressed saveset, it is very likely that you will lose the
file that holds the bit; you may lose the whole saveset.) 

Most real backup systems for minicomputers or mainframes not only
don't compress, but they include redundant information to recover from
a media error. (VAX/VMS Backup XORs the blocks in a "redundancy group"
and stores that result; this allows single failures in a redundancy
group to be corrected.) 

Backups are no good if they fail when you need them!

Henry B. Messenger           STEVE   REICH   ^   THE   DESERT   MUSIC
henry_burdett_messenger@     MICHEAL  TILSON  THOMAS    ^   CONDUCTOR
   cup.portal.com            Steve Reich  and  Musicians  with Chorus
                             and members of the brooklyn philharmonic
                             text     ^     william  carlos  williams

bill@mwca.UUCP (Bill Sheppard) (11/22/90)

In article <2806@laura.UUCP> klute@heike.informatik.uni-dortmund.de (Rainer Klute) writes:
>...Bottom line: The time to write data to disk is only one side of the coin.

>  Dipl.-Inform. Rainer Klute      klute@irb.informatik.uni-dortmund.de

I agree, but disks are relatively cheap; I'd rather spend $10 for an extra
set of 10 disks and have the software write at full I/O speed than save the
$10 and have the backup take an extra 30 minutes while it saves me two disks
in compression...ideally, the compression scheme can quickly sense whether
the file is likely to be compressed much (ASCII, non-compressed graphics
data) and implement a quick, reasonably efficient algorithm (lempel-zev
would take too long to do on the fly). If the first few hundred bytes of
data are fairly random binary then skip compression...the 10-20% savings
aren't worth the time it takes to compress.

I used BackupST to back up my drive and I'm quite happy with the results.
I do have a couple of suggestions for enhancements:

	1)  An option to verify the backup would be nice (in the same menu as
		list/backup/restore) - slightly more security than having write verify
		on, and doesn't take much longer.

	2)	A screen after the backup listing total bytes backed up/restored,
		total files/folders, Mbytes/minute, and any other pertinent or
		interesting data.

	3)	(If not already present) some error-correction data added to the
		backup to allow recovery in the event of a few bad bytes on the
		backup media.

Please don't take these suggestions as criticism; the program is superb as is.
-- 
################################################################################
#  Bill Sheppard  --  bills@microware.com  --  {uunet,sun}!mcrware!mwca!bill   #
#  Microware Systems Corporation  ---  OS-9: Seven generations beyond __/_!!   #
#######Opinions expressed are my own, though you'd be wise to adopt them!#######

bill@mwca.UUCP (Bill Sheppard) (11/22/90)

In article <36073@cup.portal.com> Henry_Burdett_Messenger@cup.portal.com writes:
>Compression should never, ever be used in backups...
>
>Henry B. Messenger

I disagree. How about "Compression should never, ever be used in backups 
*** for which the lowest possible chance of non-recovery is needed ***."

My prior experience says that I (personally) am not too likely to accidentally
delete files/kill my hard disk, nor have equipment failure resulting in loss
of data. Prior experience also shows a very small chance of a floppy backup
being corrupted. The chance of both of these happening is therefore
miniscule. (Again, I'm speaking solely for myself and my own level of risk
tolerance). Therefore, I would not be uncomfortable with the slightly higher
degree of risk imposed by using compression on my backup floppies.

I agree that the option for non-compression should be available, and that for
any files/data which I must be certain is as secure as possible that I should
use the safest backup method possible. But leave that option up to the user.



-- 
################################################################################
#  Bill Sheppard  --  bills@microware.com  --  {uunet,sun}!mcrware!mwca!bill   #
#  Microware Systems Corporation  ---  OS-9: Seven generations beyond __/_!!   #
#######Opinions expressed are my own, though you'd be wise to adopt them!#######

fred@prisma.cv.ruu.nl (Fred Appelman) (11/24/90)

>I used BackupST to back up my drive and I'm quite happy with the results.
>I do have a couple of suggestions for enhancements:
>
>	1)  An option to verify the backup would be nice (in the same menu as
>		list/backup/restore) - slightly more security than having write verify
>		on, and doesn't take much longer.

	Can you be more specific on this point. What would you like to have
	happen during this verify. Do everything you do in a restore, except
	writing the results to disk?

>	3)	(If not already present) some error-correction data added to the
>		backup to allow recovery in the event of a few bad bytes on the
>		backup media.

	The error correcting data is minimal. On every disk is a disk header.
	Every backup action has got a random number assigned to it. This
	number is stored, together with the disk number on every disk. 
	Everytime a disk is changed, these numbers are checked. The archive
	index, which is vital for a restore action, is stored both at the
	beginning of the archive, and at the end of the archive. If neither
	of these archive index's can not be read, the archive can not be
	restored. In the archive index every file is tagged with a 
	starting position in the archive and a length. If there are some 
	bad bytes on the backup media, this file is just skipped. The program
	will continue with the next file.

	If you have any further suggestions about what error-correction data
	could be added please let me know.
>
>Please don't take these suggestions as criticism; the program is superb as is.

	Thank you.
-- 
Fred J.R. Appelman, 3D Computer Vision, Utrecht University
AZU, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands.
Telephone: +31-30-506710 Fax: +31-30-513399
e-mail: fred@cv.ruu.nl or appelman@cs.unc.edu

bill@mwca.UUCP (Bill Sheppard) (11/27/90)

In article <fred.659451769@prisma> fred@prisma.cv.ruu.nl (Fred Appelman) writes:
->->I used BackupST to back up my drive and I'm quite happy with the results.
->->I do have a couple of suggestions for enhancements:
->->
->->	1)  An option to verify the backup would be nice...
->	Can you be more specific on this point. What would you like to have
->	happen during this verify. Do everything you do in a restore, except
->	writing the results to disk?

Exactly - this would confirm that there were no media problems, as well as
add some peace of mind that the indices are intact...

->->	3)	(If not already present) some error-correction data..

->	The error correcting data is minimal...

The error correcting code would ideally allow the backup to be recovered even
if a full sector is bad; of course, this would require some fairly extensive
error correction code, and I'm certainly not the person who can suggest how it
be done! PC Tools and Fastback (for MS-DOS) suggest that you can have up to
eighty sectors bad and still fully recover a backup, so it can be done, but
I don't know what the cost in speed/backup size would be. I suspect there are
some freely-published papers in the academic/Usenet world which describe
algorithms for doing this (similar to compression algorithms - perhaps the
two can be combined).
->-- 
->Fred J.R. Appelman, 3D Computer Vision, Utrecht Universi
->e-mail: fred@cv.ruu.nl or appelman@cs.unc.edu


-- 
################################################################################
#  Bill Sheppard  --  bills@microware.com  --  {uunet,sun}!mcrware!mwca!bill   #
#  Microware Systems Corporation  ---  OS-9: Seven generations beyond __/_!!   #
#######Opinions expressed are my own, though you'd be wise to adopt them!#######

Jon_Clarke@kcbbs.gen.nz (Jon Clarke) (12/24/90)

Re: Compression.

I agreee there should _never_ be compression on backup files. I use
Diamond Back for the ST and until yesterday used Fastback on the PS2. I
used compression on Fastback and you guessed it the jolly thing screwwed
up so I went to restore my files and my fastback died. So I went to the
dealer to get another copy of Fastback and wouldn't you know it I still
can not back up my PS2.

So the moral of the story is just standard backup technics. I love Diamond
Back and also Mega-a-minute I recommend them to any one with large drives
they need to back up.

 Jon Clarke

 +---------------+-----------------+----------------------+----------+
 |  o( )  STaTus | The Atari BBS   | Node 1:+64-9-606-067 | PDN soon |
 | /  /\   BBS.  | in Auckland,NZ  | Node 2:+64-9-608-485 | MBBS V3  |
 +---------------+-----------------+----------------------+----------+