[comp.sys.amiga.programmer] ramdrive repacking

meranda@iguana.cis.ohio-state.edu (deron meranda) (02/05/91)

Does anyone know about any programs which can "repack" a ramdrive?

I use my RAM: rather heavily and have noticed that my main memory
becomes fragmented.  I have determined that most of the time this
is caused by the ram device itself as files are created and later
deleted.  This is not much of a surprise to me and appears to be
an expected behavior with the lack of virtual memory.  However, it
also seems quite possible for the files in the ram drive to be
repacked (perhaps by user request) after more memory has been
freed by other processes.

Since the ramdrive.device is not a "real" disk device, I have no
idea of its storage format.  If a program does not already exist
and no one else would like to tackle writing a ramdrive repacker,
I might attempt one myself.  But, I would like any hints,
documentation, etc. that anyone might have about the ramdrive.

( The largest problem I foresee is dealing with outstanding locks.
  I would think that locks due to assigns could be easily
  remaped.  What about the others?  Any advice from DOS experts? )

Thanks!

Deron E. Meranda  ( meranda@cis.ohio-state.edu )

cmw1725@tamsun.tamu.edu (Christopher Walton) (02/05/91)

I would suggest to just use RRamdisk from The Other Guys.
It works nicely, and better than the ramdrive.device drives..
This is my oppinion.

dac@prolix.ccadfa.oz.au (Andrew Clayton) (02/06/91)

In article <88002@tut.cis.ohio-state.edu>, deron meranda writes:

> Does anyone know about any programs which can "repack" a ramdrive?

I believe that you can just do a request for a huge amount of memory
(something insane, like 4Gigabytes), and that will force the OS to clean up
the fragmented memory list, trying to give back a contiguous chunk of
memory. Make the routine not return an error (especially a guru!), and that
should do it.

I've heard of this program existing, but I've never seen it. I don't know if
the method will work either, but it sounds plausible.

Dac
--

peterk@cbmger.UUCP (Peter Kittel GERMANY) (02/07/91)

In article <18a42c4c.ARN295c@prolix.ccadfa.oz.au> ccadfa.cc.adfa.oz.au!prolix!dac@munnari.OZ.AU writes:
>In article <88002@tut.cis.ohio-state.edu>, deron meranda writes:
>
>> Does anyone know about any programs which can "repack" a ramdrive?
>
>I believe that you can just do a request for a huge amount of memory
>(something insane, like 4Gigabytes), and that will force the OS to clean up
>the fragmented memory list, trying to give back a contiguous chunk of
>memory.

No, this is not the solution. The OS only can flush out libraries
and other stuff where the use count is zero, but it can't rearrange
memory chunks of a fragmented RAM disk to one big chunk. It simply
cannot *move* memory. - For the special case of the RAM disk, I can
imagine a software that knows all internals of this device and
rearranges that memory, but that doesn't sound trivial.

-- 
Best regards, Dr. Peter Kittel  // E-Mail to  \\  Only my personal opinions... 
Commodore Frankfurt, Germany  \X/ {uunet|pyramid|rutgers}!cbmvax!cbmger!peterk

meranda@iguana.cis.ohio-state.edu (deron meranda) (02/08/91)

In article <864@cbmger.UUCP> peterk@cbmger.UUCP (Peter Kittel GERMANY) writes:
>...
>   - For the special case of the RAM disk, I can
>imagine a software that knows all internals of this device and
>rearranges that memory, but that doesn't sound trivial.

Well, does there exist any documentation at all on the internals of the
RAM drive?  Is somebody at CBM-Amiga willing to part with enough info
on this so that I can try to write a program that does this?  I realize that
these internals might change, but I can always check version #'s before
it attempts the repack.

Any docs, tech notes, or hints would be greatly appreciated.

Deron E. Meranda  ( meranda@cis.ohio-state.edu )

dac@prolix.ccadfa.oz.au (Andrew Clayton) (02/08/91)

In article <864@cbmger.UUCP>, Peter Kittel GERMANY writes:

> In article <18a42c4c.ARN295c@prolix.ccadfa.oz.au> ccadfa.cc.adfa.oz.au!prolix!dac@munnari.OZ.AU writes:
> >In article <88002@tut.cis.ohio-state.edu>, deron meranda writes:
> >
> >> Does anyone know about any programs which can "repack" a ramdrive?
> >
> >I believe that you can just do a request for a huge amount of memory
> >(something insane, like 4Gigabytes), and that will force the OS to clean up
> 
> No, this is not the solution. The OS only can flush out libraries
> and other stuff where the use count is zero, but it can't rearrange
> memory chunks of a fragmented RAM disk to one big chunk. It simply

Oh, ok. Thankyou for clearing up my misconception.

Dac
--
 _l _  _   // Andrew Clayton. Canberra, Australia.         I Post  .
(_](_l(_ \X/  Send mail to dac@prolix.pub.uu.oz.au                . .  I am.

pochron@rt6.cs.wisc.edu (David Pochron) (02/09/91)

In article <11795@helios.TAMU.EDU> cmw1725@tamsun.tamu.edu (Christopher Walton) writes:
>I would suggest to just use RRamdisk from The Other Guys.
>It works nicely, and better than the ramdrive.device drives..

The only problem with RRamDisk is it is nearly useless to all us Amiga owners
who have a combination of 16 and 32 bit memory and want the ram disk only in
16 bit memory.  Although the preallocated, fixed space of CBM's ramdrive.dev
is limiting, I can at least re-order the memory lists when I boot and make sure
it is in 16 bit memory.  With a 3000, this isn't necessary of course.

Also, VD0: is very useful if you have 16/32 bit mem combos, since it does
things very cleverly and allocates out of the C00000 memory first if it
finds it.

If they would add an extra flag bit to the mountlist entry that would force
the device to alloc out of memory in the last MemList field, I could use it.
(I *would* use it!)

-- 

       -- David M. Pochron   | "Life's a blit,
                             |  and then you VBI."
pochron@garfield.cs.wisc.edu |

darren@cbmvax.commodore.com (Darren Greenwald) (02/25/91)

In article <88002@tut.cis.ohio-state.edu> <meranda@iguana.cis.ohio-state.edu> writes:
>Does anyone know about any programs which can "repack" a ramdrive?
>
>I use my RAM: rather heavily and have noticed that my main memory
>becomes fragmented.  I have determined that most of the time this
>
>Since the ramdrive.device is not a "real" disk device, I have no
>idea of its storage format.  If a program does not already exist
>Thanks!
>
>Deron E. Meranda  ( meranda@cis.ohio-state.edu )

FYI:

	RAM: - the ram-handler (allocates memory dynamically; not a device).
	RAD: - the ramdrive.device (allocates a big chunk of contiguous memory)

So the answer to the original question is, RAD: doesn't have this problem,
but it eats up a big chunk of contiguous memory up-front.  RAM: can fragment
your memory, but it only uses what it needs.  A give, and take situation.
I suppose that a ram drive like RAM: could try to defragment itself, but
you can easily get into a no-win situation; you start trading off
non-contiguous chunks of memory for contiguous ones, and still end-up
eating large chunks of contiguous memory, leaving little ones free.

	---Darren---

meranda@frankenstein.cis.ohio-state.edu (deron meranda) (02/28/91)

In article <19286@cbmvax.commodore.com>
  darren@cbmvax.commodore.com (Darren Greenwald) writes:
>In article <88002@tut.cis.ohio-state.edu>
  <meranda@iguana.cis.ohio-state.edu> writes:
>>...
>
>    RAM: - the ram-handler (allocates memory dynamically; not a device).
>    RAD: - the ramdrive.device (allocates a big chunk of contiguous memory)
>
> So the answer to the original question is, RAD: doesn't have this problem,
> but it eats up a big chunk of contiguous memory up-front.  RAM: can fragment
> your memory, but it only uses what it needs.  A give, and take situation.
> I suppose that a ram drive like RAM: could try to defragment itself, but
> you can easily get into a no-win situation; you start trading off
> non-contiguous chunks of memory for contiguous ones, and still end-up
> eating large chunks of contiguous memory, leaving little ones free.
>
>	 ---Darren---

Thanks for recognizing the difference between RAD: and RAM:.  Perhaps
I wasn't clear on my original posting.  It really _wouldn't_ make much
sense for the ramdrive to defragment itself by moving all the files
into contiguous memory chunks - as you noticed.

However, I assume that the ramdrive uses something similar to blocks
to divide the files into small pieces, and links them via some kind
of memory pointer (perhaps even BPTR's).  Except for the controversy
over fast and chip memory (or virtual memory if anything supports it),
it should make no difference how these blocks are scattered throughout
memory from the ramdrive's perspective - i.e. contiguous chunks don't
mean much.

Now consider what the rest of the system sees - namely the free system
memory chunks.  It is desirable for these to be as large as possible.
Lets use a real simple picture for a senerio:

          `R'  is memory used by the ramdrive
          `.'  is free system memory
          `x'  is memory used by something else

   1)  Initial conditions:

       ..RRR.xxx.xxx..RR.............R...

   2)  System gets heavily loaded:

       xxRRR.xxx.xxxxxRR.xxxxx.xxxxxxRxxx

   3)  A program appends data to a RAM: file:

       xxRRR.xxx.xxxxxRR.xxxxxRxxxxxxRxxx
                              ^

   4)  The system load lightens again:

       ..RRR.xxx.xxx..RR......R......R...

Now, that one block used by the ramdrive is cutting that big free
memory block in half!  And YES -- This does happen often to me.

What I would like, is upon recognizing that condition, is to execute
(perhaps manually) a program which attempts to move that lone block
into some smaller free memory chunk.  It would be perfectly acceptable
to sacrifice system performance while that moving is being done.
Note, that the ramdrive is still technically just as fragmented -
but the "free memory" is not as fragmented.

Does this clear up what I meant?  I seems to me that this should be
very possible without too many difficulties -- if one has some
insights into the ramdrive formats itself.  Can anyone help or
comment on this idea?

Thanks in advance.

Deron E. Meranda  ( meranda@cis.ohio-state.edu )

markv@kuhub.cc.ukans.edu (03/12/91)

In article <88966@tut.cis.ohio-state.edu>, meranda@frankenstein.cis.ohio-state.edu (deron meranda) writes:
> In article <19286@cbmvax.commodore.com>
>   darren@cbmvax.commodore.com (Darren Greenwald) writes:
>>In article <88002@tut.cis.ohio-state.edu>
>   <meranda@iguana.cis.ohio-state.edu> writes:
>>>...
> Now, that one block used by the ramdrive is cutting that big free
> memory block in half!  And YES -- This does happen often to me.

Yes, this is a common problem.  One way I've found that works is (if
you have the RAM), copy the RAM disk into its own sub-dir, delete the
old files and rename the files back to the root.  This works to an
extent assuming you have to big RAM.

It should be noted that the 2.0 RAM: "packs" small files by putting
multiple files smaller than 512 bytes into one block, so that really
small files (like ENV: files) dont gobble up lots of RAM.
  
> What I would like, is upon recognizing that condition, is to execute
> (perhaps manually) a program which attempts to move that lone block
> into some smaller free memory chunk.  It would be perfectly acceptable
> to sacrifice system performance while that moving is being done.
> Note, that the ramdrive is still technically just as fragmented -
> but the "free memory" is not as fragmented.

I'm working on a replacement AllocMem() that can be configured to
behave differently depending on the task that calls it.  My original
goal was to have RAM-Handler and RAMB0 allocations come out of the
slowest non-CHIP memory (such as 16 bit Fast memory on a mixed RAM
2000 that has 32 bit memory), but I could exetend it to "secretly"
allocate bigger chunks of RAM an parcel those as as needed (kind of
like malloc()) to a particular task.

> Does this clear up what I meant?  I seems to me that this should be
> very possible without too many difficulties -- if one has some
> insights into the ramdrive formats itself.  Can anyone help or
> comment on this idea?

Most of this stuff is internal to the RAM-Handler.  There is no
documentation to its internals.	 All it has to do is respond properly
to DOS packets send to its process.
 
> Thanks in advance.
> 
> Deron E. Meranda  ( meranda@cis.ohio-state.edu )
-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Mark Gooderum			Only...		\    Good Cheer !!!
Academic Computing Services	       ///	  \___________________________
University of Kansas		     ///  /|         __    _
Bix:	  mgooderum	      \\\  ///  /__| |\/| | | _   /_\  makes it
Bitnet:   MARKV@UKANVAX		\/\/  /    | |  | | |__| /   \ possible...
Internet: markv@kuhub.cc.ukans.edu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~