451061@UOTTAWA.BITNET (Valentin Pepelea) (08/27/89)
Well, I'm gonna loose net access in a week or so, so its time for me to
make my official Goodbye. But before that, perhaps I should introduce myself,
eh?
I'm now finally a Computer Engineer with a freshly printed diploma rolled into
a plastic tube, making its way through the Canadian Post Corporation. :-)
I've owned an Amiga 1000 for three and a half years now, and used it mostly
for telecommunications, editing, flight-simulations :-), and my fourth year
project.
So Goodbye folks!
But wait. I'm not about to go with a whisper, I have to go with a Bang! Well,
that project I mentioned, it was to take an operating system wich does not
support virtual memory, and implement it. That's right, your Amiga finally
supports VM! Now you know what I meant when I said I was betting my iron ring
on it.
The nice thing is, I got 4 credits for it. A 4-month term course is usually
worth 3, 4 if a lab is included. Needless to say, I'm pretty happy to get that
for a weekend hack. If you are entering 4th year this fall, I suggest you pick
out a project too if you have the option. You'll spend 2 to 3 times what you
normally spend on a regular course, but you'll learn 5 times more, too. And it
looks damned good on a resume.
"All right Valentin, so just how much time did it take you then?"
Well, if I knew the 680x0 and Amiga OS back then as I do know, it would have
taken me a weekend. My project also included some hardware design, namely
the building of a 68020/68851 board which plugs into the slot of 68020. (into
a Lucas board) I had limited success with it. My Lucas board was edgy just by
itself; I had to run it at 20 MHz with a 7474 in U9 to get it to boot one out
of 3 times. The MMU board got things so noisy, my Amiga would crash every 2 to
5 minutes. :-( If got so fed up with it, I sold off the LUCAS board.
Getting back to the time question, I figured out and solved theoretically all
problems in March, got the program written at the end of June, and
demonstrated it to my prof yesterday. It took so long for two reasons. The
first is the hardware; I had to borrow an A2500 to test the virtual handler
on, and got one for a week in June. Unfortunately, a heat wave hit Ottawa
back then, so I only got to hack on it for about four days back then. The
Second time I got the 2500 was on the first week of August, right on final
exam week, when David Buck, my beta tester, went to Siggraph for a week. So
I got to hack on his jewel for another 3 days. Then the last bugs were stomped
by transmitting the new handlers through modem, and then listening to David's
description of the latest crash symptoms! If count up the days, it only took
me a total of about 2 weeks to write and debug the thing.
The second problem is that I rate as an ENTP on the Briggs-Meyer personality
test. Basically, I need new projects to drive me on, and as soon as I solve
the challenges in them, I lose interest. Well, I lost interest on this one in
March. :-( I'm pretty happy now though. :-)
Anyway, if you undertake a project make sure the hardware you need is at your
disposal. I had two backup plans for mine. I first wanted to make a 68070
(Signetics) plug-in board, but by October 88, it still didn't come out.
Backup plan number one was to build a Lucas board and design a 68851 addition
to it. Backup plan number two was to use an Amiga 2500. I knew Commodore was
about to come up with it.
"What's the big deal? Virtual memory is taught in Operating Systems 301."
Well, if we had any other microcomputer such as an Atari St, Macintosh or
I*M PC, all we would have to do upon a page fault is call the disk I/O
routines and modify the translation tables. But on the Amiga, we have a
real-time message based multitaskig operating system. The keyword here is
message based. Upon a page fault, we are in supervisor mode and multitasking
is disabled, so we cannot send any messages to the task that controls the
hard disk.
The challenge of this project was to understand the system, hardware and
software, and solve the above problem. It is understanding that took most of
the time, reading tons and tons of documentation, and filtering out the
fluffy stuff.
"How well does virtual memory work?"
Pretty well, really. My beta tester, David Buck, runs SmallTalk which he
ported himself. He says its fine, though not as fast as when running out
of physical memory, of course. The current version has an incredibly aging
algorithms though. It's really the bottleneck for now. Every time a page
fault occurs, the thing is called, wasting somewhere between 50 and 100
miliseconds. So loading a large program into virtual memory will take an
eternity. Get yourself a second drive exclusively for virtual memory is a
good idea. That way the drive's head does not have to move around for each
page fault when loading a file.
Why is it so slow? Hint: it's written in C. Actually the real
reason is that although the general concept behind the aging and page
replacement algorithms is correct, the implementation is not. I can speed up
the thing drastically (like one houndred thousand percent) easely, but since
I lost access to the 2500, I cannot implement it. David is working like crazy
on his Master's thesis right now, so I do not have anything to test new
versions on. I'm lucky David allowed me to test the thing over the phone
lately. And then I myself am hopping between Ottawa and Montreal looking for
the perfect job.
"So, is virtual memory compatible with everything?"
Well, it is with all software that follows the rules. Let me quote from the
Amiga ROM Kernel Manual:
"ALL INFORMATION THAT IS ACCESSED BY INTERRUPTS AND/OR OTHER TASKS HAS TO BE
PUBLIC OR LOCKED IN MEMORY. This applies to both code and data."
Emphasis is theirs, not mine. (I might have misquoted, slightly.) Now, code
which accesses data from within Forbid() or Disable() calls also exercice
interrupt characteristics, so they also have to abide by this rule. To test
out how many programs do not, I replaced all calls to Forbid() in the Exec
memory functions with calls to semaphores. Then I put in a functions which
checks wether multitasking has been forbiden or interrupts were disabled. I
suprised to see how many failed. Among others were Run, DiskChange, Ed and
my favorite editor, TxEd. :-(
Needless to say, I immediately modified AllocMem() to return virtual memory
only if specifically asked to. MEMF_VIRTUAL is bit 3. (LSB=0) Now only
programs which tinker themselves with the memory lists within Forbid() calls
would fail. These are extremely rare, but they do exists. Like Avail.
A WHOLE NEW CLASS OF GURUS
That's right. Now you too, if you have an MMU equiped Amiga can enjoy a few
more Guru numbers. Like:
A0000000 Interrupts were disabled
A0000001 Multitasking was forbiden
A200000x Unrecognized exception stack frame (If you have a wierd CPU)
A300000x Unable to allot MEMF_CHIP ram, or OpenDevice()
...
etc.
I'm sure many of you will try out Virtue only to meet the Guru more often!
Now, I have to admit I haven't tested this thing out too roughly. Since my
time is very limited, and my beta tester is falling asleep writing his thesis,
it is not likely to get tested any more for a long time.
"All right Valentin, I'm sold! Where can I get it?"
A similar virtual memory handler for the Macintosh sells for $295. And it is
not a demand paged implementation like mine. So send $395 to 2420 Boul Du
Carmel, Trois-Rivieres Que., Canada G8Z 3S2. :-) :-)
Actually, since so many programs would fail if MEMF_VIRTUAL memory was
returned by default by AllocMem(), the usefulness of this project is cut by an
order of magnitude. Meaning, I doubt it is viable to be distributed as a
commercial product. Your suggestion of what I should do with it is welcome. I'm
currently applying to a couple of Amiga devellopers for employment, so if my
employers want to do anything with it, it theirs! It will probably end up in
an Amiga Transactor article though, but you input is welcome.
"But I only have an A500. Can still run the thing?"
Sure, If you want to get an instant Guru! :-) But seriously, if you have
an A1000 and a Lucas board, you too can try you luck at wire-wrapping your
self a 68851 board. I'll even give you the design.
When I first started this project, I inteded to build a Signetic 68070 board
instead. This chip had been mentioned in a Byte issue of 1986, so I though
it was ready by then. The 68070 is a CMOS version of the 68000 with an MMU
on board. It also has a serial port, A/D converter and a few other things
that are useful in embedded systems. It's only the MMU that's interesting for
us though. The chips is out by now, but I don't know if it is suitable for
our purpose. I assume it is.
One idea would probably be for me to give out the 68851 (68030/68040) version
of Virtue freely, that way devellopers can start supporting virtual memory.
Then I sell a 68070 board to the rest of the world. The only problem with that
is that I want to spend my time creating neat hacks, not shuffling paper. I'd
rather leave that boring part to some marketroid instead. So my quest for the
perfect jobs continues. Now I better post this:
SalesPitch(ON);
If you or your company are looking for a bright young engineer, with an
interest in digital design and the development, testing and support of
system software, then call me at
(613) 231-7474 until August 31st, or at
(514) 768-0927 on September 1st and beyong.
I'm particularly font of tasks involving operating systems, real-time
systems and OS tools. Email me, and I'll send you a resume pronto.
SalesPitch(OFF);
Boy that felt good! I should do this more often. If you're going into 4th year
and are thinking about a project yourself, I'd be glad to mail you my project
to get you started on your right foot. Another neat hack would be to implement
memory protection. You wouldn't have the compatibility problems that I do.
Although many programs will fail with memory protection because they allocate
public structures on their own stack for example, they are in clear outright
violation of system rules, unlike the ho-hum useage of Forbid().
But just like I needed the Exec source code to understand and modify some
internal things, you'll need some AmogaDOS source code to modify the
functions that start up new tasks. In a task structure, only the location
of the stack is provided, so an MMU routine can let that task access its own
stack, but it will now be able to access its data hunks. Up to you to solve
solve that. Pretty easy though. Give me an A2500 free, along with the proper
source code, and I'll do it myself!
"Gee Valentin, did you do this all by your-self?
Yes mom. An look mom, no hands! Actually I have to thank these people for
their contribution to this project:
Markus Wandel <dtsteen@dahlia.waterloo.edu>, an EE coop student at Waterloo,
disassembled the Exec and distributed it publicly. He also explained me
passages of interest which I did not fully comprehend. Markus is going on work
term in September, if you're lucky, you could still get him.
Neil Webber <palladium!nw@alliant.com>, of Epoch Systems, has worked on porting
Unix to 680x0 platforms. He gave me insight on the 68851's operation when I was
stuck.
David Buck <buck@scs.carleton.ca>, a Master's student at Carleton and BNR
employee loaned me his A2500 and served as beta tester. In the last few weeks,
I was sending him updates by modem and then testing and debugging the thing
by phone! This nice guy also wrote a ray tracer for the Amiga and placed it
into the public domain.
Ah, yes, another thing before attempting a project. Make sure you got experts
at your disposition. Virtual memory might not be here without the help of
these guys.
Well, enough rambling for now, my fingers are starting to hurt. If you can,
please post this announcement on other nets. Want technical details? Put the
questions on the net, and you'll get them. Want to get your hands on the
source code? Hire me, and it's yours!
Valentin
___________________________________________________________________________
"An operating system without Name: Valentin Pepelea
virtual memory is an operating Phonet: (613) 231-7476 until Aug 31
system without virtue." (514) 768-0927 Sept 1, 1989
Bitnet: 451061@Uottawa.bitnet
- Ancient Inca Proverb Planet: 451061@acadvm1.UOttawa.CA
Usenet: Use <cunyvm.cuny.edu> gate or
(obsolete .signature) cornellc.cit.cornell.edu gate