mks@cbmvax.commodore.com (Michael Sinz) (05/31/91)
*** 68040 CPU Compatibility Warning *** Now that the 68040 CPU is available, it will not be long before it is available on the Amiga. In fact, in many Amiga magazines there are already ads for 68040 CPU cards. With the 68040, you get a much faster and more powerful CPU. It has 4K of cache memory for instructions and another 4K for data. The reason these are two separate caches is so that the CPU core can access data while it is accessing instructions. (That is, it can do both *at the same time*) Just the fact that the caches are so much larger can give software that loads and then runs code some problems. However, this is not the worst case. The 68040 data cache has a mode that makes the machine run *much* faster in most cases. It is called CopyBack. CopyBack means that when a program writes data to memory, it goes into the cache but *not* into the physical RAM. That means that if something was to read that RAM and it did not go through the data cache on the 68040, it will see old data. The two main examples of this are DMA devices and the instruction reading of the CPU itself. This means that even if the instruction cache is cleared, if you write to memory and then try to execute that code it may not be in physical RAM yet and the instruction read from RAM will be the stale data. Simply put: If you have the CPU store instructions into memory that you wish to execute later, you *must* clear the caches if you wish to work with 68040 caching modes. AmigaOS 2.0 correctly clears the caches as needed after it does the LoadSeg() of a program. Applications need to do the same if they create code at run-time. One such example was the article on multiple processes in AmigaMail. What is needed is that just before the call to CreateProc() that a call to the EXEC V37 CacheClearU() function is executed. In C that would be: extern struct ExecBase *SysBase; : : : /* We have just created a fake seglist and wish to start it */ /* If we are in 2.0, call CacheClearU() before CreateProc() */ if (SysBase->LibNode.lib_Version >= 37) CacheClearU(); /* Now do the CreateProc() call... */ proc=CreateProc(... /* whatever your call is like */ ...); : : : For those of you programming in assembly: * ******************************************************************************* * * Check to see if we are running in V37 ROM or better. If so, * we want to call CacheClearU() to make sure we are safe on future * hardware such as the 68040. This section of code assumes that * a6 points at ExecBase. a0/a1/d0/d1 are trashed in CacheClearU() * cmpi.w #37,LIB_VERSION(a6) ; Check if exec is >= V37 bcs.s TooOld ; If less than V37, too old... jsr _LVOCacheClearU(a6) ; Clear the cache... TooOld: * ******************************************************************************* * The above will keep the code working pre-2.0 but will do the correct operations in 2.0. Note that while this would mean that your code could not work on a 68040 without the final 2.0 release, it is not much of a concern since pre-2.0 versions of the OS could not correctly run with the 68040 and will not work if the 68040 caches are turned on in CopyBack mode. Note that CreateProc() is not the only time this could be a problem. Whenever you create code (or load code) that is not 100% done only via LoadSeg() you will need to call CacheClearU(). Many input.device handlers have been known to allocate and copy up the handler code and then exit back to the system. These programs also need to have this call in them. The other major case that can cause problems is with DMA devices. Since DMA devices read/write data to memory directly, caches need to be flushed as needed. For example, if a DMA device was about to read RAM via DMA, it would need to call CachePreDMA() to make sure that caches have written to memory and that all is safe. If a DMA device is about to write to memory, it should call CachePreDMA() *before* the write, to make sure that no cache data is still left to be written to RAM, then do the DMA, and finally call CachePostDMA(). In addition, CachePreDMA() and CachePostDMA() gives the OS the chance to tell the DMA device that the physical addresses and memory sizes are not the same. While currently this can not happen, the future may bring such things as virtual memory and thus it would be needed. See the autodocs for more information on these calls. -- Michael Sinz Operating Systems Development Group Commodore-Amiga, Inc. /----------------------------------------------------------------------\ | /// Michael Sinz - Amiga Software Engineer | | /// Operating System Development Group | | /// BIX: msinz UUNET: rutgers!cbmvax!mks | |\\\/// | | \XX/ Quantum Physics: The Dreams that Stuff is made of. | \----------------------------------------------------------------------/
vinsci@nic.funet.fi (Leonard Norrgard) (06/01/91)
>Note that CreateProc() is not the only time this could be a >problem. Whenever you create code (or load code) that is not 100% >done only via LoadSeg() you will need to call CacheClearU(). >Many input.device handlers have been known to allocate and copy >up the handler code and then exit back to the system. These >programs also need to have this call in them. NOTE: If you use SAS/C and link with cback.o you'll have to rewrite cback.a to fix this. As it is now it is a likely crash under '040. (cback.a allocates a piece of memory and copies code to it, to be run at program exit). -- Leonard
kaba@wintermute.north.de (Kai Bartels) (06/01/91)
mks@cbmvax.commodore.com (Michael Sinz) writes: >CopyBack. CopyBack means that when a program writes data to >memory, it goes into the cache but *not* into the physical RAM. >That means that if something was to read that RAM and it did not >go through the data cache on the 68040, it will see old data. >The two main examples of this are DMA devices and the instruction >reading of the CPU itself. This means that even if the Another important case where that mode can be deadly is in a multi-processor machine. Has Motorola or C= taken any precautions in this direction. It'd be really nice to run ADos2.0 or SVR4 on two or three processors! Kai -- "I'm beat, I'm torn, Shattered and tossed and worn, Too shocking to see" <CS> BITNET: g14b@dhbrrz41 + UUCP: kaba@wintermute.north.de Snail: Kai Bartels + Hudemuehler Str. 37 + 2800 Bremen 41 + FRG
mks@cbmvax.commodore.com (Michael Sinz) (06/03/91)
In article <1991Jun1.175626.3234@wintermute.north.de> kaba@wintermute.north.de (Kai Bartels) writes: >mks@cbmvax.commodore.com (Michael Sinz) writes: >>CopyBack. CopyBack means that when a program writes data to >>memory, it goes into the cache but *not* into the physical RAM. >>That means that if something was to read that RAM and it did not >>go through the data cache on the 68040, it will see old data. >>The two main examples of this are DMA devices and the instruction >>reading of the CPU itself. This means that even if the >Another important case where that mode can be deadly is in a multi-processor >machine. Has Motorola or C= taken any precautions in this direction. It'd be >really nice to run ADos2.0 or SVR4 on two or three processors! Yes, the 68040 has support for this. It is called bus snooping. The only thing it does not do is snoop the bus between its own caches! (That is one of the bigger mistakes, along with the fact that you can only Push back the data cache *and* clear it. You can not just push it back) /----------------------------------------------------------------------\ | /// Michael Sinz - Amiga Software Engineer | | /// Operating System Development Group | | /// BIX: msinz UUNET: rutgers!cbmvax!mks | |\\\/// | | \XX/ Quantum Physics: The Dreams that Stuff is made of. | \----------------------------------------------------------------------/
dix@clinet.fi (Risto Kaivola) (06/03/91)
kaba@wintermute.north.de (Kai Bartels) writes: >mks@cbmvax.commodore.com (Michael Sinz) writes: >>CopyBack. CopyBack means that when a program writes data to >>memory, it goes into the cache but *not* into the physical RAM. >>That means that if something was to read that RAM and it did not >>go through the data cache on the 68040, it will see old data. >>The two main examples of this are DMA devices and the instruction >>reading of the CPU itself. This means that even if the >Another important case where that mode can be deadly is in a multi-processor >machine. Has Motorola or C= taken any precautions in this direction. It'd be >really nice to run ADos2.0 or SVR4 on two or three processors! >Kai I thought that Motorola had implemented bus-snooping to maintain cache-coherency. Strangely enough, this can't keep the two caches coherent with regard to each other. In the event of an external write accessing cached memory 68040 invalidates the corresponding cache entry. I remember that I read in somewhere 68040 would be able to act as the "source" of data if an external bus master (such as another processor) wanted to read cached data. Please correct me if I'm wrong. Risto >-- >"I'm beat, I'm torn, Shattered and tossed and worn, Too shocking to see" <CS> >BITNET: g14b@dhbrrz41 + UUCP: kaba@wintermute.north.de >Snail: Kai Bartels + Hudemuehler Str. 37 + 2800 Bremen 41 + FRG -- Risto Kaivola (dix@clinet.fi) or (Risto.Kaivola@f123.n220.z2.FIDONET.ORG)
grr@cbmvax.commodore.com (George Robbins) (06/04/91)
In article <1991Jun3.163847.10733@clinet.fi> dix@clinet.fi (Risto Kaivola) writes: > kaba@wintermute.north.de (Kai Bartels) writes: > > >mks@cbmvax.commodore.com (Michael Sinz) writes: > > I thought that Motorola had implemented bus-snooping to maintain > cache-coherency. Strangely enough, this can't keep the two caches > coherent with regard to each other. In the event of an external write > accessing cached memory 68040 invalidates the corresponding cache > entry. I remember that I read in somewhere 68040 would be able to > act as the "source" of data if an external bus master (such as another > processor) wanted to read cached data. Please correct me if I'm wrong. The root problem is that Motorola's mental memory model doesn't seem to include the I=D=virtual=real=supervisor=user mode that AmigaDOS requires, but is oriented towards a more general "unix" model. In some ways, the AmigaDos software model represents an arbitrary "simple machine" model that could be mapped fairly closely to the original 68000 hardware, but is still not 100% inline with the overall 680X0 family architecture as it has developed. This does not keep it from working, but makes it requires more work and sometimes compromises to take advantage of new architectural features. -- George Robbins - now working for, uucp: {uunet|pyramid|rutgers}!cbmvax!grr but no way officially representing: domain: grr@cbmvax.commodore.com Commodore, Engineering Department phone: 215-431-9349 (only by moonlite)
daveh@cbmvax.commodore.com (Dave Haynie) (06/04/91)
In article <22139@cbmvax.commodore.com> grr@cbmvax.commodore.com (George Robbins) writes: >In article <1991Jun3.163847.10733@clinet.fi> dix@clinet.fi (Risto Kaivola) writes: >> kaba@wintermute.north.de (Kai Bartels) writes: >> >mks@cbmvax.commodore.com (Michael Sinz) writes: >> I thought that Motorola had implemented bus-snooping to maintain >> cache-coherency. Strangely enough, this can't keep the two caches >> coherent with regard to each other. The caches sure don't snoop each other. Cache coherency doesn't mean that you no longer have to treat I and D space properly, it's mainly a mechanism to properly support copyback caching. >The root problem is that Motorola's mental memory model doesn't seem >to include the I=D=virtual=real=supervisor=user mode that AmigaDOS >requires, but is oriented towards a more general "unix" model. Actually, it pretty much does work as well under AmigaOS as UNIX, except for a few caveats. AmigaOS doesn't support I=D anymore than UNIX, though it's possible to write code in AmigaOS that sets I=D (of course, you CAN do just about anything in AmigaOS, since there's no MMU protection, but that doesn't make it proper). The super=user issue isn't an I-Cache issue, though it is in the 68030's data cache. The '030's Write Allocate mode is designed to handle this, and it does, but unfortunately this hits upon what we consider a bug in the data cache mechanism: a longword write to a longword port gets cached, regardless of the hardware caching mode, and will hit when read. That's not true with write allocate off, or with caching managed by the MMU. So A3000's, the only C= sponsored Amigas with 68030s and uncachable longword ports, need the MMU around to work real well (the MMU isn't the only solution, but it is the most efficient). >In some ways, the AmigaDos software model represents an arbitrary >"simple machine" model that could be mapped fairly closely to the >original 68000 hardware, but is still not 100% inline with the >overall 680X0 family architecture as it has developed. The main problem is that AmigaOS hasn't fully adopted the use of the MMU, while all modern UNIX system have. The UNIX software architecture isn't required to build a smooth 680x0 system, but the fact that it uses the MMU for everything helps smooth things out considerably, since most of the cache problems are more easily or completely solved via MMU than via hardware of some kind. -- Dave Haynie Commodore-Amiga (Amiga 3000) "The Crew That Never Rests" {uunet|pyramid|rutgers}!cbmvax!daveh PLINK: hazy BIX: hazy "This is my mistake. Let me make it good." -R.E.M.
mks@cbmvax.commodore.com (Michael Sinz) (06/04/91)
In article <1991Jun3.163847.10733@clinet.fi> dix@clinet.fi (Risto Kaivola) writes: >kaba@wintermute.north.de (Kai Bartels) writes: > >>mks@cbmvax.commodore.com (Michael Sinz) writes: >>>CopyBack. CopyBack means that when a program writes data to >>>memory, it goes into the cache but *not* into the physical RAM. >>>That means that if something was to read that RAM and it did not >>>go through the data cache on the 68040, it will see old data. >>>The two main examples of this are DMA devices and the instruction >>>reading of the CPU itself. This means that even if the >>Another important case where that mode can be deadly is in a multi-processor >>machine. Has Motorola or C= taken any precautions in this direction. It'd be >>really nice to run ADos2.0 or SVR4 on two or three processors! > >>Kai > >I thought that Motorola had implemented bus-snooping to maintain >cache-coherency. Strangely enough, this can't keep the two caches >coherent with regard to each other. In the event of an external write >accessing cached memory 68040 invalidates the corresponding cache >entry. I remember that I read in somewhere 68040 would be able to >act as the "source" of data if an external bus master (such as another >processor) wanted to read cached data. Please correct me if I'm wrong. The 68040 can do this, if the bus is always connected to it. However, the Amiga bus design is actually a set of disjoint buses. This gives the system the ability to do a BLIT while DMA in Zorro-III space and still have the CPU run code out of motherboard RAM. That also means that not all bus transactions are seen by the 68040 so bus snooping is not possible. Also, the main warning is for the people who write programs that allocate and copy up code into memory. Since, as you have stated, the 68040 can not snoop between its own two caches, you need to flush the copyback data cache to make sure that it is in memory and then clear the instruction cache to make sure that it sees the new memory (just in case it was already in the cache) The 68040 in copyback mode will break 99.99% of any code that is self-modifying and does not do the CacheClearU() call in Exec. /----------------------------------------------------------------------\ | /// Michael Sinz - Amiga Software Engineer | | /// Operating System Development Group | | /// BIX: msinz UUNET: rutgers!cbmvax!mks | |\\\/// Programming is like sex: | | \XX/ One mistake and you have to support it for life. | \----------------------------------------------------------------------/
mks@cbmvax.commodore.com (Michael Sinz) (06/04/91)
In article <22146@cbmvax.commodore.com> daveh@cbmvax.commodore.com (Dave Haynie) writes: >In article <22139@cbmvax.commodore.com> grr@cbmvax.commodore.com (George Robbins) writes: >>In article <1991Jun3.163847.10733@clinet.fi> dix@clinet.fi (Risto Kaivola) writes: >>> kaba@wintermute.north.de (Kai Bartels) writes: >>> I thought that Motorola had implemented bus-snooping to maintain >>> cache-coherency. Strangely enough, this can't keep the two caches >>> coherent with regard to each other. > >The caches sure don't snoop each other. Cache coherency doesn't mean that >you no longer have to treat I and D space properly, it's mainly a mechanism >to properly support copyback caching. The reason is that they did not implement the hardware to snoop between the two caches. The implemented the snooping for the main CPU itself and both caches go through the same external bus. In either case, there is also the much greater chance of having the instruction cache still think it has valid data for an address since it is now 4K long rather than 256 bytes. >>The root problem is that Motorola's mental memory model doesn't seem >>to include the I=D=virtual=real=supervisor=user mode that AmigaDOS >>requires, but is oriented towards a more general "unix" model. > >Actually, it pretty much does work as well under AmigaOS as UNIX, except for >a few caveats. AmigaOS doesn't support I=D anymore than UNIX, though it's >possible to write code in AmigaOS that sets I=D (of course, you CAN do just The I/D issues are just as complex for UNIX as they are for AmigaDOS. In general, Unix applications do not do things like allocate memory and build code in that memory space. However, at the UNIX kernel level, the 68040 in copyback mode has proven to cause major problems. It turns out that UNIX not only needs to do what we do in the cache work, they need to worry about this caches every time a page fault happens. If a UNIX system is very busy page-faulting (and thus swapping to disk) the copyback mode is actually more of a bother than a real speed benefit. >>In some ways, the AmigaDos software model represents an arbitrary >>"simple machine" model that could be mapped fairly closely to the >>original 68000 hardware, but is still not 100% inline with the >>overall 680X0 family architecture as it has developed. Actually, the Amiga memory model works rather well with the new processors. We do not make use of the fact that code and data memory spaces could be physically different address spaces. (Along with user and supervisor spaces this makes for 4 separate 4Gig memory maps!) Now, we have all of the memory mapped to the same place, but so do most (all?) UNIX systems. It is rather nasty when you can not access the code memory space with a data fetch/store since it makes the loading of code rather interesting and the use of data within the code imposible. >The main problem is that AmigaOS hasn't fully adopted the use of the MMU, >while all modern UNIX system have. The UNIX software architecture isn't >required to build a smooth 680x0 system, but the fact that it uses the MMU >for everything helps smooth things out considerably, since most of the >cache problems are more easily or completely solved via MMU than via hardware >of some kind. While some of the "cache this but not that" is much easier with the MMU it does not, in any way, reduce the need for cache coherency code. The MMU can not magically force data that is being loaded (for example a what LoadSeg() does) to make sure it is in the instruction cache when the code is executed or make sure that dirty data is not in the caches when it is referenced. (I really wish that were possible, but...) The 68040 (and future processors) will continue in the trend to push the I and D spaces further apart. The main reason is for system performance. (If I and D are 100% independant, you can do a fetch on both at the same time) While most system designs will not have 100% separate I and D memory subsystems, there well may be fancy RAM controllers that can feed the CPUs faster. Software will need to continue to advance in its dealings with instruction and data space issues. The Amiga OS just happens to give enough flexibility at the application level that applications have been written that will need to understand the issues of cache coherency and separate I and D caches. /----------------------------------------------------------------------\ | /// Michael Sinz - Amiga Software Engineer | | /// Operating System Development Group | | /// BIX: msinz UUNET: rutgers!cbmvax!mks | |\\\/// | | \XX/ Quantum Physics: The Dreams that Stuff is made of. | \----------------------------------------------------------------------/
bdb@becker.UUCP (Bruce D. Becker) (06/09/91)
In article <22154@cbmvax.commodore.com> mks@cbmvax.commodore.com (Michael Sinz) writes: |The 68040 (and future processors) will continue in the trend to push |the I and D spaces further apart. The main reason is for system performance. |(If I and D are 100% independant, you can do a fetch on both at the same time) |While most system designs will not have 100% separate I and D memory |subsystems, there well may be fancy RAM controllers that can feed the |CPUs faster. Software will need to continue to advance in its dealings |with instruction and data space issues. The Amiga OS just happens to |give enough flexibility at the application level that applications |have been written that will need to understand the issues of cache |coherency and separate I and D caches. The Amiga OS just happens to give so little protection at the application level that applications have been written that will need to understand the issues of cache coherency and separate I and D caches. 8^) -- ,u, Bruce Becker Toronto, Ontario a /i/ Internet: bdb@becker.UUCP, bruce@gpu.utcs.toronto.edu `\o\-e UUCP: ...!utai!mnetor!becker!bdb _< /_ "Ferget yer humanity, do the poot" - devo
daveh@cbmvax.commodore.com (Dave Haynie) (06/10/91)
In article <107299@becker.UUCP> bdb@becker.UUCP (Bruce D. Becker) writes: >In article <22154@cbmvax.commodore.com> mks@cbmvax.commodore.com (Michael Sinz) writes: >|The 68040 (and future processors) will continue in the trend to push >|the I and D spaces further apart. The main reason is for system performance. > The Amiga OS just happens to > give so little protection at the application level that applications > have been written that will need to understand the issues of cache > coherency and separate I and D caches. 8^) Unless it's trying to get away with self-modifying code, an application does not need to know about cache issues. If it is doing that kind of thing, it has the choice of "know about it" or "fail". In a protected OS, your choice should be limited to "fail", since the OS will maintain code segments as read only. So in either case, if you're doing the right thing, you have no problems and need to take no special considerations. The programs that NEED to know about cache issues are programs that are doing OS type things, such as device drivers. > ,u, Bruce Becker Toronto, Ontario -- Dave Haynie Commodore-Amiga (Amiga 3000) "The Crew That Never Rests" {uunet|pyramid|rutgers}!cbmvax!daveh PLINK: hazy BIX: hazy "This is my mistake. Let me make it good." -R.E.M.
dix@clinet.fi (Risto Kaivola) (06/12/91)
daveh@cbmvax.commodore.com (Dave Haynie) writes: >In article <107299@becker.UUCP> bdb@becker.UUCP (Bruce D. Becker) writes: >>In article <22154@cbmvax.commodore.com> mks@cbmvax.commodore.com (Michael Sinz) writes: >>|The 68040 (and future processors) will continue in the trend to push >>|the I and D spaces further apart. The main reason is for system performance. >> The Amiga OS just happens to >> give so little protection at the application level that applications >> have been written that will need to understand the issues of cache >> coherency and separate I and D caches. 8^) >Unless it's trying to get away with self-modifying code, an application does >not need to know about cache issues. If it is doing that kind of thing, it >has the choice of "know about it" or "fail". In a protected OS, your choice >should be limited to "fail", since the OS will maintain code segments as read >only. So in either case, if you're doing the right thing, you have no problems >and need to take no special considerations. While on the issue of code segments, I have a modest proposal. I hope that this hasn't been discussed before, but: Before 2.0 goes into ROM, would it be necessary for the Commodore to add a new code hunk, the type of which would be something like "pure code". This would probably benefit future operating system designs in which MMU would be used. In the current state of affairs there are a great number of programs that place data in the code hunk(s). In connection with the announcement of the new code hunk, the fact that it should contain nothing but code would be specifically stressed. Otherwise it would be exactly like the current code hunk thus making it relatively simple for Commodore to add the code to support the new hunk in the program loader (in 2.0, that is). Developers putting data in the new code hunk should get what they reserve: almost certain unfunctionality in the future operating system versions. Developers wanting to be compatible with an MMU-utilizing OS (as far as it is possible) might start to use the new code hunk. Well, I was perhaps a bit too verbose, but anyway, what do you think? >The programs that NEED to know about cache issues are programs that are doing >OS type things, such as device drivers. >> ,u, Bruce Becker Toronto, Ontario >-- >Dave Haynie Commodore-Amiga (Amiga 3000) "The Crew That Never Rests" > {uunet|pyramid|rutgers}!cbmvax!daveh PLINK: hazy BIX: hazy > "This is my mistake. Let me make it good." -R.E.M. Risto -- Risto Kaivola (dix@clinet.fi) or (Risto.Kaivola@f123.n220.z2.FIDONET.ORG)
jesup@cbmvax.commodore.com (Randell Jesup) (06/13/91)
In article <1991Jun11.231325.8198@clinet.fi> dix@clinet.fi (Risto Kaivola) writes: > Before 2.0 goes into ROM, would it be necessary for the Commodore to >add a new code hunk, the type of which would be something like "pure code". Good idea, unfortunately about 6-8 months too late for ROM. Please submit an enhancement request to the bugs group or by mail to suggestions@ cbmvax (or bugs@cbmvax). -- Randell Jesup, Jack-of-quite-a-few-trades, Commodore Engineering. {uunet|rutgers}!cbmvax!jesup, jesup@cbmvax.cbm.commodore.com BIX: rjesup Disclaimer: Nothing I say is anything other than my personal opinion. "No matter where you go, there you are." - Buckaroo Banzai