ignatz@chinet.UUCP (ignatz) (10/20/86)
Memory allocation problems exist with both Microport and the SCO Xenix compiler, kids. Just ran into it tonight when trying to get a program working that heavily depends on using sbrk(). FYI, the traditional version of sbrk() will extend the current data space by the requested byte increment, returning the old 'break', or end-of-data-address, value. This memory was contiguous, and could also be released to the OS by using a negative increment in another sbrk() call. This call, and the related brk() call, comprise the OS interface to basic memory managment that is used in more general-purpose packages such as malloc(). A common use of the direct sbrk() interface is to provide direct stack managment in such programs as the shell; you get a memory access violation on a stack push, then do an sbrk() to extend the stack (if you can), and continue processing. The Microport version of sbrk() works properly--or at least, the documentation claims it does--in the small memory configuration. HOWEVER, in the Large memory model, it's got the following exotic behavior (the following from the Microport documentation): For *any* sbrk() call with a positive increment, **all** of the rest of the addresses in the current segment are skipped, and the entire incremental allocation is provided in the *next* segment. Not only that, but you are explicitly warned that addresses will not be necessarily contiguous with previous sbrk() call values, because of this--there may be 'holes' for the skipped memory--and that return values may not necessarily have address relationships with the current break value. The Xenix System V manual is less blatant about the whole thing, but it's there, too. It states that in large model programs, if the requested increment is greater than the number of unallocated bytes remaining in the current segment, then it skips to the next segment for the entire request. While this is a bit nicer than the Microport default of always skipping, it still leaves that bloody non-contiguous window. Also, as for the Microport version, the result of sbrk(0) is not truly indicative of anything more than a marker for the previous memory location. In both cases, handling of the sbrk() call with a negative increment reflects similar deviations from what was previously considered 'normal' Unix(Tm, and all that) behavior. Now, I understand the problems inherent with dealing with a segmented architecture and memory management--I was fighting this, with a MMU, in 1981. STILL--this behavior is both a terrible deviation from the original behavior of the Unix sbrk(), and a violation of the current proposed IEEE Std. 1003.1. Now, granted, the other bible--The Sys V Interface Document, from AT&T--seems to beg the issue of brk() and sbrk() (As I remember--my SVID docs are at work right now, and I'm at home; corrections will, I am sure, be provided!!); but nevertheless, it seems obvious to me that if you're going to try and say it's Unix, then it ought to provide downward compatibility in the behavior of already-extant system services. And the standard, while not accepted, is still pretty reliable. The upshot? SCO and Microport had better get on the bandwagon and provide routines that work. I'm sorry, I know it may not be easy, but then nobody ever said it was going to be easy, did they?... -- Dave Ihnat Analysts International Corporation aicchi!ignatz or wlcrjs!ignatz (w) 882-4673 (h) 784-4544
andrew@amdahl.UUCP (Andrew Sharpe) (10/21/86)
In article <646@chinet.UUCP>, ignatz@chinet.UUCP (ignatz) writes: > Memory allocation problems exist with both Microport and the SCO Xenix > compiler, kids .... > Also, as for the Microport version, the result of sbrk(0) is > not truly indicative of anything more than a marker for the previous > memory location. This is not quite correct. The System V/286 version (for sbrk(0) in large model) will return a memory location that is the _beginning_ of the next segment, so that the pointer may be remembered and used later after an sbrk(+n). Now, Microport is supposed to be using System V/286. This is how it works in the System V/286 kernel, available from AT&T. If Microport has changed the behavior of brk() and sbrk(), that's a shame, because we worked very hard (in conjunction with AT&T) to make the memory management of SystemV/286 act (as much as possible) like the VAX. This was one of the charters of the 286 sanctioned port. -- Andrew Sharpe !{ihnp4,decwrl}!amdahl!andrew -------------------------------------------------------- The opinions expressed above do not reflect the views of the employees, nor the management, of Amdahl Corporation --------------------------------------------------------
ignatz@aicchi.UUCP (Ihnat) (10/25/86)
I recently posted a bit to the net about the apparently anomalous behavior of sbrk() under Microport Sys V and Xenix. I have, as you might guess, received a number of responses to both the primary item in my original posting, and to some incidental comments I made. Possibly the simplest item to address is the observation made in article <4004@amdahl.UUCP> by Andrew Sharpe on my comment that the value returned isn't to be considered more than a marker: This is not quite correct. The System V/286 version (for sbrk(0) in large model) will return a memory location that is the _beginning_ of the next segment, so that the pointer may be remembered and used later after an sbrk(+n). He goes on to comment that they worked quite hard to emulate the Vax behavior, (which I'm sure they did), and that he hopes that Microport didn't change anything; and I'm sure they did not. I simply repeated a caveat in the man page that warned the user not to consider this address to be something the program can use in address calculations, because it won't be meaningful in terms of contiguous memory address calculations with previous values returned by calls to sbrk(). In light of this fact, no program with claims to portability should consider the returned value anything more than a "magic cookie" to be passed to subsequent sbrk(+/- n) calls. Now, as to the other responses I've received via mail...I'd truly forgotten how rapidly people can type; often long before the brain can engage. There's some line noise somewhere between the brain and fingers of some people that translates into sarcastic and abusive aspersions on the intelligence, experience, and professional competence of the target (me). Now, mind you, I did receive intelligent, well-written queries that simply asked if I knew the problems inherent in a memory architecture implemented on a segmented machine, with a memory managment unit that doesn't lend itself to a flat address space. I just want some of the apparently technically competent people who sent the *other* type of response to be aware that you can question someone else's conclusions or statements without turning it into a contest of egos or a professional challenge. Now...back to issues. Yes, I *do* know the problems with the 286, and segmented architecture, and the screwy memory managment addressing. I know just how hard it is/would be to recreate the flat address space that the traditional sbrk() attempted to provide. However, it can be done, and must be if the portable operating system standard is to be followed. No, every reference need not be checked; the huge model Microsoft compiler generates inline code to build the segment/offset reference on the fly, and it's not "500 times slower" than the corresponding small model that assumes that references are within the current 64K segment. It's more inefficient, certainly; but it works well enough to use. I definitely think that the 286 architecture could have used some more thought when it was designed--say, put those two mode bits in the high- order bits of the segment, instead of the low end, and then provide a memory incrementing scheme that allowed the offset increment overflow to bump the associated segment; you could then easily have modelled a flat contiguous address space. But working with what we've got, it would make sense to offer a version that either follows the Xenix model--allocates increments in the current segment until the request would go off the end-- or jumps to a new segment, but guarantees that all addresses from the old break value to the new segment are actually mapped to valid real memory. It would then be the responsibility of the compiler, or the programmer, not to fall into the 64K offset wraparound trap; but at least you could count on contiguous addresses. (Yes, I know--contiguous meaning you go from offset 0-64K, then increment the upper 14 bits of the segment while preserving the two mode bits...) I didn't really want to go into a discussion of the whole issue; I assumed that anyone reading the original posting would realize that I understood the issues, and was pointing out that the model, as implemented, changed the basic assumptions of the "classic" Unix brk()/sbrk() behavior; and, particularly, clobbers the traditional shell stack allocation method. Oh, well...I'll individually answer the people who were either kind enough to write decent letters, or were hasty enough to write flames. But this is my "technical" explanation. I don't intend to say anything more on the net about this; please mail to me if you wish to express any other opinions or ideas. Of course, if something truly significant is brought up, I'll summarize, but otherwise, this is really an issue that's probably run its course in terms of net discussion. -- Dave Ihnat Analysts International Corporation (312) 882-4673 ihnp4!aicchi!ignatz || ihnp4!homebru!ignatz
tim@ism780c.UUCP (Tim Smith) (10/25/86)
sbrk has to behave funny because of program that assume a linear
address space. Many program that want N bytes do the following:
char * ptr;
ptr = sbrk(0);
brk(ptr+N);
To make these work correctly, sbrk(0) has to return the start of
the next segment.
For example, here is how brk and sbrk work on the AT&T pc6300+ (
which is a good indication of how AT&T thinks things should work
on 80286 based machines )
( all byte counts and addresses are rounded to a multiple
of 512 )
In small model, everything looks like a normal unix system
In large model,
sbrk(0) returns start of next data segment
sbrk(N) N > 0 allocates N bytes from next data segment
sbrk(N) N < 0 does this:
N = -N
while ( N >= size of last segment )
N = N - size of last segment
remove last segment
remove N bytes from last segment
brk(addr) ( where addr == byte N of segment S )
if addr in current last segment, then set
segment size to N
if addr in next segment, then same as sbrk(N)
if addr in an allocated part of a previous segment
then throw away everything above addr
else error
--
member, all HASA divisions POELOD ECBOMB
--------------
^-- Secret Satanic Message
Tim Smith USENET: sdcrdcf!ism780c!tim Compuserve: 72257,3706
Delphi or GEnie: mnementhtony@wldrdg.UUCP (Tony Andrews) (10/27/86)
From Tim Smith sdcrdcf!ism780c!tim > sbrk has to behave funny because of program that assume a linear > address space. Many program that want N bytes do the following: > > char * ptr; > ptr = sbrk(0); > brk(ptr+N); > > To make these work correctly, sbrk(0) has to return the start of > the next segment. Unfortunately, malloc wasn't fixed to properly deal with the screwy behavior of brk and sbrk. For small malloc requests, malloc asks for a 1K chunk and allocates from that. Each time it decides to grab another 1K it gets it from a NEW segment. The net effect is that malloc squanders segments and can run out of memory because of a lack of LDT entries before it even comes close to hitting other limits. Tony Andrews ...!ihnp4!onecom!wldrdg!tony Wildridge Consulting, Inc. Boulder, CO
dan@prairie.UUCP (Daniel M. Frank) (10/29/86)
>Unfortunately, malloc wasn't fixed to properly deal with the >screwy behavior of brk and sbrk. For small malloc requests, >malloc asks for a 1K chunk and allocates from that. Each time it >decides to grab another 1K it gets it from a NEW segment. The >net effect is that malloc squanders segments and can run out >of memory because of a lack of LDT entries before it even comes >close to hitting other limits. In the System V/286 port done by Intel, and further ported by Microport and AT&T (SV/AT and 6300+ Unix), there is an additional malloc library (described in malloc(3x)) which may be used instead of the usual malloc stuff. This library includes a routine called mallopt(), which allows the following parameters to be set: M_MXFAST Allocate all blocks below the size of maxfast in large groups. M_NLBLKS Each large group allocated will contain numlblks. M_GRAIN The sizes of all blocks smaller than maxfast are considered to be rounded up to the nearest multiple of grain. M_KEEP Emulates behaviour of old malloc() by keeping the data in a block intact until that block is used again. By judicious use of these calls, the segment usage behaviour, and most likely the speed, of memory allocation under SV/286 can be significantly enhanced. -- Dan Frank uucp: ... uwvax!prairie!dan arpa: dan%caseus@spool.wisc.edu