roland@sics.se (Roland Karlsson) (05/21/91)
Maybe I am not so good at mach terminology or maybe I am not so good
at the English language. Hmmm. I shall try to restate my question.
I have an UNIX application. This application consists of a static
number of parallel processes. One UNIX process is (more or less)
equal to a MACH task with one thread. The UNIX processes are created
with fork, so they have the same address space. Of particular
interest are some huge stacks. Those stacks contain references
(pointers) to other elements in the same or other stacks. Sometimes
one UNIX process would like to help another one. Then it need to get
a copy of (parts of) the first process stacks. The original and the
copy have to be placed at the same addresses. For efficiency I then
use shared memory implemented with mmap and a paging file. All
processes can then map different parts of the map file as their stack
area. Those stack areas shall all be at the same addresses though.
The processes can also map other processes stack areas at arbitrary
addresses. This way two processes can help each other to copy memory
from one processes (say P) to another (say Q). Different parts of the
stack areas can be copied in parallel from P to Q without the need to
copy via any buffer. This is the most optimal way to copy I have find in
UNIX. I have tried to implement the same behavior in MACH. But without
success. It might also be possible to do better with some "virtual
copy method". Pages could (maybe) be transferred from process P to
process Q in a lazy way without making a physical copy. The physical
copy should be created when any of P and Q write in the page.
I have tried map_fd and vm_map. For you that that want too have a look
on what I have done I do include two test programs. They are both wrong.
The vm_map even more.
This is the version of MACH I use:
2.6 MSD #1.0 (on sun3)
Roland Karlsson
----- map_fd.c ----------------------------------------------------------
#include <mach.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/file.h>
int fd;
init_memory(size)
vm_size_t size;
{
int err;
fd = open("tmpfile", O_RDWR | O_CREAT, 0666);
if (fd < 0)
{ perror("open");
exit(1);
}
err = lseek(fd, size, L_SET);
if (err < 0)
{ perror("lseek");
exit(1);
}
err = write(fd, "", 1);
if (err < 0)
{ perror("write");
exit(1);
}
err = ftruncate(fd, size);
if (err < 0)
{ perror("ftruncate");
exit(1);
}
}
char *map_memory(address, size, offset)
char *address;
vm_size_t size;
vm_offset_t offset;
{
kern_return_t ret;
boolean_t anywhere = (address == NULL);
vm_offset_t *addr = (vm_offset_t *)&address;
fprintf(stderr,
"%8smap_fd(fd=%x,off=%x,addr=%x@%x,any=%x,siz=%x)\n",
"",
fd,
offset,
*addr, addr,
anywhere,
size
);
ret = map_fd(fd, /* The file */
offset, /* Offset inside memory object */
addr, /* Pointer to in/out address */
anywhere, /* TRUE=>system free to choose address */
size /* Size */
);
if (ret != KERN_SUCCESS)
{ mach_error("vm_map: ",ret);
exit(1);
}
return address;
}
/* remote_adr is the remote address for a shared area.
local_adr is the local address for a shared area.
private_adr is the local address for a private area.
*/
main()
{
char *remote_adr = NULL, *local_adr = NULL, *private_adr = NULL;
int child;
init_memory(vm_page_size*0x20);
remote_adr = map_memory(remote_adr, vm_page_size, 0);
printf("parent: mapping remote_adr at %x\n", remote_adr);
#ifdef MAP_BEFORE_FORK
#ifdef LOCAL_ANYWHERE
local_adr = NULL;
#else
local_adr = remote_adr + vm_page_size*0x10;
#endif
local_adr = map_memory(local_adr, vm_page_size, 0);
printf("child: mapping local_adr at %x\n", local_adr);
#endif
child = fork();
if (child == -1)
{ perror("fork:");
exit(1);
}
if (child != 0)
{
printf("child: Fork\n");
sleep(10);
#ifndef MAP_BEFORE_FORK
#ifdef LOCAL_ANYWHERE
local_adr = NULL;
#else
local_adr = remote_adr + vm_page_size*0x8;
#endif
local_adr = map_memory(local_adr, vm_page_size, 0);
printf("child: mapping local_adr at %x\n", local_adr);
#endif
private_adr = map_memory(private_adr, vm_page_size, vm_page_size);
printf("child: mapping private_adr at %x\n", private_adr);
printf("child: Read %3d@%-7x (Remote)%s\n",
remote_adr[0], remote_adr,
remote_adr[0]==10 ? "" : " (should be: 10)");
printf("child: Read %3d@%-7x (Local)%s\n",
local_adr[0], local_adr,
local_adr[0]==10 ? "" : " (should be: 10)");
printf("child: Read %3d@%-7x (Private)%s\n",
private_adr[0], private_adr,
private_adr[0]==0 ? "" : " (should be: 0)");
}
else
{
printf("parent: Fork\n");
#ifndef MAP_BEFORE_FORK
#ifdef LOCAL_ANYWHERE
local_adr = NULL;
#else
local_adr = remote_adr + vm_page_size*0x10;
#endif
local_adr = map_memory(local_adr, vm_page_size, 0);
printf("child: mapping local_adr at %x\n", local_adr);
#endif
private_adr = map_memory(private_adr, vm_page_size, vm_page_size);
printf("parent: mapping private_adr at %x\n", private_adr);
remote_adr[0] = 10;
printf("parent: Wrote %d@%-7x (Remote==Local)\n",
remote_adr[0],remote_adr);
private_adr[0] = 11;
printf("parent: Wrote %d@%-7x (Private)\n",
private_adr[0],private_adr);
printf("parent: Read %3d@%-7x (Remote)%s\n",
remote_adr[0], remote_adr,
remote_adr[0]==10 ? "" : " (should be: 10)");
printf("parent: Read %3d@%-7x (Local)%s\n",
local_adr[0], local_adr,
local_adr[0]==10 ? "" : " (should be: 10)");
printf("parent: Read %3d@%-7x (Private)%s\n",
private_adr[0], private_adr,
private_adr[0]==11 ? "" : " (should be: 11)");
}
exit(0);
}
-------------------------------------------------------------------------
----- vm_map.c ----------------------------------------------------------
#include <mach_init.h>
#include <mach.h>
#include <stdio.h>
#include <sys/wait.h>
#include <servers/env_mgr.h>
#include <servers/service.h>
#include <mach/message.h>
memory_object_t memory_object=MEMORY_OBJECT_NULL;
char *map_memory(address, size, offset)
char *address;
vm_size_t size;
vm_offset_t offset;
{
kern_return_t ret;
boolean_t anywhere = (address == NULL);
vm_offset_t *addr = (vm_offset_t *)&address;
fprintf(stderr,
"%8svm_map(t=%x,ad=%x@%x,sz=%x,%x,any=%x,%x,off=%x,%x,%x,%x,%x)\n",
"",
task_self_,
*addr, addr,
size,
0,
anywhere,
memory_object,
offset,
FALSE,
VM_PROT_ALL,
VM_PROT_ALL,
VM_INHERIT_SHARE
);
ret = vm_map(task_self_, /* My port */
addr, /* Pointer to in/out address */
size, /* Size */
0, /* Mask (Should it be 0xf000000f ?) */
anywhere, /* TRUE=>system free to choose address */
memory_object, /* The default external pager */
offset, /* Offset inside memory object */
FALSE, /* TRUE => Make a copy */
VM_PROT_ALL, /* Current protection (all=>no protection) */
VM_PROT_ALL, /* Max protection (all=>no protection) */
VM_INHERIT_SHARE /* Inherit properties. */
);
if (ret != KERN_SUCCESS)
{ mach_error("vm_map: ",ret);
exit(1);
}
return address;
}
char *allocate_memory(address, size)
char *address;
vm_size_t size;
{
kern_return_t ret;
boolean_t anywhere = (address == NULL);
vm_offset_t *addr = (vm_offset_t *)&address;
fprintf(stderr,
"%8svm_allocate(t=%x,ad=%x@%x,sz=%x,any=%x)\n",
"",
task_self_,
*addr, addr,
size,
anywhere
);
ret = vm_allocate(task_self_, /* My port */
addr, /* Pointer to in/out address */
size, /* Size */
anywhere /* TRUE=>system nfree to choose address */
);
if (ret != KERN_SUCCESS)
{ mach_error("vm_allocate: ",ret);
exit(1);
}
fprintf(stderr,
"%8svm_inherit(t=%x,ad=%x@%x,sz=%x,%x)\n",
"",
task_self_,
*addr, addr,
size,
VM_INHERIT_SHARE
);
ret = vm_inherit(task_self_, /* My port */
*addr, /* Pointer to in/out address */
size, /* Size */
VM_INHERIT_SHARE
);
if (ret != KERN_SUCCESS)
{ mach_error("vm_inherit: ",ret);
exit(1);
}
return address;
}
/* remote_adr is the remote address for a shared area.
local_adr is the local address for a shared area.
private_adr is the local address for a private area.
*/
main()
{
char *remote_adr = NULL, *local_adr = NULL, *private_adr = NULL;
int child;
#ifdef ALLOCATE_MEMORY
remote_adr = allocate_memory(remote_adr, vm_page_size);
#else
remote_adr = map_memory(remote_adr, vm_page_size, 0);
#endif
printf("parent: mapping remote_adr at %x\n", remote_adr);
#ifdef MAP_BEFORE_FORK
#ifdef LOCAL_ANYWHERE
local_adr = NULL;
#else
local_adr = remote_adr + vm_page_size*0x10;
#endif
local_adr = map_memory(local_adr, vm_page_size, 0);
printf("child: mapping local_adr at %x\n", local_adr);
#endif
child = fork();
if (child == -1)
{ perror("fork:");
exit(1);
}
if (child != 0)
{
printf("child: Fork\n");
sleep(10);
#ifndef MAP_BEFORE_FORK
#ifdef LOCAL_ANYWHERE
local_adr = NULL;
#else
local_adr = remote_adr + vm_page_size*0x8;
#endif
local_adr = map_memory(local_adr, vm_page_size, 0);
printf("child: mapping local_adr at %x\n", local_adr);
#endif
private_adr = map_memory(private_adr, vm_page_size, vm_page_size);
printf("child: mapping private_adr at %x\n", private_adr);
printf("child: Read %3d@%-7x (Remote)%s\n",
remote_adr[0], remote_adr,
remote_adr[0]==10 ? "" : " (should be: 10)");
printf("child: Read %3d@%-7x (Local)%s\n",
local_adr[0], local_adr,
local_adr[0]==10 ? "" : " (should be: 10)");
printf("child: Read %3d@%-7x (Private)%s\n",
private_adr[0], private_adr,
private_adr[0]==0 ? "" : " (should be: 0)");
}
else
{
printf("parent: Fork\n");
#ifndef MAP_BEFORE_FORK
#ifdef LOCAL_ANYWHERE
local_adr = NULL;
#else
local_adr = remote_adr + vm_page_size*0x10;
#endif
local_adr = map_memory(local_adr, vm_page_size, 0);
printf("child: mapping local_adr at %x\n", local_adr);
#endif
private_adr = map_memory(private_adr, vm_page_size, vm_page_size);
printf("parent: mapping private_adr at %x\n", private_adr);
remote_adr[0] = 10;
printf("parent: Wrote %d@%-7x (Remote==Local)\n",
remote_adr[0],remote_adr);
private_adr[0] = 11;
printf("parent: Wrote %d@%-7x (Private)\n",
private_adr[0],private_adr);
printf("parent: Read %3d@%-7x (Remote)%s\n",
remote_adr[0], remote_adr,
remote_adr[0]==10 ? "" : " (should be: 10)");
printf("parent: Read %3d@%-7x (Local)%s\n",
local_adr[0], local_adr,
local_adr[0]==10 ? "" : " (should be: 10)");
printf("parent: Read %3d@%-7x (Private)%s\n",
private_adr[0], private_adr,
private_adr[0]==11 ? "" : " (should be: 11)");
}
exit(0);
}
-------------------------------------------------------------------------
--
Roland Karlsson
SICS, PO Box 1263, S-164 28 KISTA, SWEDEN Internet: roland@sics.se
Tel: +46 8 752 15 40 Ttx: 812 61 54 SICS S Fax: +46 8 751 72 30