rtidd@ccels3.mitre.org (Randy Tidd) (09/10/90)
With all these new comp.unix.* groups coming out, I hope it's appropriate to cross-post to .programmer and .internals... Anyhow, in the application i'm working on I have a series of routines that were written by someone else that do image processing (the fbm library, if you're familiar with it), including source. These routines take a file pointer as an argument, and they read the image-to-be-processed through this fp. Normally what you do is open a file and pass in a file pointer to it, or pass in stdin and pipe your image through the program through the shell. The problem is in my application, I am using a database and thus don't have files and thus I don't have file pointers either. What I *can* do is query the database for an image, and it gives me a block of memory that holds the image *exactly* as it is stored in file form. So what I have is a block of memory, having exactly the same size and contents of a file. What I have to do is pass my image processing routines a file pointer that points to this block of memory. If the routines used file *descriptors*, it wouldn't be a problem because I could just use pipes and be done with it. What I do now is query the database, get a block of memory, dump this memory to a temporary file, open the file with a file pointer, and pass the file pointer to the image processing routines. Not only is this dumb, but images can be a big as 3 megs and this is incredibly inefficient. Can anyone help me out? Randy Tidd GOOD rtidd@mwunix.mitre.org FAST #define DISCLAIM TRUE CHEAP -pick any two
cpcahil@virtech.uucp (Conor P. Cahill) (09/11/90)
In article <119609@linus.mitre.org> rtidd@ccels3.mitre.org (Randy Tidd) writes: >What I do now is query the database, get a block of memory, dump this >memory to a temporary file, open the file with a file pointer, and >pass the file pointer to the image processing routines. Not only is >this dumb, but images can be a big as 3 megs and this is incredibly >inefficient. > >Can anyone help me out? 1. best solution is to change your routines to accept a pointer to a memory area that has the image data or 2. use the pipe like you had intended and on the recieving side (the read side of the pipe file descriptor pair) use fdopen() to assign a file pointer to the file descriptor. -- Conor P. Cahill (703)430-9247 Virtual Technologies, Inc., uunet!virtech!cpcahil 46030 Manekin Plaza, Suite 160 Sterling, VA 22170
lwall@jpl-devvax.JPL.NASA.GOV (Larry Wall) (09/11/90)
In article <119609@linus.mitre.org> rtidd@ccels3.mitre.org (Randy Tidd) writes: : What I do now is query the database, get a block of memory, dump this : memory to a temporary file, open the file with a file pointer, and : pass the file pointer to the image processing routines. Not only is : this dumb, but images can be a big as 3 megs and this is incredibly : inefficient. : : Can anyone help me out? It *definitely* counts as cheating, and it's not entirely portable, but if you're desperate you can generally do something like this: #include <stdio.h> main() { char *string = "Now is the time\nfor all good men\nto come to.\n"; FILE *fake = fopen("/dev/null", "r"); char buf[512]; fake->_ptr = string; /* pointer to your memory */ fake->_cnt = strlen(string); /* length of your memory */ /* test it */ while (fgets(buf,512,fake)) { fputs(buf,stdout); } } As soon as the _cnt runs down, it resets _ptr back to _base and refills from the fd that's connected to /dev/null, so you get EOF. If this doesn't work, look in /usr/include/stdio.h to see what the fields are actually called. Like I said, it's cheating. But then again, many versions of sprintf() do the same thing in reverse. Larry
steve@wattres.UUCP (Steve Watt) (09/11/90)
In article <119609@linus.mitre.org> rtidd@ccels3.mitre.org (Randy Tidd) writes: >With all these new comp.unix.* groups coming out, I hope it's >appropriate to cross-post to .programmer and .internals... I'm not actually sure which is right, either. .programmer is probably closer. >Anyhow, in the application i'm working on I have a series of routines >that were written by someone else that do image processing (the fbm [ slurp! ] [ he doesn't have a file, but just a large block of RAM ] >Can anyone help me out? I just hacked this little piece of code together, you might find it amenable to your application: Just set f._cnt to the size of the image block. NOTE: This is terribly dependent on stdio being implemented in the "standard unix" way... Or at least how it looks on my SCO box and my DECStation. ----- ugh.c ----- #include <stdio.h> char *buf = "This is a test of the emergency broadcast system. In the event\ of a real emergency, you would never have heard this signal. Seriously.\n"; main() { FILE f; int c; f._cnt = strlen(buf); f._ptr = f._base = buf; f._flag = _IOEOF; f._file = 0; while ((c = getc(&f)) != EOF) { putchar(c); fflush(stdout); } } ----- end ----- Note also that you could just as easily have passed &f to some other function. -- Steve Watt ...!claris!wattres!steve wattres!steve@claris.com also works Don't let your schooling get in the way of your education.
roland@ai.mit.edu (Roland McGrath) (09/11/90)
The GNU C library (which I have not yet released) has `fmemopen': /* Create a new stream that refers to a memory buffer. */ extern FILE *fmemopen(char *s, size_t len, const char *modes); I believe Chris Torek's stdio for 4.4 BSD has something similar. -- Roland McGrath Free Software Foundation, Inc. roland@ai.mit.edu, uunet!ai.mit.edu!roland
thomas@uplog.se (Thomas Tornblom) (09/12/90)
In article <119609@linus.mitre.org> rtidd@ccels3.mitre.org (Randy Tidd) writes: With all these new comp.unix.* groups coming out, I hope it's appropriate to cross-post to .programmer and .internals... Anyhow, in the application i'm working on I have a series of routines that were written by someone else that do image processing (the fbm library, if you're familiar with it), including source. These routines take a file pointer as an argument, and they read the image-to-be-processed through this fp. Normally what you do is open a file and pass in a file pointer to it, or pass in stdin and pipe your image through the program through the shell. The problem is in my application, I am using a database and thus don't have files and thus I don't have file pointers either. What I *can* do is query the database for an image, and it gives me a block of memory that holds the image *exactly* as it is stored in file form. So what I have is a block of memory, having exactly the same size and contents of a file. What I have to do is pass my image processing routines a file pointer that points to this block of memory. If the routines used file *descriptors*, it wouldn't be a problem because I could just use pipes and be done with it. You can make a fp from a fd by fp = fdopen(fd, type) it would however require all the data to pass through the kernel once more. What I do now is query the database, get a block of memory, dump this memory to a temporary file, open the file with a file pointer, and pass the file pointer to the image processing routines. Not only is this dumb, but images can be a big as 3 megs and this is incredibly inefficient. Depending on the system you are using and whether you are concerned with portability it could be done by faking up an _iobuf struct (the things fp:s point at) that would hold a pointer to the block of memory and have all the rest of the members in the struct set up to some sane value. Something along the line of: (Warning this is untested and non portable) --------------- #include <stdio.h> struct _iobuf fake; FILE *getimage() { char *image; /* read the image from the db and have image point at it */ fake._ptr = fake._base = image; fake._cnt = fake._bufsiz = image_size; fake._flag = _IOREAD | _IOMYBUF; fake._file = -1; return &fake; } ---------------
reimann@uniol.UUCP (Ulf Reimann) (09/12/90)
rtidd@ccels3.mitre.org (Randy Tidd) writes: >So what I have is a block of memory, having exactly the same size and >contents of a file. What I have to do is pass my image processing >routines a file pointer that points to this block of memory. If the >routines used file *descriptors*, it wouldn't be a problem because I >could just use pipes and be done with it. Hi, one way to do it is to set up a FILE structure (see /usr/include/stdio.h for details, it's struct _iobuf) and pass it's address to the routine expecting the file pointer. I once saw that trick in some source to sscanf(). The routine creates an iobuf structure, puts the pointer to the passed string in it (for the buffer) and calls doscanf() which does the desired processing, but with a file pointer. If you have problems doing that, maybe I can write some example code for you, but I'm short on time now... Hope that helps... Ulf #include <disclaim.h> Ulf Reimann, Hummelweg 30, 2900 Oldenburg, W-Germany, phone +49 (441) 5704245 e-mail: "reimann@faramir.informatik.uni-oldenburg.de" or "reimann@uniol.uucp"
amoss@huji.ac.il (Amos Shapira) (09/12/90)
rtidd@ccels3.mitre.org (Randy Tidd) writes: >So what I have is a block of memory, having exactly the same size and >contents of a file. What I have to do is pass my image processing >routines a file pointer that points to this block of memory. If the >routines used file *descriptors*, it wouldn't be a problem because I >could just use pipes and be done with it. There are two ways: 1. Once you have a file descriptor, you can get a file pointer to it with fdopen(3s). The bad thing about a pipes-based solution is that you are limited by the size of the pipe, which is usually 4K. 2. The secnd solution, which seems to me much better, is to make what you have suggest in your subject line, i.e. create a file pointer which points to a memory location, this is how sscanf(3s) and co. works. Here is a quick hack to demonstrate what I mean: FILE * memopen (cp, len) char *cp; int len; { FILE *fp; if ((fp = (FILE *)malloc(sizeof(*fp))) == NULL) return (NULL); fp->_bufsiz = fp->_cnt = len; fp->_base = fp->_ptr = (unsigned char *)cp; fp->_flag = (short)_IOREAD _IOSTRG; return (fp); } The interpretation of fields, as far as I remember, is: _cnt: how much left to read from the buffer. _bufsiz: the buffer size set by setbuf(3s) (in the above function, might be better set to zero). _base: the pointer to the first char in the buf (used to read a new buffer). _ptr: pointer to next char to read. Refferences: Definition of the getc(3s) macro in /usr/include/stdio.h If you have access to sscanf(3s) then this is the ultimate answer. Test by reading a real file pointer (e.g. stdin) and examinning its fields between reads. >Randy Tidd GOOD >rtidd@mwunix.mitre.org FAST >#define DISCLAIM TRUE CHEAP > -pick any two Hope this helps, Amos Shapira amoss@batata.huji.ac.il
rtidd@ccels3 (Randy Tidd) (09/12/90)
Thanks for all the info posted about my "file pointer to a memory location" problem. I really got some good ideas that I hadn't come up with myself. I haven't had the time to implement them yet (<sigh> so many projects, so little time), but when I do I will be sure to post a summary of what I learned. Something I didn't mention, this code DOES has to be portable, but since I have access to the few systems that it has to port to I could always just do #ifdef's here and there. The systems it currently has to run on are: Sun 3 running SunOS 4.1 Sun3 running SunOS 4.0.3 Compaq 386 running 386/ix 2.0.2 Compaq 486 running 386/ix 2.2 Mac IIx running A/UX 1.1 Mac IIci running A/UX 2.0 Thanks to: cpcahil@virtech.uucp (Conor P. Cahill) lwall@jpl-devvax.JPL.NASA.GOV (Larry Wall) roland@ai.mit.edu (Roland McGrath) steve@wattres.UUCP (Steve Watt) thomas@uplog.se (Thomas Tornblom) amoss@huji.ac.il (Amos Shapira) Randy Tidd GOOD rtidd@mwunix.mitre.org FAST #define DISCLAIM TRUE CHEAP -pick any two
chris@mimsy.umd.edu (Chris Torek) (09/13/90)
In article <amoss.653145372@shum> amoss@huji.ac.il (Amos Shapira) writes: >FILE * >memopen (cp, len) >char *cp; >int len; >{ > FILE *fp; > > if ((fp = (FILE *)malloc(sizeof(*fp))) == NULL) > return (NULL); > > fp->_bufsiz = fp->_cnt = len; > fp->_base = fp->_ptr = (unsigned char *)cp; > fp->_flag = (short)_IOREAD _IOSTRG; > > return (fp); >} Notes: 1. SysV stdio does not have a _bufsiz field. (The buffer `size' is found by a bletcherous hack: if fp->_file is not equal to NOFILE, it uses _bufendtab[fp->_file], otherwise it assumes that the highest legal user-space address is 0x7fffffff. This is HORRIBLE.) 2. 4.[0123]BSD stdio never tests _IOSTRG. 3. _base and _ptr point to `char', not unsigned char, in many stdios. 4. 4.4BSD stdio does not have any of these fields. (The FILE structure contains the following fields: _p _r _w _flags _file _bf._base _bf._size _lbfsize _cookie _read _write _seek _close _ub._base _ub._size _up _ur _ubuf[] _nbuf[] _lb._base _lb._size _blksize _offset _p corresponds closely to _ptr, _r and _w to _cnt, and _bf to _base and _bufsiz. _flags holds similar flags to _flag, but there are differences. One of _r and _w is always 0, so that getc and putc always work. Other stdio getc's and putc's mysteriously fail if you switch from reading to writing and forget to fseek.) == GIANT CAVEAT ***WARNING*** DANGER WILL ROBINSON == ALL OF THE APPROACHES BELOW except fmemopen/funopen WILL FAIL IF THE == CODE CALLS setbuf OR setvbuf. If you want to read a memory region using the V7/4BSD stdio, try: fp->_ptr = addr; /* possibly with (unsigned char *) cast */ fp->_cnt = nbytes; fp->_flag = _IOSTRG; /* make _filbuf return error */ Under SysV, you are in trouble. You can fopen /dev/null for reading and replace _ptr and _cnt; this is likely to work. Under 4.4BSD, use fmemopen (if we put it back in) or funopen and setvbuf (which is how fmemopen is implemented). -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 405 2750) Domain: chris@cs.umd.edu Path: uunet!mimsy!chris