[comp.sys.amiga] PIPE:... should it buffer at all?

peter@sugar.UUCP (Peter da Silva) (08/27/87)

It occurred to me that the most efficient way to handle PIPE: is to use the
writer's buffer to hold the data while waiting for the reader. This, of course,
would make all requests wait until the reader is ready... but Amiga PIPE:s
don't act like UNIX pipes anyway. It would allow any buffer size to be
transferred as an atomic operation.

Alternatively, how about making PIPE: act like the UNIX pipe? That is, you
can open it and write as much as you please without waiting until you fill
it up. You can have multiple writers but only one reader. The pipe hangs
around as long as a single writer *or* reader wants it. And finally you can
reopen it without losing data.
-- 
-- Peter da Silva `-_-' ...!seismo!soma!uhnix1!sugar!peter
--                  U   <--- not a copyrighted cartoon :->

dillon@CORY.BERKELEY.EDU.UUCP (08/31/87)

>It occurred to me that the most efficient way to handle PIPE: is to use the
>writer's buffer to hold the data while waiting for the reader. This, of course,
>would make all requests wait until the reader is ready... but Amiga PIPE:s
>don't act like UNIX pipes anyway. It would allow any buffer size to be
>transferred as an atomic operation.

	This is exactly how my PIPE:, and probably other people's PIPE's
work.  Each PIPE has an associated buffer.  When there is enough free space
in the buffer, write requests are copied and returned immediately.  If there
is not enough free space, write requests are queued until  (A) enough free
space accumulates in the buffer or (B) a read request is big enough not only
to drain the buffer, but to eat at parts of pending write requests as well.

	The major problem all of us 'PIPE writers have been experiencing is
attempting to handle programs which open the same file (pipe) several times
to handle C open modes before finally settling down on one file handle.

				-Matt

michael@stb.UUCP (Michael) (08/31/87)

In article <571@sugar.UUCP> peter@sugar.UUCP (Peter da Silva) writes:
>It occurred to me that the most efficient way to handle PIPE: is to use the
>writer's buffer to hold the data while waiting for the reader. This, of course,
>would make all requests wait until the reader is ready... but Amiga PIPE:s
>don't act like UNIX pipes anyway. It would allow any buffer size to be
>transferred as an atomic operation.

Please, no. We already have any size atomic operations--the order of the
operations are specified by the order that the messages come in.

>Alternatively, how about making PIPE: act like the UNIX pipe? That is, you
>can open it and write as much as you please without waiting until you fill
>it up. You can have multiple writers but only one reader. The pipe hangs
>around as long as a single writer *or* reader wants it. And finally you can
>reopen it without losing data.

Gee, thats how I thought PIPE: worked. Besides, I like P:, which can have
multiple readers.

-- 
: Michael Gersten		seismo!scgvaxd!stb!michael
: Copy protection? Just say Pirate! (if its worth piratill t

peter@sugar.UUCP (09/01/87)

>>It occurred to me that the most efficient way to handle PIPE: is to use the
>>writer's buffer to hold the data while waiting for the reader. This, of course,
>>would make all requests wait until the reader is ready... but Amiga PIPE:s
>>don't act like UNIX pipes anyway. It would allow any buffer size to be
>>transferred as an atomic operation.
> 
>	This is exactly how my PIPE:, and probably other people's PIPE's
>work.  Each PIPE has an associated buffer...

Which means that's not how pipes work. What I suggested was not having any
buffer in the pipe structure at all, but instead using the buffer provided
by the writer in its message. That way if the program used an 8K buffer, it
would automatically transfer 8K. If the program used 256 byte buffers, it
would transfer 256 bytes. The writer would always wait for the reader to read
the entire buffer, and data would be transferred straight from the writer
to the reader. This would save memory and time, at the cost of making the
writer wait on small writes. It's certainly worth looking into, anyway.
-- 
-- Peter da Silva `-_-' ...!seismo!soma!uhnix1!sugar!peter
--                  U   <--- not a copyrighted cartoon :->

peter@sugar.UUCP (Peter da Silva) (09/03/87)

In article <92@stb.UUCP>, michael@stb.UUCP (Michael) writes:
> In article <571@sugar.UUCP> peter@sugar.UUCP (Peter da Silva) writes:
> >It occurred to me that the most efficient way to handle PIPE: is to use the
> >writer's buffer to hold the data while waiting for the reader.
> 
> Please, no. We already have any size atomic operations--the order of the
> operations are specified by the order that the messages come in.

Really? That seems to imply a certain lack of parallelism. I guess it's OK.

But that doesn't address the efficiency question.

If you're writing an 8K buffer into a 2K PIPE:, it takes 4 transfers,
with 4 sets of:

	Copy 2K to pipe....
	Copy 2K to reader....

Note that the following is not related to the previous suggestion.

> >Alternatively, how about making PIPE: act like the UNIX pipe? That is, you
> >can open it and write as much as you please without waiting until you fill
       ^^^^^^^^^^^
> >it up. You can have multiple writers but only one reader. The pipe hangs
                       ^^^^^^^^^^^^^^^^
> >around as long as a single writer *or* reader wants it. And finally you can
> >reopen it without losing data.
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> Gee, thats how I thought PIPE: worked.

Nope. If you open it, you currently wait for the reader to open it too. You
can also only have one writer. And if the writer closes it and the reader
closes it it goes away even if there's still data in it. This makes it
useless as a printer buffer (the obvious application for it: you open
PIPE:spool and write your data. Have a program read from PIPE:spool and
write to PRT:. Make PIPE:spool 16K long. Instant buffer.)

> Besides, I like P:, which can have multiple readers.

I find it hard to fathom the use of multiple readers to a single pipe. Perhaps
you can describe an application you've used this feature for.

> : Copy protection? Just say Pirate! (if its worth pirating)

How are people supposed to take you seriously if you say things like this?
-- 
-- Peter da Silva `-_-' ...!seismo!soma!uhnix1!sugar!peter
--                  U   <--- not a copyrighted cartoon :->

michael@stb.UUCP (Michael) (09/07/87)

In article <628@sugar.UUCP> peter@sugar.UUCP (Peter da Silva) writes:
>> >Alternatively, how about making PIPE: act like the UNIX pipe? That is, you
>> >can open it and write as much as you please without waiting until you fill
>       ^^^^^^^^^^^
>> >it up. You can have multiple writers but only one reader. The pipe hangs
>                       ^^^^^^^^^^^^^^^^
>> >around as long as a single writer *or* reader wants it. And finally you can
>> >reopen it without losing data.
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> 
>> Gee, thats how I thought PIPE: worked.
>
>Nope. If you open it, you currently wait for the reader to open it too. You
>can also only have one writer. And if the writer closes it and the reader
>closes it it goes away even if there's still data in it. This makes it
>useless as a printer buffer (the obvious application for it: you open
>PIPE:spool and write your data. Have a program read from PIPE:spool and
>write to PRT:. Make PIPE:spool 16K long. Instant buffer.)

Okay, lets look at unix pipes. I assume you mean Named Fifo's (pipe files
in file space vs. pipe(2) sys call). 
#1. 4K buffer--after that, you wait until someone else reads it.
#2. The system I have makes the writer wait until a reader shows up.
#3. Both multiple readers and writers are allowed (remember: The dup() call
will work for anything in unix, and fork implicitly dups every file
descriptor).
#4. You cannot reopen without losing data.

Which system are you thinkin of? 4.3? (I'm using sys3). If I understand
correctly, that system does not guarantee any atomicy of writes to pipes.

>> Besides, I like P:, which can have multiple readers.
>
>I find it hard to fathom the use of multiple readers to a single pipe. Perhaps
>you can describe an application you've used this feature for.

Look at the "tee" command in unix. P: will send identical copies of data to
a file, such as a CON: window, or another pipe (to another program), so
one program's output will go to two destination programs.

>> : Copy protection? Just say Pirate! (if its worth pirating)
>
>How are people supposed to take you seriously if you say things like this?
>-- 
>-- Peter da Silva `-_-' ...!seismo!soma!uhnix1!sugar!peter
>--                  U   <--- not a copyrighted cartoon :->

Gee, I suppose very. Seriously, I am strongly opposed to copy protection and
"machine takeovers"; I will not willingly buy any program with either. I
bought "Hex" thinking it wasn't protected, but I see now I need a better test
for protected before I buy. But, I have changed my .signature now. And when
I learn enough 68000 and amiga assembly, I will change it again.

			Michael.

-- 
: Michael Gersten		seismo!scgvaxd!stb!michael
: Copy protection? Just say "Off site backup"