[comp.lang.fortran] slow write

h3dy@vax5.cit.cornell.edu (03/22/90)

I have a large array (characters) that I read in, process,
then write out to a file.  Prior to run time, I dont know
the final number of elements - it is smaller than the original.

      write (10) array

is fast, but if the array is much smaller, I don't want to
do this.

      write (10) (array(i),i=1,n)

but an implied do is very very slow!  How can I get the
combine the advantages whilst avoiding the disadvantages?
My array is about 15000000 characters long initially.

I am embarrassed to say I tried equivalencing arrays of
different sizes, using the closest one.  Ugliness.

Thanks for the help, sorry if this appears 2x, pnews is "broken"
so I am using anunews.

sjc@key.COM (Steve Correll) (03/23/90)

In article <3673.2607e3bb@vax5.cit.cornell.edu>, h3dy@vax5.cit.cornell.edu writes:
> I have a large array (characters) that I read in, process,
> then write out to a file.  Prior to run time, I dont know
> the final number of elements - it is smaller than the original.
> 
>       write (10) array
> 
> is fast, but if the array is much smaller, I don't want to
> do this.
> 
>       write (10) (array(i),i=1,n)
> 
> but an implied do is very very slow!

Because the implied-do notation permits you to skip around within an array,
selecting non-contiguous elements, a compiler must, in the general case,
write the elements out one at a time. Some compilers recognize the special
(but frequent) case where the implied-do elements are contiguous in memory,
and can thereby economize by writing them out in one big block. Apparently
yours doesn't.

Try putting the write statement inside a subroutine and passing the array as
"character*(*)" or (for a non-character array) using an adjustable-array
declaration:

	subroutine s(c, a, n)
	character*(*) c
	real a(n)
	write(10) c
	write(11) a
	end

	...
	call s(c(1:<desired length>), a, <desired length>)
	end

There's no guarantee, but this does make it easier for the compiler to
recognize that the elements are contiguous and perform each WRITE statement
as a single operation.

Incidentally, this illustrates one reason why users may be disappointed in the
first generation of Fortran 90 compilers. If experience is a guide, it will be
some time before most compilers mature enough to recognize all the special,
optimizable cases of the new array constructs.
-- 
...{sun,pyramid}!pacbell!key!sjc 				Steve Correll

hirchert@ux1.cso.uiuc.edu (Kurt Hirchert) (03/23/90)

In article <3673.2607e3bb@vax5.cit.cornell.edu> h3dy@vax5.cit.cornell.edu writes:
>I have a large array (characters) that I read in, process,
>then write out to a file.  Prior to run time, I dont know
>the final number of elements - it is smaller than the original.
>
>      write (10) array
>
>is fast, but if the array is much smaller, I don't want to
>do this.
>
>      write (10) (array(i),i=1,n)
>
>but an implied do is very very slow!  How can I get the
>combine the advantages whilst avoiding the disadvantages?
>My array is about 15000000 characters long initially.

1. The implied DO is slow only on some machine.  Some compilers are smart
   enough to recognize that the implied DO is transferring consecutive
   storage and use the same kind of fast I/O as for the whole array
   transfer.

2. On machines where the compiler is not that good, the following trick will
   usually work:  Replace the WRITE statement above with
         CALL WRITCH(10, ARRAY, N)
   where the subroutine WRITCH is defined as follows:
         SUBROUTINE WRITCH (IUNIT, ARRAY, N)
         CHARACTER ARRAY(N)
         WRITE(IUNIT) ARRAY
         END
   The subroutine uses the whole array notation (and thus the fast I/O call),
   but the size of the array is determined at run time.  The added procedure
   call overhead makes this slower than just a write of a whole array of
   equivalent size (and thus slower than the implied DO if the compiler does
   a good job of optimization), but faster than transferring each character
   separately (which is typically what you get if the compiler _doesn't_
   do a good job of optimizing the implied DO).

   [If the original ARRAY is declared something other than CHARACTER, adjust
   the declaration in the subroutine accordingly.]
-- 
Kurt W. Hirchert     hirchert@ncsa.uiuc.edu
National Center for Supercomputing Applications

mcqueen@acf4.NYU.EDU (David M. McQueen) (03/23/90)

/* acf4:comp.lang.fortran / h3dy@vax5.cit.cornell.edu /  7:27 pm  Mar 21, 1990 */
> I have a large array (characters) that I read in, process,
> then write out to a file.  Prior to run time, I dont know
> the final number of elements - it is smaller than the original.
> 
>       write (10) array
> 
> is fast, but if the array is much smaller, I don't want to
> do this.
> 
>       write (10) (array(i),i=1,n)
> 
> but an implied do is very very slow!  How can I get the
> combine the advantages whilst avoiding the disadvantages?
> My array is about 15000000 characters long initially.
> 
> I am embarrassed to say I tried equivalencing arrays of
> different sizes, using the closest one.  Ugliness.
> 
> Thanks for the help, sorry if this appears 2x, pnews is "broken"
> so I am using anunews.
/* ---------- */

How about this:

      program main
      dimension array(large)
      .
      .
      .
      call out(array,ismall)
      .
      .
      .
      subroutine out(array,ismall)
      dimension array(ismall)
      write(10) array
      return
      end

You might want to also write out the value of ismall for the benefit of the
program which reads the file.