[net.audio] information theory tutorials

reid@Glacier.ARPA (01/23/85)

Have you ever noticed that many of the hobbyist magazines repeat a lot of
introductory stuff on a K-month cycle, where the value of K seems to
be roughly correlated to the age of the magazine. For example, Bicycling
magazine does a piece on seats about every 4 years, and Popular Photography
does a piece on how to develop your own black-and-white film about every 4
years.

Net.audio is suffering the growing pangs of a new magazine, and needs an
editorial hand (I would never dare call it "moderation"). Shall we debate
digital vs. analog? Shall we explain information theory to each other? Shall
we ask what is the best under-$300 cassette player? Sure, why not. We
faithful net.audio readers quickly learn what to read.

I can't resist, though, in the light of the recent flamage about the
theoretical limitations of digital audio and various quotes of Shannon and
Nyquist, tossing in an observation about information theory and usenet. The
various recent discussions about how it is possible to send 9600 baud
through a dialup line (and other topics equally relevant to net.audio) are
forgetting the basic definition of "information". 1000 consecutive bits of
1's are not much information. 1000 consecutive pairs of 10 bits
(10101010....) are also not much information. Neither is any repetitive
pattern. One technique used by the high-speed modems to send 10 pounds of
bits through a 5-pound pipe is to use data compression to remove redundancy
from the signal before transmission. Ordinary ascii text is highly
redundant, and can be compressed at least a factor of 2.  Modems do this
using a relatively unsophisticated statistical technique.

On the other hand, the information content of netnews is often pretty low; a
modem could just send every third article from net.audio and thereby triple
the effective baud rate without losing any real "information" in the
mathematical sense of the word.
-- 
	Brian Reid	decwrl!glacier!reid
	Stanford	reid@SU-Glacier.ARPA

newton2@ucbtopaz.CC.Berkeley.ARPA (01/24/85)

One thousand consecutive 1's aren't much information? Well, it's exactly
one kilobit of information, quite informative since it's a single message
chosen from a universe of 2E1000 possible messages assuming the most obvious
coding scheme. Sounds like the information theory tutorial should start
signing up tutees instanter.

I'd also like to hear of an example of a "high-speed" (voice circuit) modem
that actually employs the fanciful data-compression scheme alluded to
for squeezing N lbs. of info through a .5N pipe. Multilevel quadrature
amplitude modulation or phase-shift keying have proved adequate to account
for all the modem standards of which I'm aware (ranging up to 16 kB/s).

Doug Maisel