09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) (05/08/91)
> From: "William K. McFadden" <bill%thd.tv.tek.com@RELAY.CS.NET> > Subject: Re: What's up with RDAT? > Date: Wed, 1 May 1991 18:35:45 GMT > > This isn't as irrational as you might think. The problem with the DCC > compression scheme is that signal degradation occurs with each > encoding. I haven't seen the details(algorithm) of the DCC compression scheme, but if it is any decient, nothing will be lost. Afterall, look at programs such as PCZIP, UUencode, compress. ALL of those programs take programs, ASCII text files, etc and shrink them. Some better then others. But no data is lost unless the file gets corrupted. And there is no difference between a CD with music recorded on it then the CD ROMS used by computers. I mean that both are simply a long string of 1's and 0's. So a well written compression routine should not have, on the average any data loss. Dave +-----------------------------------------+ | 09nilles@cuavax.dnet.cua.edu | | Fiver.Toadflax@f329.n109.z1.FIDONET.ORG | +-----------------------------------------+ "NOTICE: Due to the shortage of ROBOTS and COMPUTERS some of our workers are HUMAN and therefore will act unpredictably when abused."
ttak@uhura.cc.rochester.edu (Tim Takahashi) (05/09/91)
In article <11954@uwm.edu> 09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) writes: >> This isn't as irrational as you might think. The problem with the DCC >> compression scheme is that signal degradation occurs with each >> encoding. > >I haven't seen the details(algorithm) of the DCC compression scheme, but >if it is any decient, nothing will be lost. Afterall, look at programs >such as PCZIP, UUencode, compress. However, PKZIP et.al. have a compression efficiency of around 2:1 (maybe more. maybe less for music type data). It is my impression that the sort of data compression that the DCC will use has a reduction factor of somewhere around 20:1. The same sort of stuff is proposed for digital photography. Somehow, it doesn't quite ring true. tim
cjc@ulysses.att.com (Chris Calabrese) (05/09/91)
09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) writes: >> From: "William K. McFadden" <bill%thd.tv.tek.com@RELAY.CS.NET> >> >> This isn't as irrational as you might think. The problem with the DCC >> compression scheme is that signal degradation occurs with each >> encoding. > >I haven't seen the details(algorithm) of the DCC compression scheme, but >if it is any decient, nothing will be lost. Afterall, look at programs >such as PCZIP, UUencode, compress. ALL of those programs take programs, >ASCII text files, etc and shrink them. Some better then others. But >no data is lost unless the file gets corrupted. And there is no difference >between a CD with music recorded on it then the CD ROMS used by computers. >I mean that both are simply a long string of 1's and 0's. So a well written >compression routine should not have, on the average any data loss. > > Dave PCzip (pkzip, etc too), compress, pack, and other such data compression schemes talked about above are designed to be lossless. The rely on being able to re-encode the data in a more efficient manner. The DCC compression, on the other hand, is not designed to be lossless. In fact, the whole point of it is to _throw_out_part_of_the_signal. The theory is that the brain just can't comprehend all that's going on and you'll hear _exactly_ the same thing, even when a good chunk of the signal is nuked. It works pretty well too. Tests on similar schemes here at Bell Labs (I didn't take place in them, so this is second hand) showed that people couldn't tell the difference (no statistically significant results). I don't know if you can do lossless digital-to-digital copying, however. That depends on how you implement it, not on the underlying compression scheme. BTW, uuencode is doesn't do any compression. In fact, files are bigger afterward. Name: Christopher J. Calabrese Brain loaned to: AT&T Bell Laboratories, Murray Hill, NJ att!ulysses!cjc cjc@ulysses.att.com Obligatory Quote: ``pher - gr. vb. to schlep. phospher - to schlep light.philosopher - to schlep thoughts.''
d87parfo@odalix.ida.liu.se (Par Fornland) (05/09/91)
09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) writes: >> From: "William K. McFadden" <bill%thd.tv.tek.com@RELAY.CS.NET> >> >> This isn't as irrational as you might think. The problem with the DCC >> compression scheme is that signal degradation occurs with each >> encoding. > >I haven't seen the details(algorithm) of the DCC compression scheme, but >if it is any decient, nothing will be lost. ... >compression routine should not have, on the average any data loss. It isn't really compression in the computerized sence, but instead it is more of the kind of compression you do e.g. to the voice of a singer in rock/popmusic, i.e. amplitude compression. In DCC you compress the amplitude so that low amplitudes have higher resolution than high ones. Thus you have virtually more bits resolution than you actually save on the DCC. This is called robust quantization in Digital Communication by Simon Haykin. The distortion increases with amplitude, just like a LP. On a CD the dist. decreases with amplitude. I would suspect that in a strong passage of _very_ complex music, one may hear a difference. -- ------- Sound's good, HiFi's better but Music is the Ultimate! But I think John Cage would disagree! ("Sound's all", he'd say...) d87parfo@odalix.ida.liu.se - Telephone (Sweden)-013-260486 Par Fornland, Bjornkarrsg. 10c:10, 582 51 Linkoping, Sweden
sethb@fid.Morgan.COM (Seth Breidbart) (05/09/91)
In article <11954@uwm.edu> 09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) writes: >> From: "William K. McFadden" <bill%thd.tv.tek.com@RELAY.CS.NET> >> This isn't as irrational as you might think. The problem with the DCC >> compression scheme is that signal degradation occurs with each >> encoding. >I haven't seen the details(algorithm) of the DCC compression scheme, but >if it is any decient, nothing will be lost. I've read articles about it in the popular press (e.g. stereophile). DCC is a lossy compression scheme. It has to be. > Afterall, look at programs >such as PCZIP, UUencode, compress. ALL of those programs take programs, >ASCII text files, etc and shrink them. Some better then others. But >no data is lost unless the file gets corrupted. And sometimes they just can't compress a file at all. But at least no data is lost, so who cares? But with music, I want one second of music per second. If this current second happens to be hard to compress, I don't want to wait 3 seconds to hear it. > And there is no difference >between a CD with music recorded on it then the CD ROMS used by computers. >I mean that both are simply a long string of 1's and 0's. So a well written >compression routine should not have, on the average any data loss. ^^^^^^^ What do you mean? Either it can lose data or it can't. If it sometimes loses data, it can't make up for it by gaining data somewhere else. (How else would you "not lose data on the average"?) Seth sethb@fid.morgan.com
bill@thd.tv.TEK.COM (William K. McFadden) (05/09/91)
In article <11954@uwm.edu> 09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) writes: >I haven't seen the details(algorithm) of the DCC compression scheme, but >if it is any decient, nothing will be lost. Afterall, look at programs >such as PCZIP, UUencode, compress. ALL of those programs take programs, >ASCII text files, etc and shrink them. Some better then others. But >no data is lost unless the file gets corrupted. This is true. However, to get the high compression ratio (6:1) needed for DCC, MUSICAM compression also has to throw away information. This is done by breaking the signal into a spectrum of critical bands. Somewhere in the neighborhood of 24 to 32 half- or third-octave bands are used in most compression schemes. This correlates well to the frequency resolution of the human ear. According to psychoacoustic principles, the presence of a signal at one frequency masks the presence of another signal near that frequency, but at a lower amplitude, such that the second signal is inaudible. There are other effects, too, such as the presence of a signal masking another that occurs within a certain time. MUSICAM and the many similar compression schemes take advantage of these psychoacoustic effects to throw away the inaudible information and replace it with noise (which isn't encoded). Hence, it sounds good even though the measured SNR is as low as 15 dB. Since normal audio measurement techniques don't give a very good measure of the sound quality of these systems, designers must use listening tests to evaluate them. In the end, the inaudibility of the compression is really up to the individual listener. -- Bill McFadden Tektronix, Inc. P.O. Box 500 MS 58-639 Beaverton, OR 97077 bill@tv.tv.tek.com, {hplabs,uw-beaver,decvax}!tektronix!videovax!bill Phone: (503) 627-6920 "SCUD: Shoots Crooked, Usually Destroyed"
rshapiro@arris.com (Richard Shapiro) (05/11/91)
In article <11954@uwm.edu> 09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) writes: >I haven't seen the details(algorithm) of the DCC compression scheme, but >if it is any decient, nothing will be lost. Afterall, look at programs >such as PCZIP, UUencode, compress. No, it isn't like these compression programs. The DCC system *does* lose information, intentionally -- it throws away anything which is, in theory, inaudible; and then dynamically reallocates bits across the frequency spectrum, putting the most bits where the most audible signal is. This encode/decode process is definitely not transparent. The differences may or may not be relevant (audible) but there will be differences.
moskowit@paul.rutgers.edu (Len Moskowitz) (05/11/91)
I suspect that tape dropout will end up as one of DCC's biggest problems. With such a high data compression ratio and slow tape speed, won't the inevitable dropouts be disastrous?
rshapiro@arris.com (Richard Shapiro) (05/13/91)
In article <11995@uwm.edu> bill@thd.tv.TEK.COM (William K. McFadden) writes: >Since normal audio measurement techniques don't give a very good >measure of the sound quality of these systems, designers must use >listening tests to evaluate them. Typical audio measurement doesn't give a very complete measure of the sound quality of *any* decent high-end equipment; responsible designers should always use listening tests. I'm encouraged that Philips recognized this point in their design of DCC. But I hope they were using analog sources along with digital ones. I'm impressed if a DCC copy of a cd is indistinguishable from the cd; I'd be more impressed if a DCC copy of an lp were indistinguishable from the lp; I'd be most impressed if a mass-produced DCC made from an analog master were indistinguishable from a mass-produced lp made from the same master (minus surface noise, of course :). > In the end, the inaudibility of the >compression is really up to the individual listener. As with everything else in audio...
Jon.Fairbairn@computer-lab.cambridge.ac.uk (Jon Fairbairn) (05/13/91)
A while ago I made a posting "Why DCC should die," because I was incensed at Philips' cynical timing of their announcement to coincide with the launch of consumer RDAT in the UK. I got hardly any response, probably because no-one had heard of DCC at the time. The posting is in the archives, so I shan't write the whole thing again, but here are some salient points. 1) As has been noted, the DCC encoding (PASC) isn't lossless. The idea is that some sounds are inaudible because they are masked by others. So the masked sounds can be left out. This relies on a naive (or perhaps willful) assumption that says that what isn't normally heard will never be heard. I don't believe this. I'm sure I'm not alone in having missed something ninety nine times and then heard it on the hundredth. OK, so maybe I needed to be reading the score to hear it, but once I know what's in the score I can hear it without reading it. Did the listening tests in the design process go to such lengths? I doubt it. I hate to think what PASC does to young people learning to listen to complex orchestral music. 2) Very little has been said about reliability. The more that data is compressed the more information is lost when one bit is lost. Unlike CD, tape wears away, so the number of bits lost increases with time. I think RDAT uses more redundancy than CD, but even then tape wear will degrade the sound eventually. What happens to DCC? 3) Even heavily compressed, the DCC tapes can only record the same length as compact cassettes. If you don't mind the effects of PASC (and for a lot of music it really will be OK), then you can use it with RDAT format to get eight hour tapes. In case anyone thinks that I believe CD and RDAT are perfect, no I don't. Jon (Jon.Fairbairn@cl.cam.ac.uk) Please don't send me email if it has to go through ukc.uk -- that costs real money to receive.
09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) (05/15/91)
> From: sethb@fid.Morgan.COM (Seth Breidbart) > Subject: Re: data compression > Date: 8 May 91 22:25:26 GMT > > And sometimes they just can't compress a file at all. But at least no > data is lost, so who cares? But with music, I want one second of > music per second. If this current second happens to be hard to > compress, I don't want to wait 3 seconds to hear it. This is a matter of 'horse power' and only a real problem when actually encoding the music. Decodeing can be done faster on the same machine or just as quickly on a slower machine. > >And there is no difference > >between a CD with music recorded on it then the CD ROMS used by computers. > >I mean that both are simply a long string of 1's and 0's. So a well written > >compression routine should not have, on the average any data loss. > ^^^^^^^ > > What do you mean? Either it can lose data or it can't. If it > sometimes loses data, it can't make up for it by gaining data > somewhere else. (How else would you "not lose data on the average"?) What I mean by that is that not all the data/music is going to appear as a worst case situation where it takes at least as much space to encode as it takes up unencoded. So it will loose data when those worst case situations are encountered. But on average the music is not a worst case and then no data would be lost. Dave +-----------------------------------------+ | 09nilles@cuavax.dnet.cua.edu | | Fiver.Toadflax@f329.n109.z1.FIDONET.ORG | +-----------------------------------------+ Money Talks. Mine Only knows how to say bye.
sethb@fid.Morgan.COM (Seth Breidbart) (05/16/91)
In article <12149@uwm.edu> 09nilles%cuavax.dnet@netcon.cua.edu (Fiver Toadflax) writes: >> And sometimes they just can't compress a file at all. But at least no >> data is lost, so who cares? But with music, I want one second of >> music per second. If this current second happens to be hard to >> compress, I don't want to wait 3 seconds to hear it. > >This is a matter of 'horse power' and only a real problem when actually >encoding the music. Decodeing can be done faster on the same machine or >just as quickly on a slower machine. Not so. If a file is random (in the Kolmogorov/Chaitin sense), then it _cannot_ be compressed without loss of information. In general, better compression algorithms do take longer, but time is no guarantee. Besides, there's only a fixed amount of 'horse power' in any particular tape deck. Proof that most files cannot be compressed: There are 2**8000000 different files 1 megabyte long. There are 2**7999990-1 different files with lengths less than 1 megabyte-11 bytes. The ratio of these is approximately 0.001. Therefore, under 0.1% of all 1 megabyte files can be compressed by at least 11 bytes (which is 0.000011% compression). >What I mean by that is that not all the data/music is going to appear as >a worst case situation where it takes at least as much space to encode as >it takes up unencoded. So it will loose data when those worst case >situations are encountered. But on average the music is not a worst case >and then no data would be lost. You don't mean "on average". You mean "usually". It's not clear whether data need be lost for real music. Stuff like 1 kHz test tones wins. I don't care. Complex orchestral music, recorded live, with hall and audience dynamics, will lose bits. I don't know if I'll be able to hear the difference, but the bit loss will be there. Seth sethb@fid.morgan.com
tonyb@juliet.ll.mit.edu (05/17/91)
In article <12182@uwm.edu> sethb@fid.Morgan.COM (Seth Breidbart) writes: >... >Proof that most files cannot be compressed: >There are 2**8000000 differend files 1 megabyte long. There are >2**7999999-1 differend files with lengths less than 1 megabyte-11 bytes. >The ratio of these is approximately 0.001. Therefore, under 0.1% of >all 1 megabyte files can be compressed by at least 1 byetes (which is >0.000011% compression). I'm not going to argue with the arithmetic here, only the logic of applying it to music. Human activity (outside of the whitehouse, anyway) is *very* far from random. Our puny brains couldn't possibly process all the stimiulus it receives if there were not strong repeated elements in everything around us. In other words, I think its misleading to scare people by showing them math that implies that a 1 hour CD with 600 megabytes or so of data is coming from the state-space that includes every possible combination of 600 million random 16 bit words. Earlier in the discussion the issue of encode/decode 'horsepower' came up. If we want to assume *lots* of it (especially at the encode end), I think its reasonable to assume that redundancies in the data could be found and exploited quite handily. Lets start with something inbetween a sine-wave test signal and a live orchestral performance: say, a solo piano piece played on a MIDI-equipped keyboard. The keyboard might have (without compression) 2 or so megabytes of samples, and a MIDI playback sequence for an hour's worth of piano playing might be another 2 megabytes (just guessing -- I bet it's smaller). Add another couple of kilobytes for parameters to set up an ambience processor, and you just accomplished 150-to-1 data compression without trying too hard at all! Progressing to the "very high horsepower" realm, I can imagine a very smart encoder listening to a *real* piano performance and sorting the piano notes out from the hall ambience, and working backwards towards the MIDI file I was just talking about. Add a few cough and program-wrinkling samples, and voila, there's your 150-to-1 compression again. For that matter, using our currently available lossless file compression techniques on the MIDI and sample files would probably up this to 1000:1. OK, so this is beyond our current state of the art, and it has nothing to do with the compression algorithms that DCC will use. But it does have something to say about how non-random music is. Back in the world of current technology, even adaptive delta modulation schemes can deliver significant compression ratios on music without loss (although apparently not significant enough to get things down to DCC bandwidth). That in itself should be a pretty clear indicator that sound we want to hear comes from well within that set of ".1% of all files" that you mentioned. All this said, I'd still rather have a lossless compression scheme for DCC. Tony Berke (tonyb@juliet.ll.mit.edu)
max@uwm.UUCP (Max Hauser) (05/18/91)
In article <12182@uwm.edu>, sethb@fid.Morgan.COM (Seth Breidbart) wrote: | | ... If a file is random (in the Kolmogorov/Chaitin sense), | then it _cannot_ be compressed without loss of information. ... I completely agree with Seth in the statement above. As this discussion seems to have a certain vigor I feel I should also remind everyone that it is a *very different topic* from the original subject at hand, audio data compression, although it may appear similar. But the character of the data is different, the algorithms are different, and the fidelity criterion is very different. First, audio data are spectacularly non-"random." Moreover, and more importantly, it is not inherently important whether information is "lost." Information is supposed to be lost. (Information is lost now, with CDs, both in the initial quantization and in the subsequent bit-error mechanisms. Even more information is "lost" in analog recording, through different mechanisms.) What is important is what you can *hear*, not bits. | ... It's not clear whether data need be lost for real music. Stuff | like 1 kHz test tones wins. I don't care. Complex orchestral music, | recorded live, with hall and audience dynamics, will lose bits. | I don't know if I'll be able to hear the difference, but the bit loss | will be there. -- Seth sethb@fid.morgan.com Artificial data like sinusoids "win" in non-perceptually-based encoding algorithms, like homomorphic and linear-predictive coding. See my recent posting to this newsgroup, <11823@uwm.edu>. Such algorithms, by the way, sound terrible even with speech and are completely unsuited to music. But it is precisely material like "complex orchestral music ... with hall and audience dynamics" that the perceptually-informed audio compression methods, as in the DCC, are designed for. (These methods in fact are inefficient with artificial data like sinusoids). So you see it all depends. We are not talking about compressing some kind of "files" of the sort familiar to software workers on the Usenet and Internet, and our fidelity criterion has little to do with whether "bits" are lost (though that is itself a fascinating and separate topic, which can and surely will continue to be discussed on this newsgroup, from many perspectives, including the Kolmogorov/Chaitin sense). The genuine issue in audio is in fact precisely whether you will "be able to hear the difference." Seth mentions above that he doesn't know. Neither do I. Max Hauser {mips,philabs,pyramid}!prls!max prls!max@mips.com Copyright (c) 1991 by Max W. Hauser. All rights reserved.