harnad@mind.UUCP (Stevan Harnad) (10/28/86)
Steven R. Jacobs (utah-cs!jacobs) of the University of Utah CS Dept has given me permission to post his contribution to defining the A/D distinction. It appears below, followed at the very end by some comments from me. [Will someone with access please post a copy to sci.electronics?] >> One prima facie non-starter: "continuous" vs. "discrete" physical processes. >I apologize if this was meant to avoid discussion of continuous/discrete >issues relating to analog/digital representations. I find it difficult >to avoid talking in terms of "continuous" and "discrete" processes when >discussing the difference between analog and digital signals. I am >approaching the question from a signal processing point of view, so I >tend to assume that "real" signals are analog signals, and other methods >of representing signals are used as approximations of analog signals (but >see below about a physicist's perspective). Yes, I realize you asked for >objective definitions. For my own non-objective convenience, I will use >analog signals as a starting point for obtaining other types of signals. >This will assist in discussing the operations used to derive non-analog >signals from analog signals, and in discussing the effects of the operations >on the mathematics involved when manipulating the various types of signals >in the time and frequency domains. > >The distinction of continuous/discrete can be applied to both the amplitude >and time axes of a signal, which allows four types of signals to be defined. >So, some "loose" definitions: > >Analog signal -- one that is continuous both in time and amplitude, so that > the amplitude of the signal may change to any amplitude at any time. > This is what many electrical engineers might describe as a "signal". > >Sampled signal -- continuous in amplitude, discrete in time (usually with > eqully-spaced sampling intervals). Signal may take on any amplitude, > but the amplitude changes only at discrete times. Sampled signals > are obtained (obviously?) by sampling analog signals. If sampling is > done improperly, aliasing will occur, causing a loss of information. > Some (most?) analog signals cannot be accurately represented by a > sampled signal, since only band-limited signals can be sampled without > aliasing. Sampled signals are the basis of Digital Signal Processing, > although digital signals are invariably used as an approximation of > the sampled signals. > >Quantized signal -- piece-wise continuous in time, discrete in amplitude. > Amplitude may change at any time, but only to discrete levels. All > changes in amplitude are steps. > >Digital signal -- one that is discrete both in time and amplitude, and may > change in (discrete) amplitude only at certain (discrete, usually > uniformly spaced) time intervals. This is obtained by quantizing > a sampled signal. > >Other types of signals can be made by combining these "basic" types, but >that topic is more appropriate for net.bizarre than for sci.electronics. > >The real distinction (in my mind) between these representations is the effect >the representation has on the mathematics required to manipulate the signals. > >Although most engineers and computer scientists would think of analog signals >as the most "correct" representations of signals, a physicist might argue that >the "quantum signal" is the only signal which corresponds to the real world, >and that analog signals are merely a convenient approximation used by >mathematicians. > >One major distinction (from a mathematical point of view) between sampled >signals and analog signals can be best visualized in the frequency domain. >A band-limited analog signal has a Fourier transform which is finite. A >sampled representation of the same signal will be periodic in the Fourier >domain. Increasing the sampling frequency will "spread out" the identical >"clumps" in the FT (fourier transform) of a sampled signal, but the FT >of the sampled signal will ALWAYS remain periodic, so that in the limit as >the sampling frequency approaches infinity, the sampled signal DOES NOT >become a "better" approximation of the analog signal, they remain entirely >distinct. Whenever the sampling frequency exceeds the Nyquist frequency, >the original analog signal can be exactly recovered from the sampled signal, >so that the two representations contain the equivalent information, but the >two signals are not the same, and the sampled signal does not "approach" >the analog signal as the sampling frequency is increased. For signals which >are not band-limited, sampling causes a loss of information due to aliasing. >As the sampling frequency is increased, less information is lost, so that the >"goodness" of the approximation improves as the sampling frequency increases. >Still, the sampled signal is fundamentally different from the analog signal. >This fundamental difference applies also to digital signals, which are both >quantized and sampled. > >Digital signals are usually used as an approximation to "sampled" signals. >The mathematics used for digital signal processing is actually only correct >when applied to sampled signals (maybe it should be called "Sampled Signal >Processing" (SSP) instead). The approximation is usually handled mostly by >ignoring the "quantization noise" which is introduced when converting a >sampled analog signal into a digital signal. This is convenient because it >avoids some messy "details" in the mathematics. To properly deal with >quantized signals requires giving up some "nice" properties of signals and >operators that are applied to signals. Mostly, operators which are applied >to signals become non-commutative when the signals are discrete in amplitude. >This is very much related to the "Heisenburg uncertainty principle" of >quantum mechanics, and to me represents another "true" distinction between >analog and digital signals. The quantization of signals represents a loss of >information that is qualitatively different from any loss of information that >occurs from sampling. This difference is usally glossed over or ignored in >discussions of signal processing. > >Well, those are some half-baked ideas that come to my mind. They are probably >not what you are looking for, so feel free to post them to /dev/null. > >Steve Jacobs > - - - - - - - - - - - - - - - - - - - - - - - - REPLY: > I apologize if this was meant to avoid discussion of continuous/discrete > issues relating to analog/digital representations. It wasn't meant to avoid discussion of continuous/discrete at all; just to avoid a simple-minded equation of C/D with A/D, overlooking all the attendant problems of that move. You certainly haven't done that in your thoughtful and articulate review and analysis. > I tend to assume that "real" signals are analog signals, and other > methods of representing signals are used as approximations of analog > signals. That seems like the correct assumption. But if we shift for a moment from considering the A or D signals themselves and consider instead the transformation that generated them, the question arises: If "real" signals are analog signals, then what are they analogs of? Let's borrow some formal jargon and say that there are (real) "objects," and then there are "images" of them under various types of transformations. One such transformation is an analog transformation. In that case the image of the object under the (analog) transformation can also be called an "analog" of the object. Is that an analog signal? The approximation criterion also seems right on the mark. Using the object/transformation/image terminology again, another kind of a transformation is a "digital" transformation. The image of an object (or of the analog image of an object) under a digital transformation is "approximate" rather than "exact." What is the difference between "approximate" and "exact"? Here I would like to interject a tentative candidate criterion of my own: I think it may have something to do with invertibility. A transformation from object to image is analog if (or to the degree that) it is invertible. In a digital approximation, some information or structure is irretrievably lost (the transformation is not 1:1). So, might invertibility/noninvertibility have something to do with the distinction between an A and a D transformation? And do "images" of these two kinds count as "representations" in the sense in which that concept is used in AI, cognitive psychology and philosophy (not necessarily univocally)? And, finally, where do "symbolic" representations come in? If we take a continuous object and make a discrete, approximate image of it, how do we get from that to a symbolic representation? > Analog signal -- one that is continuous both in time and amplitude. > Sampled signal -- continuous in amplitude, discrete in time... > If sampling is done improperly, aliasing will occur, causing a > loss of information. > Quantized signal -- piece-wise continuous in time, discrete in > amplitude. > Digital signal -- one that is discrete both in time and amplitude... > This is obtained by quantizing a sampled signal. Both directions of departure from the analog, it seems, lose information, unless the interpolations of the gaps in either time or amplitude can be accurately made somehow. Question: What if the original "object" is discrete in the first place, both in space and time? Does that make a digital transformation of it "analog"? I realize that this is violating the "signal" terminology, but, after all, signals have their origins too. Preservation and invertibility of information or structure seem to be even more general features than continuity/discreteness. Or perhaps we should be focusing on the continuity/noncontinuity of the transformations rather than the objects? > a physicist might argue that the "quantum signal" is the only > signal which corresponds to the real world, and that analog > signals are merely a convenient approximation used by mathematicians. This, of course, turns the continuous/discrete and the exact/approximate criteria completely on their heads, as I think you recognize too. And it's one of the things that makes continuity a less straightforward basis for the A/D distinction. > Mostly, operators which are applied to signals become > non-commutative when the signals are discrete in amplitude. > This is very much related to the "Heisenburg uncertainty principle" > of quantum mechanics, and to me represents another "true" distinction > between analog and digital signals. The quantization of signals > represents a loss of information that is qualitatively different from > any loss of information that occurs from sampling. I'm not qualified to judge whether this is an anolgy or a true quantum effect. If the latter, then of course the qualitative difference resides in the fact that (on current theory) the information is irretrievable in principle rather than merely in practice. > Well, those are some half-baked ideas that come to my mind. Many thanks for your thoughtful contribution. I hope the discussion will continue "baking." Stevan Harnad {allegra, bellcore, seismo, rutgers, packard} !princeton!mind!harnad harnad%mind@princeton.csnet (609)-921-7771
aweinste@Diamond.BBN.COM (Anders Weinstein) (10/29/86)
> From Stevan Harnad: > >> Analog signal -- one that is continuous both in time and amplitude. >> ... >> Digital signal -- one that is discrete both in time and amplitude... >> This is obtained by quantizing a sampled signal. > > Question: What if the >original "object" is discrete in the first place, both in space and >time? Does that make a digital transformation of it "analog"? I Engineers are of course free to use the words "analog" and "digital" in their own way. However, I think that from a philosophical standpoint, no signal should be regarded as INTRINSICALLY analog or digital; the distinction depends crucially on how the signal in question functions in a representational system. If a continuous signal is used to encode digital data, the system ought to be regarded as digital. I believe this is the case in MOST real digital systems, where quantum mechanics is not relevant and the physical signals in question are best understood as continuous ones. The actual signals are only approximated by discontinous mathematical functions (e.g. a square wave). > The image of an object >(or of the analog image of an object) under a digital transformation >is "approximate" rather than "exact." What is the difference between >"approximate" and "exact"? Here I would like to interject a tentative >candidate criterion of my own: I think it may have something to do with >invertibility. A transformation from object to image is analog if (or >>to the degree that) it is invertible. In a digital approximation, some >information or structure is irretrievably lost (the transformation >is not 1:1). > ... It's a mistake to assume that transformation from "continuous" to "discrete" representations necessarily involves a loss of information. Lots of continuous functions can be represented EXACTLY in digital form, by, for example, encoded polynomials, differential equations, etc. Anders Weinstein
harnad@mind.UUCP (Stevan Harnad) (10/29/86)
[Will someone with access post this on sci.electronics too, please?] Anders Weinstein <princeton!cmcl2!harvard!DIAMOND.BBN.COM!aweinste> has offered some interesting excerpts from the philosopher Nelson Goodman's work on the A/D distinction. I suspect that some people will find Goodman's considerations a little "dense," not to say hirsute, particularly those hailing from, say, sci.electronics; I do too. One of the subthemes here is whether or not engineers, cognitive psychologists and philosophers are talking about the same thing when they talk about A/D. [Other relevant sources on A/D are Zenon Pylyshyn's book "Computation and Cognition," John Haugeland's "Artificial Intelligence" and David Lewis's 1971 article in Nous 5: 321-327, entitled "Analog and Digital."] First, some responses to Weinstein/Goodman on A/D; then some responses to Weinstein-on-Harnad-on-Jacobs: > systems like musical notation which are used to DEFINE a work of > art by dividing the instances from the non-instances I'd be reluctant to try to base a rigorous A/D distinction on the ability to make THAT anterior distinction! > "finitely differentiated," or "articulate." For every two characters > K and K' and every mark m that does not belong to both, [the] > determination that m does not belong to K or that m does not belong > to K' is theoretically possible. ... I'm skeptical that the A/D problem is perspicuously viewed as one of notation, with, roughly, (1) the "digital notation" being all-or-none and discrete and the "analog notation" failing to be, and with (2) corresponding capacity or incapacity to discriminate among the objects they stand for. > A scheme is syntactically dense if it provides for infinitely many > characters so ordered that between each two there is a third. I'm no mathematician, but it seems to me that this is not strong enough for the continuity of the real number line. The rational numbers are "syntactically dense" according to this definition. But maybe you don't want real continuity...? > semantic finite differentiation... for every two characters > I and K' such that their compliance classes are not identical and [for] > every object h that does not comply with both, [the] determination > that h does not comply with K or that h does not comply with K' must > be theoretically possible. I hesitantly infer that the "semantics" concerns the relation between the notational "image" (be it analog or digital) and the object it stands for. (Could a distinction that so many people feel they have a good intuitive handle on really require so much technical machinery to set up? And are the different candidate technical formulations really equivalent, and capturing the same intuitions and practices?) > A symbol _scheme_ is analog if syntactically dense; a _system_ is > analog if syntactically and semantically dense. ... A digital scheme, > in contrast, is discontinuous throughout; and in a digital system the > characters of such a scheme are one-one correlated with > compliance-classes of a similarly discontinous set. But discontinuity, > though implied by, does not imply differentiation...To be digital, a > system must be not merely discontinuous but _differentiated_ > throughout, syntactically and semantically... Does anyone who understands this know whether it conforms to, say, analog/sampled/quantized/digital distinctions offered by Steven Jacobs in a prior iteration? Or the countability criterion suggested by Mitch Sundt? > If only thoroughly dense systems are analog, and only thoroughly > differentiated ones are digital, many systems are of neither type. How many? And which ones? And where does that leave us with our distinction? Weinstein's summary: >>To summarize: when a dense language is used to represent a dense domain, the >>system is analog; when a discrete (Goodman's "discontinuous") and articulate >>language maps a discrete and articulate domain, the system is digital. What about when a discrete language is used to represent a dense domain (the more common case, I believe)? Or the problem case of a dense representation of a discrete domain? And what if there are no dense domains (in physical nature)? What if even the dense/dense criterion can never be met? Is this all just APPROXIMATELY true? Then how does that square with, say, Steve Jacobs again, on approximation? -------- What follows is a response to Weinstein-on-Harnad-on-Jacobs: > Engineers are of course free to use the words "analog" and "digital" > in their own way. However, I think that from a philosophical > standpoint, no signal should be regarded as INTRINSICALLY analog > or digital; the distinction depends crucially on how the signal in > question functions in a representational system. If a continuous signal > is used to encode digital data, the system ought to be regarded as > digital. Agreed that an isolated signal's A or D status cannot be assigned, and that it depends on its relation with other signals in the "representational system" (whatever that is) and their relations to their sources. It also depends, I should think, on what PROPERTIES of the signal are carrying the information, and what properties of the source are being preserved in the signal. If the signal is continuous, but its continuity is not doing any work (has no signal value, so to speak), then it is irrelevant. In practice this should not be a problem, since continuity depends on a signal's relation to the rest of the signal set. (If the only amplitudes transmitted are either very high or very low, with nothing in between, then the continuity in between is beside the point.) Similarly with the source: It may be continuous, but the continuity may not be preserved, even by a continuous signal (the continuities may not correlate in the right way). On the other hand, I would want to leave open the question of whether or not discrete sources can have analogs. > I believe this is the case in MOST real digital systems, where > quantum mechanics is not relevant and the physical signals in > question are best understood as continuous ones. The actual signals > are only approximated by discontinous mathematical functions (e.g. > a square wave). There seems to be a lot of ambiguity in the A/D discussion as to just what is an approximation of what. On one view, a digital representation is a discrete approximation to a continuous object (source) or to a (continuous) analog representation of a (continuous) object (source). But if all objects/sources are really discontinuous, then it's really the continuous analog representation that's approximate! Perhaps it's all a matter of scale, but then that would make the A/D distinction very relative and scale-dependent. > It's a mistake to assume that transformation from "continuous" to > "discrete" representations necessarily involves a loss of information. > Lots of continuous functions can be represented EXACTLY in digital > form, by, for example, encoded polynomials, differential equations, etc. The relation between physical implementations and (formal!) mathematical idealizations also looms large in this discussion. I do not, for example, understand how you can represent continuous functions digitally AND exactly. I always thought it had to be done by finite difference equations, hence only approximately. Nor can a digital computer do real integration, only finite summation. Now the physical question is, can even an ANALOG computer be said to be doing true integration if physical processes are really discrete, or is it only doing an approximation too? The only way I can imagine transforming continuous sources into discrete signals is if the original continuity was never true mathematical continuity in the first place. (After all, the mathematical notion of an unextended "point," which underlies the concept of formal continuity, is surely an idealization, as are many of the infinitesmal and limiting notions of analysis.) The A/D distinction seems to be dissolving in the face of all of these awkward details... Stevan Harnad {allegra, bellcore, seismo, rutgers, packard} !princeton!mind!harnad harnad%mind@princeton.csnet (609)-921-7771
aweinste@Diamond.BBN.COM (Anders Weinstein) (10/31/86)
In article <20@mind.UUCP> harnad@mind.UUCP (Stevan Harnad) writes: > I suspect that some people will find Goodman's >considerations a little "dense," not to say hirsute, ... Well you asked for a "precise" definition! Although Goodman's rigor may seem daunting, there are really only two main concepts to grasp: "density", which is familiar to many from mathematics, and "differentiation". >> A scheme is syntactically dense if it provides for infinitely many >> characters so ordered that between each two there is a third. > >I'm no mathematician, but it seems to me that this is not strong >enough for the continuity of the real number line. The rational >numbers are "syntactically dense" according to this definition. But >maybe you don't want real continuity...? Quite right. Goodman mentions that the difference between continuity and density is immaterial for his purposes, since density is always sufficient to destroy differentiation (and hence "notationality" and "digitality" as well). "Differentiation" pertains to our ability to make the necessary distinctions between elements. There are two sides to the requirement: "syntactic differentiation" requires that tokens belonging to distinct characters be at least theoretically discriminable; "semantic differentiation" requires that objects denoted by non-coextensive characters be theoretically discriminable as well. Objects fail to be even theoretically discriminable if they can be arbitrarily similar and still count as different. For example, consider a language consisting of straight marks such that marks differing in length by even the smallest fraction of an inch are stipulated to belong to different characters. This language is not finitely differentiated in Goodman's sense. If, however, we decree that all marks between 1 and 2 inches long belong to one character, all marks between 3 and 4 inches long belong to another, all marks between 5 and 6 inches long belong to another, and so on, then the language WILL qualify as differentiated. The upshot of Goodman's requirement is that if a symbol system is to count as "digital" (or as "notational"), there must be some finite sized "gaps", however minute, between the distinct elements that need to be distinguished. Some examples: A score in musical notation can, if certain conventions are adopted, be regarded as a digital representation, with the score denoting any performance that complies with it. Note that although musical pitches, say, may take on a continuous range of values, once we adopt some conventions about how much variation in pitch is to be tolerated among the compliants of each note, the set of note extensions can become finitely differentiated. A scale drawing of a building, on the other hand, usually functions as an analog representation: any difference in a line's length, however fine, is regarded as denoting a corresponding difference in the building's size. If we decide to interpret the drawing in some "quantized" way, however, then it can be a digital representation. To quote Goodman: Consider an ordinary watch without a second hand. The hour-hand is normally used to pick out one of twelve divisions of the half-day. It speaks notationally [and digitally -- AW]. So does the minute hand if used only to pick out one of sixty divisions of the hour; but if the absolute distance of the minute hand beyond the preceding mark is taken as indicating the absolute time elapsed since that mark was passed, the symbol system is non-notational. Of course, if we set some limit -- whether of a half minute or one second or less -- upon the fineness of judgment so to be made, the scheme here too may become notational. I'm still thinking about your question of how Goodman's distinction relates to the intuitive notion as employed by engineers or cognitivists and will reply later. Anders Weinstein <aweinste@DIAMOND.BBN.COM>