Signals, data and information

Signals, data and information#

In this section, we will summarize description methods for the amount of data and information, present in signals.

Amount of data#

The amount of data \(D\) that is received on the retrieval on a value between \(a\) and \(b\) of a random variable \(\underline{x}\) at time instant \(t\) is defined as

\[D=-\log_{2}\left\{ \operatorname*{Prob}(a\leq\underline{x}\leq b,t)\right\} ~\text{[bit].}\]

The probability of the occurrence of a signal value between \(a\) and \(b\) can be calculated from the probability density function \(P(\underline{x},t)\) of the random variable \(\underline{x}\) at time instant \(t\):

\[\operatorname*{Prob}(a\leq\underline{x}\leq b,t)=\int_{a}^{b}P(\underline{x} ,t)dx~\text{[-].}\]

Since the probability of obtaining a signal value is unity, we have, by definition:

\[\int_{-\infty}^{\infty}P(\underline{x},t)dx=1~\text{[-].}\]

An example of a uniform probability density function is shown in Fig. 528. A uniform probability density function is characterized by its mean value \(\mu\ \)and its width \(w=\underline{x}_{\max }-\underline{x}_{\min}:\)

\[\begin{split}P(\underline{x},t) & =\frac{1}{w}~\text{if}~\left\vert \underline{x} -\mu\right\vert <\frac{w}{2},\\ P(\underline{x},t) & =0~\text{if}~\left\vert \underline{x}-\mu\right\vert \geq\frac{w}{2}.\end{split}\]

The mean value or the expectation of a random variable is defined in section Time average and ensemble average.

Fig. 529 shows two examples of a Gaussian probability density function. A Gaussian probability density function is characterized by its mean value \(\mu\ \)and its standard deviation \(\sigma:\)

\[P(\underline{x},t)=\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{ -\frac{\left( \underline{x}-\mu\right) ^{2}}{2\sigma^{2}}\right\} .\]

Bandwidth and minimum sample rate#

The power spectral density of a signal, is a measure for the frequency contents of a signal. It is defined in section Power spectral density. The bandwidth \(B\) of a signal is the width of the frequency range in which signal components can be found. The minimum required sample rate \(S\), also the Nyquist[51] rate, is two times the bandwidth of the signal:

\[S\geq2B~\text{[1/s].}\]

Crest factor#

The crest factor \(C\) is defined as the ratio of the maximum absolute value and the RMS value of a signal:

\[C=\frac{\left\vert x\right\vert _{\max}}{x_{RMS}}.\]

Data rate#

The data rate is the amount of data contained in a signal over a time span of one second:

\[R=2B\log_{2}\frac{x_{pp}}{n_{RMS}}\ \text{[bit/s],}\]

where \(n_{RMS}\) is the RMS value of the noise associated with the signal over the bandwidth of \(B\) [Hz].

In the following example, we will evaluate the data rate of an analog signal perturbed by noise.

Example

Consider and analog signal perturbed by noise. The signal is contained in a frequency band with a bandwidth \(B=100\)kHz. The RMS value of the signal \(x_{pp}\) in this frequency band equals \(2\)V. Random noise in the same frequency band is added to the signal. The RMS value \(n_{RMS}\ \)of this noise is \(0.3\)mV. The data rate \(R\) of this signal is found as

\[R=2B\log_{2}\frac{x_{pp}}{n_{RMS}}=2\times10^{5}\log_{2}\frac{2}{0.3\times10^{-3}}=2.54\times10^{6}\text{bit/s.}\]
../_images/colorCode.svg

In the next example, we will evaluate the data rate of a stereo digital audio signal.

Example

The signal comprises the audio data of two channels. The update rate is \(44\)ksps and the number of bits per sample is \(16\). The data rate \(R\) equals:

\[R=2\times16\times44\times10^{3}=1.41\times10^{6}\text{bit/s.}\]
../_images/colorCode.svg

Information rate#

The information rate \(I\) [bits/s] is often much less than the data rate. Nowadays, when using high quality audio compression techniques, the data rate of a stereo audio signal of \(1.41\)Mbit/s can be reduced to \(256\)kbit/s without noticeable errors. The data rate for speech can be much less. In general, the data rate can be reduced if:

  1. Not all data is relevant

  2. Data is redundant

Relevant signal properties#

From the above, it seems important to know which signal properties (data) should be preserved for retrieval of the information. A few examples of signals with specific signal properties are:

  • FM and PM signals:

  • Information is embedded in the momentary frequency or in the momentary phase of the signal

  • Analog composite video signal:

  • Intensity information is embedded in the signal level

  • Color information is embedded in the phase of a carrier with respect to reference burst during the black level.

Channel capacity#

According to Shannon,[10] the maximum amount of information that can be transported per second with an arbitrarily low number of errors over a linear channel with:

  • Channel bandwidth \(B\)

  • Added white Gaussian noise with power \(N\)

  • Maximum signal power level \(S\).

    is limited by the so-called channel capacity \(C\):

    \[C=B\log_{2}\left( 1+\frac{S}{N}\right) ~\text{[bit/s].}\]

This expression clearly shows the three fundamental physical limitations to the amount of information that can be processed and that have been introduces above:

  • Noise limitation: any physical system adds noise

  • Power limitation: the power of any physical signal is limited

  • Speed limitation: the rate of change of any physical signal is limited

Spectral efficiency#

The spectral efficiency \(E\) is a measure for the information rate \(I\) per unit of bandwidth. It is defined as

\[E=\frac{I}{B}~\text{[(bit/s)/Hz].}\]
Contents