Get started !
online LTE test
online C test

Updated or New
Site traffic (Aug-15)
GMSK modulation
Case of frequency correction burst
Solving modulation equations
One bit change
Normal duration burst

Information Theory
Multiple Access
OSI Model
Data Link layer
Word about ATM
Standard Reference
Reference books
Resources on Web
Mind Map
Perl resources
Magic MSC tool
Bar graph tool
C programming
ASCII table
Project Management
Simple Google box
HTML characters
Site traffic
Page view counter
5 6 9 , 6 5 9
another knowledge site

Deducing capacity theorem

Deducing capacity theorem [Under Information theory > Shannon's paper]

Let us combine our discussions from last two articles to deduce Channel capacity assuming white thermal noise and certain average power for both transmitted signals and noise.

From article on Rate in terms of noise entropy,

When transmitted signal and noise are independent and received signal is sum of transmitted signal and noise, rate of transmission is:

R = H(y)H(n)

If we would like to know the capacity of this (noisy) channel, we will have maximise the rate of transmission i.e.

C = Rmax = ( H(y)H(n) )max

n being independent of x, adjustable parameter in above equation is H(y). So,

C = H(y)maxH(n)

From article on Continuous symbols,

We saw that taking N as average noise power, maximum possible entropy of noise power is:

H(n) = W log(2πeN)

As seen earlier (mathematically), for achieving maximum entropy, signal should form Gaussian distribution. To achieve maximum H(y), we can control x so that y forms Gaussian distribution corresponding to addition of average noise power N and average transmitted power (say P). The maximum entropy achieve would be:

H(y)max = W log(2πe (P+N) )

Putting above values in capacity equation, we get (theorem 16, page 43):

C = W log(2πe (P+N) ) W log(2πeN)

That is:

References: A Mathematical Theory of Communication by Claude E. Shannon, An Introduction to Information Theory by John R. Pierce.

Copyright © Samir Amberkar 2010-11§

Continuous symbols « Theory Index » Another angle