Jump to:  Browse | Search | Email 

Browse:  most popular entries | oldest entries | most recent entries 

Information Theory, Inference, and Learning Algorithms > Information Theory

What is Shannon Fano coding?

for a court case I need to prove that there is no lossless compression algorithm that effectively compresses everything. I need to cite a regular book for this statement. Does your book contain it?

is it true that information cannot arise from non-information? Someone said: "information cannot arise from disorder by chance. It always takes (greater) information to produce information, and ultimately information is the result of intelligence"

regarding the 12 ball problem (ex 4.13). Your reasoning for being able to do it in 3 tries was that a balance is able to generate three possibilities in each weighing and 3 weighings generates 27 possibilities. Since there are only 24 cases in the 12 ball problem the 27 possibilities should be able to cover the 24 cases. I would like to apply this argument to the problem of 4 balls with one odd ball that could be either lighter or heavier. According to the above counting argument we should be able to locate the odd ball (and its oddness) in two weighings. Can you tell me how to do it?

What do I need to set up a Turbo coding lab?

Page 118 Exercise 6.3. Won't there be random bit sequences that the Arithmetic coder never generates and therefore the decoder will fail on? This is because the compression is not really perfect for any finite length. For the specific example choose the random sequence to be the (infinite) binary representation of 0.99. Won't the decoder never be able to decode this ? Asymptotically as you go for longer and longer sequences this region will get smaller and smaller, but for any finite length, the code has some residual redundancy and therefore the compressor won't generate arbitrary random bits ??

How do you calculate the BER Shannon limit curve (as in "Gallager Codes - Recent Results" fig 3a, p.5)

subsection 21.2 of ITILA you say that "M is typically chosen such that [...] 2^M is is a little bigger than S - say, ten times bigger", and in the next sentence for S of about 1 million you suggest setting M to 30, which gives 2^30 ~ 10^9, which is a thousand, not ten, times bigger than S.

Usually all information bits on a channel are equally "important". Is there any _principled_ theory about encoding data of _unequal_ importance?

What is the solution to Exercise 11.6 (p.188)?

Showing 1 to 10 of 32 entries   (Next)