Jump to:  Browse | Search | Email 

Browse:  most popular entries | oldest entries | most recent entries 

Information Theory, Inference, and Learning Algorithms > Information Theory

What is Shannon Fano coding?

for a court case I need to prove that there is no lossless compression algorithm that effectively compresses everything. I need to cite a regular book for this statement. Does your book contain it?

What is the solution to Exercise 11.6 (p.188)?

regarding the 12 ball problem (ex 4.13). Your reasoning for being able to do it in 3 tries was that a balance is able to generate three possibilities in each weighing and 3 weighings generates 27 possibilities. Since there are only 24 cases in the 12 ball problem the 27 possibilities should be able to cover the 24 cases. I would like to apply this argument to the problem of 4 balls with one odd ball that could be either lighter or heavier. According to the above counting argument we should be able to locate the odd ball (and its oddness) in two weighings. Can you tell me how to do it?

In chapter 19 you discuss the benefits of sexual reproduction versus mutation only, and point out the smaller number of generations needed to achieve optimal fitness for the sexual case. However, it's important to mention that when using evolutionary algorithms to solve problems, what's significant (when running on a single CPU) is not the number of generations, but the number of fitness evaluations. When we compare this, there are many cases where a mutation only evolutionary strategy (AKA random mutation hill-climber) outperforms a population-based GA - see this Mitchell, Holland and Forrest paper from NIPS-93, for example: http://www.cs.unm.edu/~forrest/publications/NIPS-93.ps

What's the general formula for the ball problem?

What do I need to set up a Turbo coding lab?

is it true that information cannot arise from non-information? Someone said: "information cannot arise from disorder by chance. It always takes (greater) information to produce information, and ultimately information is the result of intelligence"

Page 118 Exercise 6.3. Won't there be random bit sequences that the Arithmetic coder never generates and therefore the decoder will fail on? This is because the compression is not really perfect for any finite length. For the specific example choose the random sequence to be the (infinite) binary representation of 0.99. Won't the decoder never be able to decode this ? Asymptotically as you go for longer and longer sequences this region will get smaller and smaller, but for any finite length, the code has some residual redundancy and therefore the compressor won't generate arbitrary random bits ??

In section 28.3 "Minimum description length" aren't you talking about minimum *message* length (Wallace and others' approach) instead of minimum *description* length (Rissanen's approach)?

Showing 1 to 10 of 34 entries   (Next)