Jump to:  Browse | Search | Email 

Browse:  most popular entries | oldest entries | most recent entries 

Information Theory, Inference, and Learning Algorithms > Inference, probability theory > Variational methods

questions about Exercise 20.3 1) What does "data points come from a SINGLE separable 2-d Gaussian" mean? Does it mean the data is actually generated by a single Gaussian (centered at mean 0,0) that is axis aligned, i.e. has covariance matrix [(\sigma_1)^2, 0; 0 (\sigma_2)^2]? (Perhaps the interpretation of separable is then simply that the two axes are independent?) 2) In going from eq 20.12 to 20.13, what is the justification for why the denominator evaluates to 1/2? Aside: As a reader, I found it a big leap to go from eq 20.8, which operates on finite data, to eq 20.12, which operates on "the truth" (and here, P(x_1) would be Gaussian(0,(\sigma_1)^2), right?). 3) You mention that m = 0 is a fixed point (and I agree intuitively that this should be true given the underlying data generator), but is it supposed to be clear from the math you've derived so far? If so, please give me a hint as to where the math demonstrates this. 4) I'm uncertain why no derivation for the 2nd axis' mean is done. While the responsibility only depends on x_1, the algorithm would update on all dimensions. Which brings me to my final question.... 5) How were Figures 20.11 and 20.10 created? If you actually run soft kmeans to do this (which is how I interpret the solution), it seems you're back in real land (as opposed to ideal land). In this case, it seems one can't even say that 0 is the stable fixed point on dimension x_1 (the empirical mean might, for example, be 10-6). Also, in these figures are you simply ignoring behavior on the 2nd dimension (x_2)?

Solution of 20.5: Why the asymptote is the mean of the rectified Gaussian?

Showing 1 to 2 of 2 entries