[edit] The issue was that I was not training long enough.
I cloned this exact code. Just removed the keep_dims=True in line 91 and then plotted the output of the generator during training (even after a number of epochs). The generated distribution's mean and std have converged to the values of the real distribution (that's cool!) BUT the distribution (when plotted) does not look like a Gaussian. It does not look like the gaussian plot you have in your blog. Here is my plotting code. Any ideas why this happens?
if (epoch+1) % 1000 == 0:
plt.clf()
gen_input = Variable(gi_sampler(minibatch_size, g_input_size))
g_sample_data = G(gen_input)
d_fake_data_numpy = g_sample_data.data.cpu().numpy()
d_real_data_numpy = d_real_data.data.cpu().numpy()
p1, bins, patches = plt.hist(d_real_data_numpy.flatten(), 20, normed=1, facecolor='r', alpha=0.75)
p2, bins, patches = plt.hist(d_fake_data_numpy.flatten(), 20, normed=1, facecolor='b', alpha=0.75)
plt.show(True)
After 10,000 epochs it looks like this (red is a real distribution and blue is the generated). Despite the mean and std of the distribution being spot on, it doesnt take the shape/form of a Gaussian distribution:
I realized it starts to look pretty good after 60,000 runs (if it has not diverged. It sometimes does).
I was just not training long enough.