Denoising Diffusion From the Perspective of Two state Markov Chains and Linear Dynamical Systems
In this note, I introduce the denoising diffusion models, the state of art behind image generation platforms such as stable diffusion and Dall-E. Denoising diffusion is an approach to sampling developed in the context of generative modeling in the machine learning community by Jascha Sohl-Dickstein's group according to a quick google search. And that's about the only credit giving I have energy for in this post. The problem of sampling is to draw samples from some probability distribution $p^{data}(x)$. One instance of the problem that arises in applications is when one is agnostic to the expression $p^{data}(x)$, but has samples from this distribution. For example, you have Italian jobbed a number of Van Gogh paintings. And now you want to create your own Van Gogh style paintings. Suppose the variable $x$ lives in the set of paintings. The goal is to draw new samples from $p^{data}(x)$ based on available samples, and try as best as best possible not to generate crap lik...