Dr. Stefan Arnold

Research

Denoising Diffusion Models represent a category of generative models that employ a diffusion process in which data is gradually perturbed by adding random noise, and a parametrized reverse process tasked at iteratively denoising random noise into data.

Due to their fidelity and coverage (albeit with slower sampling), diffusion models are applied to a wide variety of generative modeling tasks, including image synthesis, image upscaling, image inpainting, and image-to-image translation such as style transfer.

By conditioning the denoising process on natural language text, diffusion models permit the generation of images that correspond to textual descriptions. This involves incorporating the intended meaning of a text into the latent representation of the diffusion model. However, natural language encompasses numerous linguistic formalisms, and it remains uncertain which of these formalisms contribute to the translation of text into images.

Using a diverse set of spatio-temporal techniques drawn from introspection analysis, my guiding research objective aims to disentangle and pinpoint which linguistic formalisms are embedded within the latent representations of diffusion models.