Details

Designing New Algorithms for Diffusion-Based Meta-Optimizers in Machine Learning

Year: 2025

Term: Winter

Student Name: Hamza Khattab

Supervisor: Junfeng Wen

Abstract: Diffusion models have achieved significant success in the field of image and video generation. However, not much work is done in other domains, leaving their potential to be largely unexplored. This is especially true in for model parameter generation, in which very few papers explore this field. We utilized an variational autoencoder to extract latent representations of model parameters before training the SDE to generate novel encodings. Our approach is also shown to effectively generate parameters using different SDEs, allowing for a wide range of experimental flexibility. In this paper, we discuss the various approaches we took to generate model parameters using a score-based SDE. Its primary purpose is to encourage discussion in this field.