Riffusion

State‑of‑the‑art generative AI model used to generate spectrogram images of music given a text prompt. These spectrograms can be converted into audio clips.

Generates high resolution spectrograms images of music from text prompts using a latent diffusion model. This model uses CLIP ViT‑L/14 as text encoder, U‑Net based latent denoising, and VAE based decoder to generate the final image.