Hello 👋

We develop latent diffusion models for video game audio generation. The system achieves two types of alignment: semantic alignment with text descriptions and temporal alignment with real-time video frame features. Through these alignments, we maintain long-horizon coherence over multi-second outputs.

The model serves as a complete audio companion for games, generating both immediate Foley effects and on-going soundscapes. It creates context-aware sound effects for player actions while maintaining background audio and music that responds to the gameplay situation.

Our approach works across diverse gaming genres, including RPGs, Action-Adventure, fast-paced FPS, and Racing games. Our distillation-based sampling approach reduces inference steps while preserving audio quality, enabling sub-150ms response times on consumer GPUs (30/40 series: RTX 3090, RTX 4090) for real-time local inference.

Game designers interested in validating and refining this approach are welcome to reach out, and we will soon release an API for game developers to integrate this technology into their development pipeline. In the near future, gamers will have direct access to audio customization features in supported games.

We study reasoning capabilities and systematic failure modes in vision-language models. Current work focuses on sparse autoencoders for mechanistic interpretation of internal model representations and understanding of emergent behaviors in multimodal reasoning.