LLM & Stable Diffusion Movie Generation Pipeline

Demos

🚀 Project Description:

This project highlights some of the movies autonomously generated by the Generative AI models. We use the Mistral 7B LLM to create a story and break it down into scenes. We can control the story providing textual guidance to the model. After that the Stable Diffusion model would generate a movie based on the scenes, creating a smooth transition between scenes using spherical interpolation between previous and current scene text vector embedding. There is a Google Colab Notebook provided, which can be used to customize the input and generate some other movies. Below you may find some demos generated with this script, as well as the link to the notebook.


Notebook:

Open Notebook

Links:

Github

🤖 Models:

LLMs: - Mistral 7B
Diffusion: - Stable Diffusion XL base 1.0

Story of love in September

Attack on titan Mikasa and Eren Alternative story

Child of Light

Kratos and Atreus in Egypt