https://shadowscythe03.github.io/tags/generative-ai/Generative AIHugo Blox Builder (https://hugoblox.com)en-usWed, 01 Jan 2025 00:00:00 +0000https://shadowscythe03.github.io/media/logo_hu_f3e4bd98189c9234.pnghttps://shadowscythe03.github.io/tags/generative-ai/https://shadowscythe03.github.io/projects/storyforge/Wed, 01 Jan 2025 00:00:00 +0000https://shadowscythe03.github.io/projects/storyforge/<p>StoryForge turns a handful of keywords into a complete narrated video — story, scene images, voice narration, and compiled MP4 — through a 7-stage generative pipeline with a Streamlit interface. Supports English, Hindi, French, German, Spanish, Italian, and Telugu.</p> <h2 id="pipeline-architecture">Pipeline Architecture</h2> <p>Each stage is a dedicated Python module; they can be run individually or end-to-end through the Streamlit UI.</p> <table> <thead> <tr> <th>#</th> <th>Stage</th> <th>Module</th> <th>Model / Tool</th> </tr> </thead> <tbody> <tr> <td>1</td> <td>Story generation</td> <td><code>story_generation.py</code></td> <td>Llama 3 via Ollama</td> </tr> <tr> <td>2</td> <td>Scene prompt extraction</td> <td><code>prompt_generator.py</code></td> <td>Llama 3</td> </tr> <tr> <td>3</td> <td>Image synthesis</td> <td><code>image_generator.py</code></td> <td>Stable Diffusion 2.1 + LoRA</td> </tr> <tr> <td>4</td> <td>Consistency verification</td> <td><code>consistancy_check.py</code></td> <td>BLIP + CLIP</td> </tr> <tr> <td>5</td> <td>Audio narration</td> <td><code>audio_generator.py</code></td> <td>gTTS</td> </tr> <tr> <td>6</td> <td>Video compilation</td> <td><code>movie_creater.py</code></td> <td>MoviePy</td> </tr> <tr> <td>7</td> <td>Interactive interface</td> <td><code>streamlit.py</code></td> <td>Streamlit</td> </tr> </tbody> </table> <p><strong>Character consistency</strong> across scenes is enforced with SameFace_Fix LoRA fine-tuning on the diffusion model. <strong>Semantic alignment</strong> between generated images and story prompts is measured using CLIP similarity scores, averaging &gt;25.0 in test runs.</p> <h2 id="technical-stack">Technical Stack</h2> <ul> <li><strong>LLM</strong>: Llama 3 (Ollama) for story and prompt generation</li> <li><strong>Image generation</strong>: Stable Diffusion 2.1, custom LoRA fine-tuning, VAE</li> <li><strong>Vision-language</strong>: BLIP (captioning), CLIP (semantic similarity)</li> <li><strong>Audio</strong>: gTTS with automatic language detection</li> <li><strong>Video</strong>: MoviePy (image transitions + audio sync, MP4/AVI output)</li> <li><strong>Framework</strong>: PyTorch, HuggingFace Transformers</li> <li><strong>UI</strong>: Streamlit with real-time progress tracking and ZIP export</li> </ul> <h2 id="demo-video">Demo Video</h2> <!-- TODO: paste your GitHub blob URL for the video here and convert to raw: blob URL: github.com/shadowscythe03/projects/blob/main/path/to/video.mp4 raw URL: raw.githubusercontent.com/shadowscythe03/projects/main/path/to/video.mp4 Then replace YOUR_RAW_VIDEO_URL below. --> <p><em>Demo video coming soon.</em></p> <!-- Uncomment once you have the raw URL: <video width="100%" controls style="border-radius: 8px; margin: 1rem 0;"> <source src="YOUR_RAW_VIDEO_URL" type="video/mp4"> Your browser does not support the video tag. </video> --> <h2 id="project-report">Project Report</h2> <!-- TODO: Upload your project report PDF and replace the src URL below. Recommended: upload as a GitHub Release asset on this repo, then use: https://docs.google.com/viewer?url=<raw-pdf-url>&embedded=true --> <p><em>Project report coming soon.</em></p> <!-- TODO: once your PDF is hosted, replace the line above with: <iframe src="https://docs.google.com/viewer?url=YOUR_PDF_URL_HERE&embedded=true" width="100%" height="850px" style="border: 1px solid #e5e7eb; border-radius: 8px; display: block; margin: 1rem 0;"> </iframe> [Download PDF](YOUR_PDF_URL_HERE) --> <hr> <p><strong>Course</strong>: ES667 — Deep Learning, IIT Gandhinagar (Jan–Apr 2025) · 4-member team · Supervised by Prof. Anirban Dasgupta</p>