StoryWeaver

AI Video Generation Platform

StoryWeaver is an AI Video Generation platform featuring an interactive node-based UI that allows users to visually script, generate, and stitch video clips together—enabling the creation of cohesive long-form AI videos. Structured as a Turborepo monorepo, the system leverages Topological Sort algorithms to map and safely resolve complex video node generation sequences. It solves the critical problem of visual consistency in AI-generated content by extracting the last frame of each clip and feeding it as context to the next generation step.

Architecture & How It Works

The project is structured as a Bun + Turborepo monorepo with four workspaces: studio (the React Flow canvas), client (the user dashboard), server (Hono API on Bun), and shared (Zod schemas + TypeScript interfaces). Users visually script video narratives as node graphs in the studio. The server resolves node dependencies via topological sort, builds context-aware prompts for Google Gemini (Veo), and chains clips together by extracting and forwarding the last frame of each clip to the next generation step—ensuring visual continuity across the entire video.

1. Iterative AI Video Pipeline (Frame Extraction)

The core innovation is the Consistency Mechanism. AI video generators produce isolated clips with no awareness of prior context. StoryWeaver solves this by extracting the last frame of each generated clip via extractLastFrame.ts and passing it as an image input to the next generation step. This forces Gemini to start the next clip exactly where the previous one ended.

// Consistency pipeline in the server
async function generateChainedVideo(nodes: SceneNode[]) {
  let previousFrame: Buffer | null = null;

  for (const node of topologicalSort(nodes)) {
    // Build prompt with optional image context
    const prompt = buildGCPVideoPrompt(node, previousFrame);
    
    // Generate clip via Gemini (Veo)
    const operation = await gemini.generateVideo(prompt);
    
    // Poll until complete, then upload to GCS
    const clip = await pollOperation(operation.id);
    await uploadToGCS(clip);
    
    // Extract last frame for next iteration
    previousFrame = await extractLastFrame(clip);
  }
}

2. Visual Scripting Engine (React Flow)

The studio workspace provides an infinite canvas powered by React Flow where users create node graphs representing video narratives. State is managed via Zustand for predictable, performant updates:

●Scene Nodes: Each node represents a scene with a prompt, duration, and style parameters. Nodes are connected via edges that define the generation order.
●Topological Resolution: The server resolves the node graph using topological sort to determine the correct execution sequence, respecting dependencies between clips.
●Real-Time Progress: Generation progress is streamed to the client via Server-Sent Events (SSE), with TanStack Query managing cache invalidation and UI updates through Zustand.
// SSE stream → Zustand store → React Flow node status update

3. Async Operation Management & Cloud Storage

Video generation is inherently asynchronous—Gemini may take minutes per clip. The server tracks every generation as an “Operation” persisted in the videoOperationsTable via Drizzle ORM (PostgreSQL). A polling mechanism checks Gemini's status endpoint; once complete, the server downloads the asset and transfers it to Google Cloud Storage (GCS) for permanent hosting. This decoupled pattern means the client never waits synchronously—it subscribes to status changes and renders completed clips as they arrive.

Tech Stack & Tools Used

Frontend & UI

React & Vite: Component-based SPA with fast HMR across both studio and client workspaces.
React Flow: Infinite canvas library powering the visual node-based scripting engine.
TanStack Router & Query: Type-safe routing and Suspense-first data fetching with cache management.
Zustand: Lightweight state management for the studio canvas and generation progress.
TailwindCSS & Radix UI: Utility-first styling with accessible, unstyled primitives.

Backend, AI & Infrastructure

Hono on Bun: Ultra-fast API framework running on the Bun runtime for the execution orchestrator.
Google GenAI (Gemini/Veo): AI video generation engine receiving context-aware prompts from the server.
Google Cloud Storage: Permanent asset hosting for generated video clips and extracted frames.
PostgreSQL & Drizzle ORM: Type-safe database layer tracking video operations, projects, and user data.
Turborepo & Bun: Monorepo orchestration with shared Zod schemas for end-to-end type safety.

Architecture Diagram

📐Monorepo Structure & Video Pipeline

[ ARCHITECTURE OVERVIEW ]

┌─────────────────────────────────────────────────────────────┐
│                    MONOREPO (Turborepo + Bun)               │
│                                                             │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌──────────────┐   │
│  │ studio  │  │ client  │  │ server  │  │   shared     │   │
│  │ Canvas  │  │ Dash    │  │ API     │  │ Zod Schemas  │   │
│  │ Engine  │  │ Board   │  │ Engine  │  │ TS Interfaces│   │
│  └────┬────┘  └────┬────┘  └────┬────┘  └──────┬───────┘   │
│       │            │            │               │           │
│       └────────────┴─────┬──────┴───────────────┘           │
│                          │                                   │
│              End-to-End Type Safety via shared/              │
└──────────────────────────┼──────────────────────────────────┘
                           │
          ┌────────────────┼────────────────┐
          ▼                ▼                ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐
│   STUDIO     │  │   CLIENT     │  │      SERVER           │
│   React Flow │  │   TanStack   │  │      Hono on Bun      │
│              │  │   Router     │  │                        │
│ Node Canvas  │  │   Dashboard  │  │  ┌──────────────────┐  │
│ ┌──────────┐ │  │   Library    │  │  │ Video Pipeline   │  │
│ │ Scene    │ │  │   Projects   │  │  │                  │  │
│ │ Nodes    │ │  │              │  │  │ 1. Receive nodes │  │
│ │   │      │ │  │              │  │  │ 2. Build prompt  │  │
│ │   ▼      │ │  │              │  │  │ 3. Call Gemini   │  │
│ │ Prompt   │ │  │              │  │  │ 4. Poll status   │  │
│ │ Nodes    │ │  │              │  │  │ 5. Extract frame │  │
│ │   │      │ │  │              │  │  │ 6. Chain next    │  │
│ │   ▼      │ │  │              │  │  │ 7. Upload GCS   │  │
│ │ Output   │ │  │              │  │  └────────┬─────────┘  │
│ │ Nodes    │ │  │              │  │           │            │
│ └──────────┘ │  │              │  │  ┌────────▼─────────┐  │
│              │  │              │  │  │  Drizzle ORM     │  │
│  Zustand     │  │  TanStack    │  │  │  PostgreSQL      │  │
│  State Mgmt  │  │  Query       │  │  │  videoOpsTable   │  │
│              │  │              │  │  └──────────────────┘  │
└──────────────┘  └──────────────┘  └──────────────────────┘

Video Generation Pipeline (Consistency Mechanism):

  [Node Graph] → buildGCPVideoPrompt.ts → Gemini (Veo)
       │                                       │
       │                                       ▼
       │                              Generated Clip N
       │                                       │
       │                          extractLastFrame.ts
       │                                       │
       │                                       ▼
       │                              Last Frame (PNG)
       │                                       │
       └──── Next Node ◄── Image Input ────────┘
                    │
                    ▼
           Gemini (Veo) generates Clip N+1
           starting from exact last frame
                    │
                    ▼
           Upload to Google Cloud Storage
                    │
                    ▼
           videoOperationsTable updated
           status: "completed"

  Result: Visually consistent, long-form AI video
  from independently generated clips

Local Execution & Testing

The monorepo uses Bun as its package manager and runtime. Turborepo orchestrates parallel builds and dev servers across all workspaces. You'll need environment variables for Gemini, GCS, and PostgreSQL configured in server/.env and studio/.env:

# Install all monorepo dependenciesbun install

# Start the entire stack (Server + Studio + Client)bun run dev

# Start individual workspacesbun run dev:server # Backend API onlybun run dev:client # Dashboard + Studio only

# Database: Push Drizzle schema to PostgreSQLcd server && bunx drizzle-kit push:pg

Testing Strategy

●Monorepo Tests: Run bun run test to execute all test suites across every workspace via Turborepo's task pipeline.
●Type Safety: Run bun run type-check to validate TypeScript across the entire monorepo, catching schema mismatches between shared, server, and client.
●Environment Validation: Ensure GEMINI_API_KEY, GCS_BUCKET_NAME, and DATABASE_URL are set before running the server to avoid silent failures.