AI-Powered Content Generation Pipeline

Problem

Creating consistent YouTube content across multiple channels requires scriptwriting, narration, video editing, thumbnail creation, and upload scheduling — work that typically takes a content team. The goal was to automate the entire pipeline from topic selection to published video.

Solution

Built an end-to-end content automation pipeline that manages 6 YouTube channels across different niches, with 228+ videos produced and uploaded. The pipeline generates scripts using Claude, converts them to narrated audio via Edge TTS (322+ voices), assembles videos with stock footage and Ken Burns effects using FFmpeg, and uploads on a scheduled basis through the YouTube Data API.

The system runs as a scheduled batch process — twice daily — handling the full pipeline from script to upload without manual intervention. Semantic topic deduplication ensures the system never produces a video too similar to one it’s already made, even when the topic is phrased differently. OAuth resilience with pre-flight token health checks prevents upload failures from expired credentials.

Architecture

Topic Sourcing → Script Generation → TTS Narration → Video Assembly → Upload
      │                  │                  │               │            │
  Google Trends    Claude Sonnet        Edge TTS        FFmpeg      YouTube API
  Reddit/RSS      Niche Prompts       322+ Voices     Ken Burns     OAuth 2.0
  GNews API       JSON Output         Timing Sync     Crossfades    Scheduling

Each channel has its own YAML configuration: niche topics, voice selection (rate/pitch), visual templates, and upload schedule. The orchestrator coordinates all steps, tracks pipeline results, and manages a throttled backlog drain system that processes queued videos at a sustainable rate to avoid API rate limits.

Key Decisions

Per-channel isolation. Each channel runs independently with its own content queue, OAuth credentials, and scheduling. If one channel fails, the others continue uninterrupted.

Semantic topic deduplication. Early versions used exact-match title checking, which missed semantically similar topics (“The Fall of the Roman Empire” vs “Why Rome Collapsed”). The pipeline now uses semantic similarity scoring to detect conceptual overlap, preventing repetitive content even when topics are phrased differently. This is critical for maintaining channel quality at scale.

OAuth resilience with pre-flight health checks. YouTube’s API requires OAuth tokens that expire, and a failed upload after spending compute on script generation, TTS, and video assembly wastes the entire pipeline run. Pre-flight token health checks validate credentials before starting any work. If a token is expired or revoked, the system refreshes or flags it before burning resources on content that can’t be uploaded.

Throttled backlog drain. When channels accumulate a backlog (holidays, API outages, new channel ramp-up), the system drains it at a controlled rate rather than dumping dozens of videos at once. This avoids YouTube’s spam detection heuristics and maintains a natural publishing cadence.

Edge TTS over paid services. Microsoft Edge TTS provides 322+ free voices with customizable rate and pitch. This eliminates per-video API costs that would make the pipeline uneconomical at scale.

Hardware-accelerated encoding. FFmpeg uses macOS VideoToolbox for H.264 encoding, significantly reducing render times compared to software encoding.

Results

6 YouTube channels across different niches (AI/tech, finance, geopolitics, history, legal, ambient)
228+ videos produced and uploaded
Semantic topic deduplication preventing conceptual duplicates across the entire video history
OAuth resilience with pre-flight token health checks before every pipeline run
Throttled backlog drain system for sustainable upload pacing
AI script generation via Claude with niche-specific prompt engineering
Analytics tracking integration for monitoring video performance
Topic sourcing from Google Trends, Reddit, RSS feeds, and GNews
Thumbnail generation with Pillow
Scheduled twice-daily batch processing (6 AM and 4 PM EST)

How This Scales

Quality gates — Pre-upload validation checking audio levels, duration targets, and metadata completeness. Videos that fail get flagged for review instead of auto-uploading.
Analytics feedback loop — YouTube Analytics API integration to track which topics/thumbnails perform best, feeding insights back into topic sourcing.
A/B thumbnail testing — Generate multiple thumbnail variants and rotate to optimize click-through rates.
Multi-platform distribution — Extend uploads to TikTok, Instagram Reels, and Shorts from the same source content.
Client service — Package the pipeline as a white-label service for content creators who want automated channels in their niche.

Tech Stack

Orchestration: Python, pipeline orchestrator with result tracking
Content: Claude API (script generation), niche-specific prompt engineering
Deduplication: Semantic similarity scoring (topic-level, not just title match)
Audio: Edge TTS (322+ voices), FFmpeg audio concatenation
Video: FFmpeg with VideoToolbox HW acceleration, Pillow (thumbnails)
Visuals: Pexels, Pixabay, Wikimedia Commons stock fetchers
Upload: YouTube Data API v3, OAuth 2.0 with pre-flight health checks
Analytics: YouTube Analytics integration for performance tracking
Infrastructure: Scheduled twice-daily batch processing, Mac Mini