Architecture Overview

Brief to Bison Launch
in One Command

GTM Orchestrator is the pipeline glue that takes a campaign brief from "new campaign" to "ready for human checkpoint before launch." Every intermediate CSV and state is inspectable on disk and in Sheets throughout.

19
Stage Runners
12
External Tools
28
Skills
6
List Sources
6
Checkpoints
6 List Sources Feed Stage 1
Every campaign starts with a brief. The brief's list_source field determines how leads enter the pipeline.
🔍
DiscoLike
Lookalike discovery from customer seed domains
📍
Google Maps
Local SMB scraping for geo-targeted campaigns
📋
UCC Direct
Government filing data — MCA leads, business liens
💼
LinkedIn Jobs
Hiring signal companies via Apify actor scraping
📢
LinkedIn Ads
Ad-spend signal companies from Ad Library
📄
Custom CSV
Clutch, G2, ProductHunt, BBB, follower lists, scrapers
19 Stage Runners, 6 Phases
Each stage is a Python class extending StageRunner. Stages write to SQLite, mirror to Sheets, and track via GitHub Issues.
Phase 1 — Discovery + Normalization
01
🔍
Discover
Routes by list_source — DiscoLike, GMaps, UCC, LinkedIn Jobs/Ads, or Custom CSV. 3 stage classes handle routing.
02a
📐
Clay Format
Normalize columns to Clay-compatible schema. Maps arbitrary column names, deduplicates.
02b
🧪
Clay Enrich
Clay enrichment pass — website derivation, firmographics, company metadata.
02c
🔗
Clay Join Back
Merge Clay enrichment results back into the main data table.
02.5
🛡️
Domain MX Prefilter
Block security-gateway domains (Mimecast, Proofpoint, Barracuda) before wasting ICP qualification tokens.
Phase 2 — Qualification + Company Enrichment
03
🎯
ICP Qualify
AI qualification via ICP prompt + optional DiscoLike validate/discogen signal extraction. Filters non-ICP leads.
AI
03b
🏢
Enrich Company
Account-level enrichments after qualify — tech stack, funding, recent news, buying signals. Runs via Trigger.dev or Python fallback.
Phase 3 — People Finding + Email Waterfall
04
👤
People Find
AI Ark finds decision-maker contacts matching brief's buyer titles. Returns name + LinkedIn URL.
04b
💧
Email Waterfall
5-step waterfall: Supabase cache → MillionVerifier validate → Kitt parallel find → Icypeas → LeadMagic. Only resultcode 1 (ok) = sendable.
04c
✉️
MX Filter
MX record validation + mail provider detection + provider classification. Blocks invalid domains.
VALIDATE
04d
📇
Enrich Contact
Contact-level enrichments after MX filter — LinkedIn scrape, role signals. Trigger.dev or Python fallback.
Phase 4 — Segmentation + Copy Application
06
📊
Segment
AI segmentation with confidence threshold. Low-confidence leads → tiebreak CSV checkpoint for human review.
CHECKPOINT
07
✍️
Apply Copy
Renders copy variables from brief's _copy.json against real lead data. Copy was generated at briefing time — this is deterministic application.
07b
🎨
Personalize
Optional per-lead AI personalization — situation_line, value_line, cta_soft. 10-lead approval loop, budget gate (default 500 leads).
OPTIONAL
Phase 5 — QA + Scorecard + Launch
08
🔬
QA Suite
3 sub-stages: rendered semantic QA (AI), mechanical checks (spam + syntax), variable conversion ({{DOUBLE}} → {SINGLE}).
CHECKPOINT
SC
📋
Scorecard
8-dimension list quality gate. Grade F = hard block (exit 7). Grade D = checkpoint. Grade C+ = proceed to launch.
GATE
09
🚀
Bison Launch
7 sub-stages: create → copy upload → inbox selection → leads → route → standards → human gate. Exit 5 at 9c (inboxes) and 9g (launch).
HARD GATE
9a
Create campaigns
9b
Upload copy
9c
Inbox selection ⏸️
9d
Upload leads
9e
Route leads
9f
Standards check
9g
Human gate 🛑
How Data Moves Through the System
Every stage reads its input, transforms, writes to SQLite + Sheets, and hands off to the next stage.
Brief
List Source
Clay Normalize
MX Prefilter
ICP Qualify
Company Enrich
People Find
Email Waterfall
MX Filter
Contact Enrich
Segment
Tiebreak ⏸️
Apply Copy
Personalize
QA Suite
Scorecard Gate
Bison Launch
Human Gate 🛑
Input
Processing
AI-Powered
Checkpoint
Gate
Output

Every stage writes to all four observability channels simultaneously.

🗄️
SQLite
Source of truth — campaign.db per campaign. Every stage writes rows.
📊
Google Sheets
Live mirror via gws CLI. Human-readable window into pipeline state.
🏷️
GitHub Issues
Per-stage label lifecycle: running → complete / checkpoint / failed.
🔭
Langfuse
LLM observability — campaign + stage spans, token tracking, cost attribution.
🗃️
Supabase
Enrichment persistence — campaign_contacts, enrichments tables. Cross-campaign intelligence.
12 Integrated Services
Python 3.12 orchestrates everything. All LLM calls route through lg-llm-runtime — never direct API calls.
🔍 DiscoLike Discovery
Lookalike company discovery from seed domains. Seed = client's customers/targets, not client's own domain.
🏛️ AI Ark Discovery
Company + people search. Global Bun CLI. People search has no emails — use waterfall post-discovery.
🧪 Clay Enrichment
Data normalization + enrichment. 29-command CLI across 13 groups. Handles column mapping, dedup, website derivation.
📧 Bison / EmailBison Delivery
Campaign management + email sending. Workspace 3 (LeadGrow). npm global CLI. 3-email sequence standard.
Trigger.dev Enrichment
Durable TypeScript enrichment workflows per slug. Python fallback when unavailable. Two CLIs: trigger-dev-pp-cli + lg-trigger.
🗃️ Supabase Persistence
Enrichment persistence + campaign contacts tables. Per-row upsert, reconnect pattern. Cross-campaign intelligence layer.
📊 Google Sheets Mirror
Live mirror of SQLite state via gws CLI. Auth: mitchell@leadgrow.ai. Human-readable pipeline window.
🏷️ GitHub Issues State Machine
Per-stage issues with label-based lifecycle. 15-label vocabulary. PATCH-based atomic transitions. Env guard for tests.
🔭 Langfuse Observability
LLM call tracing with campaign + stage spans via contextvars. Batch flush mode. Cost accumulator remains source of truth.
🧠 lg-llm-runtime Runtime
All LLM calls route through this. No direct model API calls. Local editable at ../lg-llm-runtime. Cost tracking via tracked_run.
MillionVerifier Validation
Email validation real-time API. Only resultcode 1 (ok) = sendable for primary campaigns. Catch-all preserved for low-volume.
💧 Kitt + Icypeas + LeadMagic Waterfall
Email waterfall providers. Kitt (parallel find + inline validation), Icypeas, LeadMagic. Invalid-email convergence tracking prevents wasted credits.
Hooks, MCP Servers + Safety Rails
Claude Code hooks fire automatically on tool use events. MCP servers extend Claude's reach to external services. Together they form the safety and automation backbone.

Claude Code Hooks 5 ACTIVE

🔒 protect-env.js

Trigger: PreToolUse → Read | Edit | Write | Glob | Grep | Bash
Blocks any shell command that reads or references .env files. Forces all env access through Python launcher scripts that load .env programmatically. Prevents accidental secret exposure in tool output.

📧 guard-gmail-send.js

Trigger: PreToolUse → Bash
Guards against accidental Gmail sends. Intercepts gws CLI calls that would send email and requires explicit confirmation. Prevents unintended outbound messages from automation.

🤖 agent-context-snapshot.js

Trigger: PreToolUse → Agent
Auto-generates .claude/agent-context/snapshot.md before every subagent spawn. Contains git branch, recent commits, working tree changes, and active tasks. Ensures every subagent has workspace awareness.

✨ prettier-format.js

Trigger: PostToolUse → Edit | Write
Auto-formats files after every edit or write operation. Keeps code style consistent without manual intervention. Runs via Bun for speed.

🔌 plugin-sync.js

Trigger: SessionStart
Synchronizes plugin state at session start. Ensures all Claude Code plugins and their skills are available and up-to-date before work begins.

Hook Lifecycle — When Each Fires
Session Start
plugin-sync
Any Tool Call
protect-env
guard-gmail
agent-snapshot
Execute
prettier-format
Blocks dangerous ops
Injects context
Post-processing

🌐 MCP Server 1 CONNECTED

All external services (DiscoLike, Supabase, LeadMagic, Google Drive) accessed via CLI or Python SDK — not MCP. Only AutoContext runs as MCP for codebase intelligence.

🧠 AutoContext STDIO
Codebase intelligence. Local Python binary. Provides semantic code understanding for agent context.
📝 Agent Logging

SubagentStop hook on campaign-doctor agent auto-logs session completions to leadgrow-hq/automation/logs/agents/. Provides audit trail for autonomous agent runs.

28 Skills Across 6 Categories
Skills are Claude Code slash commands that wrap complex multi-step workflows into single invocations.
📥 Intake 2
  • icp-onboarding Interview + crawl + CSV → icp-profile.yaml
  • web-scraping 3-level waterfall: Spider → Firecrawl → Firecrawl JS
🔬 Research 3
  • research-gate Freshness check before campaign work
  • research-chain 11-step FIND chain, step-addressable, staleness detection
  • voice-of-market Reddit prospect-language mining for copy research
✍️ Strategy + Copy 5
  • campaign-strategy 15+ campaign ideas from ICP + playbook
  • briefing-maker Synthesize ICP + research → pipeline brief
  • enrichment-selector Pick enrichments for brief
  • campaign-copywriting Single-agent copy → _copy.json at briefing time
  • 1to1-copy Per-lead ABM: Spider research → segment → emails
📋 List Building 8
  • discolike Lookalike discovery from seed domains
  • ai-ark-companies Industry/geo company discovery
  • ai-ark-people Decision-maker contacts at companies
  • google-maps Local SMB lists
  • linkedin-jobs Hiring signal companies (Apify)
  • linkedin-ads Ad-spend signal companies
  • competitor-engagers Competitor LinkedIn post engagers
  • list-normalizer Any CSV → Clay-compatible format
⚙️ Campaign Ops 5
  • bison Campaign lifecycle: create, launch, refill, score
  • signatures Bulk inbox signature management
  • yolo-mode Skip all checkpoints except Gate 9g
  • weekly-rhythm Mon/Wed/Fri operational cadence
  • deliverability Spam/bounce/blacklist incident response
🛠️ Authoring + Dev 2
  • prompt-creator Scaffold enrichment/classifier prompts, validate, register
  • research-builder Build search pattern processes, test across tiers
🎯 Quality + Offers 3
  • experiment-design Single-variable A/B tests pre/post-launch
  • 3-offers 3-tier offer package (Hormozi framework)
  • lead-magnet 10 archetypes scored on 4-criterion rubric
How the System Thinks
Mental models every team member needs to internalize.

⏸️ Checkpoints

Stages raise CheckpointPending (exit code 5) when human judgment needed. Operator edits the surfaced file (YAML/CSV/JSON), then runs gtm-orchestrator resume. Partial resolution valid — unfilled rows re-surface. 6 checkpoint locations: enrich, segment, copy, QA, inboxes, launch gate.

🛫 Pre-flight

Before pipeline starts: validates 15 GitHub labels exist in the repo + Supabase namespace is bootstrapped. Blocks the run if missing — prevents mid-pipeline failures from infrastructure gaps.

🧪 Dry-run Mode

--dry-run simulates all Bison calls with synthetic IDs and skips gate 9g. Use for testing pipeline wiring end-to-end without touching production systems or requiring human confirmation.

📜 Brief Immutability

Brief is frozen at campaign init. Since v4.0, copy (_copy.json) is generated at briefing time by the campaign-copywriting skill and included in the frozen brief. Stage 7 (Apply) deterministically renders variables — no LLM calls at runtime for copy.

💰 Cost Tracking

Every LLM call via tracked_run auto-increments the cost accumulator. gtm-orchestrator status shows per-stage + grand total. Langfuse provides span-level detail but cost_accumulator remains the billing source of truth.

🗄️ State Model

SQLite per campaign is the source of truth. Google Sheets is the human-readable mirror (visibility window). GitHub Issues track per-stage lifecycle via label state machine. Supabase stores cross-campaign enrichment data.

0
SuccessPipeline completed or stage finished normally
5
Checkpoint PendingHuman edit needed → resume after fixing
6
Standards FailedBison campaign settings don't match — fix in UI
7
Scorecard BlockGrade F — list quality too low to proceed
Dual Supabase Architecture
Two independent Supabase adapters serve different purposes. Both are non-blocking — a Supabase outage never aborts a pipeline run. SQLite remains the source of truth.
🪞
SupabaseMirror
Dashboard Tables
Pushes pipeline state to public schema for the operator dashboard. Per-client table isolation via table_prefix. Swallows all exceptions — logs warning, returns None.
sync_stage() — batch upsert at stage complete. Writes campaign_slug, stage, status, row_count, cost_usd.
sync_lead_snapshot() — per-row streaming for Stage 5 (enrich) + Stage 7 (copy) only. Real-time lead-level visibility.
sync_campaign() — campaign-level status, current stage, lead count, total cost.
🗃️
SupabasePersistence
Knowledge Base
Targets leadgrow_knowledge schema for cross-campaign intelligence. Retries failed writes via queue (not swallow-only). Auto-reconnects on stale connections for 60min+ pipeline runs.
Enrichments table — per-row upsert during enrichment loops. Slugged by enrichment type.
Campaign contacts — normalize/qualify/people/waterfall/segment stages write rows.
Retry queue — failed writes queued in memory, batch-retried. Never blocks pipeline.
Design Rule
SQLite is truth. Supabase is cloud mirror. Never swap.
Offline Safe
from_env() returns None if env vars absent. Pipeline works offline with zero behavior change.
Per-Client Isolation
table_prefix creates namespaced tables per client. No cross-contamination.
GitHub Issues as Pipeline State Machine
Every pipeline stage gets a GitHub Issue. Labels track lifecycle. Checkpoint instructions post as comments. The GitHub Projects board gives table-level visibility into every campaign's CSV stages.
Issue Lifecycle — Label State Machine
Stage starts
status:running
status:checkpoint
status:running
status:complete
On failure at any point:
status:failed
pending
running
checkpoint
complete
failed

🏷️ 15-Label Vocabulary

Derived from CANONICAL_STAGE_NAMES at runtime. Sub-stages grouped: 02a/02b/02c/02.5 → stage:normalize, 04b/04c → stage:people, 07b → stage:copy. Plus 5 status labels. Pre-flight validates all labels exist before first run.

🎫 Three Label Dimensions

Every issue carries exactly 3 labels:
stage:qualify — which pipeline stage
status:running — current lifecycle state
campaign:rbs-mca-v1 — which campaign
PATCH-based atomic transitions swap all 3 at once.

⏸️ Checkpoint Comments

When a stage raises CheckpointPending, the adapter posts a comment with: the file to edit, the action to take, and the resume command. Operator gets notified via GitHub, edits the file, resumes.

📋 Campaign Parent Issue

One parent issue per campaign tracks the full run. Includes client, campaign slug, list source, segments. Checkpoint comments accumulate as the pipeline hits pause points. Closed at launch. Sidecar file (.github_issue) ensures idempotency.

📊 GitHub Projects — Table Visibility

GitHub Projects board turns issues into a sortable, filterable table. Each row = one pipeline stage for one campaign. Columns show status labels as colored badges. Filter by campaign:slug to see one campaign's progress, or by status:checkpoint to see all blocked stages across all campaigns.

📋
Table View
All stages × all campaigns. Sort by status, filter by client.
📊
Board View
Kanban by status: pending → running → checkpoint → complete.
🔔
Notifications
Checkpoint comments trigger GitHub notifications. Operator acts from phone.
🛡️
Best-Effort
All GitHub calls wrapped in try/except. Pipeline continues if GitHub is down.
Vercel Web Dashboard
Next.js 14 App Router app deployed to Vercel. Gives operators a live view of every campaign without touching the terminal.

📋 Campaign List View

All active campaigns with status, lead count, cost, and last activity. Server-rendered with ISR (60s revalidation). Data sourced from Google Sheets CAMPAIGN_SPREADSHEET_MAP env var.

🔎 Campaign Detail View

Per-campaign stage table — each of 19 stages shows row count, status (complete/idle), and timing. GitHub Issues panel surfaces open checkpoints that need operator action.

Next.js 14
App Router, server components only (no client JS). ISR 60s revalidation on all data.
📊 Google Sheets API
googleapis SDK reads campaign Sheets tabs. Service account JSON from base64 env var (server-side only).
🏷️ GitHub API
Fetches open Issues with checkpoint labels. Native fetch with ISR revalidation.
🎨 LeadGrow Design
Dark mode design tokens via CSS custom properties + Tailwind for layout. Matches brand palette.
Runtime: Bun + Vercel
Client JS: Zero
Revalidation: ISR 60s
Secrets: Server-only (no NEXT_PUBLIC_)
Routes: / (static) + /campaign/[slug] (dynamic)
From v1.0 to v4.0 in 30 Days
Shipped April 21 → May 20, 2026. 54 phases across 4 major versions.
v1.0
April 21, 2026
Full Pipeline Shipped
9 stages: discovery → normalize → qualify → people → enrich → segment → copy → QA → launch. DiscoLike default. SQLite state. Bison integration with human gate.
v2.0
April 30, 2026
Intelligence + List Building
Added 20+ skills: research chain, voice-of-market, AI Ark, Google Maps, LinkedIn Jobs/Ads, competitor engagers. Per-lead personalization. Email waterfall. MX prefilter. Scorecard gate.
v3.0
May 10, 2026
Full GTM Operating System
ICP ingestion, stateful research chain, copy + QA system, list source skills as standalone invocable commands. Google Sheets hygiene tabs. Web dashboard on Vercel.
v4.0
May 20, 2026
Pipeline v2 Architecture
Split enrichment (company vs contact), copy-at-briefing (no runtime LLM for copy), Trigger.dev durable workflows, Supabase persistence, GitHub Issues state machine, Langfuse observability, pre-flight checks, template repo bootstrap.
Running the Pipeline

▶️ Start / Resume

gtm-orchestrator run <campaign_dir>
Starts or resumes from last completed stage. Add --dry-run to simulate Bison calls.

⏸️ Resume After Checkpoint

gtm-orchestrator resume <campaign_dir>
Edit the checkpoint file, then resume. Partial resolution OK.

📊 Status + Cost

gtm-orchestrator status <campaign_dir>
Per-stage progress + token/cost totals.

📨 Score Replies

gtm-orchestrator score-replies <campaign_dir>
Post-launch: classify replies into 9 categories. SHA-256 dedup.