Skip to main content
Version: 0.6.3

ClaudusBridge

🤖 567+ AI Tools
🔌 MCP Protocol
🏗️ 50+ Categories
🔒 100% Local
👁️ Multi-Agent + Watcher
🧠 Cognition Tier
💬 Native Editor Chat
🎨 Image Gallery

ClaudusBridge is an AI-powered Unreal Engine 5 editor plugin that turns the editor itself into a multi-AI workstation. 567+ tools are exposed over the Model Context Protocol (MCP) to any MCP-compatible client (Claude Code, Codex CLI, Cursor, Windsurf, Claude Desktop, …) — and a dedicated dashboard tab inside Unreal lets you talk to Claude, ChatGPT, or Gemini directly from the engine without leaving the editor. Provider replies, external execution-agent activity, and tool-result evidence all share one conversation.

Version 0.6.x ships:

  • First-boot UX (0.6.3) — open the editor and start typing. The provider runtime auto-launches at PostEngineInit, the self-heal then auto-attaches the last-selected provider, and by the time the user types into the dashboard their AI is ready to answer. No /runtime start, no /login retry — Claudus is the conversation.
  • Multi-agent auto-link (0.6.3) — when a provider emits an action_request inside an active multi-agent workflow but skips the explicit step-update tool (observed in ChatGPT tests), the most-recently-updated active workflow with a pending step is found automatically and the step is auto-completed with the result as evidence. Different providers (Claude, ChatGPT, Gemini) have different dispositions — the auto-link papers over that so every workflow finishes regardless of style.
  • agent_login auto-resolve (0.6.3) — provider_mode=claudus no longer requires an explicit base_url. The plugin reads <LOCALAPPDATA>/ClaudusBridge/ProviderRuntime/dispatch.json and derives the URL automatically, so external MCP clients (Claude Code, Codex, Cursor, Windsurf, custom JSON-RPC) don't need to know the runtime port.
  • International polish (0.6.3) — every emitted label is now English ("Full Access" / "Limited Access"). Backward-compatible lowercase Spanish input aliases still work.
  • The Cognition tier — workflow checkpoints, structured action-failure observations, continuation-batch savings telemetry, and observability over the local provider-runtime auth state (presence + mtime, never tokens).
  • Native editor conversation — a dashboard tab and matching Output Log commands (/login, /provider, /model, /connect, /access full | limited, /logs claudus | all) so the engine console is the chat interface. Provider-specific log categories (LogClaudusClaude, LogClaudusChatGPT, LogClaudusGemini, LogClaudusUser, LogClaudusAgent, LogClaudusSystem) route every message to its own native log channel.
  • Image generation + native texture vision — claudus_generate_gallery_image creates raster images as native Unreal Texture2D assets under Content/Gallery/&lt;Topic&gt;/. claudus_show_texture_asset opens any Texture2D in the native Texture Editor; claudus_describe_texture_asset returns a provider-side visual description without routing through an external vision agent.
  • Multi-AI transcript — providers (Claude / ChatGPT / Gemini through the Claudus runtime) and external execution agents (Claude Code, Codex, Cursor, Windsurf, …) share one chat. Providers reason; execution agents claim [Claudus Action Request] blocks and run real filesystem / shell / build work; results flow back as evidence.
  • Self-healing provider pipeline — restart the runtime out of band and the editor re-reads dispatch.json before the next request goes out, so LogHttp warnings about dead ports never appear and the user sees a clean answer instead of an error.

The earlier 0.5.0 baseline still applies: full multi-agent coordination with per-session event subscriptions, an agent_message bus for agent-to-agent messaging, and a mandatory visual watcher sub-agent that observes the editor through /preview?bare=1 while the primary agent works with the user. The verification gap — tools say success based on API return values but the visual reality may disagree — is closed by design.

The plugin also ships a persistent shared knowledge base under Saved/ClaudusBridge/memory/ that every agent (Claude Code, Codex CLI, Cursor, Windsurf, …) reads and writes. The plugin gets more useful with every session instead of re-teaching the same lessons. An ergonomic CLI helper cb.py is auto-generated on every connect and collapses the repetitive curl + JSON-RPC + Python-unwrap pattern into single-line subcommands.

From Blueprint scripting to Material creation, Actor management to Niagara VFX, ClaudusBridge turns natural language into editor actions, with an always-on internal watcher and a self-improvement loop that learns from every session.

Who It's For​

TargetUse Case
Solo DevelopersAccelerate prototyping by describing what you want instead of clicking through menus
Studios & TeamsAutomate repetitive editor tasks, batch operations, and pipeline workflows; multiple agents working in parallel without stepping on each other
Technical ArtistsScriptable Material and Niagara workflows via AI commands; visual watcher confirms the result matches the intent
Educators & StudentsLearn UE5 concepts with AI-guided, interactive exploration — see exactly what the agent does through the WebRTC stream

Key Features​

  • 567+ Tools across 50+ categories (Blueprints, Materials, Actors, Niagara, Widgets, Sequencer, Animation, Mesh, Audio, Physics, Cognition, and more)
  • Native Editor Conversation (0.6.x) — A dedicated Claudus dashboard tab inside the editor plus matching Output Log commands let you talk to Claude / ChatGPT / Gemini without leaving Unreal. /login chatgpt | claude | gemini, /provider, /model, /connect, /access full | limited, /logs claudus | all. Six provider-specific log categories (LogClaudusClaude, LogClaudusChatGPT, LogClaudusGemini, LogClaudusUser, LogClaudusAgent, LogClaudusSystem) keep the conversation routed through native UE channels
  • Multi-AI Shared Transcript — Providers (Claude / ChatGPT / Gemini through the local Claudus runtime) and external execution agents (Claude Code, Codex CLI, Cursor, Windsurf, Claude Desktop) post into the same conversation. Providers reason and emit [Claudus Action Request] blocks; execution agents claim them via claudus_get_action_requests and run the real filesystem / shell / build / out-of-root work; results flow back through claudus_provide_action_result as evidence
  • Companion Packages for External AI Agents (0.6.x) — claudus_get_companion_package(host) generates a complete install package for Claude Code, Codex CLI, Cursor, Windsurf, Claude Desktop, or the Gemini CLI in one call: agent profile in the host's native format, MCP config snippet, one-line bootstrap script, INSTALL.md, and shared protocol reference. The agent walks in already understanding the action contract, EYES MANDATORY rule, memory + checkpoint patterns, and access-mode protocol — no custom prompting needed
  • Documentation Pack + Token-Efficient Workflow (0.6.x) — claudus_get_documentation_pack lets fresh agents bootstrap themselves without provider-side re-teaching. Combined with mixed-tier delegation (claudus_delegate_to_model), speculative pre-fetch, auto-continuation batching, and claudus_cost_summary / claudus_speculation_stats / claudus_continuation_stats telemetry, the typical multi-agent run spends a small fraction of the inferences a naive provider-only loop would consume
  • Image Gallery (0.6.x) — claudus_generate_gallery_image creates ChatGPT-backed raster images as native Unreal Texture2D assets under compact topic folders (Content/Gallery/&lt;Topic&gt;/). claudus_save_image_to_gallery accepts existing base64 / local image files. Provider-authored image_generation action requests auto-execute in Full Access
  • Native Texture Vision (0.6.x) — claudus_show_texture_asset opens any Texture2D in the native Texture Editor. claudus_describe_texture_asset returns a provider-side visual description of a texture without routing through an external vision agent. preview_texture_asset is the light PNG-evidence fallback
  • Cognition Tier (0.6.x) — Workflow checkpoints (claudus_checkpoint_workflow + claudus_list_workflow_checkpoints) snapshot asset state under Saved/ClaudusBridge/WorkflowCheckpoints/. Every tool failure writes a structured memory entry tagged action_failed + tool:&lt;name&gt; + severity:&lt;high|low&gt;. claudus_continuation_stats reports projected saved inference ms from auto-continuation batching with a USD estimate. claudus_get_last_auth_state exposes a sanitized provider-auth snapshot (presence + mtime, no tokens)
  • Access Mode Controls (0.6.x) — Full Access by default; switch with /access limited, /access full, claudus_set_access_mode, or the compatibility claudus_press_access_button. Allowlisted read-only tools auto-run in limited mode; mutating tools wait for full access. Every transition is audited in the transcript with timestamp, requester, reason, and old/new mode
  • Self-Healing Provider Pipeline (0.6.1 → 0.6.3) — Restart the Node provider runtime out of band and the editor recovers transparently. A cheap stat() of dispatch.json at the entry of every chat request detects the new port via mtime and swaps BaseURL before libcurl ever touches the dead socket — no LogHttp warnings, no 2-second connect timeout, no visible recovery message. Failure-path rediscovery + retry remain as a safety net
  • cb Ergonomic CLI — Auto-generated Saved/ClaudusBridge/cb.py collapses curl + JSON-RPC + Python-unwrap into single-line subcommands (cb spawn, cb tx, cb capture, cb verify, cb remember, …). The preferred interface for any agent
  • Persistent Shared Knowledge Base — Saved/ClaudusBridge/memory/ accumulates tribal knowledge across all agents and sessions. Topic files + _log.jsonl + auto-curated _index.md. Five MCP tools: memory_remember / memory_recall / memory_topics / memory_search / memory_consolidate
  • EYES MANDATORY — Every agent driving the editor MUST use the streaming captures. API success ≠ visual success. Every editor tool returns success based on whether the API call completed, NOT whether the result actually looks right. The plugin enforces the rule throughout the auto-generated docs the agent reads on every connect
  • Multi-Agent Coordination — Per-session subscription queues, publish_agent_event bus for agent-to-agent messaging, channel-based broadcast or targeted delivery
  • Iterative Dialogue Pattern — Make a small change, look at it, decide next change, repeat. The watcher publishes plain-text observations on the agent_message bus; the primary reads them between user turns and acts on visual ground truth
  • Dual-Watcher A/B Parity — Spawn TWO watchers in parallel (one per browser extension) for high-stakes parity verification
  • Pixel Streaming Readiness Gate — pixel_streaming_status MCP tool gives every agent a single source of truth for whether the editor's WebRTC pipeline is up before any capture
  • Persistent Watcher Daemon — Production-grade Python daemon (Saved/ClaudusBridge/claudusbridge_watcher.py) with pattern rules (compile loops, PIE crash-on-start, actor leaks); pure stdlib, JSON logs, exponential-backoff reconnect
  • Always-On Internal Watcher — Built-in C++ watcher (FCBInternalWatcher) that runs from plugin load, no setup required
  • Self-Improvement Loop — record_feedback writes to Saved/ClaudusBridge/feedback.jsonl and broadcasts on the agent_message bus; the canonical signal for plugin evolution
  • Robust Transform Parser — spawn_actor / spawn_basic_shape accept location/rotation/scale as arrays, objects, or top-level scalars (and uniform scale: 2 for all axes); top-level scalars override array form so callers can patch one axis
  • Neural Cortex Architecture — Tools organized as interconnected neurons: perception, navigation, creation, and engineering regions
  • AI Vision System — /preview WebRTC stream as the primary realtime view; structured perception tools (ai_perceive, ai_vision_stream, get_vision, get_cognitive_map, scan_360, proximity_alert) for spatial awareness without pixel I/O
  • Smooth Navigation — WASD-style camera movement, orbit, look_at_actor, fly_to_actor with collision avoidance (never teleport)
  • NNE GPU Inference — ONNX model loading via DirectML Tensor Cores (load_nne_model, get_nne_status, list_nne_runtimes)
  • Open Protocol — Works with any MCP-compatible AI client; Codex CLI / rmcp-strict transport fully supported (HTTP 202 Accepted on notifications/initialized)
  • 100% Local — No cloud, no API keys, no subscriptions. One-time purchase, runs entirely on your machine
  • Full C++ Source — Complete source code included, fully extensible
  • Real-Time Dashboard — Monitor connections, session stats, multi-agent / watcher status, and recent activity from within the editor
  • Cross-Platform — Windows, macOS, and Linux support

How It Works​

AI Client (Claude Code, Codex CLI, Cursor, Windsurf, Claude Desktop, ...)
    |
    | 1. HTTP POST /mcp (JSON-RPC 2.0, Streamable HTTP, MCP spec 2025-03-26)
    v
ClaudusBridge MCP Server (Port 3000)
    |
    | 2. Per-session event queue + agent_message bus
    |    (subscribe_events, poll_editor_events, publish_agent_event)
    v
Router -> Neural Cortex -> 40+ Manager classes
    |
    | 3. Executes on GameThread
    v
Unreal Engine Editor API + /preview WebRTC stream
    |
    | 4. Internal C++ watcher inspects every event, records feedback
    | 5. Sub-agent watcher observes /preview?bare=1, publishes observations
    v
Your Project (Blueprints, Materials, Actors, Niagara, Sequencer, ...)
  1. You describe what you want in natural language to your AI client
  2. The AI translates your request into MCP tool calls
  3. ClaudusBridge executes the commands in the Unreal Editor
  4. The watcher confirms that the visual result matches the intent
  5. You see results instantly through the WebRTC /preview stream

Quick Example​

"Create a Blueprint actor with a BeginPlay event that prints 'Hello World' after a 2-second delay"

The AI will automatically:

  1. Subscribe its primary session to agent_message and spawn the visual watcher sub-agent
  2. Call read_blueprint to inspect the target
  3. Call add_event_node to add BeginPlay
  4. Call add_node to create a Delay node and PrintString
  5. Call connect_pins to wire the execution flow
  6. Call compile_blueprint to validate
  7. Drain poll_editor_events to read the watcher's observation confirming the graph compiled green and the Print String node is wired correctly

All without you touching the editor — and with visual ground truth instead of blind tool-success assumptions.

Supported Domains​

DomainToolsDescription
Blueprint Nodes18Add, delete, configure graph nodes
Pin Connections3Wire execution and data flow
Events2Standard and custom Blueprint events
Variables3Create, delete, configure variables
Functions5Create functions with inputs/outputs
Inspection7Read Blueprint structure, analyze graphs
Actors23Spawn, transform, configure actors and components
Assets17Find, duplicate, rename, delete assets
Materials30Create materials, expressions, instances
Editor48Viewport, PIE, capture_editor_window, console
World9Levels, lights, physics
Niagara30Particle systems, emitters, renderers
Widgets29UMG UI creation, manipulation, animations, events
Sequencer12Cinematics and animation tracks
Procedural Mesh38Runtime geometry creation and CSG
MetaSound15Procedural audio graphs
Editor Events4poll/subscribe + publish_agent_event + record_feedback
Neural Cortex12ai_perceive, ai_navigate, ai_create, ai_modify, ai_inspect, ai_build, ai_scene, ai_blueprint, ai_media, ai_engineer, ai_evaluate, ai_brain_map
AI Vision7ai_vision_stream, ai_vision_identify, NNE GPU inference
And 25+ more...180+Animation, Physics, Landscape, Source Control, MVVM, BlueprintLisp...

Next Steps​