Overview

Prompt: "Cyberpunk city, raining, neon lights, 4k"

**UELTX2: Curated Generation** is a streamlined bridge between **Unreal Engine 5** and **Lightricks' LTX-2** generative video model.

Unlike external tools (Runway, Sora) that require cloud subscriptions and break your pipeline, UELTX2 runs **locally** on your hardware via ComfyUI. It allows you to generate animated placeholders, dynamic textures, and cinematic animatics directly inside the Editor Viewport.

Architecture

Understanding how the pieces fit together is crucial for stability.

The plugin sends a JSON payload to `localhost:8188`. ComfyUI processes the Diffusion Transformer (DiT) steps, saves an `.mp4` to disk, and UELTX2 auto-imports it as a `FileMediaSource`.

Phase 1: Backend Setup

Before opening Unreal, you must configure the AI server.

1. Install ComfyUI

Download the standalone Windows version to avoid Python dependency issues.

2. Install Required Nodes

UELTX2 relies on specific custom nodes to handle video encoding and quantization.

• Open ComfyUI Manager (Install it if you haven't).
• Search for and install: ComfyUI-VideoHelperSuite (Essential).
• Search for and install: ComfyUI-GGUF (For VRAM optimization).

3. Download Models

Place the LTX-2 weights in your checkpoints folder.

# Recommended: GGUF Quantized (16GB - 24GB VRAM)
C:\ComfyUI\models\checkpoints\LTX-2-dev-Q4_K_M.gguf  (example)

# Optional: Full Precision (24GB+ VRAM)
C:\ComfyUI\models\checkpoints\ltx219BNextGenAIVideo_ltx2FP8VERSION.safetensors  (example)

Phase 2: Plugin Setup

1. Installation

Steps to integrate into your C++ project:

1. Close Unreal Engine.
2. Copy the 'UELTX2' folder to 'YourProject/Plugins/'.
3. Right-click YourProject.uproject -> Generate Visual Studio Files.
4. Build the solution.

2. Configuration

Once compiled, open the Editor and navigate to Project Settings > Game > LTX-2 Generation.

Text-to-Video Workflow

The standard generation method for cinematic placeholders.

1. Start your local ComfyUI server (run run_nvidia_gpu.bat).
2. In UE5, click the LTX-2 Icon in the main toolbar.
3. When the panel opens, enter your prompt (e.g., "A spinning golden coin, 4k, loopable").
4. Click Generate.
5. Wait for the OnGenerationComplete notification. The file will appear in your Content Browser automatically.

Image-to-Video (Texture Animating)

The "Curated Generation" power-feature. Turn static screenshots into dynamic elements.

Right-Click Action

Simply Running Right-Click > Scripted Actions > Animate with LTX-2 on any texture will create a 2-second video variation of that texture, preserving color and composition.

Mastering the Workflow

A deep dive into the interface, prompt engineering for LTX-2, and converting static assets into motion.

1. The Interface Panel

The UELTX2 Editor Utility Widget is your command center. It is designed to be dockable anywhere in the Unreal Editor layout (we recommend docking it next to the World Outliner).

2. Prompt Engineering for LTX-2

LTX-2 is a Diffusion Transformer. Unlike older U-Net models, it understands scene composition better but requires specific "trigger words" to get the best motion.

Key Tokens for Motion

• "Slow pan right" - Control Camera
• "Zoom in" - Dynamic depth
• "Fluid motion" - Smooth liquids/smoke
• "Loopable" - Good for textures

3. Context-Aware Generation (I2V)

The most powerful feature of UELTX2 is generating motion from existing project assets. This runs a different pipeline that preserves the color palette and composition of your source image.

Resolution Warning

4. Advanced Configuration (JSON Hacking)

Want to change the Seed, Steps, or Resolution? You don't need to recompile C++. Tweak the "Brain" of the plugin directly in the content folder.

Navigate to: Plugins/UELTX2/Content/Templates/LTX2_T2V.json

{
  "9": {
    "inputs": {
      "seed": 123456789,      <-- Change default seed
      "steps": 30,            <-- Increase for quality (e.g. 50)
      "cfg": 3.0,             <-- Guidance Scale
      "sampler_name": "euler",
      "denoise": 1.0
    },
    ...
  },
  "8": {
    "inputs": {
      "width": 768,           <-- Set custom width
      "height": 512,          <-- Set custom height
      "length": 49            <-- Frame count (approx 2s @ 24fps)
    }
  }
}

Note: Increasing resolution logic exponentially increases VRAM usage. 768x512 uses approx 10GB. 1280x720 requires 24GB+.

"Living Texture" Workflow for I2V

Turn static screenshots into dynamic, loopable materials in one click.

Step-by-Step Guide

2

Set Source in Panel

In the UELTX2 Panel, click the "Set Source from Selection" button.

UELTX2_Panel

⊕ Set Source from Selection

SOURCE ACTIVE

Subtle smoke movement, loopable...

GENERATE

Pro Tips for LTX-2 I2V

Rapid Pre-Visualization Workflow for T2V

Generate "Instant Animatics" to block out cinematic timing before 3D production begins.

# The Workflow

"Niagara Source" Workflow for VFX

Generate fluid simulations (Fire, Smoke, Magic) and inject them immediately into Niagara.

# The Workflow

Troubleshooting & Diagnostics

Solve connection refusals, memory spikes, and missing assets.

1. Quick Diagnostics

2. VRAM & Out of Memory (OOM)

LTX-2 is a heavy model. If Unreal Engine crashes or ComfyUI reports CUDA Out of Memory, check your configuration against this gauge.

Optimization Checklist

• Limit Frame Rate: In UE5 Editor Preferences, enable "Use Less CPU when in Background" to free resources for ComfyUI.
• Close High-VRAM assets: Close Metahuman blueprints or massive levels in UE5 before generating.
• Use GGUF: Installing the ComfyUI-GGUF node is the #1 way to fix memory crashes.

3. The "Red Node" Problem

If the console logs "Success" but no video appears, drop the template JSON into ComfyUI manually to verify integrity.

4. Unreal Engine Quirks