VJing and Video DJing: Resolume Arena Setup, DXV3/HAP Codecs, GPU Acceleration and Video Outputs.

Introduction to VJing and Video DJing

The integration of visual stimuli into electronic music performances has evolved from simple static projection into a complex ecosystem of real-time interaction. Within this discipline, there are two distinctly different conceptual and technical approaches:

1. Traditional VJing (Visual Performance): A visual artist (VJ) improvises, mixes and creatively manipulates visual content (pre-rendered clips, procedural generators, live cameras and video effects), interpreting the music’s energy and structure independently but in coordination with it.
2. Video DJing (Audio-Visual DJing): The DJ synchronises or triggers video content directly associated with each music track from the booth, integrating the audio and video signals through automation, faders or unified software.

Both approaches depend on one fundamental pillar: the computer’s ability to decode multiple high-definition video streams (1080p, 4K) simultaneously with zero latency and without experiencing frame drops.

---

Resolume Arena and Avenue Architecture

Resolume is the reference media server in the live events and VJing industry. Its internal architecture is organised hierarchically into:

• Clips: Media files or generative sources assigned to cells in a grid (matrix).
• Layers: Independent playback channels with visual volume controls (opacity), blend modes such as Add, Multiply, Screen and Luma Key, and dedicated effects chains.
• Composition: The overall sum of all layers mixed on a canvas with a specified master resolution.
• Advanced Output: The routing engine that cuts the composition into sections (slices) and sends them to multiple physical outputs (LED screens, projectors, reference monitors or SDI cards).

To maintain a constant rate of 60 frames per second (FPS) with dozens of active layers, Resolume delegates graphics processing to the GPU (Graphics Processing Unit). However, the most common bottleneck is not the graphics card’s raw power, but the choice of video codec.

---

GPU-Decompressed Intraframe Codecs: DXV3 vs HAP vs ProRes vs H.264

Interframe vs Intraframe Compression

Consumer video codecs, such as H.264 or H.265 / HEVC, are designed to minimise disk space through interframe compression. These codecs store a complete key frame (I-frame) at set intervals and only store the vector motion changes in the intermediate frames (P-frames and B-frames).

• Negative impact live: If you try to play an H.264 clip backwards, change its speed abruptly or perform scratching, the CPU must decompress dozens of preceding frames to reconstruct a single image. This overwhelms the processor and creates unacceptable stuttering.

Professional media servers require intraframe codecs, where each frame is compressed as an independent image. To maximise performance, they also use codecs decompressed by the GPU’s compute cores rather than the CPU.

Codec Comparison Table for Live Video

Codec	Compression Type	Decoding	Alpha Channel (Transparency)	Recommended Use Case
DXV3 Normal	Intraframe (GPU)	GPU	Optional	Standard use in Resolume (maximum performance).
DXV3 High Quality	Intraframe (GPU)	GPU	Optional	Graphics with fine gradients (avoids banding).
HAP / HAP Alpha	Intraframe (GPU)	GPU	Yes (HAP Alpha)	Cross-platform standard (TouchDesigner, Modul8).
HAP Q	Intraframe (GPU)	GPU	No	Maximum colour fidelity in GPU compression.
Apple ProRes 422/4444	Intraframe (CPU)	CPU	Yes (ProRes 4444)	Intermediate rendering or editing on macOS.
H.264 / H.265	Interframe (GOP)	CPU / ASIC	No	PROHIBITED for smooth live VJing.

The DXV3 codec (proprietary to Resolume) uses algorithms that let the graphics card load compressed textures directly into VRAM and decompress them in parallel using its shader architecture, freeing the CPU from any substantial workload.

---

Native Resolution, Scaling, Refresh Rate and Visual Latency

Refresh Rate and VSync

The smoothness of visual transmission is measured in frames per second (FPS). In professional show environments, the refresh standard is 60 Hz (60 FPS) or 59.94 Hz in SDI broadcast environments.

• Tearing: Occurs when the GPU sends a new frame to the screen while it is halfway through its horizontal scanning cycle. It appears as a horizontal line of discontinuity.
• VSync (Vertical Synchronisation): Forces the GPU to wait for the monitor’s vertical synchronisation pulse before swapping the image buffers (framebuffers). Enabling VSync is essential to eliminate tearing on giant screens.

Visual Chain Latency (Input Lag)

The total latency from the DJ or VJ triggering an event to the photons reaching the screen consists of:

$$\text{Total Latency} = \text{Input Latency (MIDI/OSC)} + \text{GPU Rendering Latency} + \text{LED Processor/Projector Latency}$$

To maintain precise rhythmic synchronisation, GPU rendering latency in Resolume must remain below 16.6 ms (one frame at 60 FPS).

---

Dedicated GPU Assignment and Video Output Setup

GPU Management on Laptops and Desktop Towers

On computers with processors that include secondary integrated graphics (for example, Intel Iris Xe or AMD Radeon Graphics) alongside a dedicated GPU (NVIDIA GeForce/RTX or AMD Radeon Pro), the operating system may assign rendering to the integrated graphics by default to save power.

1. Windows: You must go to Display settings > Graphics, select Avenue.exe or Arena.exe and force High performance mode assigned to the dedicated NVIDIA/AMD GPU.
2. NVIDIA Control Panel: Under Manage 3D Settings, set the preferred graphics processor to Nvidia High-Performance Processor and disable power-saving technologies (Power Management Mode = Prefer Maximum Performance).

Physical Output Connections: HDMI, DisplayPort and SDI

• DisplayPort 1.4 / 2.1: Native output on desktop cards. It can transmit 4K at 120Hz or 8K at 60Hz. When converting to HDMI to connect LED processors, use active DisplayPort-to-HDMI adapters (passive adapters limit resolution and pixel-clock stability).
• HDMI 2.0 / 2.1: A common standard on laptops and video consoles. Supports 4K 60Hz (HDMI 2.0) or 4K 120Hz (HDMI 2.1). Susceptible to physical disconnection if cables with mechanical locking are not used.
• 3G-SDI / 12G-SDI: Professional broadcast standard with bayonet-locking BNC connectors. Allows cable runs of up to 100 metres without signal loss. Requires dedicated video output cards such as the Blackmagic DeckLink (PCIe) or UltraStudio (Thunderbolt 3) ranges.

---

EDID Management and Video Matrix Switchers

In festival and venue setups, connecting and disconnecting screens can cause the operating system to dynamically rearrange the connected displays, disrupting Resolume’s mapping.

• EDID (Extended Display Identification Data): Information the screen sends to the graphics card to report its native resolution, supported refresh rates and colour space.
• EDID Emulators / Passthrough: Hardware devices (or features built into NVIDIA Quadro/RTX Enterprise cards and Novastar processors) that simulate the constant presence of a connected screen with a fixed EDID. This ensures that even if the main HDMI cable is unplugged, the GPU continues sending the signal to the port at the fixed resolution and rate, preventing Resolume’s canvas from collapsing.
• Video Matrix Switchers: Hardware devices, such as Blackmagic ATEM or the Roland XS series, that let you alternate routing between the VJ signal, the backup computer and live cameras with clean switching and no black frames (seamless switching).