Nvidia VFX Nodes for Video Upscaling, Frame Generation and SDR-to-HDR, for VFX Professionals

Source: Comfy Org•

Nvidia VFX Nodes for Video Upscaling, Frame Generation and SDR-to-HDR, for VFX Professionals

Upscale, interpolate, and convert to HDR locally on your RTX GPU, with better EXR support and color management built for VFX pipelines.

Nvidia’s RTX video effects now run inside ComfyUI as custom nodes. The models are fast, lightweight, ship with the nodes, and run locally on your Nvidia RTX GPU.

  • The RTX Video Super Resolution node upscales images and video frames up to 4×, either by a multiplier or to target dimensions up to 8192 px. The same node can also denoise or deblur a source without changing its resolution.

The RTX Video Super Resolution node upscales images and video frames up to 4×, either by a multiplier or to target dimensions up to 8192 px. The same node can also denoise or deblur a source without changing its resolution.

  • The RTX Video Frame Generation node creates new frames between adjacent pairs, for frame-rate conversion or slow motion. It works at 2× to 16× or at a specific timestep.

The RTX Video Frame Generation node creates new frames between adjacent pairs, for frame-rate conversion or slow motion. It works at 2× to 16× or at a specific timestep.

  • The RTX Video HDR node converts SDR images to BT.2020 HDR data, with HLG or PQ output. Contrast, saturation, middle gray, and peak luminance from 400 to 2000 nits are all adjustable, and a debanding pass is on by default.

The RTX Video HDR node converts SDR images to BT.2020 HDR data, with HLG or PQ output. Contrast, saturation, middle gray, and peak luminance from 400 to 2000 nits are all adjustable, and a debanding pass is on by default.

With the NVIDIA VFX Nodes, you can now run a full professional grade VFX pipeline end-to-end. Chaining the nodes from Video Super Resolution → Frame Generation → Video HDR is a quick and simple way to level up your footage and prepare it for the rest of your VFX pipeline.

The RTX Video Super Resolution node is a great option for efficient upscaling and has 19 modes that can handle different kinds of source material.

  • Standard modes upscale typical compressed footage.

Standard modes upscale typical compressed footage.

  • High-bitrate modes upscale clean or lossless sources without unnecessary artifact suppression.

High-bitrate modes upscale clean or lossless sources without unnecessary artifact suppression.

  • Streaming modes are tuned for broadcast and webcam streams.

Streaming modes are tuned for broadcast and webcam streams.

  • Denoise and deblur modes clean up noise, compression artifacts, or softness and keep the input resolution.

Denoise and deblur modes clean up noise, compression artifacts, or softness and keep the input resolution.

  • A plain bicubic mode is also available when you don’t need AI processing.

A plain bicubic mode is also available when you don’t need AI processing.

Most modes run from LOW to ULTRA, trading speed for detail. An optional 10-bit encoding preserves more precision. The frame count and frame rate are unchanged, which sets your footage up for the next step in the workflow.

The RTX Video Frame Generation node offers two ways to generate new frames within an input video. Frame-rate multiplier mode fills every gap evenly, from 2× to 16×. To keep the original duration, encode the output at the source frame rate times the multiplier. Specific-timestep mode inserts one frame per pair at any point between the two frames, which suits custom timing in downstream workflows. The Quality parameter runs from LOW, MEDIUM and HIGH. Shot changes are detected automatically, and a manual option marks a known cut. Like RTX Video Super Resolution, the node offers an optional 10-bit encoding for more precision.

In the final step of the workflow, use the RTX Video HDR node to convert SDR images to BT.2020 HDR at their original dimensions. You can tune the conversion with three tone controls. The Contrast and saturation parameters work as they do in any grade. Middle gray sets the reference level for midtones, which decides how bright the overall image sits. Peak luminance tells the node how bright the brightest highlights should get. Set it to match the display you’re grading for, anywhere from 400 to 2000 nits. A debanding pass is on by default, which helps smooth out the visible steps that can show up in gradients like skies when SDR footage is stretched to a wider range. The output is a standard ComfyUI image with no color-space metadata, so downstream Create Video nodes need to be told whether the data is HDR or HDR PQ. The color management and EXR support in this release also handle that step.

Finally, the node pack’s example_workflows folder has a starter workflow for each node: image upscaling, video upscaling, frame generation, and HDR conversion, plus one for converting longer videos to HDR using the loop nodes. To see all three nodes working together, try the full VFX pipeline below.

Nvidia VFX Nodes Workflow

In addition to Nvidia’s VFX nodes, we’re improving support for EXR files format. EXR is the standard format in VFX pipelines, and this release makes it much easier to work with in ComfyUI.

  • The Save Image (Advanced) node can now write half-float (16-bit) EXRs, now the default, as well as full-float (32-bit) EXRs.

The Save Image (Advanced) node can now write half-float (16-bit) EXRs, now the default, as well as full-float (32-bit) EXRs.

  • The Load Image node has been improved to ensure images remain intact. It now reads half-float and full-float EXRs. Superwhites above 1.0 and negative values come through untouched instead of being silently clipped, so the full range of data is preserved.

The Load Image node has been improved to ensure images remain intact. It now reads half-float and full-float EXRs. Superwhites above 1.0 and negative values come through untouched instead of being silently clipped, so the full range of data is preserved.

  • Color management gets an upgrade as well. The Convert Image Color Space node handles sRGB, scene-linear Rec.709, HDR HLG and PQ (Rec.2020), ARRI LogC3 (EI 800) and ACEScct.

Color management gets an upgrade as well. The Convert Image Color Space node handles sRGB, scene-linear Rec.709, HDR HLG and PQ (Rec.2020), ARRI LogC3 (EI 800) and ACEScct.

EXRs also get a major quality-of-life improvement: they are now easier to inspect and to download. Since web browsers can’t display EXR files, our new asset management system attaches a JPEG preview to every EXR output. The preview is visible in the save and load nodes, in the Assets panel, and in the Jobs history. The small sun button in the load and save nodes also opens the EXR file in the HDR Viewer for more comfortable viewing.

Remark: Previews are only available when ComfyUI starts with the command line argument--enable-asset. The flag is set by default in Comfy Desktop since version 1.1.7 and in ComfyUI portable since 0.40.0.

Remark: Previews are only available when ComfyUI starts with the command line argument--enable-asset. The flag is set by default in Comfy Desktop since version 1.1.7 and in ComfyUI portable since 0.40.0.

Finally, when a save node outputs a sequence of images, the new Download images action lets you pick frames, or hit “Download all”, and download them as a single zip.

  • Update ComfyUI to the latest version.

Update ComfyUI to the latest version.

Install the Nividia VFX custom nodes from the ComfyUI Manager.

  • Load the example workflows from the custom nodes templates, or download the workflows here.

Load the example workflows from the custom nodes templates, or download the workflows here.

Try it on Comfy Cloud, and let us know what you make. As always, enjoy creating!

Try the Workflow on Comfy Cloud

No posts

What this article says