PATH, Python 3 with a virtual environment, a CUDA-enabled PyTorch build, Real-ESRGAN (upscaler), Practical-RIFE or rife-ncnn-vulkan (frame interpolation), an NVIDIA GPU, and free disk equal to roughly 3× your source size.-vsync 0 from a variable-frame-rate (VFR) source breaks all frame math downstream, because VFR means the file does not hold a fixed number of frames per second.
What to do: Install the tools and record the exact build strings before touching any video.
Why: Every failure later in this guide is diagnosed by comparing what the tool *says it is* with what you assumed. GPU encoders, CUDA wheels and model flags are all build-specific, so the first thing you need is a baseline.
C:\video\; Linux/macOS users substitute /opt/video/.nvidia-smi. AMD/Intel users should use the Vulkan builds of the same tools — the CLI flags differ, so check the repository README.libx264/libx265 and, ideally, NVENC. Confirm the encoder list before planning an encode.du -sh / dir before rendering an hour of footage.ffmpeg -version | head -n 3
ffmpeg -hide_banner -encoders | grep -i nvenc
nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
On Windows PowerShell the second and third lines become:
ffmpeg -hide_banner -encoders | Select-String nvenc
nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv
How to check: torch.cuda.is_available() must print True. The encoder grep must list at least h264_nvenc and hevc_nvenc. If the encoder list is empty, you can still finish the pipeline — use libx265 in Step 2 — but the encode will run on the CPU.
Checklist before Step 1:
ffmpeg -version prints a build string (record it in your notes)nvidia-smi shows the GPU and driver versiontorch.cuda.is_available() returns Truepython inference_realesrgan_video.py --help prints a flag listpython inference_video.py --help prints a flag listExact version numbers, download URLs and flag names change between releases — check the official repository READMEs for the versions you install.
What to do: Measure the input and write down four numbers: width, height, frame rate, frame count. Then decide the scale factor and the target frame rate.
Why: Upscaling multiplies pixels by the square of the factor (2× scale = 4× the pixels), and interpolation multiplies frame count by the multiplier. Both numbers must be known *before* rendering, because they determine disk usage, VRAM pressure and how long the job will take.
ffprobe -v error -select_streams v:0 \
-show_entries stream=width,height,r_frame_rate,avg_frame_rate,nb_frames,codec_name \
-of default=noprint_wrappers=1 input.mp4
Expected output for a typical source:
codec_name=h264
width=1280
height=720
r_frame_rate=30/1
avg_frame_rate=30/1
nb_frames=300
Example: r_frame_rate=30/1 and nb_frames=300 means 10 seconds of video. Target -s 2 upscale → 2560×1440. Target --multi=2 interpolation → 600 frames at 60 fps, still 10 seconds. Budget for 300 input PNGs and 600 output PNGs.
If r_frame_rate and avg_frame_rate differ (for example 30000/1001 vs 24000/1001), the file is variable frame rate. Normalise it before anything else:
ffmpeg -i input.mp4 -vsync cfr -r 30 -c:v libx264 -crf 16 -an input_cfr.mp4
-vsync cfr forces constant frame rate, which is what makes "300 frames = 10 seconds" true.
If you only need a fragment of a long recording, cut it first — trimming with a GUI wrapper is covered in this FFmpeg GUI trimming guide. Cutting before upscaling saves hours, because everything downstream scales with duration.
How to check: Re-run ffprobe on input_cfr.mp4. r_frame_rate and avg_frame_rate must now be identical. Write the four numbers into a notes.txt next to the project folder — you will compare them against the output in Step 3.

What to do: Run four discrete stages. Each stage writes to its own folder, so a crash never forces you to restart from the beginning.
Why: Frame sequences are the only intermediate format every tool agrees on. Real-ESRGAN and RIFE both consume and produce image folders or short clips, and neither knows about your audio track or container — FFmpeg reattaches those at the end.
Stage A — extract frames losslessly.
mkdir -p work/in work/up work/out
ffmpeg -i input_cfr.mp4 -vsync 0 work/in/%06d.png
-vsync 0 prevents FFmpeg from duplicating or dropping frames during extraction. Duplicated frames poison motion estimation in Stage C: RIFE will interpolate between two identical frames and produce nothing.
Stage B — upscale on the GPU.
python inference_realesrgan_video.py \
-i work/in -o work/up \
-n realesr-animevideov3 -s 2 \
--tile 256
-s 2 is the scale factor; --tile splits each frame into tiles so VRAM fits. Flag names differ across releases — confirm with --help if a flag is rejected.
Stage C — interpolate.
python inference_video.py --multi=2 --video=input_cfr.mp4 --output=work/interp
Or, in the frame-folder workflow: rife-ncnn-vulkan -i work/up -o work/out -m <model> -n 2. --multi=2 doubles the frame count; --multi=4 quadruples it. Check the repository README for available model names.
Decision table — which stage runs first:
| Order | GPU cost | Use when |
|---|---|---|
| Interpolate → upscale | Motion estimation runs on small frames; more frames to upscale | VRAM is the bottleneck, or the source is low-resolution |
| Upscale → interpolate | Motion estimation runs on sharp, high-resolution frames; fewer frames to interpolate | Output sharpness matters more than VRAM, e.g. fast camera pans |
Settle this with a measurement, not with an opinion: render a 10-second sample both ways and time each stage.
time python inference_realesrgan_video.py -i work/in -o work/up -n realesr-animevideov3 -s 2
Record the wall-clock number and the frames-per-second line the tool prints. Multiply by your real duration to get the full render estimate.
Stage D — reassemble and encode.
ffmpeg -framerate 60 -i work/out/%06d.png \
-i input_cfr.mp4 \
-map 0:v:0 -map 1:a? \
-c:v hevc_nvenc -preset p5 -rc vbr -cq 23 -pix_fmt yuv420p \
-c:a copy -shortest output_1440p60.mp4
-framerate 60 tells FFmpeg the input sequence is 60 fps — get this wrong and the video plays at double speed or half speed. -map 1:a? copies the original audio if one exists; the ? makes a missing audio track non-fatal.
If hevc_nvenc is unavailable, swap in -c:v libx265 -crf 20 -preset medium.
How to check: Count the frames. Expected output count = input frame count × multiplier.
ls work/out | wc -l # Linux/macOS
(Get-ChildItem work\out).Count # PowerShell
Example: 300 input frames × 2 = 600. If you see 599 or 601, one frame was dropped or duplicated at a stage boundary — re-run that stage on the affected 10-second chunk rather than on the whole file.

What to do: Confirm the container metadata, compare the upscaled result against a reference, and freeze the exact commands you used into a script.
Why: A file that plays is not the same as a file that is correct. Metadata catches the frame-rate and duration mistakes; a similarity metric catches a stage that silently degraded the image; a script catches the fact that you will not remember the flags next week.
ffprobe -v error -select_streams v:0 \
-show_entries stream=width,height,r_frame_rate,nb_frames,duration \
-of default=noprint_wrappers=1 output_1440p60.mp4
Example success output:
width=2560
height=1440
r_frame_rate=60/1
nb_frames=600
duration=10.000000
Duration must match the source, width/height must be exactly the scale factor × source, and nb_frames must equal the number you calculated in Step 2.
Next, compare against a reference. Upscale one frame with your chosen settings, then measure similarity between the final output and a baseline encode at the same resolution:
ffmpeg -i output_1440p60.mp4 -i baseline_1440p60.mp4 -lavfi ssim -f null -
The last line reports SSIM ... All:<value>. Write the number down, change one flag, re-measure: this is the only reliable way to tell whether --tile 256 or a different interpolation model actually helped. Treat any single measurement without a fixed baseline as meaningless — the same discipline described in this guide to testing and evaluating AI models.
Finally, inspect the frame where interpolation usually fails — the fastest pan or the biggest cut. Extract it and look at it:
ffmpeg -ss 00:00:05 -i output_1440p60.mp4 -frames:v 1 check_frame.png
How to check: check_frame.png must open at 2560×1440 with no ghosting halo around moving edges. Ghosting means the motion model could not resolve the displacement — split the clip at the cut and process the segments separately.
Save the whole run as run.sh with the four commands from Step 2, and commit it next to the output. That is your reproducibility record.
Problem. Encoding fails immediately with an "unknown encoder" message.
Unknown encoder 'hevc_nvenc'ffmpeg -hide_banner -encoders | grep -i nvenc. If nothing is listed, either install a build with NVENC or fall back to CPU encoding: -c:v libx265 -crf 20 -preset medium. The pipeline output is identical; only the encode time changes.Problem. The upscaler starts, then stops with a CUDA assertion.
AssertionError: Torch not compiled with CUDA enabledtorch.cuda.is_available() returns False and every GPU call falls through. python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
If it prints False, reinstall PyTorch from the index URL matching your CUDA version (check the official PyTorch install selector for the exact command), then re-run the check until it prints True.
Problem. Upscaling dies partway through a long clip.
CUDA out of memory. Tried to allocate <N> GiB--tile 128 instead of --tile 256), and process the source in 10-second chunks with a shell loop, running ffmpeg -ss <start> -t 10 -i input_cfr.mp4 ... for each chunk and concatenating the results afterwards. Confirm the flag name with --help — it differs between releases.You now have a working four-stage pipeline — probe, extract, upscale, interpolate, re-encode — with a measurement step that tells you which stage order is cheaper on your hardware. From here, two directions: automate the chunk loop into a script that processes a whole folder, or move on to generating the source clips programmatically with Remotion-based AI video automation.
Do I need a GPU to upscale video with a neural network?
No, but the difference is decisive. Real-ESRGAN and RIFE both run on CPU if PyTorch is installed without CUDA, and both become impractically slow for anything longer than a few seconds. Use the CPU path only to validate the pipeline on a 10-second sample.
Should I upscale first or interpolate first?
Run a 10-second sample both ways and compare wall-clock time and output quality. Interpolating first keeps motion estimation on small frames and suits tight VRAM; upscaling first gives the motion model sharp input and suits fast pans. The decision table in Step 2 lists the trade-off.
Why does my upscaled video play at the wrong speed?
The -framerate value in the reassembly command does not match the real frame rate of your sequence. If you interpolated --multi=2 on a 30 fps source, the sequence is 60 fps and -framerate 60 is mandatory. Verify with ffprobe on the output — duration must match the source.
Does frame interpolation fix motion blur or a stuttering source?
No. Interpolation synthesises intermediate frames between existing ones; it cannot recover detail that was never captured, and it amplifies artefacts on VFR sources. Normalise to constant frame rate first, as shown in Step 1, and expect ghosting around cuts — split the clip at each cut instead of interpolating across it.
Status: practical guide. Some statements come from vendors; verify in your environment.
Time to complete: 45–60 minutes of setup, plus render time that you measure on your own 10-second sample before committing to a full render. Difficulty: Intermediate
Machine data: Citation Map JSON · JSON-LD · Semantic Embeddings — separate stage.
| Entity | Type | Role |
|---|---|---|
| улучшить качество видео нейросетью | тема/инструмент | упоминается в заголовке, TL;DR или разделах гайда |
| апскейл видео ИИ | тема/инструмент | упоминается в заголовке, TL;DR или разделах гайда |
| интерполяция кадров | тема/инструмент | упоминается в заголовке, TL;DR или разделах гайда |
| VividFrame-GPU | тема/инструмент | упоминается в заголовке, TL;DR или разделах гайда |
| улучшение видео на видеокарте | тема/инструмент | упоминается в заголовке, TL;DR или разделах гайда |
| Windows | платформа/технология | упоминается в заголовке, TL;DR или разделах гайда |
| Linux | платформа/технология | упоминается в заголовке, TL;DR или разделах гайда |
| macOS | платформа/технология | упоминается в заголовке, TL;DR или разделах гайда |
| CUDA | платформа/технология | упоминается в заголовке, TL;DR или разделах гайда |
| Vulkan | платформа/технология | упоминается в заголовке, TL;DR или разделах гайда |
No independent confirmation is provided; source claims and editorial conclusions are separated below.
| Source | Type | Role |
|---|---|---|
| No external links | — | — |
Cite as: AI Release. Как улучшить качество видео нейросетью на GPU: апскейл, интерполяция кадров и настройка пайплайна. ai-release.org.