AMD H.264 Acceleration in Windows 10 VM

[cvbx]

Hello Team NoMachine!

So far I have been very pleased with NoMachine, maybe we can figure out the only bit that separates me from Elysium:

I am running the latest NoMachine Server on a Windows 10 VM.
The VM is virtualized with QEMU (Proxmox) and the GPU (AMD Radeon RX480, 8GB VRAM) is properly and exclusively passed through to the VM. GPU Drivers are installed, and 3D acceleration has been confirmed via running several 3D applications.

What does not seem to work is NoMachines H.264 GPU acceleration in that scenario. Taking a look at the servers’ session logs reveals:

NXAGENT – Version 7.1.3

Copyright (C) 2001, 2019 NoMachine.
See http://www.nomachine.com/ for more information.

Session: Starting session at Tue Feb 23 16:38:57 2021.
Info: Agent running with pid 8136.
Info: Slave server running with pid 3708.
Info: Listening to slave connections on port 12001.
Info: Display running with pid 4520.
Info: Audio server started with pid 9664.
Info: Audio client started with pid 9968.
Info: Display server started with pid 6924.
Session: Session started at Tue Feb 23 16:38:57 2021.
Info: Events handler running with pid 2480.
Info: Screen blanking running with pid 9448.
Info: Using SSE3 for screen analysis.
Info: Screen capture running with pid 7920.
Info: Screen analysis running with pid 7924.
Info: Using desktop duplication for screen updates.
8136 7920 16:39:32 634.220 GPUEngine: Configuring GPU resources.
Info: Using screen size 1920×1080.
Info: Display server for 96195FCCABCE9371F4D107D3AB41B85E connected on Tue Feb 23 16:39:32 2021.
Info: Audio server for 96195FCCABCE9371F4D107D3AB41B85E connected on Tue Feb 23 16:39:32 2021.
Info: Audio client for 96195FCCABCE9371F4D107D3AB41B85E connected on Tue Feb 23 16:39:32 2021.
Info: Intel Quick Sync H.264 acceleration is not supported.
Info: Please consider updating your Intel drivers.
Info: Using AMD VCE 1.4 hardware encoder.
VCE: ERROR!: Surface not allocated.
8136 7920 16:39:33 273.220 Vce14EncodeData: ERROR! Data encoding failed.
8136 9708 16:39:33 273.220 Writer/Writer: WARNING! Failing writer for FD#2376.
Info: Using DirectX for screen updates.
8136 7920 16:39:53 310.828 GPUEngine: Configuring GPU resources.

Is there anything I can do to get hardware encoding running in my scenario?

[cvbx]

I solved it myself. So NoMachine was actually not able to properly talk to the display adapter.

The solution was to completely disable the VM Display within Proxmox by setting it to “None”.
Additionally, you must either

(1) hook up a physical display to the GPU that is being passed through – even if you exclusively VNC into it
(2) buy a 5€ display dummy adapter for HDMI/DisplayPort and the like. Those things emulate a connected display.

This deactivates the basic Microsoft display adapter.
After I hooked up a display to a GPU output, everything was fine. GPU acceleration and everything is just running as expected now.

[cvbx]

Alternative

(3) Install a virtual display driver (disencouraged)
Virtual display drivers are a royal pain. Only use as last resort.

[Carin]

Hi cvbx,

Thank you for letting us know, we are glad you’ve found the solution!

[Author]

Posts