Hardware H.264 encoding failure on Intel HD 4000 – libmfx assertion error

[kermu]

Hi,

I am experiencing issues with hardware acceleration on a Thinkpad X230 (Intel HD 4000 / Ivy Bridge) running Linux Mint 21.2 (Ubuntu 22.04 base). NoMachine fails to use hardware encoding and falls back to software mode.

It appears that “libmfx” (Intel Media SDK) has compatibility issues with the “i965” driver on this specific architecture. While the “i965” driver supports VA-API encoding, the “libmfx” wrapper used by NoMachine seems to pass parameters that the driver rejects.

When testing with the Intel Media SDK samples (sample_encode), I get a crash due to an assertion failure. This happens because the library attempts to set a quality level that is out of range for the Ivy Bridge hardware.

Testing Environment:

Hardware: Intel Core i5-3320M / HD Graphics 4000 (Ivy Bridge)
OS: Linux Mint 21.2 / Ubuntu 22.04
NoMachine server Version: 9.3.7, 8.9.1
NoMachine client Version: 9.3.7
VA-API Driver: i965-va-driver-shaders (v2.4.1)

Direct sample_encodel log (libmfx 1.35 from Ubuntu 22.04 repo and 1.32 form https://old-releases.ubuntu.com/ubuntu/pool/universe/i/intel-mediasdk/libmfx1_20.1.0-0ubuntu1_amd64.deb):

libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva error: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: va_openDriver() returns 0
…
Target usage: balanced
libva info: VAEncMiscParameterBufferQualityLevel.quality_level (4) out of range (max 2).
sample_encode: …/mfx_h264_encode_vaapi.cpp:2989: virtual …: Assertion `0x00000000 == vaSts’ failed.
Aborted (core dumped)

As seen in the logs, the hardware correctly reports a max quality level of 2, but the library tries to use 4, causing a hard assertion failure.

My questions:

1. Is there a way to force NoMachine to use pure VA-API (via libva) instead of going through the “libmfx” wrapper?
2. Can I manually limit the quality_level (Target Usage) in node.cfg to prevent this assertion failure?
3. If I disable NoMachine’s internal “libmfx.so”, it defaults to software encoding. Is there any workaround to make it recognize the “i965” VA-API backend directly?

Any help or guidance on how to enable hardware encoding for this legacy Intel hardware would be greatly appreciated.

Best regards
Krzysztof Szwaba

[Britgirl]

Hi,

Is there a way to force NoMachine to use pure VA-API (via libva) instead of going through the “libmfx” wrapper?

Unfortunately, no, although there is good news. We are currently working on support for the VA-API encoder. This will be made available in the one of our releases next year.

Can I manually limit the quality_level (Target Usage) in node.cfg to prevent this assertion failure?

No, but NoMachine does not use this buffer. It’s worth noting that MFX/VPL Intel HW encoders are mostly made for iHD drivers, not i965.

If I disable NoMachine’s internal “libmfx.so”, it defaults to software encoding. Is there any workaround to make it recognize the “i965” VA-API backend directly?

Are you asking how to changing to i965 from iHD? Please check the article in our knowledge base which contains everything you need to know about Intel HW encoding and currently supported API, http://www.nomachine.com/AR09O00938, but there’s also the LIBVA_DRIVER_NAME environment variable where you can set whatever driver you want to be loaded. You can try that.

[Author]

Posts