NoMachine on Windows: “Error 22: Invalid argument”

[hababeton]

I’m running NoMachine Client 8.16.1 on Windows and trying to authenticate with my YubiKey via PKCS#11. Depending on which DLL I point to, I see one of two behaviors:

When using OpenSC’s PKCS#11

Module: C:\Program Files\OpenSC Project\OpenSC\pkcs11\opensc-pkcs11.dll

After “Connected…” message the session window simply closes without any further prompt or error.

When using YubiKey’s PKCS#11

Module: C:\Windows\System32\ykcs11.dll

Connection aborts with:

Could not connect to the server.

Error is 22: Invalid argument

Relevant log excerpt

Info: Starting NoMachine version 8.16.1.
…
17352 8152 2025-04-23 16:45:50 203.782 DaemonLogin/DaemonLogin: Smartcard api module is(nil).
17352 13972 2025-04-23 16:45:50 232.382 DaemonClientApplication: WARNING! Session terminated abnormally.
17352 13972 2025-04-23 16:45:50 232.382 DaemonClientApplication: WARNING! Error is 22, ‘Invalid argument’.
Warning: Connection to 10.2.2.106 port 4000 failed at 16:45:50 232.382.
Warning: Error is 22, ‘Invalid argument’.
…
ClientSession: Failing reason is ‘Could not connect to the server.

Error is 22: Invalid argument’.

Questions

Has anyone seen “Error 22: Invalid argument” from NoMachine when loading a PKCS#11 module?

Is there a known compatibility issue with OpenSC or YubiKey’s DLL on Windows?

Are there any additional debug flags or registry settings that could shed light on why NoMachine reports the module as nil?

Any guidance or pointers to Windows‐specific PKCS#11 configurations for NoMachine would be greatly appreciated!

[Britgirl]

Support for authentication with smart card has been set-up by relying on the Public Key Infrastructure (PKI) and using an OpenSC compatible smart card, and using the SSH key method (see more about this here: https://kb.nomachine.com/DT07S00230#4, there is also a nice comparison table in chapter 2 which shows you what is supported).

The alternative is to use password authentication. However, we currently have a TR open related to USB forwarding and the remote desktop which shows the login window.

https://kb.nomachine.com/TR05U10850

[Author]

Posts