Hello mrosing,
The error means that some internal PAM module failure occured. PAM errors are usually written to /var/log/auth.log on Ubuntu host. Please, check if you can find anything useful there. Can you share the output of ‘grep -r “nxexec” /var/log/auth.log’?
You can also try modifying PAM configuration for NX protocol authentication, residing in /etc/pam.d/nx file. If the problem doesn’t occur when SSH protocol is in use, you can try to reuse SSH PAM configuration with NX. You can achieve that by running as root:
cp /etc/pam.d/nx /etc/pam.d/nx.ori
cp /etc/pam.d/sshd /etc/pam.d/nx
Let us know if this solves the issue.