Wayland is not supported in the NoMachine virtual desktops, you can only use x11 desktops.
At the moment there is no possibility to add more Desktop types to the list, there is Feature Request for this: https://www.nomachine.com/FR09L02833.
However you can still create separate different .nxs connection file for the specific session type.
Create first connection file to start Gnome 3D as default on RHEL8 choose New Desktop and then click “Create a new virtual desktop” and select option “Save this settings in a connection file”;
Create second connection file to start Gnome Classic and then click “Create a new custom session”-> “Run the following command: env GNOME_SHELL_SESSION_MODE=classic /bin/dbus-launch –exit-with-session gnome-session –session gnome-classic” + important is to select options “Run the command in a virtual desktop” and “Save this settings in a connection file”