I’m encountering the same error during an install on XP. After clicking ‘Next’ the error dialog window pops up as per the OP’s screenshot, and upon clicking ‘OK’ it cancels the setup process with a screen saying:
Warning
Updating to a different server type not supported
With the only buttons as ‘Back’ and ‘Cancel’
Using NoMachine v5.1.9_7 (free)
It successfully installed on my Windows 8.1 machine, but I needed it to be installed on my XP machine.