If you can explain better what you need to do exactly, we can point you to the right place on our website 🙂
Besides the NoMachine compression protocol, which is not public anymore, since version 4, there is not a single NoMachine API, but many different pieces: NoMachine server, the node, the player, which interact with each other using the NoMachine libnx core.
Additionally, the server offers a “pluggable scripts” interface which lets programmers add their own extensions to the server, so that their custom functions are executed before, during and after each session startup and shutdown.
This server interface is described in the document “NoMachine Enterprise Products – Server Guide”, which you can find here: https://www.nomachine.com/DT07M00090, in section 9.
It’s also possible to run the nxplayer from the command line and customize the session file (NXS configuration). More about this is here: https://www.nomachine.com/AR04O00923
If this documentation is not enough, don’t hesitate to explain what you’d like to achieve.