Node.js subagent

[flytech-ben]

Recently I implemented a solution for monitoring various external APIs using oauth authentication with multiple different credentials (for different customers). This solution is implemented on a single NetXMS agent for data collection and customer nodes in NetXMS console can query their metrics using "source node" of the DCI with this single agent.

First I started implementing the solution using PowerShell scripts for ExternalMetric and ExternalTable. This implementation had several problems, the main issue was performance: Some metrics are requested every 30 seconds, for several hundred different parameter values. Calling the PowerShell runtime for every requested metric caused a huge load on the central agent's system. Also, some metrics / external APIs do not return their value immediently and need some time for response, which does not work very well for ExternalMetric.

So I consolidated some of the metrics and wrote scripts for ExternalMetricProvider which did improve the system load, but I still had no solution for ExternalTable as this do not have a provider option.

Finally I dropped everything I had and re-implemented the PowerShell scripts as JavaScript and started running all of them in a single Node.js instance. The system load dropped to an insignificant level as the gathering of the metrics and tables is not very CPU intensive and Node.js handles asychronous calls to external APIs very well. Also, the single Node.js process is running continously in contrast to frequently starting and terminating PowerShell runtime processes.
The collected metrics and tables are stored in a SQLite database. This way the NetXMS agent can query the metrics and tables using the built-in SQLite driver and the DBQUERY subagent which is also very lightweight and super-fast without any significant load on the CPU. This works a bit similar to Push parameters using the local cache.

All in all, everything is running pretty smoothly at the moment but is quite cumbersome to set up. This can be considered as an idea for a subagent which embeds scripting ability within the NetXMS agent using Node.js and extends the function of ExternalMetric, ExternalTable and ExternalMetricProvider.

[flytech-ben]

I realized that there is Python-subagent for scripting, but I can't find any examples how to use it or any documentation. Unfortunately I don't know much Python but I'm happy to learn.

[Filipp Sudanov]

To mention, tables now can work in provider mode - there's new syntax for defining tables in agent config, what we need there is PollingInterval parameter.

Also, did you check if Web Service functionality could do the job for you? It's possible to use that from NXSL scripts.

I actually did not know that there's python subagent, judging by number of commits it's not actively developed, will check with developers in a while on it's status.