NetXMS 3.0 released

Victor Kirhenshtein

Hi all!

NetXMS 3.0 is officially out! There are few visual changes and lot of them under the hood. Development focus was on improving system stability and performance, especially for large networks: we have improved TimescaleDB support which is now production ready; zones can have multiple proxies for both redundancy and load sharing; embedded scripting language - NXSL - is up to ten times faster then in previous version.
Upgrade should be smooth in most cases, but there are few changes that could potentially break things:

We have switched from TRE to PCRE as regular expression engine. In most cases this should go unnoticed, but in certain cases regular expressions may stop working or produce unexpected results.
Some bit fields in NXSL classes (most notably flags in Node class) had changed their meaning. If your script use those fields please check NXSL documentation for changes.
Agent policies were merged with templates - all existing policies will be converted into separate templates, and new policies has to be created under template object.
SMS drivers replaced by notification channels (so it is now possible to have more than one channel for messaging). Existing driver configuration should be automatically converted to channel named SMS.

Starting with this release we are changing versioning scheme. From now on product version consists of two numbers - major version and minor version. Third number represents build (patch) number. After release there will be no new features in current version, only critical bug fixes. It is guaranteed that all components with same major and minor versions are interoperable. We will publish patched versions (with new build number) only for affected components. So current product version is 3.0, release build is 2258, and next version will be 3.1.

Full change log for version 3.0:

Event tags
ICMP polls for collecting ICMP response time statistic on server side
User agents (desktop support application)
Raw DCI values saved in history along with corresponding transformed values
New DCI data types - "32 bit counter" and "64 bit counter"
Collected DCI data recalculation based on stored raw values and current transformation settings
Agent always requires encryption unless RequireEncryption parameter explicitly set to off
Default values for missing custom attributes in macros (in form %{attr:default})
Support for macros in threshold values
Improved network discovery filter script capabilities
Internal server parameters for counting client sessions
New hook script UpdateInterface
Node creation option "create as zone proxy"
JSON support in NXSL
Improved NXSL performance
Null values interpreted as "false" in NXSL logical expressions
Added method executeSSHCommand in NXSL class "Node"
Cleanup of collected DCI data by housekeeper can be disabled
Service group support in Tuxedo subagent
Improved Enduro/X compatibility
Improved Cisco Aironet (former Airespace) driver
Added driver for Cisco Nexus switches
Duplicate IP address detection during network discovery
Improved I/O performance parameters in Windows agent
Optional JSON format for log files
Option to use stdout as log output device
Switched to pcre (from libtre) as regular expression matching engine
Event parameters exposed as writable "Event" class attributes in NXSL
New method "addParameter" in NXSL class "Event"
"Origin timestamp" for events
Object management functions in NXSL implemented as object methods
SNMP functions in NXSL implemented as transport methods
Range access for strings and arrays in NXSL via [:] operator
Access to VLAN information from NXSL
Object tool type "Agent Table" now used to read agent tables and "Agent List" to read agent lists
Fixed issues:
    NX-183 (Implemented responsible users for objects)
    NX-725 (Add alarm comments from NXSL)
    NX-900 (Server started during nxdbmgr check forced repair)
    NX-1021 (Add event SYS_SERVER_STARTED)
    NX-1102 (Implement event groups)
    NX-1199 (SNMP credentials per-zone)
    NX-1219 (Add 'created' and 'last login' columns to User Manager)
    NX-1271 (Implement internal server topology map)
    NX-1307 (After a node is deleted, its tunnel can't be unbound)
    NX-1386 (Implement units for server variables)
    NX-1426 (Transformation scripts for SNMP traps)
    NX-1479 (DCI poll spread after NetXMS server/service restart)
    NX-1562 (Add "tunnel only" option to node's agent communication configuration)
    NX-1601 (Log Policy editor can't parse xml with unknown event code)
    NX-1643 (Extra parameters for event generated by log parser)

Best regards,
Victor

jermudgeon

Excellent! Thanks for all the work.

Tursiops

Thanks to the team for all the effort put into this. Great Release!

Just a practical note on this one:

We have switched from TRE to PCRE as regular expression engine. In most cases this should go unnoticed, but in certain cases regular expressions may stop working or produce unexpected results.

A side effect of this change is that if you are matching a regular expression, even without a capture group, $1, $2, etc. will be reset to null values.
An transform script example of where that might cause an issue:

if ( $1 ~= "^Hello\s+World" ) {
  return $1;
} else {
  return "Something Else";
}

In prior versions of NetXMS, if the DCI returned "Hello World", the transform script would have returned exactly that.
With version 3, the transform script will return null.

You will need to change your script to something like this:

value = $1;
if ( value ~= "^Hello\s+World" ) {
  return value;
} else {
  return "Something Else";
}

tomaskir

@tursiops said in NetXMS 3.0 released:

Just a practical note on this one:

We have switched from TRE to PCRE as regular expression engine. In most cases this should go unnoticed, but in certain cases regular expressions may stop working or produce unexpected results.

A side effect of this change is that if you are matching a regular expression, even without a capture group, $1, $2, etc. will be reset to null values.
An transform script example of where that might cause an issue:
if ( $1 ~= "^Hello\s+World" ) {
  return $1;
} else {
  return "Something Else";
}
In prior versions of NetXMS, if the DCI returned "Hello World", the transform script would have returned exactly that.
With version 3, the transform script will return null.

You will need to change your script to something like this:
value = $1;
if ( value ~= "^Hello\s+World" ) {
  return value;
} else {
  return "Something Else";
}

Is this a known and expected change or a bug?
This will break A LOT of my NXSL scripts...

normalcy

In the console, when deploying agent packages with the package manager I'm getting an error status "failed" "Agent's version doesn't match package version after upgrade".

The 3.0 agents seem to be installed on the clients and look to be working (polls work and send back 3.0 agent version) but summary tables show 3.0-2258-gc5516133f2 agent verison, would that be cause of error if 'package version' is something like 3.0-2258 only?

brettmilfos

Just upgraded and have found an issue with SMS.

It created the migrated notification channel but did not work.

I tried to create a new notification channel, but there is nothing to select in the Driver name drop down.

Thanks, Brett.

Tursiops

@normalcy said in NetXMS 3.0 released:

In the console, when deploying agent packages with the package manager I'm getting an error status "failed" "Agent's version doesn't match package version after upgrade".

The 3.0 agents seem to be installed on the clients and look to be working (polls work and send back 3.0 agent version) but summary tables show 3.0-2258-gc5516133f2 agent verison, would that be cause of error if 'package version' is something like 3.0-2258 only?

Yes, that is correct.
The next agent release is going to fix that. The hash shouldn't be in the version name anymore.
Alternatively, you can remove the package from package manager, edit your NPI file and re-import it.
In the NPI file, change the "VERSION" line to match the one you're seeing on the agents right now. That way when NetXMS checks post-upgrade, it will find that the version matches what it expects to see.

Tursiops

@tomaskir said in NetXMS 3.0 released:

@tursiops said in NetXMS 3.0 released:
Just a practical note on this one:

We have switched from TRE to PCRE as regular expression engine. In most cases this should go unnoticed, but in certain cases regular expressions may stop working or produce unexpected results.

A side effect of this change is that if you are matching a regular expression, even without a capture group, $1, $2, etc. will be reset to null values.
An transform script example of where that might cause an issue:
if ( $1 ~= "^Hello\s+World" ) {
  return $1;
} else {
  return "Something Else";
}
In prior versions of NetXMS, if the DCI returned "Hello World", the transform script would have returned exactly that.
With version 3, the transform script will return null.

You will need to change your script to something like this:
value = $1;
if ( value ~= "^Hello\s+World" ) {
  return value;
} else {
  return "Something Else";
}
Is this a known and expected change or a bug?
This will break A LOT of my NXSL scripts...

That was an expected change at the time (we were running the beta) and broke several scripts in our system, hence I thought I'd point this out by giving an example.
I'd recommend going through the scripts prior to upgrading.

batnator

Thanks for the work and this great software which makes my life as an admin easier.

Victor Kirhenshtein

@brettmilfos said in NetXMS 3.0 released:

Just upgraded and have found an issue with SMS.

It created the migrated notification channel but did not work.

I tried to create a new notification channel, but there is nothing to select in the Driver name drop down.

Thanks, Brett.

How did you install server? On what platform?

Best regards,
Victor

lindeamon

hi victor,

keep up the good work.

Lindeamon

Victor Kirhenshtein

@tomaskir said in NetXMS 3.0 released:

@tursiops said in NetXMS 3.0 released:
Just a practical note on this one:

We have switched from TRE to PCRE as regular expression engine. In most cases this should go unnoticed, but in certain cases regular expressions may stop working or produce unexpected results.

A side effect of this change is that if you are matching a regular expression, even without a capture group, $1, $2, etc. will be reset to null values.
An transform script example of where that might cause an issue:
if ( $1 ~= "^Hello\s+World" ) {
  return $1;
} else {
  return "Something Else";
}
In prior versions of NetXMS, if the DCI returned "Hello World", the transform script would have returned exactly that.
With version 3, the transform script will return null.

You will need to change your script to something like this:
value = $1;
if ( value ~= "^Hello\s+World" ) {
  return value;
} else {
  return "Something Else";
}
Is this a known and expected change or a bug?
This will break A LOT of my NXSL scripts...

That's complicated issue. Current implementation is a bit messy with $n variables after matching. Consider the following code:

v = "error 43";
if (v ~= "(error|warning) ([0-9]+)")
	println("Step 1: $1=".$1." $2=".$2);
if (v ~= "(error .*|info ([0-9]+))")
	println("Step 2: $1=".$1." $2=".$2);

In NetXMS 2.2 $2 will be equal "43" on step 2, which is kind of misleading. It was because implementation in 2.2 silently skipping unmatched capture groups. Implementation in 3.0 resets all unused $n variables, which has side effect of clearing $1 if there were no capture groups at all. Looks like it was bad decision to use same names for capture groups and script arguments.

I want to gradually clean up inconsistencies in NXSL, and that could lead to break up of some things. I think that for now I can introduce server configuration variable (off by default) that will prevent resetting of unused capture group variables. Also I will introduce separated naming for script arguments and capture groups so scripts could be migrated over time to non-overlapping variable naming. Also for transformation scripts I think it is worth passing raw value not only as first argument to the script but also as global variable, like $rawValue.

Best regards,
Victor

vir_db

Good morning.

just upraded to 3.0: more or less all hosts reports this alarm: Status of DCI xxxx (NetXMS Agent: sqlite-hosts) changed to UNSUPPORTED. (seems it comes from "Local Database" template)

just the server itself reports:

Status of DCI 697 (Internal: Server.ThreadPool.Usage(SYNCER)) changed to UNSUPPORTED
Status of DCI 696 (Internal: Server.ThreadPool.LoadAverage(SYNCER,1)) changed to UNSUPPORTED
Status of DCI 695 (Internal: Server.ThreadPool.CurrSize(SYNCER)) changed to UNSUPPORTED
Status of DCI 694 (Internal: Server.ThreadPool.Load(SYNCER)) changed to UNSUPPORTED

looking at Data Collection Configuration, those DCIs doesn't exists.

Also tried to delete the database (PGSQL) and have a fresh start, but the results is the same.

How can I solve?

thanks alot and best regards

tomaskir

@victor said in NetXMS 3.0 released:

That's complicated issue. Current implementation is a bit messy with $n variables after matching. Consider the following code:
v = "error 43";
if (v ~= "(error|warning) ([0-9]+)")
	println("Step 1: $1=".$1." $2=".$2);
if (v ~= "(error .*|info ([0-9]+))")
	println("Step 2: $1=".$1." $2=".$2);
In NetXMS 2.2 $2 will be equal "43" on step 2, which is kind of misleading. It was because implementation in 2.2 silently skipping unmatched capture groups. Implementation in 3.0 resets all unused $n variables, which has side effect of clearing $1 if there were no capture groups at all. Looks like it was bad decision to use same names for capture groups and script arguments.

I want to gradually clean up inconsistencies in NXSL, and that could lead to break up of some things. I think that for now I can introduce server configuration variable (off by default) that will prevent resetting of unused capture group variables. Also I will introduce separated naming for script arguments and capture groups so scripts could be migrated over time to non-overlapping variable naming. Also for transformation scripts I think it is worth passing raw value not only as first argument to the script but also as global variable, like $rawValue.

Best regards,
Victor

The problem here is that there are likely thousands of scripts all around the Forums, Wiki and the internet that this change breaks.
Every single one would have to be updated - which is unrealistic.
For new users who find some regex-based scripts on the internet, they will not work, and they don't know why.

Existing implementations (scripts) will start working differently upon upgrade, without any knowledge of the system operators
This will be VERY difficult to track down, as this will not produce script errors, rather things will just work differently.
This means there can be a threshold script which will "pretend" to still work, but in reality will not work.
It puts burden on system operators to review every script they have upon upgrade to check if it still works properly

What I personally would propose:
There are far less scripts using regex capture groups than just regex.
So if a change is needed, only regex capture group should be changed, without affecting regex operations without capture groups (as is the case now).

What about capturing regex returning arrays with captures?

matchArrray = "test" ~= "t(e)(s)t";

println(matchArrray [0] . matchArrray [1]);
for (match : matchArrray) {
  print(match);
}

This would clean up the reuse of $1, $2, etc. without breaking ALL regex handling.
The change would only affect a smaller part of community, so less impact.

Victor Kirhenshtein

@vir_db said in NetXMS 3.0 released:

Good morning.

just upraded to 3.0: more or less all hosts reports this alarm: Status of DCI xxxx (NetXMS Agent: sqlite-hosts) changed to UNSUPPORTED. (seems it comes from "Local Database" template)

just the server itself reports:

Status of DCI 697 (Internal: Server.ThreadPool.Usage(SYNCER)) changed to UNSUPPORTED

Status of DCI 696 (Internal: Server.ThreadPool.LoadAverage(SYNCER,1)) changed to UNSUPPORTED

Status of DCI 695 (Internal: Server.ThreadPool.CurrSize(SYNCER)) changed to UNSUPPORTED

Status of DCI 694 (Internal: Server.ThreadPool.Load(SYNCER)) changed to UNSUPPORTED

looking at Data Collection Configuration, those DCIs doesn't exists.

Also tried to delete the database (PGSQL) and have a fresh start, but the results is the same.

How can I solve?

thanks alot and best regards

You van safely ignore them. It is cosmetic issue in default templates, we will fix that in next patch release.

Best regards,
Victor

vir_db

Thanks Viktor,

just another issue: seems that the upgrade brok the grafana plugin. I can get alerts, but I cannot collect any graphs (for ex. CPU Usage)

Alex Kirhenshtein

Plugin in grafana repo will be updated shortly.
In a meantime you can update it directly from git.

@vir_db said in NetXMS 3.0 released:

Thanks Viktor,

just another issue: seems that the upgrade brok the grafana plugin. I can get alerts, but I cannot collect any graphs (for ex. CPU Usage)

vir_db

Hi Alex,
do you mean this git? https://github.com/netxms/grafana

seems last commit was in june:

Commits · netxms/grafana

Grafana integration module for NetXMS. Contribute to netxms/grafana development by creating an account on GitHub.

GitHub (github.com)

paul

Fantastic work Victor and the team - including those who helped in the testing.

I will wait for the first patch release - or Saturday - whichever comes first

Victor Kirhenshtein

@tomaskir said in NetXMS 3.0 released:

@victor said in NetXMS 3.0 released:
That's complicated issue. Current implementation is a bit messy with $n variables after matching. Consider the following code:
v = "error 43";
if (v ~= "(error|warning) ([0-9]+)")
	println("Step 1: $1=".$1." $2=".$2);
if (v ~= "(error .*|info ([0-9]+))")
	println("Step 2: $1=".$1." $2=".$2);
In NetXMS 2.2 $2 will be equal "43" on step 2, which is kind of misleading. It was because implementation in 2.2 silently skipping unmatched capture groups. Implementation in 3.0 resets all unused $n variables, which has side effect of clearing $1 if there were no capture groups at all. Looks like it was bad decision to use same names for capture groups and script arguments.

I want to gradually clean up inconsistencies in NXSL, and that could lead to break up of some things. I think that for now I can introduce server configuration variable (off by default) that will prevent resetting of unused capture group variables. Also I will introduce separated naming for script arguments and capture groups so scripts could be migrated over time to non-overlapping variable naming. Also for transformation scripts I think it is worth passing raw value not only as first argument to the script but also as global variable, like $rawValue.

Best regards,
Victor
The problem here is that there are likely thousands of scripts all around the Forums, Wiki and the internet that this change breaks.
Every single one would have to be updated - which is unrealistic.
For new users who find some regex-based scripts on the internet, they will not work, and they don't know why.

Existing implementations (scripts) will start working differently upon upgrade, without any knowledge of the system operators
This will be VERY difficult to track down, as this will not produce script errors, rather things will just work differently.
This means there can be a threshold script which will "pretend" to still work, but in reality will not work.
It puts burden on system operators to review every script they have upon upgrade to check if it still works properly

What I personally would propose:
There are far less scripts using regex capture groups than just regex.
So if a change is needed, only regex capture group should be changed, without affecting regex operations without capture groups (as is the case now).

What about capturing regex returning arrays with captures?
matchArrray = "test" ~= "t(e)(s)t";

println(matchArrray [0] . matchArrray [1]);
for (match : matchArrray) {
  print(match);
}
This would clean up the reuse of $1, $2, etc. without breaking ALL regex handling.
The change would only affect a smaller part of community, so less impact.

I've changed regexp expressions to return array with capture groups instead of boolean value. It still sets $n variables for compatibility but does not erase unused variables (so if there is no capture groups in regexp $1 will be left intact). As a side effect it is now possible do directly access capture group from regexp. For example the following code

text = "error 43";
errorCode = (text ~= "error ([0-9]+)")[1];
println(errorCode);

will print 43.

Best regards,
Victor

NetXMS Forum

NetXMS 3.0 released

Commits · netxms/grafana