NetXMS Forum

I've experienced the same issue for the past few versions of netXMS. 

Operating System:
Windows 11 Pro

Currently using client version 4.5.2

Theme: Windows Dark

Good Evening!

Could you please clarify if the time filter option in EPP is using the Server time, or the local agent time? 

Please also note there appears to be a typo as of version 4.5.2 in the condition -> Events window. Inverse rule (as spelled on the other condition windows ) appears to be spelled "Invertse". I haven't seen this spelling on any of the other windows.

Thanks,
Chris

I'll be updating this using correct syntax for NetXMS v5.0.

This could be a whole range of issues, but I would first try to use your netxms server to ping the device you are trying to reach via agent. If that works, I would then check the devices firewall settings to make sure the port is open.

Hi Filipp, Thank you for the clarification! That makes me less stressed out. I was getting worried I would have to rewrite everything I've written so far.

Labdien, NetXMS komanda!

Es neesmu pārliecināts, vai nav par vēlu, bet es cerēju sniegt dažus ieteikumus V5.0 sintakses izmaiņām saistībā ar:

https://track.radensolutions.com/issue/NX-2343

Virknes savienošana — funkcijas pieprasījums
Ja iespējams, es ieteiktu mainīt saiti uz "+", nevis ".". Šādi Python saista virknes, un es domāju, ka tas radītu mazāk problēmu, mainot veidu, kā piekļūt atribūtiem un metodēm izskatās, ka tas mainīsies. Var palaist datu tipa pārbaudi, un, ja tips ir virkne, "+" tiks savienots, un, ja tips ir skaitlis, tas veiktu pievienošanas funkciju.

Man patīk lietot "." "->" vietā atribūtiem un metodēm, jo tas man šķiet vairāk kā Python. (Man patīk python, ja jūs nevarat pateikt.)

Jautājumi
Vai būs pārejas periods, lai mēs varētu migrēt esošos skriptus un filtrus uz jauno sintaksi?
Piemēram, ar virkņu savienošanu, pievienojot ".." vai "+" kā vēlamo un dokumentēto veidu virkņu savienošanai.

Funkciju pieprasījumi

• Pievienojiet brīdinājumu kompilēšanas funkcijai, lai cilvēki zinātu, ka viņiem ir kods, kas tiks bojāts turpmākajos laidienos, kas ir jāatjaunina.

• Pievienojiet brīdinājuma uznirstošo logu klienta startēšanas laikā, kas informē, ka mums ir kods, kas tiks bojāts turpmākajos laidienos.

• Pievienojiet rīku cilnei Rīki, kas pārbaudīs, vai katrā skriptā nav novecojuša koda, un nodrošinās skriptu sarakstu un to atrašanās vietu. Piemēram: "EPP noteikumam Nr. 21 ir viens vai vairāki novecojuši metožu izsaukumi filtrēšanas skriptā"

Tulkots no angļu valodas, izmantojot Google tulkotāju

Hello NetXMS Team,

I'm not sure if it's too late, but I was hoping to give some suggestions for V5.0 Syntax changes with regards to 

https://track.radensolutions.com/issue/NX-2343

String Concatenation - Feature Request
I would suggest changing concatenation to "+" instead of ".." if possible. This is how Python concatenates strings and I think it would lead to less problems with the change to the way accessing attributes and methods looks like it's going to change. A check for data type could be run and if the type is a string the "+" would concatenate, and if the type is a number, it would perform the addition function.

I like using "." instead of "->" for attributes and methods as it feels more like Python to me. ( I like python if you can't tell.) 

Questions
Will there be a transition period to give time for us to migrate existing scripts and filters to the new syntax? 
For example, with string concatenation, adding either the ".." or "+" as the preferred and documented way of performing string concatenation. 

Feature Requests

• Add a warning to the compile function letting people know they have code that will be broken in future releases that needs to be updated.
• Add a warning popup on client startup that lets us know we have code that will be broken in future releases.
• Add a tool to the Tools tab that will check each script for deprecated code and provide a list of scripts and where they are located. For Example: "EPP Rule #21 has one or more deprecated method calls in Filtering Script"

Acknowledge Alert
When an alarm is acknowledged or sticky acknowledged in NetXMS, send an API call via curl to Opsgenie to acknowledge the alert.

Server Action:
Name: OpsGenie Acknowledge Alarm
Type:  Execute command on management server
Command:

C:\\Windows\\System32\\curl.exe -v -X POST https://api.opsgenie.com/v2/alerts/%1/acknowledge\?identifierType=alias -H "Content-Type: application/json" -H "Authorization: GenieKey XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" -d "{ \\"user\\": \\"netXMS Admin\\", \\"source\\": \\"netXMS\\", \\"note\\":\\"Acknowledged via Alert API from netXMS \\n Alarm ID:%1 \\n Time: %t\\" }"

 
Macros Used:
     ·         %1 – Alarm Source Event Name + GUID
     ·         %t - %t - Event’s timestamp is a form day-month-year hour:minute:second.
 
Event Processing Policy Rule
Conditions:
 Event Code:
     ·         Xms_Alarm_Ack
     ·         Xms_Alarm_StickyAck
Actions:
 Execute Server Action called Opsgenie Acknowledge Alarm

Comments: Acknowledge OpsGenie alert when NetXMS alarm is acknowledged

Close Alert
When an alarm is resolved in NetXMS, send an API call via curl to Opsgenie to close the alert.
Server Action:

Name: OpsGenie Close Alert
Type:  Execute command on management server
Command:

C:\\Windows\\System32\\curl.exe -v -X POST https://api.opsgenie.com/v2/alerts/%1/close\?identifierType=alias -H "Content-Type: application/json" -H "Authorization: GenieKey XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" -d "{ \\"user\\": \\"netXMS Admin\\", \\"source\\": \\"netXMS\\", \\"note\\":\\"Alert Closed via Alert API from netXMS \\n Alarm ID:%1 \\n Time: %t\\" }"

Macros Used:
·         %1 – Alarm Source Event Name + GUID
·         %t - %t - Event’s timestamp is a form day-month-year hour:minute:second.
 
Event Processing Policy Rule
Conditions:
            Event Code:
     ·         Xms_Alarm_Resolve

Actions:
Execute Server Action called Opsgenie Close Alert

Comments: Close OpsGenie alerts for resolved Alarms
 

Known issues/limitations:
·         Alert status change in OpsGenie does not change alarm status in NetXMS.
·         Custom OpsGenie routing rules may cause priority to be assigned improperly if priority tag is not used for severity calculation in OpsGenie.
 
Notes:
Creating additional custom attributes and filtering scripts can give better control over what events result in OpsGenie notifications. For example, servers and switches may be a higher priority than desktops or laptops.

To Acknowledge or close your OpsGenie Alert we have to modify a built in script to post new events.

Script Name: Hook::AlarmStateChange
Code:

sub main()
{
 eventserver = FindNodeObject($node, 100);
 global alarmstate = $alarm->state;
 global sourceAlarm = $alarm->id;
 global eventname = $alarm->eventName;
 global sourceobj = FindObject($alarm->sourceObject);
 global nameobj = sourceobj->guid;


 switch(alarmstate)
 {
/* Alarm State is Outstanding */
     case "0":
         break;
/* Alarm State is Acknowledged */
     case "1":
         PostEvent(eventserver, "Xms_Alarm_Ack","ACKALARM", eventname . nameobj);
         break;
/* Alarm State is Resolved */
     case "2":
         PostEvent(eventserver, "Xms_Alarm_Resolve","RSLVALARM", eventname . nameobj);
         break;
/* Alarm State is Sticky Acknowledged */
     case "17":
         PostEvent(eventserver, "Xms_Alarm_Ack","SACKALARM", eventname . nameobj);
         break;
 }
}

We also have to create these events in Configuration->Event templates

Alarms Needed:
·        Xms_Alarm_Ack
    o    Message: ALARM ACK - Alarm: %1
    o    Write to log: TRUE
    o    Description: Event created when an Alarm is acknowledged
    o    Severity: Normal
·        Xms_Alarm_StickyAck
    o    Message: ALARM STICKY_ACK - Alarm: %1
    o    Write to log: TRUE
    o    Description: Event created when an Alarm is sticky acknowledged
    o    Severity: Normal
·        Xms_Alarm_Resolve
    o    Message: ALARM RESOLVED - Alarm: %1
    o    Write to log: TRUE
    o    Description: Event created when an Alarm has been resolved
    o    Severity: Normal

Event Processing Policy

Conditions:
Event Codes:
·        SYS_NODE_DOWN
Filtering Script:

if (IsAlerted@$node == "Yes" )
{
    return true;
}
else
{
     return false;
}

The filtering script will check for a Custom Attribute on the node called IsAlerted. If the Custom Attribute does not exist you may see script error alarms. This filter is optional and is used to prevent notification spam. You can set the attribute to “No” for your entire network and then set the attribute to “Yes” for the nodes you wish to be alerted.

Actions:
We will assign two actions to this Event Processing Policy. The first action will be to create an alarm in NetXMS. The second action will be to execute our server action called OpsGenie Create Alert.
Alarm configuration
    ·        Message: %m
    ·        Alarm key: NODE_DOWN_%i
    ·        Alarm severity: From event
    ·        Alarm timeout: 0
Comments:  Create Alarm and OpsGenie Alert when Node is Down

Make sure to save your EPP Rules.

Hello,
Through extensive setup and testing, I have been able to implement an integration with OpsGenie using NetXMS server actions and the OpsGenie API. This is a one-way integration currently and I may work on a two-way integration in the future.

Key Components:

• OpsGenie API
• Custom Scripts in Script Library
• Event Processing Policy Rules

The most basic functionality using this would be to create an OpsGenie alert when a node is down. This guide will walk you through setting up functionality to create, acknowledge, and close OpsGenie alerts from inside the NetXMS client. I won’t go through the steps necessary to set up and opsgenie API integration but the link to the official guide for that will be posted below.

OpsGenie API Setup
You can obtain an API key by following the instructions here:  https://support.atlassian.com/opsgenie/docs/create-a-default-api-integration/
Additional API Docs can be found here: https://docs.opsgenie.com/docs/alert-api

Create Alert
To create an alert in OpsGenie we will need to set up an event processing policy that will call a server action.

Server Action:
Name: OpsGenie Create Alert
Type: Execute command on management server
Options: None

C:\\Windows\\System32\\curl.exe -v -X POST https://api.opsgenie.com/v2/alerts -H "Content-Type: application/json" -H "Authorization: GenieKey XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" -d "{ \\"message\\": \\"%Z - %m - %n\\", \\"alias\\": \\"%N%g\\", \\"description\\":\\"Ip Address: %a \\n \\n GUID: %g \\n \\n Severity: %S \\n \\n Event Name: %N \\n Event Source Object Name: %n \\n Time: %t \\n Zone Name: %Z\\", \\"entity\\":\\"%n\\", \\"priority\\":\\"%[ConvertSeverity]\\" }"

Important Notes:
·        Message will be the title of the OpsGenie Alert. In this example we have the Zone, the Event Name, and the Object Name for easy identification when receiving mobile alerts.
·        You MUST use \\ to escape the characters in the message or the request will not submit properly.
·        To be able to acknowledge and close OpsGenie alerts from inside NetXMS, you MUST use an alias that can be referenced. We will use %N%g for our alias. This will give us a unique Alias of EventName+GUID of the source object.
·        In multi-tenant MSP scenarios you can set “Zone Name: %Z” inside the command in order to route the alerts to the appropriate OpsGenie teams.
Macros used:
    ·        %n - Name of event source object. Name of interface when interface name is generated using macros.
    ·        %Z - Zone name of event source object.
    ·        %t - Event’s timestamp is a form day-month-year hour:minute:second.
    ·        %g - Globally unique identifier (GUID) of event source object.
    ·        %N - Event’s name.
    ·        %a - IP address of event source object.
    ·        %S - Event’s severity code as text.

You can use additional macros for event processing: https://www.netxms.org/documentation/adminguide/event-processing.html#macros-for-event-processing

Custom Scripts:
This script is necessary because the severity codes for NetXMS are the opposite of OpsGenie. You can create this script in the script library.

Script ( ConvertSeverity )

sub main()
{
 prioritylevel = "P5";
   switch($event->severity)
   {
   case 0:
        prioritylevel = "P5";
        break;
   case 1:
        prioritylevel = "P4";
        break;
   case 2:
        prioritylevel = "P3";
        break;
   case 3:
        prioritylevel = "P2";
        break;
   case 4:
        prioritylevel = "P1";
        break;
   };
   return prioritylevel;
}

Thank you for the info!

Good Morning/Evening, 

I just wanted to reach out and make sure we don't need to worry about the recently disclosed libcurl vulnerability CVE-2023-38545. This is a buffer overflow vulnerability in the socks5h proxy. The vulnerability has been patched in libcurl 8.4.0

Affected libcurl versions: 7.69.0 through 8.3.0

Just checking in to see if this is something that is being looked at. I didn't see this added as an issue on https://track.radensolutions.com/issues yet.

Can we get an updated grafana plugin that accepts dashboard variables? The current plugin is set to be deprecated for grafana cloud and grafana 11 users soon as grafana is no longer using angular in favor of react.

https://grafana.com/docs/grafana/latest/developers/angular_deprecation/#why-are-we-deprecating-angular-support

https://grafana.com/docs/grafana/latest/developers/angular_deprecation/angular-plugins/#netxmshttpsgrafanacomgrafanapluginsradensolutions-netxms-datasource

According to Grafana:
"For data source plugins the query editor and config options will likely need a total rewrite."

Just wanted to follow up on this. I discovered the deprecation when I was attempting to figure out how to use dashboard variables with the plugin. It doesn't appear that that plugin supports dashboard variables.

Sorry for the delay! I've been in the hospital, but I'm all better now. 

We restarted the server shortly after this happened but only because it stopped returning the correct parameters to grafana SQL calls and we realized the disk space was low.

I haven't been able to duplicate this issue, but I think I discovered what caused the problem. Our server needed to be restarted as it was utilizing too much of the allocated resources. I believe this was causing the unexpected behavior as the database was also not updating values that were set for the object_properties.name field.
For the sequence of events. 

1.) Agent Disabled on endpoint for alarm creation -> Alarm created via EPP {status=0}
      1.1) Alert is created in OpsGenie
2.) Alarm marked as acknowledged by user -> HOOK:AlarmStateChange is executed -Confirmed via trace()- {status=1}
      2.1) Event is processed and opsgenie alert is acknowledged via cUrl
3.) Alarm Enabled on endpoint -> Alarm terminated via EPP  -> HOOK:AlarmStateChange is executed -Confirmed via trace()- {status=3}
      3.1) Alert is closed in OpsGenie
4.) Agent Disabled on endpoint again for alarm creation -> Alarm created via EPP {status=0} -> Log shows that HOOK:AlarmStateChange has executed via trace and Alarm has {status=3}
      4.1) Alert is created in Opsgenie via cUrl
      4.2) Alert is marked as closed in opsgenie via cUrl

I think this is more of a server related problem as we were running low on disk space at the time.

Is there any development currently happening on the Grafana plugin? I checked github and noticed an issue was posted that AngularJS has been deprecated by Grafana in favor of React. 

https://grafana.com/docs/grafana/latest/developers/angular_deprecation/#why-are-we-deprecating-angular-support

https://grafana.com/docs/grafana/latest/developers/angular_deprecation/angular-plugins/#netxmshttpsgrafanacomgrafanapluginsradensolutions-netxms-datasource

According to Grafana:
"For data source plugins the query editor and config options will likely need a total rewrite."

I was able to get the external alerting to work with OpsGenie finally and I think I discovered a bug in the Alarm generation.

I am using the following code inside HOOK:AlarmStateChange 

sub main()
{
    eventserver = FindNodeObject($node, 100);
    global alarmstate = $alarm->state;
    global eventname = $alarm->eventName;
    global sourceobj = FindObject($alarm->sourceObject);
    global nameobj = sourceobj->guid;

    trace(0, "Alarm State" . alarmstate . " Alarm:" . eventname . nameobj);
    switch(alarmstate)
    {
/* Alarm State is Outstanding */
        case "0":
            break;
/* Alarm State is Acknowledged */
        case "1":
            PostEvent(eventserver, "Xms_Alarm_Ack","ACKALARM", eventname . nameobj);
            break;
/* Alarm State is Resolved */
        case "2":
            PostEvent(eventserver, "Xms_Alarm_Resolve","RSLVALARM", eventname . nameobj);
            break;
/* Alarm State is Sticky Acknowledged */
        case "17":
            PostEvent(eventserver, "Xms_Alarm_StickyAck","SACKALARM", eventname . nameobj);
            break;
    }
}

I originally had code for state 3 ( terminated ). This is where the bug seems to occur.
When the Alarm is created HOOK:AlarmStateChange runs. This is not a big deal as that allows us to call scripts when an event is created as it should have a state of 0. I noticed via the trace() function that some of the alarms were being created with a state of 3 instead of a state of 0.

Currently we have an EPP that uses a server action to send a CURL request to the OpsGenie API for alert generation. We have Opsgenie integrated directly with Teams, and Jira for ticket management. I would be happy to post a guide on how to do this if anybody else wants to know how we implemented it.

The code for state 3 ( terminated ) created an event "Xms_Alarm_Term". This event would be processed by the EPP and use a server action to send a CURL request to OpsGenie to close the alert. The practical implications of the alarm being created with a state of 3 is that the alert is created in OpsGenie and Closed in Opsgenie at exactly the same moment.

I suspect the problem is in alarm.cpp -> Copy constructor.

I could be wrong, but it appears that code is part of the alarm constructor that duplicates alarms if the event occurs again. I believe the reason this happens is the line (378 )

m_state = src->m_state;

That code appears to set the new alarm state as the state of the source alarm. This is only an issue if the source alarm is terminated as it creates a new copy of the alarm with a state of 3.

I could very well be wrong about this, but it makes sense in my head.