"SYS_NODE_DOWN" Event not processed bei EPP

[victor]

So that's the problem - server found another event that is considered root cause and so mark this event as correlated. Correlated events do not pass through event processing policy. You can see root cause event ID in event log and check what is root cause event (and if it was correct).

Best regards,
Victor

[user54119]

where can I see the root cause event ID exactly? Or should I send you the log (but not in the public forum)?

[24-Apr-2015 16:26:31.220] [DEBUG] EVENT 28 (ID:1852235 F:0x0001 S:4 TAG:"" CORRELATED) FROM node.domain.com: Node down
[24-Apr-2015 16:26:31.220] [DEBUG] Event::expandText(event=0x2a57620 sourceObject=392 template='Interface "%2" changed state to DOWN (IP Addr: %3/%4, IfIndex: %5)' alarmMsg='(null)' alarmKey='(null)')
[24-Apr-2015 16:26:31.220] [DEBUG] CorrelateEvent: event SYS_IF_DOWN id 1852234 source node.domain.com [392]
[24-Apr-2015 16:26:31.220] [DEBUG] CorrelateEvent: finished, rootId=1852235

[victor]

As it turns out event ID and root cause event ID are not shown in event log. I've fixed it for next release, but currently you can check server log for message like

CorrelateEvent: event SYS_NODE_DOWN id ...

followed by

CorrelateEvent: finished, rootId=...

Number after rootId= will be ID of root cause event. You can find it either in server log or select directly from database by running SQL

SELECT * FROM event_log WHERE event_id=<id>

Best regards,
Victor

[user54119]

Thanks Victor. I see in all test cases the same root event ID. In the DB I saw it is a "SYS_NETWORK_CONN_LOST" event (Event Code 50) several days ago.
I don't understand why this very old event is relevant in the event correlation. Of course the NetXMS server has network connection.

[tomaskir]

@user54119 said in "SYS_NODE_DOWN" Event not processed bei EPP:

Thanks Victor. I see in all test cases the same root event ID. In the DB I saw it is a "SYS_NETWORK_CONN_LOST" event (Event Code 50) several days ago.
I don't understand why this very old event is relevant in the event correlation. Of course the NetXMS server has network connection.

It seems you have Beacon Probing enabled, and your NetXMS server see Beacon Probe hosts as down.

Check your server config variables regarding Beacon Probing.

[user54119]

Yes, Beacon Probing is enabled:

BeaconHosts: 172.16.100.8
BeaconPollingInterval: 1000
BeaconTimeout: 1000

It's configured like this for a few months. The Beacon Host ist the core switch.

I tested, I can ping the core switch ip from the command line on the NetXMS server.

Now I changed the IP address to reversed order: "8.100.16.172". And now I get the alarms of SYS_NODE_DOWN events again! When I switch back to normal IP address, I get a "SYS_NETWORK_CONN_LOST event".
Is this a similar bug as in the PING subagent of V2.0-M3?
And why are only the "SYS_NODE_DOWN" events correlated with this and not all other events too? I get alarms from the same nodes when a DCI threshold is active but not when node down, strange for me.

[tomaskir]

Definitely a bug
I registered it as #817 on bug-tracker.

As to why no correlation for other events - Victor will have to answer that.

[user54119]

Thanks Tomas. Ok let's have a look what Victor says.

Dani

[victor]

Only topology events are correlated - data collection events can be originated elsewhere (from script DCI for example) and may or may not be affected by network connectivity.

Best regards,
Victor

[user54119]

ok, thanks Victor, so for the moment I leave the BeaconHost IP in reverse order till V2.0-M5.

[victor]

Just fixed this. If quick fix is needed, just replace file src/server/core/beacon.cpp with attached one and recompile server.

Best regards,
Victor

---

beacon.cpp

[user54119]

Wow, thank you very much for the quick fix. I just tested and I can confirm it's fixed.

Thanks
Dani