netxmsd segmentation faults
-
NetXMS Version 2.0.3
Running on Ubuntu 14.04.4 LTS, using packages from "deb http://packages.netxms.org/ubuntu trusty main".nxdbmgr comes back clean. But I guess that doesn't mean it couldn't be some database corruption bringing NetXMS to fall.
-
Are you using "Interface::setExpectedState" NXSL functions by any chance?
There was a bug that would cause segfaults that is fixed in develop branch.If not, your best bet in this case is to contact official Raden support, so they can help you out.
https://www.netxms.org/contact/ -
Hi,
also 2.0.3 may crash on receiving some SNMP traps (fixed in develop branch as well).
Best regards,
Victor -
We're not using the NXSL function for the ExpectedState (yet), but we certainly have some SNMP traps coming in.
Maybe some device we're monitoring or added recently is having issues and just happens to be sending one of those "bad" traps.
Guess I'll wait for 2.0.4 and see what happens. -
-
I may have found the culprit. There have been two segfaults right after the same type of DCI alarm was triggered. And a third one just now, "live" while I was connected. An alert popped up relating to that type of DCI and NetXMS died together with displaying the message. However, there were several segfaults without a directly related log message. So to confirm, I've made some config changes and am now monitoring to see if the segfaults actually stop.
Will update the post once I have some more data (i.e. no segfault in the next 24 hours would be a good sign).
-
Looks like that was it. No more segfaults.
I have attached the template that is causing the problem.After some digging in the database (prior to removing the template from all relevant devices) I found that the devices (old Intel Modular servers) appear to send "SCM Comm Error" as response to random values at times. The template includes a number of Integer and Int64 fields, including in SNMP tables. Not sure if that's what is causing the problem, i.e. NetXMS receiving "SCM Comm Error", trying to put that into an Int64 field?
-
Template seems normal, nothing unusual. Even if agent returns texts in some fields server should handle it correctly. There were some fixes in 2.0.4, you can try to re-enable this template when 2.0.4 will be available. If it will not help, then the only way to debug issue will be to run netxmsd under debugger and wait for crash.
Best regards,
Victor -
I decided to re-enable the template in 2.0.5 and unfortunately the issue persists.
Once I enable the template, the NetXMS server crashes within a few hours.
When I disable the template again, NetXMS runs stable and without issues.This is driving me nuts... I'm not exactly a code/debug whiz, so what do I have to do to get some debugging data that would help with fixing this?
-
Could you please run netxmsd under gdb and send backtrace after crash? Here is the instruction: https://wiki.netxms.org/wiki/Running_NetXMS_under_debugger
Best regards,
Victor
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login