<![CDATA[netxmsd segmentation faults]]>Hi,

For the last couple of days the netxmsd process on our server has been segfaulting randomly. Randomly in that debug logs ( at level 8 ) do not hint at the same thing being done prior to the crash. However it does usually take a couple of hours before the process crashes.

Not sure how to troubleshooting this?
I've been reading up on gdb and am following some suggestions from this post: http://stackoverflow.com/questions/16169022/debugging-a-running-daemon-using-gdb
Not sure if that is going to produce any useful output at all.

Anything else I can do to troubleshoot this?

Thanks

]]>https://forum.netxms.org/topic/4459/netxmsd-segmentation-faultsRSS for NodeFri, 13 Nov 2026 14:49:39 GMTFri, 13 May 2016 04:23:58 GMT60<![CDATA[Reply to netxmsd segmentation faults on Thu, 25 Aug 2016 14:59:55 GMT]]>Could you please run netxmsd under gdb and send backtrace after crash? Here is the instruction: https://wiki.netxms.org/wiki/Running_NetXMS_under_debugger

Best regards,
Victor

]]>https://forum.netxms.org/post/20778https://forum.netxms.org/post/20778Thu, 25 Aug 2016 14:59:55 GMT<![CDATA[Reply to netxmsd segmentation faults on Mon, 22 Aug 2016 23:04:32 GMT]]>I decided to re-enable the template in 2.0.5 and unfortunately the issue persists.
Once I enable the template, the NetXMS server crashes within a few hours.
When I disable the template again, NetXMS runs stable and without issues.

This is driving me nuts... I'm not exactly a code/debug whiz, so what do I have to do to get some debugging data that would help with fixing this?

]]>https://forum.netxms.org/post/20753https://forum.netxms.org/post/20753Mon, 22 Aug 2016 23:04:32 GMT<![CDATA[Reply to netxmsd segmentation faults on Fri, 27 May 2016 07:26:27 GMT]]>Template seems normal, nothing unusual. Even if agent returns texts in some fields server should handle it correctly. There were some fixes in 2.0.4, you can try to re-enable this template when 2.0.4 will be available. If it will not help, then the only way to debug issue will be to run netxmsd under debugger and wait for crash.

Best regards,
Victor

]]>https://forum.netxms.org/post/19956https://forum.netxms.org/post/19956Fri, 27 May 2016 07:26:27 GMT<![CDATA[Reply to netxmsd segmentation faults on Mon, 16 May 2016 21:58:26 GMT]]>Looks like that was it. No more segfaults.
I have attached the template that is causing the problem.

After some digging in the database (prior to removing the template from all relevant devices) I found that the devices (old Intel Modular servers) appear to send "SCM Comm Error" as response to random values at times. The template includes a number of Integer and Int64 fields, including in SNMP tables. Not sure if that's what is causing the problem, i.e. NetXMS receiving "SCM Comm Error", trying to put that into an Int64 field?

IntelModular.xml

]]>https://forum.netxms.org/post/19882https://forum.netxms.org/post/19882Mon, 16 May 2016 21:58:26 GMT<![CDATA[Reply to netxmsd segmentation faults on Mon, 16 May 2016 00:56:50 GMT]]>I may have found the culprit. There have been two segfaults right after the same type of DCI alarm was triggered. And a third one just now, "live" while I was connected. An alert popped up relating to that type of DCI and NetXMS died together with displaying the message. However, there were several segfaults without a directly related log message. So to confirm, I've made some config changes and am now monitoring to see if the segfaults actually stop.

Will update the post once I have some more data (i.e. no segfault in the next 24 hours would be a good sign).

]]>https://forum.netxms.org/post/19866https://forum.netxms.org/post/19866Mon, 16 May 2016 00:56:50 GMT<![CDATA[Reply to netxmsd segmentation faults on Sun, 15 May 2016 14:32:03 GMT]]>I can provide you with built packages from the develop branch if you wish.

]]>https://forum.netxms.org/post/19865https://forum.netxms.org/post/19865Sun, 15 May 2016 14:32:03 GMT<![CDATA[Reply to netxmsd segmentation faults on Sun, 15 May 2016 11:09:51 GMT]]>We're not using the NXSL function for the ExpectedState (yet), but we certainly have some SNMP traps coming in.
Maybe some device we're monitoring or added recently is having issues and just happens to be sending one of those "bad" traps.
Guess I'll wait for 2.0.4 and see what happens.

]]>https://forum.netxms.org/post/19864https://forum.netxms.org/post/19864Sun, 15 May 2016 11:09:51 GMT<![CDATA[Reply to netxmsd segmentation faults on Sat, 14 May 2016 10:25:36 GMT]]>Hi,

also 2.0.3 may crash on receiving some SNMP traps (fixed in develop branch as well).

Best regards,
Victor

]]>https://forum.netxms.org/post/19846https://forum.netxms.org/post/19846Sat, 14 May 2016 10:25:36 GMT<![CDATA[Reply to netxmsd segmentation faults on Fri, 13 May 2016 12:26:34 GMT]]>Are you using "Interface::setExpectedState" NXSL functions by any chance?
There was a bug that would cause segfaults that is fixed in develop branch.

If not, your best bet in this case is to contact official Raden support, so they can help you out.
https://www.netxms.org/contact/

]]>https://forum.netxms.org/post/19832https://forum.netxms.org/post/19832Fri, 13 May 2016 12:26:34 GMT<![CDATA[Reply to netxmsd segmentation faults on Fri, 13 May 2016 11:53:20 GMT]]>NetXMS Version 2.0.3
Running on Ubuntu 14.04.4 LTS, using packages from "deb http://packages.netxms.org/ubuntu trusty main".

nxdbmgr comes back clean. But I guess that doesn't mean it couldn't be some database corruption bringing NetXMS to fall.

]]>https://forum.netxms.org/post/19830https://forum.netxms.org/post/19830Fri, 13 May 2016 11:53:20 GMT<![CDATA[Reply to netxmsd segmentation faults on Fri, 13 May 2016 09:41:18 GMT]]>We need more information.

What version of NetXMS are you using?
What OS?

Are you compiling from source using using .deb packages?

]]>https://forum.netxms.org/post/19829https://forum.netxms.org/post/19829Fri, 13 May 2016 09:41:18 GMT