Errors Starting 2.2.7

[victor]

Hi,

so "show watchdog" actually shows empty list? Can you enter any command in nxadm after that?
Could you please create process dump for netxmsd.exe (using Task Manager) and provide it to us for debug?

Best regards,
Victor

[StanHubble]

The show watchdog displays the dashes and then hangs, (Ctrl-C) will drop you back to command line.
I can restart nxadm after and do a show queue or show thread, but sho stat, sho watchdog, or show tunnel will hang.

I created the process dump but it is almost 10GB.  Zipped it is 500MB.  Will this forum accept an upload that big?

[victor]

@stanhubble said in Errors Starting 2.2.7:

I created the process dump but it is almost 10GB.  Will this forum accept an upload that big?

I don't think so. I'll send you upload link via PM.

Best regards,
Victor

[StanHubble]

I hate #$%@#$  windows.......
removed and reapplied the latest set of windows updates  and 10 reboots later
, re-allow netxms thru windows firewall rules (even though it is disabled) and held my left foot while drinking a beer
and like magic netxmsd starts properly.

I think it was the beer that did it.

[victor]

I doubt it's Windows bug, more likely just lucky timing. I checked provided process dump - it is deadlocked in a code related to applying templates and sending events. There were execution errors for template auto apply scripts - in that case server generates event - and this event generation from within auto apply code somehow caused deadlock. I will investigate it further. As a workaround I suggest to check auto-apply scripts for possible errors.

Best regards,
Victor

[StanHubble]

Yeah ! was too optimistic.....it died after about 12 hours and wont restart.
I have checked the DCI's throwing errors and they were complaining about null values.  They were all dependent dci's that used a GETDCIVALUEBYDESCRIPTiON function in the transformation script.

[victor]

Would you like to test build with experimental patch that changes some internal synchronization mechanism?

Best regards,
Victor

[StanHubble]

yes please.

[victor]

This is download link for both minimal and full installers: https://cloud.netxms.org/index.php/s/yWU2wcC3c4WO2Y6

Best regards,
Victor

[StanHubble]

Thanks for the help.  It has been 24 hrs now since I installed 2.2.7.17 and everything is seems to be running fine.

Start up was longer than it used to be, but I can live with that.
I still have a number of errors on some Template and Container Autobind scripts during startup that behave normally when the system is up and running.  These are complaining about null values on variables that are populated either from a GETDCIVALUEBYDESCRIPTION function or a custom attribute of the node being tested. 

Now with 2.2.7 recognizing Node types of "Virtual" I can change some of these autobind scripts key on this instead, but there is a problem.  The config poll is correctly identifying actual virtual machines (if they have an agent) both on VMware and Hyper-V.  But it is also identifying Hyper-V Host machines as virtual (both 2.2.6 and 2.2.7 agents).
I don't have visibility on any VMware hosts machines to say if it is the case with them as well.

Thanks again
Stan

[victor]

There is issue registered for incorrect Hyper-V host detection: https://track.radensolutions.com/issue/NX-1472. Possible reason is that we are using CPUID "hypervisor" bit to detect if agent is running within a VM, and I suppose that Hyper-V host OS is in fact privileged VM, so agent runs inside virtual environment anyway. We will add additional checks in 2.2.8 to report host as physical machine.

Best regards,
Victor