Server crash
-
-
@xenth said in Server crash:
I just noticed something very disturbing, the polling isn't going properly.
My set intervals aren't being listened to, I'll give you an example, this is one of my workstations.
I have set the interval to 60 seconds, yet it does NOT poll every 60 seconds
It should poll every 60 seconds, yet when I view the history:
This is a VERY big problem as well, is it related to my other problem?
Yes, most likely they are related. If you take a look at a result of "show queues" console command, you can see quite big number (330) in data collector queue. This means that at the moment when you type this command 330 requests for collecting data was waiting for processing, because all data collectors was busy. You can try to increase number of data collectors to 40 or 50 (by changing server's parameter NumberOfDataCollectors), this may help a bit, but will not remove the problem completely.
Some additional questions: do you have SNMP on the nodes? Do you use SNMP for data collection?
Best regards,
Victor -
Hi,
I set the Numberofdatacollectors to 90 and the data collector queue is now at 150ish, should I keep increasing the number of collectors?
Some of the workstations have windows XP on them with the SNMP service but I am not running any dci's for those,
I am however monitoring several routers/modems/etc with snmp.To be more precise: About 7 or 8 nodes with SNMP that I am monitoring with an average of about 6-7 dci's per SNMP node.
Thank you for your time.
-
Hi!
Further increase of data collectors will not help - we have similar problem at one of our customer's site, and it's related to SNMP. I'll publish updated version of NetXMS server today, it may help.
Best regards,
Victor -
-
-
Very interesting information... Thank you for reporting!
Best regards,
Victor -
You're welcome, all is not solved unfortunately
On all the workstations I am monitoring I have disabled status polling, personally I think that's where the problem is coming from.
When everyone is working on the machines (and they are reachable by netxms) everything is fine with the current settings.
However at the moment (it's 20:00 here) I am seeing a queue of around 100 for the datacollector pollers and the same problems are happening again, I can't delete anything, timeouts and I have to reboot the netxms service.
You see what's happening is this:
08:00 - 17:00 - Server requests data from agents and gets replies - no queues, no issues.
17:00 - 08:00 - Server requests data from agents and the packets are blocked by the firewall because the destination adres does not exist at that moment - queues and lots of issues.-UPDATE: Here's the routers log message for when it tries to poll offline nodes if it helps:
Exceed MAX incomplete, sent TCP RSTPutting my workstations back on status polling is not an option because they go down so frequently
-
It can be a problem for NetXMS, because if during status poll it determines that node is down, data collection will not be scheduled for that node. But, if there are no status polls, NetXMS will try to establish TCP connection for each parameter to be collected. With connection timeout about 1 minute, it can easily cause long data collection queues.
What is the reason for disabling status polls? If you don't want alarms, mails, etc. for SYS_NODE_DOWN events, you can just block them in the first rule of event processing policy.
Best regards,
Victor -
Hi
That makes a lot of sense.
But, if I block those specific nodes from the policy that will only block them from my alarm list (which is what I want) but they will still be marked with a red dot in front of them, and in the network summary they will be displayed as critical nodes.
This is the specific reason I've blocked status polling on those nodes.
Basically what I wanted to achieve:
Only important nodes need to be marked with a red dot and displayed as critical in the network summary, is there any other way to achieve this then?Thanks again for all your work, I appreciate it.
-
Node status calculated based on two sources - active alarms for that node and status of child objects (usually interfaces). If you wish that node remains in NORMAL status event if some or all interfaces goes down, you should change status propagation algorithm for child objects of that node. You can select "fixed value" propagation algorithm for interface objects and set value to NORMAL.
Best regards,
Victor -
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login