NetXMS Forum

I would like to add that version 2.0.3 seems a bit more forgiving of these errors and doesn't always mark the dci entry as inactive.    But, I believe it still does sometimes.

We rely on the wonderful ExternalParametersProvider feature of NetXMS extensively.  Most of the time the values we are after are relatively static so we often only run the script every 10 minutes.

Unfortunately, when a node reboots we see a race condition where NetXMS declares many of the variables that came from ExternalParametersProvidor as unsupported.  It then disables that DCI variable.

For example, if we have an ExternalParametersProvider that provides license info from a USB Licensing Hasp plugged into the node, it might return a variable "HaspID" and another variable "HaspType".    In the DCI tables for a node, we would create an entry that pulls the Agent Variable "HaspID".    That entry works just fine.  But, after a reboot, many of the nodes that reference that type log alarms and declare that DCI Entry as unsupported.    We then have to individually go to each node and manually re-enable them.    Sometimes, it will generate a new alarm and re-disable that entry a few minutes later.  If we do a configuration poll and re-enable, it often solves the problem until the node reboots.    We usually try to match the polling interval of the DCI entry to the frequency specified in the ExternalParametersProvider function.  For example, if we use 600 seconds for the ExternalParametersProvider definition, we'll also set the polling interval of the DCI entry to 600.

It can be fairly painful after Microsoft Patch Tuesday when 50 machines have rebooted.  we have to go in and re-enable the dci entry on each node and re-poll.

Is there something we need to change in our configuration to solve this issue?

We see this issue on different versions of netXMS.  For example, it occurs on 1.2.17 and on 2.0.3

Thanks for the workaround.   

It does seem to me that it is a bug.  But, I can use the workaround for now.  Fortunately, that workaround also affects the export to Excel.

We often use the DCI Summary tables feature to summarize configuration information.  For numeric data, the summary tables often try to simplify the way large numbers are displayed.  A number like 192123456 will be shown as 192 M.  Unfortunately for us, it treats string fields that look like numbers as numbers.  In our case, we are collecting the serial number of a licensing hasp.    That serial number is 9 digits long and is entirely numeric.    We would like to see the entire serial number.

Is there a way of treating a string object as a string and not trying to simplify the number?    Or is there a way of temporarily turning off the simplification feature for all numbers?

We are on Version 2.0.3 of netXMS

Just in case it would be helpful to someone else, I'm posting the WMI Query I use to get the total number of disk errors for my hard drive.  I have a system that I stumbled on that had a disk that was failing.  I found the SMART Data was fairly useless, but Windows has a special class that gave me the data I was interested in - MSFT_StorageReliabilityCounter.  If you google it, you'll see that it has a bunch of useful info.  I decided that ReadErrorsTotal would be a good indicator of a problem drive.

Unfortunately, the query tended to return two entries for a given DeviceID that were almost identical, so I arbitrarily chose the one that ended with the text "reliabilitycounter".  The other one on my system had some sort of SCSI Id that I wasn't sure would be consistent.  On my system without any "where" clause, the query returns 4 entries - two for DeviceId 0 and two for DeviceId 1.  I'm assuming those correspond to physical drives 0 and 1.  If you want to monitor another drive instead of 0, just change the "0" in the "Where" statement to the other drive number.  You could have multiple queries, one for each drive on your system.

WMI.Query(root\microsoft\windows\storage,SELECT * FROM MSFT_StorageReliabilityCounter where '(DeviceId=0) AND (ObjectID like "%reliabilitycounter")',ReadErrorsTotal)

This query returns an unsigned 64 bit integer.  I have the frequency set to 3600 seconds and 90 days of history in NetXMS.

I hope this helps someone so that they don't have to go through hours of research to get this data, and they don't waste too much time trying to use the SMART data which declared my drive that has 4248 total read errors as perfectly fine - even though it is barely functioning.

I suspect the issues with not preserving was self-inflicted.  We have an internal distribution process that was used in the past.   

I notice that there appear to be registry settings that show the current location.  On machines where our internal process was used I don't see those settings.   

I'm assuming that if I create registry entries that match the current locations of the files, then NetXMS agent deployments will use those entries to decide where to place the files.

Victor - do you need any additional information from me to diagnose this?

Victor - do you have any followup to my post?

I am very interested in using the centralized upgrade feature of NetXMS.

But, we have a non-standard directory structure where we like to deploy items:

  For the binaries and highly static info, we would like to use c:\programs\NetXms\NetXmsA\ as the path.
  For configuration files, and less static info we like to use c:\programdata\NetXMS\NetXMA\...  (This is where we would put the agent config files)

This structure allows us to keep the root directory clean. 

I tried the upgrade feature and found that it ignored the existing structure and installed everything in default locations and even duplicated the service creation.

Is there some way of specifying alternative locations?  For example, could I edit the npi file and add a /DIR=c:\programs\netxms\netxmsa to specify a different install location.  Since it doesn't appear that you have an install option to specify the config file location, if I were to put a symbolic link in the install location that points to the agent file in its location, would that work?

If there is no current method, could this be considered as an enhancement?

Bill

That OID doesn't exist.  The closest I can get is .1.3.6.1.4.1.9.5  There is nothing between .1.3.6.1.4.1.9.3 and .1.3.6.1.4.1.9.9 on this switch.  Below is a portion of the long snmp walk for .1.3.6.1.4.1.9.5

SNMPv2-SMI::enterprises.9.3.6.14.0 = INTEGER: 1
SNMPv2-SMI::enterprises.9.3.6.15.0 = Timeticks: (0) 0:00:00.00
SNMPv2-SMI::enterprises.9.9.6.1.1.1.1.10.3.20.1.80.10.2.18.8.50585 = Counter32: 1880
SNMPv2-SMI::enterprises.9.9.6.1.1.1.1.10.3.20.1.80.10.2.18.8.50750 = Counter32: 611

I am running NetXms 2.0.M5 and have a network of Cisco switches.

The switches all have the Catalyst-generic driver associated with them.

If I view the switch forwarding database of a switch that has a node connected, I see in the node column that the database is showing that node.

But, if I go to the node and select "Find Switch Port", it reports an indirect connection to the wrong switch.  Often the switch it reports has a 2nd node that is communicating with the original node, so the original node appears on the trunk between the switches.   

If I try at a different time with the same "Find Switch Port" on the same node, I may get the error "Connection Port information cannot be found".  I can sometimes resolve this 2nd error by forcing topology polls on all the switches (not just the ones involved), but it doesn't last - eventually it returns to the state that displays the error message "Connection Port information cannot be found"

The switches involved don't directly connect to each other, they have a 10GBit connection to a pair of core switches.

I've attached some screen shots and the topology network map that NetXMS generated (which is accurate).  The example node is highlighted on the topology map.  I've also exported the topology map of the switch that the node is connected to because the screen shots only show a part of the info.    In the file Indirect-Switch.png I've highlighted the indirect entry.  The port associated with that entry is a 10Gbit trunk back to one of the core switches.

The node in this example uses only an Ping for polling, but I have the same problem with nodes that use the full NetXMS agent.

Let me know if you need additional info.

Thanks in advance for your assistance.

Bill

---

Node-FDB-Full

The workaround worked well.    I have a good topology map now.

But, I'm still having problems with the "Find switch port" functionality.    It shows me indirect information, but not the switch & port the device is directly connected to.  When I know the switch that it belongs to, I can do a level 2 topology view and see the node is associated with a switch port.    But, if I try from the node, it tends to show the switch and trunk port of a node that has another server talking to the original node. 

Because this sounds like an unrelated issue, I'm going to start a post in the General Support section and provide the documentation there.

Thank you so much for you prompt help on this original request.  Hopefully you can help me on the other issue.

Bill

I have a network consisting of Cisco 2960S with 4900M switches at the core.  The 4900M switches are using the GENERIC driver.    Because of this, I believe that it is preventing me from effectively using the "Find Switch Port" commands because NetXMS can't figure out the entire network.

I had the same problem on NetXMS version 1.2.17, but upgraded to 2.0.M5 to see if the switch was supported on the newer version.

What would be involved with supporting this switch?

I'm attaching an Object Details screen print of the switch and an SNMP walk.  The SNMP walk file contains two different walks, the first walk was just using the community name.  The 2nd walk was a vlan snmp walk of vlan 101 produced by appending @101 to the community name.

It appears this switch's snmp info is similar to the 2960S switches which do work.  Is there a way I can trick NetXMS into thinking it is a supported switch while I wait for this switch to be supported?

Thanks in Advance

Bill

---

snmpwalk.txt

Thank you Victor.  I suspect you had a good reason to disable static entries.  It is too bad that Static entries come from different sources.  The ones I care about come from the Security feature of the switch.  You can see them with the SHOW MAC ADDRESS-LIST SECURE.  If you use SHOW MAC ADDRESS-LIST STATIC you get a bigger list. 

The Static ones include odd CPU port types.  Maybe there was a need to filter them out.  Perhaps in your solution if you could consider filtering out the CPU-based addresses, or at a minimum allowing the learned SECURE addresses (assuming you can detect them), the solution won't have a lot of side-effects.  I agree with making it an option.  Please make sure you publish that option somewhere when it is implemented  

Here is an example on the same switch:

#show mac address-table secure
          Mac Address Table
-------------------------------------------

Vlan    Mac Address      Type        Ports
----    -----------      --------    -----
 103    00c0.b75b.421e    STATIC      Gi1/0/2
 102    101f.742d.9c07    STATIC      Gi1/0/22
 102    101f.742d.9c45    STATIC      Gi1/0/6
 102    101f.742d.b239    STATIC      Gi1/0/20
 102    9c8e.9919.9bc5    STATIC      Gi1/0/10
 102    9c8e.992e.555f    STATIC      Gi1/0/18
 102    9c8e.992f.defd    STATIC      Gi1/0/8
 102    9c8e.992f.ee9f    STATIC      Gi1/0/16
 102    9c8e.9930.d013    STATIC      Gi1/0/4
 101    101f.742d.9c06    STATIC      Gi1/0/21
 101    101f.742d.9c44    STATIC      Gi1/0/5
 101    101f.742d.b238    STATIC      Gi1/0/19
 101    9c8e.9919.9bc4    STATIC      Gi1/0/9
 101    9c8e.992e.555e    STATIC      Gi1/0/17
 101    9c8e.992f.defc    STATIC      Gi1/0/7
 101    9c8e.992f.ee9e    STATIC      Gi1/0/15
Total Mac Addresses for this criterion: 16

#show mac address-table static
          Mac Address Table
-------------------------------------------

Vlan    Mac Address      Type        Ports
----    -----------      --------    -----
 All    0100.0ccc.cccc    STATIC      CPU
 All    0100.0ccc.cccd    STATIC      CPU
 All    0180.c200.0000    STATIC      CPU
 All    0180.c200.0001    STATIC      CPU
 All    0180.c200.0002    STATIC      CPU
 All    0180.c200.0003    STATIC      CPU
 All    0180.c200.0004    STATIC      CPU
 All    0180.c200.0005    STATIC      CPU
 All    0180.c200.0006    STATIC      CPU
 All    0180.c200.0007    STATIC      CPU
 All    0180.c200.0008    STATIC      CPU
 All    0180.c200.0009    STATIC      CPU
 All    0180.c200.000a    STATIC      CPU
 All    0180.c200.000b    STATIC      CPU
 All    0180.c200.000c    STATIC      CPU
 All    0180.c200.000d    STATIC      CPU
 All    0180.c200.000e    STATIC      CPU
 All    0180.c200.000f    STATIC      CPU
 All    0180.c200.0010    STATIC      CPU
 All    ffff.ffff.ffff    STATIC      CPU
 103    00c0.b75b.421e    STATIC      Gi1/0/2
 102    101f.742d.9c07    STATIC      Gi1/0/22
 102    101f.742d.9c45    STATIC      Gi1/0/6
 102    101f.742d.b239    STATIC      Gi1/0/20
 102    9c8e.9919.9bc5    STATIC      Gi1/0/10
 102    9c8e.992e.555f    STATIC      Gi1/0/18
 102    9c8e.992f.defd    STATIC      Gi1/0/8
 102    9c8e.992f.ee9f    STATIC      Gi1/0/16
 102    9c8e.9930.d013    STATIC      Gi1/0/4
 101    101f.742d.9c06    STATIC      Gi1/0/21
 101    101f.742d.9c44    STATIC      Gi1/0/5
 101    101f.742d.b238    STATIC      Gi1/0/19
 101    9c8e.9919.9bc4    STATIC      Gi1/0/9
 101    9c8e.992e.555e    STATIC      Gi1/0/17
 101    9c8e.992f.defc    STATIC      Gi1/0/7
 101    9c8e.992f.ee9e    STATIC      Gi1/0/15
Total Mac Addresses for this criterion: 36
#

Thank you in advance for your hard work and your attention on this item.

Bill

I just read in the following post that there is a difference between the topology view of the FDB database and the view from the tools menu because the topology view is filtered for only dynamic addresses where the tools view walks the SNMP tree.

https://www.netxms.org/forum/general-support/switch-fdb-differences/

What is the reasoning for removing static addresses from the topology view and does that affect the "Find switch port" query.

When I use the tools version which walks the tree, I get only one entry -- the entry for the aggregate port, but it shows only one MAC address.  Maybe this is ok also if the purpose is to see what switch it connects to.  Strangely when I run the same tools->info-FDB query on a root switch, it returns nothing.

My main concern is to get "Find Switch Port" working so it identifies local connections on these switches, but I would be interested in understanding the reasoning for the different behavior on Topology query and the tools-based query.

I have a site that has about 20 Cisco 2960S switches and a couple Cisco 4900M switches that I have NetXMS monitoring.

NetXMS is using the CATALYST-GENERIC Driver so it appears that the switches are being recognized.

I am on NetXMS version 1.2.17 at this site.

The problem I'm running into is that NetXMS seems to only be aware of some of the MAC addresses on the switch.  If I try to find a switch port, it never reports a direct connection.  It always reports that a given node is indirectly connected to a remote switch.

In researching the problem, I've determined that NetXMS is only storing "dynamic" MAC addresses.  It ignores "static" MAC addresses.  We use Cisco's security features where it learns what mac addresses are directly connected to a port and only lets those addresses on that port.  This prevents someone from plugging in a foreign device that we don't know about.  Apparently, even though those addresses are "learned", Cisco classifies them as Static.

Strangely when I snmpwalk the switches, all the mac addresses appear and I don't see anything obvious that would distinguish between static addresses and dynamic addresses.  When I researched cisco literature, I couldn't even find a reference for using SNMP to look at whether an address is static or dynamic.  So, I'm puzzled at why the static addresses are missing.

Is there are way of configuring NetXMS to store all the addresses?  I would really like to be able to use the "Find switch port" feature on an object to see the local port it is connected to.  Seeing that it indirectly connects to a root switch in the core isn't helpful since all my devices connect to one of the two root switches.

I've attached the following files to demonstrate the problem.  I picked a simple 24 port gigabit switch, that has an additional 2 ports of ten-Gigabit.  We always use one of the Ten Gigabit ports as a trunk to the other switches.  typically the gigabit ports are connecting to servers and other nodes in the same rack.

1. An SNMP walk of the 3 VLANs and typical OIDs used for mapping the MAC table to switch ports.  I used the following reference to get those OIDs:
   • http://www.cisco.com/c/en/us/support/docs/ip/simple-network-management-protocol-snmp/44800-mactoport44800.html
2. A screen capture from Object Details of the switch
3. An Excel Spreadsheet in which I compared the output of NetXMS's exported Switch Forwarding Database to the list produced by telneting into the switch and running a "show mac address-list".  I sorted both of those items by MAC address and inserted blank lines whenever appropriate to keep the MAC addresses matching.  When looking at that comparison, you can see that all the "static" entries are missing from NetXMS's data.  There are other items missing from NetXMS and from the Cisco output, but I consider that related to timing and just noise because there is no consistent pattern.

Please let me know if you need additional info.

---

Snmpwalks-common-mac-related.txt

Tocumen-Edge-1-Mac-table.csv

While your solution fixed my problem for the Cisco 2960S switches, my Cisco 4900M switch (which identifies itself as a 4500 series switch still uses a generic driver.

The Object ID=.1.3.6.1.4.1.9.1.917

I assume that the detection logic isn't aware of this switch.  Is there a way of tricking NetXMS into thinking it is another switch that is supported to see if it works? 

This is the software I'm running on the switch:
System Description=Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-IPBASEK9-M), Version 12.2(53)SG9, RELEASE SOFTWARE (fc2) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2013 by Cisco Systems, Inc. Compiled Tue 29-Jan-13 03:04 by p

I have two of these switches at the core of my network (they are 10GBit switches).  It would be nice if NetXMS recognized them.

Let me know if there is anything I can provide that would help.

Thanks in advance.

Thanks again Victor for your quick response.

Your solution solved my problem.

The driver is now showing as Catalyst-Generic and the other vlans are being looked at.

Here is a copy of the new log:

[07-Jan-2015 17:59:56.465] Log file opened
[07-Jan-2015 17:59:56.481] [INFO ] Database driver "mysql.ddr" loaded and initialized successfully
[07-Jan-2015 17:59:56.904] [INFO ] Network device driver "AIRESPACE" loaded successfully
[07-Jan-2015 17:59:56.904] [INFO ] Network device driver "AT" loaded successfully
[07-Jan-2015 17:59:56.904] [INFO ] Network device driver "BAYSTACK" loaded successfully
[07-Jan-2015 17:59:56.919] [INFO ] Network device driver "CATALYST-2900XL" loaded successfully
[07-Jan-2015 17:59:56.935] [INFO ] Network device driver "CATALYST-GENERIC" loaded successfully
[07-Jan-2015 17:59:56.935] [INFO ] Network device driver "CISCO-ESW" loaded successfully
[07-Jan-2015 17:59:56.950] [INFO ] Network device driver "CISCO-SB" loaded successfully
[07-Jan-2015 17:59:56.950] [INFO ] Network device driver "DELL-PWC" loaded successfully
[07-Jan-2015 17:59:56.950] [INFO ] Network device driver "ERS8000" loaded successfully
[07-Jan-2015 17:59:56.966] [INFO ] Network device driver "H3C" loaded successfully
[07-Jan-2015 17:59:56.966] [INFO ] Network device driver "MIKROTIK" loaded successfully
[07-Jan-2015 17:59:56.966] [INFO ] Network device driver "NETSCREEN" loaded successfully
[07-Jan-2015 17:59:56.982] [INFO ] Network device driver "NTWS" loaded successfully
[07-Jan-2015 17:59:56.982] [INFO ] Network device driver "PING3" loaded successfully
[07-Jan-2015 17:59:56.982] [INFO ] Network device driver "PROCURVE" loaded successfully
[07-Jan-2015 17:59:56.982] [INFO ] Network device driver "SYMBOL-WS" loaded successfully
[07-Jan-2015 17:59:56.997] [INFO ] Network device driver "UBNT" loaded successfully
[07-Jan-2015 18:00:03.376] [INFO ] Listening for SNMP traps on UDP socket 0.0.0.0:162
[07-Jan-2015 18:00:03.392] [INFO ] NetXMS Server started
[07-Jan-2015 18:00:03.392] [INFO ] Listening for client connections on TCP socket 0.0.0.0:4701
[07-Jan-2015 18:00:03.392] [INFO ] Listening for client connections on TCP socket :::4701
[07-Jan-2015 18:00:03.392] [INFO ] Listening for mobile device connections on TCP socket 0.0.0.0:4747
[07-Jan-2015 18:00:03.392] [INFO ] Listening for mobile device connections on TCP socket :::4747

Thanks for your quick response.

It looks like the GENERIC driver is being selected.

Does GENERIC mean completely Generic or is it CATALYST_GENERIC (which is mentioned in the Wiki)? 

• If it is Catalyst_generic, then what can I do to determine why it is only paying attention to VLan 1?
• If it is completely generic, then what is the next step to determine why it isn't selecting the Catalyst drivers?    I did verify that snmpget returns a value ("9") for .1.3.6.1.4.1.9.5.1.2.14.0

Is there a way of determining which drivers are loaded?  I don't see any network drivers in the server log (which I posted in my first message).

When I do a configuration poll, this is the output:

[2015-01-07 08:04:45] **** Poll request sent to server ****
[2015-01-07 08:04:45] Poll request accepted
[2015-01-07 08:04:45] Starting configuration poll for node Edge-Switch-1
[2015-01-07 08:04:45] Checking node's capabilities...
[2015-01-07 08:04:45]    Checking SNMP...
[2015-01-07 08:04:45]    SNMP agent is active (version 2c)
[2015-01-07 08:04:45]    System description changed to Cisco IOS Software, C2960S Software (C2960S-UNIVERSALK9-M), Version 15.0(2)SE6, RELEASE SOFTWARE (fc2) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2014 by Cisco Systems, Inc. Compiled Wed 09-Apr-14 03:09 by prod_rel_team
[2015-01-07 08:04:45]    System name changed to 0xee1400
[2015-01-07 08:04:46]    LLDP node ID changed to 4@6C9CEDEE1400
[2015-01-07 08:04:46] Capability check finished
[2015-01-07 08:04:46] Checking interface configuration...
[2015-01-07 08:04:46] Interface configuration check finished
[2015-01-07 08:04:46] Checking node name
[2015-01-07 08:04:47] Node name is OK
[2015-01-07 08:04:47] Finished configuration poll for node Edge-Switch-1
[2015-01-07 08:04:47] Node configuration was changed after poll
[2015-01-07 08:04:47] **** Poll completed successfully ****

I'm not sure about what you mean by verview, but I did see an object details option on the view menu.  When I've tried to pull that up before, I just got a blank page.    I tried a couple times, and this time I received the following on the general page.

ID=472
GUID=c1116b44-1ee4-8b40-9f0b-b905c296c3f1
Class=Node
Status=Normal
Primary Host Name=10.3.101.1
Primary IP Address=10.3.101.1
System Description=Cisco IOS Software, C2960S Software (C2960S-UNIVERSALK9-M), Version 15.0(2)SE6, RELEASE SOFTWARE (fc2) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2014 by Cisco Systems, Inc. Compiled Wed 09-Apr-14 03:09 by prod_rel_team
SNMP sysName=0xee1400
SNMP Object ID=.1.3.6.1.4.1.9.1.1208
Bridge Base Address=6C:9C:ED:EE:14:00
Driver=GENERIC
Boot Time=2014-12-10 09:22:49

I've been using NetXMS for a while, but recently installed version 1.2.17 on a Windows Server 2012 machine at a site that uses Cisco 2960S switches and 4500 switches. 

I use primarily 3 VLans for most of the traffic and don't use vlan 1.  NetXMS doesn't seem to show any information except for vlan 1.  When I look at the forwarding database, I only see vlan 1 related info.

I know that Cisco uses a modified form of the community string for vlans and have tried that without any better results.

I'm skeptical that the Catalyst driver is loading or working.  I just restarted the NetXMS server and don't see any mention about loading the network drivers.

[07-Jan-2015 01:35:32.010] Log file opened
[07-Jan-2015 01:35:32.026] [INFO ] Database driver "mysql.ddr" loaded and initialized successfully
[07-Jan-2015 01:35:38.711] [INFO ] Listening for SNMP traps on UDP socket 0.0.0.0:162
[07-Jan-2015 01:35:38.727] [INFO ] Listening for client connections on TCP socket 0.0.0.0:4701
[07-Jan-2015 01:35:38.727] [INFO ] Listening for client connections on TCP socket :::4701
[07-Jan-2015 01:35:38.727] [INFO ] NetXMS Server started
[07-Jan-2015 01:35:38.727] [INFO ] Listening for mobile device connections on TCP socket :::4747
[07-Jan-2015 01:35:38.727] [INFO ] Listening for mobile device connections on TCP socket 0.0.0.0:4747

I'm curious if this is a bug in the current version, or something odd about my configuration.

I did search to see how to determine which driver it is trying to use, but the articles are very old and refer to things I can't find on my system.

We don't install NetXMS in the standard "Program files" location for Windows.  We try to keep Windows Operating System stuff separated from the applications we install.  So, we install on C:\programs\NetXMS\NetXMS\  (Yes NetXMS is listed twice).  Within that directory, we have the standard installation (bin, database, doc, etc, lib directories).  Within the lib directory is a ndd directory which contains 19 files (including the catalyst.ndd and some other cisco ndds).

We are using SNMP polling for the switches and seem to be able to get SNMP data in DCI and SNMP Trap configuration.

Could you provide guidance on how to determine which NetXMS driver is in use for a switch?  Also, any other tips on how to debug this issue would be appreciated.

Bill