Event generated but could not see it in event_log

[victor]

Hi,

configuration on agent side seems to be ok, and SendTrap record in log confirms that event is being generated. Check that you run agent with debug level 6 or higher and try to find record similar to "sending message CMD_TRAP" after SendTrap() record. If you'll find it, it mean that message was actually sent to server and problem is on server side. If not, problem is on agent side.

Best regards,
Victor

[Nagav]

Hi Victor

I have made debug level of agent to 9 and restated the agent.
But I could not see message ""sending message CMD_TRAP" after SendTrap() record" in agent log file

In agent log I could see only following two line
[05-Aug-2015 03:16:15.483] [DEBUG] LogParser: new data avialable in file "/apps/uspsqa1/switch/site/log/sys/SunOSsyslg.log"
[05-Aug-2015 03:16:15.484] [DEBUG] SendTrap(): event_code=0, event_name=IST_SYSLG1_RECORD, num_args=1, arg[0]="Fatal error " arg[1]="(null)" arg[2]="(null)"

Should I do any change to get "sending message CMD_TRAP" after SendTrap() record
Please help

Regards
Naga

[victor]

Hi,

most likely it indicated that there are no server connection accepting traps. Can you share agent configuration file (nxagentd.conf)? Also, how this node is configured in NetXMS server and what is configuration poll output for it?

Best regards,
Victor

[Nagav]

Hi Victor

Attached nxagentd.conf file and word doc that has screenshot of node property and node configuration poll details

Regards
Naga

---

nxagentd_conf.txt

Node property and configuration poll.docx

[victor]

Hi,

your agent version (2.0-M2) has a bug that can cause incorrect server access if server address listed in multiple categories (Servers, ControlServers, MasterServers). Try to comment out Servers and ControlServers options in nxagentd.conf leaving only MasterServers.

Best regards,
Victor

[Nagav]

Hi Victor

Tried your suggestion, still I could see only two line in agent log file
[06-Aug-2015 05:00:21.367] [DEBUG] LogParser: new data avialable in file "/apps/uspsqa1/switch/site/log/sys/SunOSsyslg.log"
[06-Aug-2015 05:00:21.367] [DEBUG] SendTrap(): event_code=0, event_name=IST_SYSLG1_RECORD, num_args=1, arg[0]="fatal error" arg[1]="(null)" arg[2]="(null)"

Regards
Naga

[victor]

Can you post agent config for verification please?

Best regards,
Victor

[Nagav]

Hi

Attached updated agent configuration

Thanks
Naga

---

Updated_agt_conf.txt

[Nagav]

Hi

Just now I noticed that

if I comment out server name in server and control list
I see following logs in agent and DCI's did not collect data

[06-Aug-2015 05:20:36.789] [DEBUG] Connection from 127.0.0.1 rejected
[06-Aug-2015 05:20:36.789] [DEBUG] Incoming connection from 127.0.0.1
[06-Aug-2015 05:20:36.789] [DEBUG] Connection from 127.0.0.1 rejected
[06-Aug-2015 05:20:36.789] [DEBUG] Incoming connection from 127.0.0.1
[06-Aug-2015 05:20:36.789] [DEBUG] Connection from 127.0.0.1 rejected
[06-Aug-2015 05:20:36.790] [DEBUG] Incoming connection from 127.0.0.1
[06-Aug-2015 05:20:36.790] [DEBUG] Connection from 127.0.0.1 rejected
[06-Aug-2015 05:20:36.790] [DEBUG] Incoming connection from 127.0.0.1
[06-Aug-2015 05:20:36.790] [DEBUG] Connection from 127.0.0.1 rejected
[06-Aug-2015 05:20:36.791] [DEBUG] Incoming connection from 127.0.0.1
[06-Aug-2015 05:20:36.791] [DEBUG] Connection from 127.0.0.1 rejected
[06-Aug-2015 05:20:36.791] [DEBUG] Incoming connection from 127.0.0.1
[06-Aug-2015 05:20:36.791] [DEBUG] Connection from 127.0.0.1 rejected
[06-Aug-2015 05:20:36.792] [DEBUG] Incoming connection from 127.0.0.1
[06-Aug-2015 05:20:36.792] [DEBUG] Connection from 127.0.0.1 rejected
[06-Aug-2015 05:20:36.792] [DEBUG] Incoming connection from 127.0.0.1
[06-Aug-2015 05:20:36.792] [DEBUG] Connection from 127.0.0.1 rejected

Thanks
Naga

[victor]

So, there are two problems. First, you should add 127.0.0.1 to list of MasterServers for agent running on NetXMS server. Second, you should check that primary host name for node is not set to 127.0.0.1, but to real IP address.

Best regards,
Victor

[Nagav]

Excellent Victor, thanks for your support
After making the changes suggested by you, I could see approriate log messages in both agent log and server log

Agent log:
[06-Aug-2015 23:59:32.636] [DEBUG] [session:0] GetTableValue(): result is 0 (SUCCESS)
[06-Aug-2015 23:59:32.637] [DEBUG] [session:0] Sending message CMD_REQUEST_COMPLETED (size 2880)
[06-Aug-2015 23:59:33.055] [DEBUG] LogParser: new data avialable in file "/apps/uspsqa1/switch/site/log/sys/SunOSsyslg.log"
[06-Aug-2015 23:59:33.055] [DEBUG] SendTrap(): event_code=0, event_name=IST_SYSLG1_RECORD, num_args=1, arg[0]="fatal error" arg[1]="(null)" arg[2]="(null)"
[06-Aug-2015 23:59:33.055] [DEBUG] [session:0] Sending message CMD_TRAP (size 144)

Server log:
g-2015 23:59:06.692] [DEBUG] StatusPoll(s1): bAllDown=false, dynFlags=0x00000001
[06-Aug-2015 23:59:06.694] [DEBUG] StatusPoll(s1 [112]): boot time set to 1438269511 from agent
[06-Aug-2015 23:59:06.695] [DEBUG] Finished status poll for node s1 (ID: 112)
[06-Aug-2015 23:59:33.056] [DEBUG] AgentConnectionEx::onTrap(): Received trap message from agent at 10.80.226.33, node ID 112
[06-Aug-2015 23:59:33.056] [DEBUG] AgentConnectionEx::onTrap(): trapID is valid
[06-Aug-2015 23:59:33.056] [DEBUG] Event from trap: 100066
[06-Aug-2015 23:59:33.056] [DEBUG] CorrelateEvent: event IST_SYSLG1_RECORD id 44 source s1 [112]
[06-Aug-2015 23:59:33.056] [DEBUG] CorrelateEvent: finished, rootId=0
[06-Aug-2015 23:59:33.056] [DEBUG] EVENT 100066 (ID:44 F:0x0001 S:0 TAG:"") FROM s1: fatal error
[06-Aug-2015 23:59:33.057] [DEBUG] Event 44 match EPP rule 27
[06-Aug-2015 23:59:46.486] [DEBUG] Updating maps...

Also could see event being logged in event_log table

Regards
Naga