Find text string within log file

[user59290]

Hi there,

Can you please advise how I can configure netxms to monitor a folder on a node, this folder contains log files and if the log files contains a specific text string "Failed to connect to FX Server" an alert is generated?

I have looked at Logwatch but I have no idea how to make this work, or if its even the correct parser to use.

Can someone help me?

Many thanks, Damien

[victor]

Hi,

yes, logwatch is a correct choice. Parser could be following:

<parser>
   <file>/path/to/log/file</file>
   <rules>
      <rule>
         <match>Failed to connect to FX Server</match>
         <event>FX_SERVER_CONNECT_FAILED</event>
      </rule>
   </rules>
</parser>

and add the following to nxagentd.conf:

# this is main section
SubAgent = logwatch.nsm

[LOGWATCH]
Parser = /path/to/parser.xml

you have to create event FX_SERVER_CONNECT_FAILED on server (of course you can name it differently) - it will be generated each time this string will be found in log file.

Best regards,
Victor

[user59290]

Hi there,

On the remote agent I have adjusted the conf file as described and in C:\NetXMS I have created a parser1.xml file

remote agent conf
<parser>
<file>C:\test\logs</file>
<rules>
<rule>
<match>TEST ERROR LOG</match>
<event>100000</event>
</rule>
</rules>
</parser>

parser1.xml
<parser>
<file>C:\test\logs</file>
<rules>
<rule>
<match>TEST ERROR LOG</match>
<event>100000</event>
</rule>
</rules>
</parser>

When you say I need to create an event , do you mean a DCI for the remote node?  Thanks

[user59290]

sorry I made an error in my copy and paste, the remote agent config is defined as:

#
# NetXMS agent configuration file
# Created by server installer at Tue Oct 11 15:24:54 2016
#

LogFile = {syslog}
MasterServers = 127.0.0.1, 192.168.11.207
FileStore = C:\NetXMS\var
RequireAuthentication = yes
SharedSecret = HIDDEN
SubAgent = winperf.nsm
SubAgent = portcheck.nsm
SubAgent = logwatch.nsm
# Below is log parsers definitions
*LOGWATCH
Parser = C:\NetXMS\parser1.xml

[user54119]

Hi Damien

Change in your nxagentd.conf '*LOGWATCH' to '[LOGWATCH]'.
With your parser xml the string 'TEXT ERROR LOG' in your Logfile 'C:\test\logs' would generate an NetXMS event 100000. So this event 100000 must exist in your event configuration in NetXMS. You don't need a DCI for that.

best regards
Dani

[dmna007]

Hi, just to let you know, I have this working if I point the parser directly at a file:

<parser>
  <file>C:\FX_CHUB\LOGS\test.txt</file>
  <rules>
      <rule>
        <match>TCP Command Socket Failed Error</match>
        <event>LOGWATCH_PLXCENTRALHUB_176.74.191.177</event>
      </rule>
  </rules>
</parser>

I have created the event in the main Netxms console and an email is sent when the match string is inserted into the txt file, so that's great. 

The problem I now have is the logs within this folder are all variable names, is it possible to point the parser at a folder opposed to a direct file?  This way any log file in this file will be parsed looking for the string to match.

I have tried <file>C:\FX_CHUB\LOGS\*</file>  and <file>C:\FX_CHUB\LOGS\</file>

But this does not seem to work.  Many thanks for your help

[victor]

Hi!

You can use macros for forming date/time based file name (possible macros can be found here: http://www.cplusplus.com/reference/ctime/strftime/) or you can use output of shell script as file name using backticks, like this:

<file>C:\FX_CHUB\LOGS\`C:\gen_file_name.cmd`</file>

In this example file name will be taken from output of script C:\gen_file_name.cmd

Best regards,
Victor

[tickett]

• I am trying this (dynamic filename) and struggling- I have tried;
  • using a wildcard * (didn't expect it to work, but worth a longshot)
• adding a macro (see below)
• using backticks with an echo command directly
• using backticks with a batch file which executes an echo command

<macro name="timestamp">%Y_%m_%d</macro>

The log file seems to show whatever I enter and doesn't appear to show a "resolved" value. Is this normal? Regardless new log entries don't appear to be picked up.

2019.04.11 09:31:02.328 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_@{timestamp}.txt" stopped
2019.04.11 16:33:00.121 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_`echo %date:~6,4%_%date:~3,2%_%date:~0,2%`.txt" started
2019.04.11 16:38:34.340 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_*.txt" stopped
2019.04.11 16:38:34.906 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_`c:\netxms\etc\date.bat`.txt" started

I have the parser working fine for the Windows Event Log.

Any ideas? TIA

[victor]

Yes, this is normal. On startup agent logs unexpanded file name. You should also see something like this when file is found:

2019.04.12 12:03:13.608 *D* [logwatch          ] File "/tmp/test-20190412.log" (pattern "/tmp/test-`date +%Y%m%d`.log") successfully opened

Are you sure your commands return correct values? You may also try to use cmd /c "command" within backticks (because echo for example is not an external executable but cmd's internal command).

Also, if you only need current date/time you may use date/time macros without external commands, like this:

<file>C:\inetpub\wwwroot\website\logs\Log_%Y_%m_%d.txt</file>

Best regards,
Victor

[tickett]

Thanks Victor, putting the wildcards straight in the filename as you suggested worked perfectly;

<file>C:\inetpub\wwwroot\website\logs\Log_%Y_%m_%d.txt</file>