DCI last values don't store history data

xelander

I tried (see attached config.log), but the issue is still present.
The query is the same as before:

200806 15:54:06    266 Prepare  SELECT idata_timestamp,idata_value,raw_value FROM idata_2069 WHERE item_id=? ORDER BY idata_timestamp DESC LIMIT 4096
                   266 Execute  SELECT idata_timestamp,idata_value,raw_value FROM idata_2069 WHERE item_id=683 ORDER BY idata_timestamp DESC LIMIT 4096
                   266 Close stmt

I was wondering if it could be a client issue, instead of a server one.
After all, if the server had a problem SELECT'ing data from the DB, almost nothing would work...

config.zip

Filipp Sudanov

What's the operating system where the client is running? What's client version? Is it downloaded from netxms web site?
For testing - may be you could try the client on another machine, preferrably on a different OS.

xelander

I tried the Windows client (nxmc-3.4.284-win32-x64.zip) both on Windows 7 and on Windows 10, but I also tried the web interface (nxmc-3.4.284.war), everything downloaded from the netxms website, so it seems the issue doesn't depend on the client OS.
I still have to try on Linux...

xelander

Today I also tried on Linux CentOS 8, (nxmc-3.4.284-linux-gtk-x64.tar.gz) but nothing changed: no historical data.This is so strange that now I wonder if I'm doing something wrong...

Filipp Sudanov

Does it happen that there are no historical data for all DCIs in system, or just for some DCIs?

xelander

I've not configured many DCIs yet, but it seems no DCI shows historical data.
I've added a couple of other DCI on a different switch (3Com vs CISCO), but still no historical data in the GUI.

Even default DCIs for the NetXMS server itself show no historical data (see attachments).
The only strange thing I'm noticing now, is the <<ERROR>> value for the only DCI coming from the NetXMS agent on the server.

Filipp Sudanov

One more thing to have a look at - console writes it's own log file in user's home folder in \.nxmc\data\.metadata
Can you share what you have there?

xelander

Here they are.

nxmc-metadata-windows7.zip

nxmc-metadata-centos8.tgz

Filipp Sudanov

Did you try to open the dci graph / history during the duration of that log?

xelander

Yes, of course.

Filipp Sudanov

Let's try to get historical data from java API using nxshell - this way we could see if the issue is with the server or the GUI.

You will need to recompile to get nxshell, for that please run

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64

add then run your ./configure command with the following keys added:

--with-jdk=/usr/lib/jvm/java-11-openjdk-amd64 --with-client

make & make install should produce nxshell executable in your bin folder.

Then make a file, say collected_data.py

from java.util import Date
data = s.getCollectedData(200, 2000, Date(Date().time - 60 * 60 * 1000), Date(), 100, HistoricalDataType.valueOf('PROCESSED'))
print data
for v in data.values:
    print v.timestamp.toString() + " = " + str(v.value)

replace 200 with a node ID and 2000 with a DCI ID.

Then run the following (-u username, same as in management console and -P is the password):

nxshell -H 127.0.0.1 -u admin -P ""  collected_data.py

and share the output. This script script should return datapoints for a give DCI for last hour.

xelander

Hi, here they are.
Node 2069, DCI 682

nxshell -H 127.0.0.1 -u admin -P XXXX collected_data.py
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/usr/local/lib/netxms/nxshell.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.PySystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
DciData [nodeId=2069, dciId=682, dataType=INT32, valuesSize=0]

Node 2069, DCI 683

nxshell -H 127.0.0.1 -u admin -P XXXX collected_data.py
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/usr/local/lib/netxms/nxshell.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.PySystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
DciData [nodeId=2069, dciId=683, dataType=INT32, valuesSize=0]

It seems nxshell doesn't get any data too

collected_data.py

Filipp Sudanov

Can we possible get remote access to your netxms server? If yes, please send a message to [email protected]

If no, please collect server log at debuglevel = 8. Start console during the log and open the graph and history of a DCI.
You can change debut level on the fly:
nxadm -c "debug 8"
will change it

nxadm -c "debug"
will print current debug level

xelander

Yes, I can set up a remote access for you, thank you very much.
I have to setup some port forwarding on our firewall: do you only need SSH or also port 4701 for the console?
Can you tell me the public IP address you will be using to connect?

In the meanwhile, I changed the debug level and opened the history and graph of DCI 682 and 683 of node 2069, see attached log.I think they are at line 5744, timestamp 2020.08.11 11:21:55.828

netxmsd.zip

Filipp Sudanov

Sent you a PM.

Meanwhile - it's recommended to check netxms log for *E* and *W* messages - there is one SQL query error, which is not related to issue discussed here, we will have a look. And there's also a warning about autobind script.

xelander

The SQL errors are related to two new groups I was creating (see screenshot), perhaps I made something wrong.
About the autobind script, I don't know, I'm new to NetXMS and I'm still exploring all its features.
I remember opening the script library, perhaps I unvoluntarily tampered with the scripts.

Filipp Sudanov

There was one thing we missed - after recompiling --with-mariadb --with-mariadb-compat-headers, database driver in netxmsd.conf should have been updated, but mysql.ddr was left there.
The correct configuration is:

DBDriver = mariadb.ddr

P.S. pls disable system user in netxms - it's currently enabled.

xelander

Oh! I'm really sorry, I completely missed this step :-[ .
Thank you very, very much for your support and for your time.
P.S. I disabled the system user, thank you.

logic_sam

BIG HUG FOR Filipp Sudanov

NetXMS on CentOS 8
MariaDB

Data was not written in the history.
I was able to see "last values" but no history, no graphs.
netxms data collection was not working (i add this for google searches)

Thanks to Filipp :

i learned to look for datas in "idata_xxx" tables
i recompiled with mariadb options
i changed netxmsd.conf to "DBDriver = mariadb.ddr"

God bless you Filipp !

for the records :

./configure --with-server --with-mariadb --with-mariadb-compat-headers --with-agent
/usr/local/etc/netxmsd.conf
DBDriver = mariadb.ddr

NetXMS Forum

DCI last values don't store history data