Slow GUI

[Egert143]

Hello

What could cause slow gui response, curent version 5.0.5?

For example if i navigate the object tree, then sub folders opening speed is sometimes very slow. and clicking on random node, then information tabs load slow sometimes. Is it more sql problem ?

show dbstats:

SQL query counters:
   Total .......... 4922790645
   SELECT ......... 458044306
   Non-SELECT ..... 4464754025
   Long running ... 0
   Failed ......... 12131046
Background writer requests:
   DCI data ....... 1779941668
   DCI raw data ... 1983730639
   Others ......... 10013301

show stats:
Objects............: 60590
   Nodes...........: 7909
   Interfaces......: 46960
   Access points...: 1492
   Sensors.........: 0
Collectible DCIs...: 23509
Active alarms......: 5610
Uptime.............: 123 days,  6:38:56

Egert

[uldis]

Hi, could you please also share 'show threads' outputs? One while delay is observed and one perhaps just before?

[Egert143]

show threads:


MAIN
   Threads.............. 64 (64/512)
   Load average......... 0.00 0.00 0.00
   Current load......... 0%
   Usage................ 12%
   Active requests...... 0
   Scheduled requests... 1
   Total requests....... 4667321
   Thread starts........ 0
   Thread stops......... 0
   Wait time EMA........ 1 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

AGENT
   Threads.............. 4 (4/256)
   Load average......... 0.00 0.00 0.00
   Current load......... 0%
   Usage................ 1%
   Active requests...... 0
   Scheduled requests... 0
   Total requests....... 40449212
   Thread starts........ 1
   Thread stops......... 1
   Wait time EMA........ 0 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

POLLERS
   Threads.............. 762 (600/1200)
   Load average......... 581.33 450.22 410.93
   Current load......... 51%
   Usage................ 63%
   Active requests...... 392
   Scheduled requests... 0
   Total requests....... 3069782028
   Thread starts........ 31888
   Thread stops......... 31726
   Wait time EMA........ 8 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 7
   Queue size SMA....... 0
   Queue size SD........ 0

FILE-TRANSFER
   Threads.............. 2 (2/16)
   Load average......... 0.00 0.00 0.00
   Current load......... 0%
   Usage................ 12%
   Active requests...... 0
   Scheduled requests... 0
   Total requests....... 0
   Thread starts........ 0
   Thread stops......... 0
   Wait time EMA........ 0 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

NPE
   Threads.............. 1 (1/1024)
   Load average......... 0.00 0.00 0.00
   Current load......... 0%
   Usage................ 0%
   Active requests...... 0
   Scheduled requests... 0
   Total requests....... 0
   Thread starts........ 0
   Thread stops......... 0
   Wait time EMA........ 0 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

DATACOLL
   Threads.............. 600 (600/1200)
   Load average......... 7.95 11.07 11.24
   Current load......... 0%
   Usage................ 50%
   Active requests...... 3
   Scheduled requests... 0
   Total requests....... 2207466859
   Thread starts........ 6
   Thread stops......... 6
   Wait time EMA........ 0 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

CLIENT
   Threads.............. 16 (16/2048)
   Load average......... 2.61 1.36 1.19
   Current load......... 56%
   Usage................ 0%
   Active requests...... 9
   Scheduled requests... 0
   Total requests....... 163569444
   Thread starts........ 21
   Thread stops......... 21
   Wait time EMA........ 0 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

SYNCER
   Threads.............. 4 (4/4)
   Load average......... 0.00 0.00 0.43
   Current load......... 0%
   Usage................ 100%
   Active requests...... 0
   Scheduled requests... 0
   Total requests....... 228080205
   Thread starts........ 0
   Thread stops......... 0
   Wait time EMA........ 193 ms
   Wait time SMA........ 137 ms
   Wait time SD......... 65 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

DISCOVERY
   Threads.............. 8 (8/16)
   Load average......... 0.00 0.00 0.00
   Current load......... 0%
   Usage................ 50%
   Active requests...... 0
   Scheduled requests... 0
   Total requests....... 3105
   Thread starts........ 0
   Thread stops......... 0
   Wait time EMA........ 2 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

SCHEDULER
   Threads.............. 1 (1/64)
   Load average......... 0.00 0.00 0.00
   Current load......... 0%
   Usage................ 1%
   Active requests...... 0
   Scheduled requests... 1
   Total requests....... 43036
   Thread starts........ 0
   Thread stops......... 0
   Wait time EMA........ 0 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

MOBILE
   Threads.............. 4 (4/256)
   Load average......... 0.00 0.00 0.00
   Current load......... 0%
   Usage................ 1%
   Active requests...... 0
   Scheduled requests... 0
   Total requests....... 0
   Thread starts........ 0
   Thread stops......... 0
   Wait time EMA........ 0 ms
   Wait time SMA........ 0 ms
   Wait time SD......... 0 ms
   Queue size EMA....... 0
   Queue size SMA....... 0
   Queue size SD........ 0

[uldis]

Hi, thank you for your feedback. Client should not be doing any requests to server when browsing objects in object tree. When client starts, it reads information about all objects from server and keeps cache of that in it's memory. When state of objects changes on the server, server sends updates to the client and client updates it's in-memory copy.
So first thing to check - if there's enough of free mem on PC where client is running, may be it starts swapping. Also to check - CPU load on the PC, may be java consumes a lot of CPU and this slows things down.

To check, may be client still requests something from the server and waits for it - on the server side it's possible to see client-related messages in log, lines look like this:

2024.11.05 13:12:25.297 *D* [client.session.0  ] Sending message CMD_REQUEST_COMPLETED (64 bytes)

you can do
debug client.* 6
this would show messages for all connected clients. In our example 0 in tag client.session.0 means that it's client session number 0. So you can do

debug client.session.0 6
and you will see messages in log which are related only to that specific client session.

On unrelated note, please check your server logs to see why so many SQL queries are failing.

Failed ......... 12131046

[Egert143]

PC usage is relatively low. The i7-10875H CPU is utilized at about 20%, and out of 32GB of RAM, 40% is in use. This is the total PC usage.
In the debug log, I see a large amount of traffic, for example:

2024.11.06 09:55:42.798 *D* [client.session.7   ] Sending message CMD_NODE_DCI (16 bytes)
2024.11.06 09:55:42.809 *D* [client.session.7   ] Received message CMD_GET_DATA_COLLECTION_SUMMARY
2024.11.06 09:55:42.809 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (48 bytes)
2024.11.06 09:55:42.826 *D* [client.session.7   ] Received message CMD_GET_DATA_COLLECTION_SUMMARY
2024.11.06 09:55:42.826 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (48 bytes)
2024.11.06 09:55:42.864 *D* [client.session.7   ] Sending compressed message CMD_OBJECT_UPDATE (1256 bytes)
2024.11.06 09:55:42.867 *D* [client.session.7   ] Sending compressed message CMD_OBJECT_UPDATE (912 bytes)
2024.11.06 09:55:42.870 *D* [client.session.7   ] Sending compressed message CMD_OBJECT_UPDATE (912 bytes)
2024.11.06 09:55:42.873 *D* [client.session.7   ] Sending compressed message CMD_OBJECT_UPDATE (888 bytes)
2024.11.06 09:55:42.875 *D* [client.session.7   ] Sending compressed message CMD_OBJECT_UPDATE (928 bytes)
2024.11.06 09:55:43.179 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:43.179 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:43.181 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:43.181 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:43.181 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:43.181 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:43.182 *D* [client.session.7   ] Received message CMD_GET_NODE_DCI_LIST
2024.11.06 09:55:43.183 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:43.183 *D* [client.session.7   ] Sending message CMD_NODE_DCI (16 bytes)
2024.11.06 09:55:43.194 *D* [client.session.7   ] Received message CMD_GET_DATA_COLLECTION_SUMMARY
2024.11.06 09:55:43.194 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (48 bytes)
2024.11.06 09:55:43.204 *D* [client.session.7   ] Received message CMD_GET_DATA_COLLECTION_SUMMARY
2024.11.06 09:55:43.204 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (48 bytes)
2024.11.06 09:55:43.517 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:43.517 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:43.519 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:43.519 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:43.519 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:43.519 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:43.520 *D* [client.session.7   ] Received message CMD_GET_NODE_DCI_LIST
2024.11.06 09:55:43.520 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:43.521 *D* [client.session.7   ] Sending message CMD_NODE_DCI (16 bytes)
2024.11.06 09:55:43.536 *D* [client.session.7   ] Received message CMD_GET_DATA_COLLECTION_SUMMARY
2024.11.06 09:55:43.536 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (48 bytes)
2024.11.06 09:55:43.557 *D* [client.session.7   ] Received message CMD_GET_DATA_COLLECTION_SUMMARY
2024.11.06 09:55:43.557 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (48 bytes)
2024.11.06 09:55:43.657 *D* [client.session.7   ] Sending compressed message CMD_OBJECT_UPDATE (1328 bytes)
2024.11.06 09:55:44.061 *D* [client.session.7   ] Sending compressed message CMD_EVENTLOG_RECORDS (192 bytes)
2024.11.06 09:55:44.223 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:44.223 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:44.224 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:44.224 *D* [client.session.7   ] Received message CMD_UNLOCK_NODE_DCI_LIST
2024.11.06 09:55:44.225 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:44.225 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:44.226 *D* [client.session.7   ] Received message CMD_GET_NODE_DCI_LIST
2024.11.06 09:55:44.226 *D* [client.session.7   ] Sending message CMD_REQUEST_COMPLETED (32 bytes)
2024.11.06 09:55:44.227 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (728 bytes)
2024.11.06 09:55:44.228 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (600 bytes)
2024.11.06 09:55:44.229 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (600 bytes)
2024.11.06 09:55:44.231 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (896 bytes)
2024.11.06 09:55:44.232 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (776 bytes)
2024.11.06 09:55:44.234 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (752 bytes)
2024.11.06 09:55:44.235 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (1040 bytes)
2024.11.06 09:55:44.236 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (760 bytes)
2024.11.06 09:55:44.237 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (792 bytes)
2024.11.06 09:55:44.238 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (568 bytes)
2024.11.06 09:55:44.239 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (568 bytes)
2024.11.06 09:55:44.240 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (576 bytes)
2024.11.06 09:55:44.240 *D* [client.session.7   ] Received message CMD_GET_DATA_COLLECTION_SUMMARY
2024.11.06 09:55:44.242 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (776 bytes)
2024.11.06 09:55:44.242 *D* [client.session.7   ] Sending compressed message CMD_REQUEST_COMPLETED (1776 bytes)
2024.11.06 09:55:44.243 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (776 bytes)
2024.11.06 09:55:44.245 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (792 bytes)
2024.11.06 09:55:44.246 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (792 bytes)
2024.11.06 09:55:44.247 *D* [client.session.7   ] Sending compressed message CMD_NODE_DCI (656 bytes)

Not sure if it was a coincidence, but as soon as I started debugging, the GUI felt a lot more responsive, with objects loading almost instantly. However, after restarting the client, the GUI returned to being slow, and re-enabling debug mode did not improve speed this time, so it was likely random.

In the NetXMS server log, I see endless entries like this:

2024.11.06 10:03:56.006 *E* [db.drv             ] SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_197008 WHERE item_id=? AND idata_timestamp BETWEEN ? AND ? ORDER BY idata_timestamp DESC"): [Microsoft][SQL Server Native Client 11.0]Invalid cursor state
2024.11.06 10:03:56.006 *E* [db.drv             ] SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_197008 WHERE item_id=? AND idata_timestamp BETWEEN ? AND ? ORDER BY idata_timestamp DESC"): [Microsoft][SQL Server Native Client 11.0]Invalid cursor state
2024.11.06 10:03:56.006 *E* [db.drv             ] SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_197008 WHERE item_id=? AND idata_timestamp BETWEEN ? AND ? ORDER BY idata_timestamp DESC"): [Microsoft][SQL Server Native Client 11.0]Invalid cursor state
2024.11.06 10:03:56.006 *E* [db.drv             ] SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_197008 WHERE item_id=? AND idata_timestamp BETWEEN ? AND ? ORDER BY idata_timestamp DESC"): [Microsoft][SQL Server Native Client 11.0]Invalid cursor state
2024.11.06 10:03:56.006 *E* [db.drv             ] SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_197008 WHERE item_id=? AND idata_timestamp BETWEEN ? AND ? ORDER BY idata_timestamp DESC"): [Microsoft][SQL Server Native Client 11.0]Invalid cursor state
2024.11.06 10:03:56.006 *E* [db.drv             ] SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_197008 WHERE item_id=? AND idata_timestamp BETWEEN ? AND ? ORDER BY idata_timestamp DESC"): [Microsoft][SQL Server Native Client 11.0]Invalid cursor state
2024.11.06 10:03:56.006 *E* [db.drv             ] SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_197008 WHERE item_id=? AND idata_timestamp BETWEEN ? AND ? ORDER BY idata_timestamp DESC"): [Microsoft][SQL Server Native Client 11.0]Invalid cursor state

This appears to be what's causing the failed SQL queries.
I also asked other NetXMS client users, and they reported a slow GUI as well, which they noted likely started with version 5.

[victor]

Just to make sure we understand setup correctly: you are using desktop client on Windows workstation, connecting to remote server that is also on Windows and using MS SQL database?

Best regards,
Victor

[Egert143]

Yes, that's correct.