Performance with Custom attributes
-
Hello geeks,
This is a design question... We have our own HW that communicates over SNMP. So we created a brand new server driver on the NetXMS side so that it collects all info we need thru "Custom attributes". It works. My concern falls into Performance of such solution. Currently, we are talking about approximately 250 Custom attributes and about approximately hundreds of devices to monitor simultaneously.
Do you have a general idea how such monitoring configuration will work in respect to let say performance of PostgreSQL database? We're currently using an alternative NMS and it is performing not very well when it comes into database.
Would you have a general advice when these "Custom attributes" are the best approach and when we better think about changing/enriching the NetXMS DB model and putting some tables in.
Thank you for advices.
Yarda.
-
Hi!
It all depends on how often you change and access attributes. NetXMS server caches lot of things, including custom attributes. For example, when you cange custom attribute from the driver, it will not change database immediately, but just mark object as modified. Background sync thread writes changes from objects to database periodically (once per minute by default). Reading custom attributes never involve database access - they read from memory. Drawback of this is of course increased memory usage. Custom attributes stored as strings, so it's not very effective when you have numeric data.
I would go with custom attributes. I also see couple of simple changes to allow easier creation of complicated drivers: we can add "driver data" field to Node object (simple pointer to void) where driver will be able to store it's data in any convenient format, and two additional calls in the driver - saveToDatabase and loadFromDatabase - to be called on node object saving and loading to allow save and restore of driver specific data.
Best regards,
Victor -
One of our businesses is to poll these hundreds of devices every five minutes. Basically, most of the data are numbers. Polling is not critical for DB storage, we do not keep history. Your answer is fair enough, I do not see a problem.
The second important business of ours is of course processing of traps.
And here is a buggy case from our current system:
When we poll statuses and listen to traps, we sometimes face to a situation of losing the proper state due to problem with collision of transactions. Is the state of objects and its propagation to parent objects kept in memory as well? When saying parents I mean artificial objects / logical view on our monitored devices. Typically, we group devices geographically. country (artificial object) -> city (artificial object) -> device (real HW with a driver in NetXMS).
Yarda.
-
Hi!
Yes, server keep entire object tree in memory, and all status calculations done in memory. Objects with changed statuses are marked as modified, and later saved to database by background sync thread. We also actively use queues for processing, so you can experience a slight delay in trap processing for example, but they will not be lost and will be processed in correct order.
Best regards,
Victor
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login