NetXMS Forum

Hi NetXMS team,

recently, I was investigating the usage of TimescaleDB for the underlying NetXMS database. The system works fine, however when taking a look at the implementation, I noticed that it only uses the feature of Hypertables and does not use Continuous Aggregation options.

I tried to set up Continuous Aggregation on my local instance of NetXMS by adding VIEWs to the DB schema, using the TimescaleDB tutorials as a guide (https://docs.timescale.com/latest/using-timescaledb/continuous-aggregates).

The SQL code for my tests is added in the file continuous_aggregation_for_netxms_tsdb.sql.

For testing, I only used the idata_sc_default table and left all DCIs in the default storage class. The SQL code basically creates five "layers" of continuous aggregation, each with decreasing granularity compared to the previous one:

• Older than 1 hour: Calc average over 3 minutes intervals (idata_continuous_3minutes_gt_1hour)
• Older than 1 day: Clac average over 5 minutes intervals (idata_continuous_5minutes_gt_1day)
• Older than 1 week: Clac average over 10 minutes intervals (idata_continuous_10minutes_gt_1week)
• Older than 1 month: Clac average over 30 minutes intervals (idata_continuous_30minutes_gt_1month)
• Older than 3 months: Clac average over 1 hour intervals (idata_continuous_1hour_gt_3months)

All these VIEWs are then combined in one single VIEW idata_continuous, including the original data for the most recent 60 minutes.

The VIEW itself works as expected, however, making NetXMS actually read the data through this VIEW it not possible without changing NetXMS sourcecode, as by default when using TSDB syntax, NetXMS reads the data directly from the storage class (idata_sc_XYZ) tables and not from the combined idata VIEW.

-----------------------------------------------

Now for my actual feature request (sorry btw. for the verbose introduction  ;D)

Would it be possible to introduce this as an optional feature? As a first implementation, I thought about a globally definable policy for multiple levels of Continuous Aggregation (possibly even define individual aggregation functions?), which could be configured via the NXMC GUI. As this feature does only use VIEWs, which are mostly read-only, they do not alter any materialized data in the DB and thus could even be configured on an existing database without breaking the system.

The most obvious advantage of this feature would be a huge performance boost for read queries that request large amounts of data over long period of time (e.g. 1 year of 60s data points) where it is of greater interest to observe a certain trend of data instead of the actual detailed datapoints. This would especially be useful in combination with Grafana, where it is quite common to increase the size of the displayed timeframe to investigate certain data-trends instead of being interested in exact values of very old data.

One minor downside of this would be increased storage consumption, as those VIEWS are updated periodically by TimescaleDB and thus store additional data next to the original values.

As for the actual implementation, there are two problems which I would consider hard to overcome - How to handle string/text based data values and how to handle table DCIs?

Anyway, this is just meant as an idea and to provide some suggestions for potential ways to implement it.

Hoping for your feedback,
best regards
fbu

---

continuous_aggregation_for_netxms_tsdb.sql

While this approach is certainly an alternative to mine, it does not help with the core problem, which is the DCI Data Recalculation:

Unfortunately, you cannot pass a timestamp parameter the functions PushDCIData, which could define where to put the new value in the DCI table. Therefore, it will always put the new value to the actual current timestamp.

When looking at the sourcecode, I would see the following location, at which this problem could be addressed:

In line 114 of src/server/core/dci_recalc.cpp, the recalculateValue(value) function is called, which calls the function transform(value, elapsedTime) in line 2151 of src/server/core/dcitem.cpp.
In this function, in line 999 of the same file (dcitem.cpp), a ScriptVMHandle object is created, using the function CreateServerScriptVM(...). This function CreateServerScriptVM(...) could have an optional timestamp parameter, which would tell the script VM to simulate an execution at the given point in time, defined by the given parameter.

The value for this timestamp parameter could be extracted from the ItemValue &value (which is the current DCI value to be recalculated) of the calling function, using value.getTimeStamp().

This approach, however, would require refactoring the implementation of NXSL functions, like time(), GetDCIValue(...) or GetDCIRawValue(...) as their behavior should then be adapted, accordingly.

(I'm sorry, if I have some weird ideas, which might sound reasonable to me but not for others  ;D)

Dear NetXMS team,

I seem to have found a scenario related to DCI Data Recalculation, during which the server (Version 3.2-400) does not behave as expected. The scenario is as follows:

For some of my DCIs I use secondary DCIs, which feature a longer retention period and a bigger polling interval, to avoid storing data with fine granularity over a long period of time.

These secondary DCIs use different aggregation scripts, based on what kind of data the primary DCI returns. The most common script is a simple average calculation, like this:

return GetAvgDCIValue($node, FindDCIByDescription($node, "gateway.RTT"), time()-3600, time());

You can see that (with a polling interval of 1 hour) the secondary DCI simply returns the average value of the primary DCI for the last hour.

This method works perfectly fine, as long as the server is running.

And this is where the problem comes in:

If for some reason, the server was down and the secondary DCIs could not poll any data (the primary DCIs still gathers data, as it is coming from a NetXMS Agent, which stores the data on its local DB during server downtime), I want to use the DCI Data Recalculation feature, to recalculate the secondary DCI data, based on the data that the primary DCI could still gather during the downtime via the Agent.

However, it seems like the NXSL function time() always returns the actual current timestamp, instead of the timestamp of the value to recalculate.
Just to make it a little clearer: Imagine the current timestamp is X and the server should recalculate a DCI value at the timestamp Y (with Y < X). The transformation script shown above is supposed to calculate the average of the primary DCI for the period of one hour before timestamp Y and not X.

This behavior leads to the situation that for every recalculated value of the secondary DCI, the time() function returns timestamp X, meaning the recalculated average value will always be the one for the latest one hour period (relative to actual present), which is not correct.

One way I could see to resolve this problem is to have an option for the DCI Data Recalculation, which makes the server always take the timestamp of the value that is currently being recalculated and (temporarily) set this as the current timestamp, which will then be returned by the time() function, just for the process of data recalculation.

Additionally, I could also think of an option, which allows to "fill gaps" in the data (e.g. caused by server downtime). Maybe by checking the polling interval and generating the missing values. If no RAW values are present, the transformation script could still be executed. For a script like the one above, this would still return values to the final data.

Hopefully, these thoughts will be helpful to you.

Thank you and kind regards,
fbu

Now, for my final update:

After some hours of digging, I found out that NGINX was the reason for the entire problem: It was not configured to pass the POST data to the proxied API server.

Thus, there is no problem with the original API code - I guess that the scenario, as stated in my second update, may be related to the entity's content being empty, once it is read (Somewhere, I read about it being a stream, rather than a storage variable, but I'm not 100% sure about that)

Anyway, I apologize for causing this false alarm and thank you for your support.

Keep up the good work on this tool

Best regards,
fbu

Update #2:

Based on my previous update, I tested a workaround solution, by simply instantiating the JSONObject, based on the text value returned by the entity:

JSONObject data = new JSONObject(entity.getText());

This works perfectly fine for me, when bypassing the NGINX.

I still don't know what the actual problem is, but it works like this for me - I only need to fix the problem related to NGINX.

Thanks for your help, Victor. Hopefully the root cause of this problem can be determined.

Kind regards,
fbu

Quick update:

Previously, I was using an NGINX in front of Tomcat9 for hostname and SSL/TLS handling.

When bypassing that NGINX and going directly to the Tomcat, the entity.getText() actually returns the provided data as string. However, new JsonRepresentation(endity).getJsonObject() and .getText() still return NULL and cause the same NullPointerException, as before. (see attachment for debug messages)

I am pretty much out of ideas, right now. I'll still try to at least get a similar result with using NGINX, as I will need it in the future, but this might just be a configuration error.

Hope this helps.

Kind regards,
fbu

---

Hi Victor,

I managed to build the changed source code for the API and deploy it on tomcat. You can find the changes in the attachment "sourcecode_change.png" (lines 103 - 108).

The debug-messages are printed in the screenshot "debug_messages.png". I shows that entity.getText() returns NULL, which causes the StringReader called by getJsonObject() (now in line 107) to throw a NullPointerException (as seen in "nullpointer_cause.png".

However, I still don't know why getText() returns NULL in the first place ... I will keep searching for the cause, but I'm not feeling to optimistic about my chances.

Hope, I could help, so far.

Best regards,
fbu

---

Okay, I will try my best. Is there a way to only build the war file for the API, or do I need to rebuild the whole project?

Kind regards,
fbu

Hi Victor,

I added a screenshot of the stack trace for the NullPointerException. (Note that the date does not match, because I simply placed the request, again. But it is the same, as for the previous requests)

There are different debug outputs as well, but they are all related to other requests, this error message and stack trace are the first log lines that appear, when performing the request.

Best regards,
fbu

---

Dear NetXMS,

I am currently trying to receive data from our NetXMS Server through the REST API.

Logging in via the basic Authentication header works fine and GET requests to /objects.*, /alarms.*, /info or /predefinedgraphs endpoints returns proper data on valid and error messages on invalid requests.

However, using POST requests on the /sessions and /summaryTable/adHoc endpoints results in {"error":46} responses, when adding a JSON object to the request.

Omitting the JSON object results in an "Invalid argument" message.

Digging through the sourcecode of the latest version (3.0.2329) and cross-checking with the Tomcat9 logs, I could find out that when not adding a JSON object to the request, the logging message in line 100 of AbstractHandler.java ("No POST data in call") is being printed and the method returns the expected error object.

But: When a JSON object is added to the request, the debug message in line 105 ("POST: data = ") is not being printed, although it should be and the loglevel is set to "debug" (Other debug messages of the API are visible).

The tomcat logs then state that the internal error is caused by a NullPointerException, so my assumption would be that either new JsonRepresentation(entity) or .getJsonObject() in line 104 of AbstractHandler.java return NULL, thus causing this exception.

Is there anything special that I did not take into account or did I stumble upon a bug? (Stacktrace is attached as screenshot "tomcat9_stacktrace.png")

Just as additional information: I am using Postman (note the missing "description" field in the response, as seen in the attached screenshot "postman_error_46.png") for testing, but I also tried with cURL, giving the same error.

I would be glad, if you could help me with this 

Kind regards,
fbu

---

Hi NetXMS team,

I found a few instances of Dashboard Templates being mentioned on this forum and that they might be considered for development.

I think this would be a great addition to the tool, especially when having a network with numerous similar nodes which all should have individual dashboards.

Currently, we need to create the dashboards manually, which is very time consuming and dashboard templates would be really helpful with this.

Is there any update on this topic?

Kind regards,
fbu

To add another point:
This could also be implemented for the geographical map (World Map view). There, if you want to access the object details of a device you chose in the map, you need to navigate to "Service dependency", open its dropdown menu there and hit "Show Object details".

I think, it would be nice to have the dropdown menu entry "Show Object Details" by default available. Maybe even not only in dashboards and geo-maps, but in every occasion, where you can open a dropdown menu for a specific object.

Kind regards,
fbu

Dear NetXMS team,

as requested by Victor in my previous post under 'General Support' (https://www.netxms.org/forum/general-support/multiple-values-assigned-to-same-timestamp-with-nxapush/), I now post this topic as a feature request.

To sum up my point: I try to use nxapush to update DCI values at given timestamps, which results in multiple values being assigned to the same timestamp, as seen in the two screenshots attached to this post.

My idea would be to have an option (maybe -u) indicating that any value currently assigned to the given timestamp should be deleted and replaced by the new value, allowing a more flexible use of both nxpush and nxapush.

Best regards,
fbu

---

Okay, I will do so.

Thank you very much.

Kind regads,
fbu

Hi,

currently, if you want to display in a dashboard how the values of a certain DCI developed during a certain period of time, you use a line chart.

I think it would be cool, if it was possible to do something similar with bar- and tube-charts, as well.

Maybe even give the possibility to define intervals in which the period should be split up to and how the data in these intervals should be accumulated.

E.g.: Something like - "For the period of the last 8 months, display the sum of all data of each individual month", resulting in a chart similar to this:
https://preply.com/wp-content/uploads/2018/08/representation-bar-graph.png.

Kind regards,
fbu

Dear NetXMS team,

I am currently facing an issue with Line charts which use data from DCI, populated by the NetXMS Agent push:

The way, data collection works for these DCIs is that (e.g.) every five minutes, a script pushes 24 timestamp-value pairs (one for every one of the last full hours) to the DCI using nxapush.

This is done every five minutes, because the values might change over the course of an hour. The idea here is that it always overwrites the previously assigned values to those timestamps, and here comes the problem:

I noticed (at first in the line charts) that there are multiple different values assigned to equal timestamps, resulting in vertical steps within the graph, as seen in 'double_value_assign.png'.

Then, I checked the history data of the DCI and as seen in 'multiple_value_assign1.png', there are in fact multiple instances of timestamp-value pairs, which have the same timestamp.

(I marked the two entries, which refer to the right-most step in the line chart screenshot)

Is this expected behavior, or a bug?

Anyway, my assumption is that most users would expect nx(a)push to just update the value, if an entry with the given timestamp is already present on the DCI.

Kind regards,
fbu

---

Soooo, after hundreds of LoC and hundreds of grey hair later, I finally managed to get the Apereo CAS working with NetXMS Web UI.

In order to help other users, who also struggle to get this to work on their setup, I answer my questions from the original post by myself, give brief tips and try to keep it as general as possible.

You will need the following Server Configuration variables set on your NetXMS server:

"CASHost" - The hostname/IP facing to the CAS server; in case it is behind a proxy, configure the proxy let this hostname point to the CAS server (or the root folder of it's .war)
"CASPort" - The port which will be appended to 'CASHost' for the https-request (Again, when behind a proxy, use the port, which is configured in the proxy to point towards the server)
"CASService" - The 'serverId' configured in CAS, NetXMS uses to identify intself at CAS
"CASTrustedCACert" - The path to the CA certificate of the CAS server (actually, I'm not 100% sure about that, but it worked for me to just use the .crt file, which I generated for the CAS-domain)
"CASValidateURL" - The path extension to append to hostname and port to get CASHost:CASPort/CASValidateURL. At this resource, NetXMS requests CAS to validate the received service ticket. (With CAS 6.0.x, this is by default '/cas/serviceValidate')

(From this point, I will use cas.example.com as CAS-hostname and netxms.example.com as NetXMS-WebUI hostname -> this will also be its serviceId at CAS)

1. "How does the Webapp handle the SSO session?":

• After loggin into CAS at cas.example.com/cas/login?service=netxms.example.com, CAS will redirect to netxms.example.com?ticket=ST--something
• The Webapp then attempts to connect to the NetXMS server defined either in nxmc.properties or by the 'server' query parameter (otherwise, 127.0.0.1 is chosen) and provides the service ticket to be used as SSO login.
• The NetXMS server then tries to connect to the CAS server's service validation endpoint defined by the three variables above, using HTTPS and to verify its SSL certificate (here: cas.examlpe.com/cas/serviceValidate?service=netxms.example.com&ticket=ST-[...]-something
• The provided parameters 'service' and 'ticket' are used for validation. If they are valid, NetXMS tries to extract the XMS tags cas:authenticationSuccess for confirmation and cas:user to determine the username, from CAS-server's response.
• Successful login is only possible, if the username extracted from cas:user is present in the NetXMS user database, otherwise, access will not be granted.

1.1 "Is there a special URL to which CAS needs to redirect upon successful login?":
More or less: netxms.example.com?ticket=ST-[...]-something (The URL to reach the login-page of the NetXMS WebUI, extended by a query string containing at least the ticket-parameter)
This is done by default by CAS, so nothing to do here.

1.2 "Which attributes are needed from and for CAS?":
From CAS, only the service ticket (and optionally, the NetXMS server, but this can by set in nxmc.properties)
To CAS, serviceId (CASService configuration variable) and ticket, additionally, the username in CAS should also be present in NetXMS user database

1.3 "Is SLO (single log out) also implemented?":
So far, I did not see anything like this. After logging out from a NetXMS SSO session, the current service ticket may be expired (depending on the CAS configuration), but the TGT (Ticket granting ticket) is not, so logging in again via the CAS interface will not prompt the login dialog of CAS, until cas.example.com/cas/logout is requested.

2 "How is the user management handled?: "
The username extracted from the validation response (cas:user, which is basically the principal id) is used to determine the representing user within the NetXMS database. Basically, it is just as logging in as this user.

2.1 "Does te user still need to be present in the NetXMS database?: "
Yes 

2.2 "If not, how are access rights handled?: "
As 2.1 returns 'Yes', this is pretty self explaining, given that you know how NetXMS internal user-/group management works.

3 "Can the official documentation be extended with this topic?":
I don't know, they could refer to this forum thread, maybe  ;D

Hopefully, this helps some people setting up their CAS/NetXMS infrastructure. (Btw.: Debug level 9 in server logs gives some good information about this. Keep an eye open for SSO error codes (0,-1,-2,-10 ... -20 ...) and messages - their handling can be checked in the file 'cas_validator.cpp' of the sourcecode, which can provide some help in determining, what is going on).

The file 'ApplicationWorkbenchAdvisor.java' in [SourceCodeRoot]/webui/webapp/Core/src/org/netxms/ui/eclipse/console/ shows, how the login method is chosen, depending on the URL parameters provided upon accessing the login page of the WebUI.

Another tip: Service ticket validation needs to be done via HTTPS - a self-signed certificate is sufficient in a testing environment.

Feel free to ask, if questions come up 

Thank you and kind regards,
fbu

Hi all,

maybe someone can help me with the following:

I am currently trying to set up a SSO (Single sign on) solution for NetXMS using Apereo CAS. After several days of research and digging, I stumbled across a few lines in the WebUI source code, indicating that it has native SSO support, which was then confirmed by the change log of version 1.2.8 (https://www.netxms.org/forum/announcements/netxms-1-2-8-released/).

However, I could not find a single line of documentation nor a single forum thread on this topic, which is why I started one, by myself.

Currently, the following are the main questions related to this:

1. How does the Webapp handle the SSO session?
   -> Is there a special URL to which CAS needs to redirect upon successful login?
   -> Which attributes (if any) are needed from and for CAS?
   -> Is SLO (single log out) also implemented?
2. How is the user management handled?
   -> Does the user still need to be present in the NetXMS database?
   -> If not, how are access rights handled (maybe via a special group?)
3. Can the official documentation be extended with this topic? It would really be cool to have a proper guide on how to set this up

I would be really happy, if someone could find the time to help me with this topic 

Thank you and kind regards,
fbu

Dear NetXMS team,

here is another thing that came to my mind when creating line charts in NetXMS:

Currently, NetXMS provides the ability to create "stacked" line chart diagrams, however, it only allows to stack all or none of the data from the source DCIs.

I think it would be useful to manually choose, which data to stack on top of each other and to have a more user friendly way of defining the stacking order (right now it is defined by the order the DCI sources).

Imagine the following: We have the four metrics TCP upstream, UDP upstream, HTTP upstream and overall upstream (downstream equivalent). To combine all four within one diagram, it should be possible to only stack UDP on top of TCP, as HTTP is a subset of the TCP traffic and the overall upstream is the superset of all three (which would only be added as a reference line).

What do you think about this idea?

Thanks and kind regards,
fbu 

Hi NetXMS team,

here's another idea of a potentially useful feature addition:

A NXSL function which allows to manually enable and disable DCIs or set their states. The function could be defined such as "setDCIStatus(dci, status)", where dci would be the DCI-object and status an integer with 0=Active, 1=Disabled and 2=Not Supported.

Maybe "setDCIStatusByID(dciID, node, status)", with dciID being an integer representing the DCI ID and node being a node-object as optional parameter, would also be possible.

The idea came to my mind, because a fairly large set of the nodes within my current setup is receiving monitoring data via NXAgent extension scripts from a centralized database, and are initially created via a big NXSL script under "Infrastructure Services", based on this data, as it is impossible to have a direct monitoring communication with NetXMS - not even with the use of NXAgent and proxy agents.

Because these devices might change in their states and different metrics temporarily might not return data any more, it would be useful to have a NXSL function, to disable those metrics via a script, as the "unsupported"-detection is also not working with this setup.

I know, template automatic apply rules can be used for this, however, once a template is un-applied from a node, the DCIs on it are gone and there is no way to access their history data via the management console until the template is re-applied, again.

What do you think, could this be useful if so, would it be difficult to implement?

Kind regards,
fbu