Migrate V2.2.17 on PostgreSQL to new server with V3.0 on Timescale-DB

[Filipp Sudanov]

Hi!

NetXMS has a tool called nxdbmgr that has an option for database migration. It is launched like this:

nxdbmgr migrate old.configuration.file

The idea is that this tool is launched on the system you are migrating to. In the regular netxms configuration file on that system you have all the configuration for the new database. And you have another configuration file that has connection parameters to the database you are migrating from - that file name is given to nxdbmgr as parameter.
Before migration you have to initialize the new database by running nxdbmgr init.
netxms server should be stopped so no new data appears in the database during migration.
Migration is only supported on the same version of the database. So you can install 2.2.17 on the new system, do the migration and the do the upgrade.
A more recommended way is to upgrade first and migrate then (in that case you would use newer nxdbmgr that possibly has some bugs fixed) - but for that you would need to compile libprce32, so I would try the above approach first.

[user54119]

Thanks for your information Filipp

But is Timescale DB really supported in V2.2.17 for migration purposes? Then I would install 2.2.17 on new server with Timescale directly. And upgrade after DB migration.

Or would it be better to install V2.2.17 on new server with DB on PostgreSQL, than update to V3.0.x and then Migration to Timescale?
What ist the current most recommendet DB engine for NetXMS? Still PostgreSQL oder already Timescale?Is performance in NetXMS really better on Timescale? (my main point to change db engine)

[victor]

Hi,

recommended database engines are PostgreSQL and TimescaleDB, depending on deployment size. For small deployments (few thousands of DCIs) you most likely won't get any significant performance gains. The larger your deployment is the more performance improvement you'll see.

Support for TimescaleDB in 2.2.17 was still experimental, and I would not recommend to do migration to TimescaleDB using 2.2.17. I would do as following:

1. Install new server and initialize TimescaleDB database
2. Upgrade old database to 3.0 using new nxdbmgr
3. Migrate old database to new one

We are planning to add in-place conversion from plain PostgreSQL to TimescaleDB but I'm not sure when it will be ready.

Best regards,
Victor

[user54119]

thanks Victor for your information and suggestions.
Deployment is not so big. But in the past I saw the huge difference in speed between grafana graphs (on influx) and NetXMS graphs. So I hoped Timescale would speed up graphing in NetXMS. Or is this not the point where we can see more performance with Timescale in NetXMS?

[victor]

Yes, TimescaleDB could increase graphing speed. You have to experiment. If you have enough disk space you can do migration from one to another quite fast and check the difference.

Best regards,
Victor

[user54119]

i tried to compile netxms-3.0.2357 on openSUSE LEAP 15.1 but got error from configure 'cannot find pcre.h'.
Then installed package 'pcre-devel' (version 8.41) from package-manager (YaST), configure and get 'error: libpcre32 is requred'.Then downloaded  https://ftp.pcre.org/pub/pcre/pcre-8.43.tar.gz
./configure --prefix=/opt/pcre --disable-static --disable-dependency-tracking --enable-pcre16 --enable-pcre32 --disable-cpp --enable-utf --enable-unicode-properties --disable-stack-for-recursionmakemake installthen configure for NetXMS with option --with-pcre=/opt/pcre but got again 'error: libpcre32 is requred

[victor]

Please share your config.log.

Best regards,
Victor

[user54119]

here I have. Thanks Victor!

---

config.zip

[victor]

Very strange. Please show output of command

file /opt/pcre/lib/libpcre32.so*

and

ldd /opt/pcre/lib/libpcre32.so

Best regards,
Victor

[user54119]

netxms:~ # file /opt/pcre/lib/libpcre32.so*
/opt/pcre/lib/libpcre32.so*: cannot open `/opt/pcre/lib/libpcre32.so*' (No such file or directory)
netxms:~ # ldd /opt/pcre/lib/libpcre32.so
ldd: /opt/pcre/lib/libpcre32.so: No such file or directory

[victor]

That means that libpcre was not properly built or installed. Are you sure that there were no compiltation or installation errors? What files are present in /opt/pcre/lib?

Best regards,
Victor

[user54119]

Thanks for your help Victor. Bu t I didnt's saw errors. Here all output of configure, make, make install: Maybee you see some problems in it.

---

pcre-build-output.txt

[Tursiops]

Looks like compiling PCRE installed the libraries in /opt/pcre/lib64, while NetXMS is looking in /opt/pcre/lib.
Not sure if it would be enough to create a symbolic link lib64 -> lib?

[user54119]

I didn't knew NetXMS is looking in /opt/pcre/lib. Made a symlink and compiling NetXMS is fine now. I'm a step forward, thanks 

[victor]

This looks like a bug in configure script - it is made with assumption that library files always installed under PCRE prefix /lib. Seems to not be the case on SUSE. I'll update configure script.

Best regards,
Victor