After binding remote agent it loosed connection with tlsv1 a decrypt error - Win

[grae]

I just deployed NetXMS in azure on Server 2022, opened up port 4700 & 4703.
Created the self signed certificates

openssl genrsa -out rootCA.key 2048
openssl req -config openssl.conf -x509 -new -key rootCA.key -days 10000 -out rootCA.crt
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -config openssl.conf

C:\NetXMS\bin>openssl x509 -req -in server.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out server.crt -days 5000 -extfile openssl.conf -extensions v3_ca

Finally my agents will show up in Agent Tunnels as Unbound, however as soon as they are bound, it will close and reset the tunnel, load new certificates and then error out with the error  SSL routines::tlsv1 alert decrypt error

2023.03.16 17:25:52.864 *W* [                   ] Tunnel with XXX.XXX.XXX.XXX closed
2023.03.16 17:26:12.565 *D* [tunnel             ] XXX.XXX.XXX.XXX: Resetting tunnel
2023.03.16 17:26:12.630 *D* [tunnel             ] XXX.XXX.XXX.XXX: Loading certificate from file
2023.03.16 17:26:12.634 *D* [tunnel             ] XXX.XXX.XXX.XXX: Certificate and private key loaded
2023.03.16 17:26:12.776 *D* [tunnel             ] XXX.XXX.XXX.XXX: TLS handshake failed (error:00000001:lib(0)::reason(1))
2023.03.16 17:26:12.777 *D* [tunnel             ] XXX.XXX.XXX.XXX: Caused by: error:0A00041B:SSL routines::tlsv1 alert decrypt error

After a few minutes it times out and ignores the certificate, and shows back up as un-bound in the console.  Any help is greatly appreciated as this will never be used over LAN or VPN and must be able to register and manage remote sites.

2023.03.16 17:30:44.891 *D* [tunnel             ] XXX.XXX.XXX.XXX: Next connection attempt will ignore agent certificate
2023.03.16 17:31:15.106 *D* [tunnel             ] XXX.XXX.XXX.XXX: TLS handshake completed
2023.03.16 17:31:15.107 *D* [tunnel             ] XXX.XXX.XXX.XXX: Server certificate subject is C=US,ST=WY,L=Cheyenne,O=Grae Inc,CN=Grae Network Monitoring
2023.03.16 17:31:15.107 *D* [tunnel             ] XXX.XXX.XXX.XXX: Server certificate issuer is C=US,ST=WY,L=Cheyenne,O=Grae Inc,CN=Grae Network Monitoring
2023.03.16 17:31:15.108 *D* [tunnel             ] XXX.XXX.XXX.XXX: Server certificate verification is disabled
2023.03.16 17:31:15.108 *D* [tunnel             ] XXX.XXX.XXX.XXX: Server certificate pinning is disabled
2023.03.16 17:31:15.108 *D* [comm.vs.3          ] Requesting metric "System.PlatformName"
2023.03.16 17:31:15.110 *D* [comm.vs.3          ] Requesting metric "System.UName"
2023.03.16 17:31:15.110 *D* [comm.vs.3          ] Requesting list "Net.InterfaceList"
2023.03.16 17:31:15.118 *D* [tunnel             ] XXX.XXX.XXX.XXX: Sending message CMD_SETUP_AGENT_TUNNEL (1)
2023.03.16 17:31:15.293 *D* [tunnel             ] XXX.XXX.XXX.XXX: Received message CMD_REQUEST_COMPLETED (1)
2023.03.16 17:31:15.293 *I* [                   ] Tunnel with XXX.XXX.XXX.XXX established
2023.03.16 17:31:15.294 *D* [tunnel             ] XXX.XXX.XXX.XXX: Tunnel is active
2023.03.16 17:31:45.302 *D* [tunnel             ] XXX.XXX.XXX.XXX: Sending message CMD_KEEPALIVE (2)
2023.03.16 17:31:45.375 *D* [tunnel             ] XXX.XXX.XXX.XXX: Received message CMD_KEEPALIVE (2)
2023.03.16 17:32:10.015 *D* [websvc             ] WebServiceHousekeeper(): running cache entry check

[grae]

Ok so I finally got it to connect, this morning I deleted everything, went line by line and here's what worked for me:

c:\NetXMS\Bin\openssl.exe

openssl genrsa -out rootCA.key 2048
openssl req -config openssl.conf -x509 -new -key rootCA.key -days 10000 -out rootCA.crt
openssl genrsa -out server.key 2048
openssl req -config openssl.conf -new -key server.key -out server.csr
openssl x509 -req -in server.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out server.crt -days 5000 -extfile openssl.conf -extensions v3_ca

I copied those certificates to c:\NetXMS\cert (I'm sure you could leave them in \bin if you wanted)

Then in my server configuration
c:\netxms\etc\netxmsd.conf

# For agent tunnels
TrustedCertificate = C:\NetXMS\cert\rootCA.crt
TrustedCertificate = C:\NetXMS\cert\server.crt
ServerCertificate = C:\NetXMS\cert\server.crt
ServerCertificateKey = C:\NetXMS\cert\server.key

The trick was to add BOTH the rootCA.crt and the server.crt as TrustedCertificate
After rebooting the server it worked!

Some things that helped was running the server executable and setting debug 4, this helped me see the connection information.

Same thing on the client side, run the c:\netxms\bin\nxagentd.exe -D6
-D4 also worked, but client side D6 gave a little more, in addition setting "nxagentd.exe -S" stopped the service so it wouldn't auto restart and have to be killed.  Made it easier to diagnose.

Anyway hope this helps someone in the future, drove me crazy for hours!

[graeChris]

Resolved