The table 'alarm_event' is full in NetXMS 4.5.3-1 version

[jbriseno]

Hi, there. five days ago, we did no able to log into the NetXMS server. The NetXMS client sent back the following message: Connection refused. I decided to look up for in the log of the system. I found the following message:

2024.02.14 16:49:51.981 *E* [db.drv            ] SQL query failed (Query = "INSERT INTO alarm_events (alarm_id,event_id,event_code,event_name,severity,source_object_id,event_timestamp,message) VALUES (?,?,?,?,?,?,?,?)"): The table 'alarm_events' is full

Any idea how to fix this database issue? Thanks in advance.

[victor]

Hi,

it looks to be disk space issue. You probably can truncate biggest tables (possible candidates are event_log and top idata_ tables). 
If you are ok with losing all alarms and event history, you can truncate tables event_log, alarms, alarm_events, and alarm_notes.

What database you are using?

Best regards,
Victor

[jbriseno]

Hi, Victor. First, I appreciate your time to reply my message. Second, even better you reply it as soon as possible. Third, we are using: mysql  Ver 15.1 Distrib 10.3.39-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

Best regards !
Jose

[jbriseno]

Before to do it. Do I have to think about the primary key on these tables? because, I had have problems with mariaDB tables when I use the Mysql TRuncate table command. Thanks in advance

[victor]

Seems to be MariaDB specific issue. I'm not an expert on it unfortunately, but seems that they have all sort of strange limits on tables. One thing you may want to look at is big_tables configuration option. Maybe you should turn it on. Or consider migration to PostgreSQL
How many records do you have in this table? And how many active alarms? It is a bit unexpected that event_log for example works fine, but alarm_events is full. This table should contain event records for active alarms only.

Best regards,
Victor

P.S. Primary key on that table is alarm_id,event_id - those are two integers, so should not be an issue as such.

[jbriseno]

Awesome ! Thanks ! I will do it.

Best regards

Jose

[jbriseno]

Victor, for your information: 

event_log table

105941216 rows

alarm_events table

55125439  rows

MariaDB [netxms]> describe alarm_events;
+------------------+---------------+------+-----+---------+-------+
| Field            | Type          | Null | Key | Default | Extra |
+------------------+---------------+------+-----+---------+-------+
| alarm_id        | int(11)      | NO  | PRI | NULL    |      |
| event_id        | bigint(20)    | NO  | PRI | NULL    |      |
| event_code      | int(11)      | NO  |    | NULL    |      |
| event_name      | varchar(63)  | YES  |    | NULL    |      |
| severity        | int(11)      | NO  |    | NULL    |      |
| source_object_id | int(11)      | NO  |    | NULL    |      |
| event_timestamp  | int(11)      | NO  |    | NULL    |      |
| message          | varchar(2000) | YES  |    | NULL    |      |
+------------------+---------------+------+-----+---------+-------+
8 rows in set (0.027 sec)

[jbriseno]

Victor. I have other question:

Do I need to stop the NetXMS server before to truncate both tables?

[victor]

Yes, it's better to stop it.

Best regards,
Victor

[bdefloo]

Hi,

Hopping over from https://www.netxms.org/forum/general-support/netxmsd-hangs-during-startup/ to say thanks for the pointer towards alarm_events which solved my issue!

I also had a huge number of rows in this table, and found the culprit in a misconfigured event processing policy which was catching too many events. You may also have something similar.

In my case I discovered it by looking at alarms with very high repeat counts. If you open the alarm details it will show the EPP rule which created it.