Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor
  • Default (Flatly)
  • No Skin
Collapse
Brand Logo

NetXMS Forum
  1. Home
  2. Support (Russian)
  3. Database writer's request queue более 1000

Database writer's request queue более 1000

Scheduled Pinned Locked Moved Support (Russian)
5 Posts 2 Posters 13.0k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • N
    N
    NAI
    wrote on last edited by
    #1

    Вопрос может быть не совсем по теме форума, но столкнулся с такой штукой:

    Естественно, netxms в рабочее время становится чуть более чем бесполезным.

    Конфигурация железа выглядит так:
    Windows 2008 с Hyper-V на ней виртуалка с debian:

    Linux version 3.2.0-4-686-pae ([email protected]) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.60-1+deb7u1

    Под Debian:

    >service netxmsd start
>iostat -xk -t 10

    Получаю вот такую замечательную картинку:

    25.07.2014 19:50:34
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0,09    0,00    0,47   35,37    0,00   64,08

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
scd0              0,00     0,00    0,00    0,00     0,00     0,00     8,00     0,00    0,73    0,73    0,00   0,73   0,00
sda               0,00    23,41    0,09   29,57     1,10   216,76    14,69     0,41   13,93   66,31   13,77  12,04  35,72

25.07.2014 19:50:44
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0,10    0,00    1,10   97,60    0,00    1,20

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
scd0              0,00     0,00    0,00    0,00     0,00     0,00     0,00     0,00    0,00    0,00    0,00   0,00   0,00
sda               0,00    53,81    0,00   73,25     0,00   519,44    14,18     1,10   15,23    0,00   15,23  13,46  98,56

25.07.2014 19:50:54
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0,00    0,00    0,40   99,60    0,00    0,00

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
scd0              0,00     0,00    0,00    0,00     0,00     0,00     0,00     0,00    0,00    0,00    0,00   0,00   0,00
sda               0,00    36,98    0,00   40,70     0,00   310,75    15,27     1,04   25,40    0,00   25,40  24,68 100,46

    окей, делаем:

    >service netxmsd start
>iostat -xk -t 10

    Получаю:

    25.07.2014 19:57:03
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0,00    0,00    0,20    0,00    0,00   99,80

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
scd0              0,00     0,00    0,00    0,00     0,00     0,00     0,00     0,00    0,00    0,00    0,00   0,00   0,00
sda               0,00     0,00    0,00    0,00     0,00     0,00     0,00     0,00    0,00    0,00    0,00   0,00   0,00

25.07.2014 19:57:13
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0,00    0,00    0,00    0,00    0,00  100,00

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
scd0              0,00     0,00    0,00    0,00     0,00     0,00     0,00     0,00    0,00    0,00    0,00   0,00   0,00
sda               0,00     0,00    0,00    0,00     0,00     0,00     0,00     0,00    0,00    0,00    0,00   0,00   0,00

25.07.2014 19:57:23
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0,00    0,00    0,10    0,00    0,00   99,90

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
scd0              0,00     0,00    0,00    0,00     0,00     0,00     0,00     0,00    0,00    0,00    0,00   0,00   0,00
sda               0,00     0,00    0,00    0,00     0,00     0,00     0,00     0,00    0,00    0,00    0,00   0,00   0,00

    Куда копать, чего гуглить? С линуксом на Вы.
    P.s. Версия netxms 1.2.14, DBDriver = sqlite.ddr

    1 Reply Last reply
    0
    • A
      A
      Alex Kirhenshtein
      wrote on last edited by
      #2

      SQLite предназначен только для совсем маленьких, тестовых систем - надо для начала сделать миграцию на нормальную базу, я советую PostgreSQL.

      В английской ветке форума я писал про процесс миграции: https://www.netxms.org/forum/general-support/database-locked-error/msg14453/#msg14453

      1 Reply Last reply
      0
      • N
        N
        NAI
        wrote on last edited by
        #3

        Попробую мигрировать.
        Хотя все равно странно - мониторится 12 нод, по 8-10 параметров (раз в минуту).

        1 Reply Last reply
        0
        • N
          N
          NAI
          wrote on last edited by
          #4

          Alex Kirhenshtein спасибо за ваши советы, поддержку.

          С переездом возникли трудности:

          1. Create a copy of existing netxmsd.conf, I'll name it "netxmsd_old.conf" in this example
          2. Create new database
          3. Init with initial schema:
            Code: [Select]
            nxdbmgr init /opt/netxms/share/netxms/sql/dbinit_pgsql.sql
            Please note, that there are different files for different database types, in this example - "dbinit_pgsql.sql" is for PostgreSQL.
          4. Adjust "netxmsd.conf" - change DBDriver, DBName, etc. to match new database
          5. (optional) run "nxdbmgr check" to verify connectivity to new database
          6. run "nxdbmgr migrate netxmsd_old.conf". This will load data from old database (using credentials in netxmsd_old.conf) and migrate it into new one (as configured in netxmsd.conf)

          п. 1 - 6 выполняются без видимых проблем.
          п3:

          postgres@netxms:~$ nxdbmgr init /usr/share/netxms/sql/dbinit_pgsql.sql
NetXMS Database Manager Version 1.2.14 Build 7873

Initializing database...
ЗАМЕЧАНИЕ:  CREATE TABLE / PRIMARY KEY создаст неявный индекс "metadata_pkey" для таблицы "metadata"
ЗАМЕЧАНИЕ:  CREATE TABLE / PRIMARY KEY создаст неявный индекс "config_pkey" для таблицы "config"
ЗАМЕЧАНИЕ:  CREATE TABLE / PRIMARY KEY создаст неявный индекс "config_clob_pkey" для таблицы "config_clob"
ЗАМЕЧАНИЕ:  CREATE TABLE / PRIMARY KEY создаст неявный индекс "users_pkey" для таблицы "users"
.....
ЗАМЕЧАНИЕ:  CREATE TABLE / PRIMARY KEY создаст неявный индекс "mapping_tables_pkey" для таблицы "mapping_tables"
ЗАМЕЧАНИЕ:  CREATE TABLE / PRIMARY KEY создаст неявный индекс "mapping_data_pkey" для таблицы "mapping_data"
ЗАМЕЧАНИЕ:  CREATE TABLE / PRIMARY KEY создаст неявный индекс "dci_summary_tables_pkey" для таблицы "dci_summary_tables"
Database initialized successfully

          п5:

          root@netxms:~# nxdbmgr check
NetXMS Database Manager Version 1.2.14 Build 7873

Checking database (excluding collected data):
* Checking zone objects...                                            [PASSED]
* Checking node objects...                                            [PASSED]
* Checking interface objects...                                       [PASSED]
* Checking network service objects...                                 [PASSED]
* Checking cluster objects...                                         [PASSED]
* Checking template to node mapping...                                [PASSED]
* Checking object properties...                                       [PASSED]
* Checking event processing policy...                                 [PASSED]
* Checking network map links...                                       [PASSED]
Database doesn't contain any errors
Database check completed

          п.7:

          root@netxms:~# nxdbmgr migrate /etc/netxmsd.conf_old
NetXMS Database Manager Version 1.2.14 Build 7873

Database driver sqlite.ddr loaded
Connected to source database


WARNING!!!
This operation will clear all configuration and collected data from database.
Are you sure? (Y/N) Y
Database successfully cleared
Migrating table config
ERROR: cannot prepare INSERT statement (??????:  ?????? ?????????? (????????? ?????????: ")")
LINE 1: INSERT INTO config ) VALUES ($1)
                           ^)
Database migration failed.

          Ладно, черт с ним со старыми данными, пытаюсь просто запустить netxms -D 9:

           netxmsd -D 9
Using configuration file "/etc/netxmsd.conf"
[26-Jul-2014 17:50:56.216] [DEBUG] Code page set to ASCII
[26-Jul-2014 17:50:56.216] [INFO ] Platform subagent "/usr/lib/netxms/linux.nsm" successfully loaded
[26-Jul-2014 17:50:56.216] [DEBUG] Debug callback set for DB library
[26-Jul-2014 17:50:56.223] [INFO ] Database driver "pgsql.ddr" loaded and initialized successfully
[26-Jul-2014 17:50:56.223] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=postgres schema=
[26-Jul-2014 17:50:56.263] [DEBUG] New DB connection opened: handle=0x9174908
[26-Jul-2014 17:50:56.263] [DEBUG] Successfully connected to database [email protected]
[26-Jul-2014 17:50:56.265] [DEBUG] Successful sync query: "SELECT var_value FROM metadata WHERE var_name='SchemaVersion'" [2 ms]
[26-Jul-2014 17:50:56.266] [DEBUG] {0x9174408} Successful prepare: "SELECT var_value FROM config WHERE var_name=?" [1 ms]
[26-Jul-2014 17:50:56.266] [DEBUG] {0x9174408} bind at pos 1: "ConnectionPoolBaseSize"
[26-Jul-2014 17:50:56.267] [DEBUG] Successful prepared sync query: "SELECT var_value FROM config WHERE var_name=?" [1 ms]
[26-Jul-2014 17:50:56.267] [DEBUG] {0x9174408} Successful prepare: "SELECT var_value FROM config WHERE var_name=?" [0 ms]
[26-Jul-2014 17:50:56.267] [DEBUG] {0x9174408} bind at pos 1: "ConnectionPoolMaxSize"
[26-Jul-2014 17:50:56.268] [DEBUG] Successful prepared sync query: "SELECT var_value FROM config WHERE var_name=?" [1 ms]
[26-Jul-2014 17:50:56.268] [DEBUG] {0x9174408} Successful prepare: "SELECT var_value FROM config WHERE var_name=?" [0 ms]
[26-Jul-2014 17:50:56.268] [DEBUG] {0x9174408} bind at pos 1: "ConnectionPoolCooldownTime"
[26-Jul-2014 17:50:56.269] [DEBUG] Successful prepared sync query: "SELECT var_value FROM config WHERE var_name=?" [1 ms]
[26-Jul-2014 17:50:56.269] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=postgres schema=
[26-Jul-2014 17:50:56.301] [DEBUG] New DB connection opened: handle=0x921adc8
[26-Jul-2014 17:50:56.301] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=postgres schema=
[26-Jul-2014 17:50:56.333] [DEBUG] New DB connection opened: handle=0x92c1258
[26-Jul-2014 17:50:56.333] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=postgres schema=
[26-Jul-2014 17:50:56.362] [DEBUG] New DB connection opened: handle=0x9367618
[26-Jul-2014 17:50:56.362] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=postgres schema=
[26-Jul-2014 17:50:56.390] [DEBUG] New DB connection opened: handle=0x940da48
[26-Jul-2014 17:50:56.390] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=postgres schema=
[26-Jul-2014 17:50:56.419] [DEBUG] New DB connection opened: handle=0x94b3e60
[26-Jul-2014 17:50:56.419] [DEBUG] Database Connection Pool initialized
[26-Jul-2014 17:50:56.420] [DEBUG] Successful sync query: "SELECT var_value FROM metadata WHERE var_name='Syntax'" [1 ms]
[26-Jul-2014 17:50:56.422] [DEBUG] {0x94b3960} Successful prepare: "SELECT var_value FROM config WHERE var_name=?" [2 ms]
[26-Jul-2014 17:50:56.422] [DEBUG] {0x94b3960} bind at pos 1: "ServerID"
[26-Jul-2014 17:50:56.422] [DEBUG] Successful prepared sync query: "SELECT var_value FROM config WHERE var_name=?" [0 ms]
[26-Jul-2014 17:50:56.423] [DEBUG] {0x94b3960} Successful prepare: "SELECT var_value FROM config WHERE var_name=?" [0 ms]
[26-Jul-2014 17:50:56.423] [DEBUG] {0x94b3960} bind at pos 1: "ServerID"
[26-Jul-2014 17:50:56.423] [DEBUG] Successful prepared sync query: "SELECT var_value FROM config WHERE var_name=?" [0 ms]
[26-Jul-2014 17:50:56.424] [DEBUG] {0x94b3960} Successful prepare: "INSERT INTO config (var_name,var_value,is_visible,need_server_restart) VALUES (?,?,?,?)" [0 ms]
[26-Jul-2014 17:50:56.424] [DEBUG] {0x94b3960} bind at pos 1: "ServerID"
[26-Jul-2014 17:50:56.424] [DEBUG] {0x94b3960} bind at pos 2: "D7CC4F0A40B2D353"
[26-Jul-2014 17:50:56.424] [DEBUG] {0x94b3960} bind at pos 3: "1"
[26-Jul-2014 17:50:56.424] [DEBUG] {0x94b3960} bind at pos 4: "0"
[26-Jul-2014 17:50:56.425] [DEBUG] Database Connection Pool maintenance thread started
[26-Jul-2014 17:50:56.449] [DEBUG] Successful prepared sync query: "INSERT INTO config (var_name,var_value,is_visible,need_server_restart) VALUES (?,?,?,?)" [24 ms]
[26-Jul-2014 17:50:56.450] [DEBUG] {0x94b3960} Successful prepare: "SELECT var_value FROM config WHERE var_name=?" [1 ms]
[26-Jul-2014 17:50:56.450] [DEBUG] {0x94b3960} bind at pos 1: "DBLockStatus"
[26-Jul-2014 17:50:56.450] [DEBUG] Successful prepared sync query: "SELECT var_value FROM config WHERE var_name=?" [0 ms]
[26-Jul-2014 17:50:56.450] [DEBUG] DBLockStatus="ERROR"
[26-Jul-2014 17:50:56.451] [ERROR] Error initializing component locks table

          Листинг /etc/netxmsd.conf:

          DBDriver = pgsql.ddr
Server = 127.0.0.1
DBLogin = postgres
DBName = netxms_db
LogFile = /var/log/netxmsd
          1 Reply Last reply
          0
          • N
            N
            NAI
            wrote on last edited by
            #5

            В продолжении темы, вопрос с загрузкой HDD под sqlite наверно снимается.

            iotop -o -a показывает

              TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
  227 be/3 root          0.00 B    312.00 K  0.00 % 83.68 % [jbd2/sda1-8]
 2402 be/4 root          0.00 B      2.57 M  0.00 %  1.85 % netxmsd -d -D 5
 2403 be/4 root          0.00 B   1500.00 K  0.00 %  0.98 % netxmsd -d -D 5

            Отключение netxms уменьшало нагрузку до 10-15%

            Помогло добавление commit=30 в fstab.

            Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
  227 be/3 root          0.00 B     96.00 K  0.00 % 28.00 % [jbd2/sda1-8]
 2402 be/4 root          0.00 B      2.48 M  0.00 %  0.89 % netxmsd -d -D 5
 2403 be/4 root          0.00 B   1256.00 K  0.00 %  0.22 % netxmsd -d -D 5
 2344 be/4 root          0.00 B      8.00 K  0.00 %  0.00 % netxmsd -d -D 5

            Хотя понятнее не становится, вроде netxms не так уж много логов пишет =\

            1 Reply Last reply
            0

            Hello! It looks like you're interested in this conversation, but you don't have an account yet.

            Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

            With your input, this post could be even better 💗

            Register Login
            Reply
            • Reply as topic
            Log in to reply
            • Oldest to Newest
            • Newest to Oldest
            • Most Votes

            • Login
            • Don't have an account? Register
            • Login or register to search.
            Powered by NodeBB Contributors
            • First post
              Last post
            0
            • Categories
            • Recent
            • Tags
            • Popular
            • Users
            • Groups