lock manager crash 4.05

[gerdus]

Error in log file:

Fatal lock manager error: Process disappeared in LockManager::acquire_shmem, errno: 22 --Invalid argument

Version: V4.0.5.3140 (from website)
OS: Linux
Mode: Super

Link to coredump

Crash Dump Info:

           PID: 2338780 (firebird)
           UID: 109 (firebird)
           GID: 116 (firebird)
        Signal: 6 (ABRT)
     Timestamp: Wed 2024-10-09 15:29:35 SAST (37min ago)
  Command Line: /opt/firebird/bin/firebird
    Executable: /opt/firebird/bin/firebird
 Control Group: /system.slice/firebird.service
          Unit: firebird.service
         Slice: system.slice
       Boot ID: 5d0055bb335e47b185c48c6b6b5936d1
    Machine ID: 5f578cbbe9b3413fb66084d017726484
      Hostname: sis002-truservplus-jhb1-001
       Storage: /var/lib/systemd/coredump/core.firebird.109.5d0055bb335e47b185c48c6b6b5936d1.2338780.1728480575000000.zst (present)
     Disk Size: 118.4M
       Message: Process 2338780 (firebird) of user 109 dumped core.
                
                Found module linux-vdso.so.1 with build-id: bcf11dcd6dd2f76c832fcd017faeceae4b3c9ffc
                Found module libChaCha.so without build-id.
                Found module libfbtrace.so without build-id.
                Found module libicui18n.so.70 with build-id: ff2dbcdd92cbe5a63d20291e295d8fed9f87d35b
                Found module libstdc++.so.6 with build-id: e37fe1a879783838de78cbc8c80621fa685d58a2
                Found module libicudata.so.70 with build-id: b1c2496dd0543023c7a19c961bb7f3abc818f465
                Found module libicuuc.so.70 with build-id: bef3ff1d70aadd68aab07d858a759360c8b919ae
                Found module fbintl without build-id.
                Found module libib_util.so without build-id.
                Found module libEngine13.so without build-id.
                Found module ISO8859-1.so with build-id: d425664f0849dcfeabefea8a9e1e77c45b39182c
                Found module ld-linux-x86-64.so.2 with build-id: 4186944c50f8a32b47d74931e3f512b811813b64
                Found module libc.so.6 with build-id: 490fef8403240c91833978d494d39e537409b92e
                Found module libpthread.so.0 with build-id: 81f46d553e2f7c999e43c3eede73a822bc8d5d93
                Found module libgcc_s.so.1 with build-id: e3a44e0da9c6e835d293ed8fd2882b4c4a87130c
                Found module libm.so.6 with build-id: a508ec5d8bf12fb7fd08204e0f87518e5cd0b102
                Found module libdl.so.2 with build-id: 6f6fe1a24b7b981e11c9a3373b806d3496d4d9d4
                Found module libtomcrypt.so.1 without build-id.
                Found module libtommath.so.0 with build-id: 309215294227ee7d2ff3d67dd726a5c47c9324b9
                Found module libfbclient.so.2 without build-id.
                Found module firebird without build-id.
                Stack trace of thread 3771838:
                #0  0x00007f222bc6d9fc __pthread_kill_implementation (libc.so.6 + 0x969fc)
                #1  0x00007f222bc19476 __GI_raise (libc.so.6 + 0x42476)
                #2  0x00007f222bbff7f3 __GI_abort (libc.so.6 + 0x287f3)
                #3  0x00007f2229f9a8e5 _ZN8fb_utils9logAndDieEPKc (libEngine13.so + 0x59a8e5)
                #4  0x00007f2229ebd148 _ZN3Jrd11LockManager3bugEPN8Firebird18CheckStatusWrapperEPKc (libEngine13.so + 0x4bd148)
                #5  0x00007f2229ec0378 _ZN3Jrd11LockManager13acquire_shmemEi (libEngine13.so + 0x4c0378)
                #6  0x00007f2229ec40a2 _ZN3Jrd11LockManager14LockTableGuardC4EPS0_PKci (libEngine13.so + 0x4c40a2)
                #7  0x00007f2229c3823c enqueue (libEngine13.so + 0x23823c)
                #8  0x00007f2229ad44bf _ZN3Jrd12GlobalRWLock8lockReadEPNS_9thread_dbEsb (libEngine13.so + 0xd44bf)
                #9  0x00007f2229a6e7ba _ZN3Jrd13BackupManager13lockStateReadEPNS_9thread_dbEs (libEngine13.so + 0x6e7ba)
                #10 0x00007f2229c533f4 _ZN3Jrd13BackupManager14StateReadGuard4lockEPNS_9thread_dbEs (libEngine13.so + 0x2533f4)
                #11 0x00007f2229c22c25 _ZN3Jrd9JProvider14internalAttachEPN8Firebird18CheckStatusWrapperEPKcjPKhPKNS_6UserIdE (libEngine13.so + 0x222c25)
                #12 0x00007f2229a9952a _ZN8Firebird17IProviderBaseImplIN3Jrd9JProviderENS_18CheckStatusWrapperENS_15IPluginBaseImplIS2_S3_NS_7InheritINS_21IReferenceCountedImplIS2_S3_NS5_INS_14IVersionedImplIS2_S3_NS5_INS_9IProviderEEEEEEEEEEEEEE29cloopattachDatabaseDispatcherEPS8_PNS_7IStatusEPKcjPKh (libEngine13.so + 0x9952a)
                #13 0x00007f222c2a6e20 _ZN8Firebird9IProvider14attachDatabaseINS_18CheckStatusWrapperEEEPNS_11IAttachmentEPT_PKcjPKh (libfbclient.so.2 + 0xa6e20)
                #14 0x00007f222c253621 _ZN8Firebird17IProviderBaseImplIN3Why10DispatcherENS_18CheckStatusWrapperENS_15IPluginBaseImplIS2_S3_NS_7InheritINS_21IReferenceCountedImplIS2_S3_NS5_INS_14IVersionedImplIS2_S3_NS5_INS_9IProviderEEEEEEEEEEEEEE29cloopattachDatabaseDispatcherEPS8_PNS_7IStatusEPKcjPKh (libfbclient.so.2 + 0x53621)
                #15 0x0000000000444256 _ZN8Firebird9IProvider14attachDatabaseINS_18CheckStatusWrapperEEEPNS_11IAttachmentEPT_PKcjPKh (firebird + 0x44256)
                #16 0x0000000000437f7a authenticate (firebird + 0x37f7a)
                #17 0x00000000004474c0 attach_database (firebird + 0x474c0)
                #18 0x00000000004497ab loopThread (firebird + 0x497ab)
                #19 0x0000000000466920 run (firebird + 0x66920)
                #20 0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #21 0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 978402:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x00007f2229fb20f0 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (libEngine13.so + 0x5b20f0)
                #3  0x00007f2229ccc1ca _ZN3Jrd8Database17garbage_collectorEPS0_ (libEngine13.so + 0x2cc1ca)
                #4  0x00007f2229ccd9c3 _ZN16ThreadFinishSyncIPN3Jrd8DatabaseELPFvS2_E0EE11internalRunEv (libEngine13.so + 0x2cd9c3)
                #5  0x00007f2229f58f40 run (libEngine13.so + 0x558f40)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 2338782:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x00007f222c37e650 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (libfbclient.so.2 + 0x17e650)
                #3  0x00007f222c24e1cb timeThread (libfbclient.so.2 + 0x4e1cb)
                #4  0x00007f222c32af70 run (libfbclient.so.2 + 0x12af70)
                #5  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #6  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771828:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771832:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3766456:
                #0  0x00007f222bc682c0 futex_wait (libc.so.6 + 0x912c0)
                #1  0x00007f222bc6f05d lll_mutex_lock_optimized (libc.so.6 + 0x9805d)
                #2  0x00007f2229c04589 _ZN8Firebird5Mutex5enterEPKc (libEngine13.so + 0x204589)
                #3  0x00007f2229c2285f _ZN8Firebird12EnsureUnlockINS_5MutexENS_13NotRefCountedEE5enterEv (libEngine13.so + 0x22285f)
                #4  0x00007f2229a9952a _ZN8Firebird17IProviderBaseImplIN3Jrd9JProviderENS_18CheckStatusWrapperENS_15IPluginBaseImplIS2_S3_NS_7InheritINS_21IReferenceCountedImplIS2_S3_NS5_INS_14IVersionedImplIS2_S3_NS5_INS_9IProviderEEEEEEEEEEEEEE29cloopattachDatabaseDispatcherEPS8_PNS_7IStatusEPKcjPKh (libEngine13.so + 0x9952a)
                #5  0x00007f222c2a6e20 _ZN8Firebird9IProvider14attachDatabaseINS_18CheckStatusWrapperEEEPNS_11IAttachmentEPT_PKcjPKh (libfbclient.so.2 + 0xa6e20)
                #6  0x00007f222c253621 _ZN8Firebird17IProviderBaseImplIN3Why10DispatcherENS_18CheckStatusWrapperENS_15IPluginBaseImplIS2_S3_NS_7InheritINS_21IReferenceCountedImplIS2_S3_NS5_INS_14IVersionedImplIS2_S3_NS5_INS_9IProviderEEEEEEEEEEEEEE29cloopattachDatabaseDispatcherEPS8_PNS_7IStatusEPKcjPKh (libfbclient.so.2 + 0x53621)
                #7  0x0000000000444256 _ZN8Firebird9IProvider14attachDatabaseINS_18CheckStatusWrapperEEEPNS_11IAttachmentEPT_PKcjPKh (firebird + 0x44256)
                #8  0x0000000000437f7a authenticate (firebird + 0x37f7a)
                #9  0x00000000004474c0 attach_database (firebird + 0x474c0)
                #10 0x00000000004497ab loopThread (firebird + 0x497ab)
                #11 0x0000000000466920 run (firebird + 0x66920)
                #12 0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #13 0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771833:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771769:
                #0  0x00007f222bc682c0 futex_wait (libc.so.6 + 0x912c0)
                #1  0x00007f222bc6f05d lll_mutex_lock_optimized (libc.so.6 + 0x9805d)
                #2  0x00007f2229c04589 _ZN8Firebird5Mutex5enterEPKc (libEngine13.so + 0x204589)
                #3  0x00007f2229c2285f _ZN8Firebird12EnsureUnlockINS_5MutexENS_13NotRefCountedEE5enterEv (libEngine13.so + 0x22285f)
                #4  0x00007f2229a9952a _ZN8Firebird17IProviderBaseImplIN3Jrd9JProviderENS_18CheckStatusWrapperENS_15IPluginBaseImplIS2_S3_NS_7InheritINS_21IReferenceCountedImplIS2_S3_NS5_INS_14IVersionedImplIS2_S3_NS5_INS_9IProviderEEEEEEEEEEEEEE29cloopattachDatabaseDispatcherEPS8_PNS_7IStatusEPKcjPKh (libEngine13.so + 0x9952a)
                #5  0x00007f222c2a6e20 _ZN8Firebird9IProvider14attachDatabaseINS_18CheckStatusWrapperEEEPNS_11IAttachmentEPT_PKcjPKh (libfbclient.so.2 + 0xa6e20)
                #6  0x00007f222c253621 _ZN8Firebird17IProviderBaseImplIN3Why10DispatcherENS_18CheckStatusWrapperENS_15IPluginBaseImplIS2_S3_NS_7InheritINS_21IReferenceCountedImplIS2_S3_NS5_INS_14IVersionedImplIS2_S3_NS5_INS_9IProviderEEEEEEEEEEEEEE29cloopattachDatabaseDispatcherEPS8_PNS_7IStatusEPKcjPKh (libfbclient.so.2 + 0x53621)
                #7  0x0000000000444256 _ZN8Firebird9IProvider14attachDatabaseINS_18CheckStatusWrapperEEEPNS_11IAttachmentEPT_PKcjPKh (firebird + 0x44256)
                #8  0x0000000000437f7a authenticate (firebird + 0x37f7a)
                #9  0x00000000004474c0 attach_database (firebird + 0x474c0)
                #10 0x00000000004497ab loopThread (firebird + 0x497ab)
                #11 0x0000000000466920 run (firebird + 0x66920)
                #12 0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #13 0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771834:
                #0  0x00007f222bc682c0 futex_wait (libc.so.6 + 0x912c0)
                #1  0x00007f222bc6f05d lll_mutex_lock_optimized (libc.so.6 + 0x9805d)
                #2  0x00007f2229c04589 _ZN8Firebird5Mutex5enterEPKc (libEngine13.so + 0x204589)
                #3  0x00007f2229c2285f _ZN8Firebird12EnsureUnlockINS_5MutexENS_13NotRefCountedEE5enterEv (libEngine13.so + 0x22285f)
                #4  0x00007f2229a9952a _ZN8Firebird17IProviderBaseImplIN3Jrd9JProviderENS_18CheckStatusWrapperENS_15IPluginBaseImplIS2_S3_NS_7InheritINS_21IReferenceCountedImplIS2_S3_NS5_INS_14IVersionedImplIS2_S3_NS5_INS_9IProviderEEEEEEEEEEEEEE29cloopattachDatabaseDispatcherEPS8_PNS_7IStatusEPKcjPKh (libEngine13.so + 0x9952a)
                #5  0x00007f222c2a6e20 _ZN8Firebird9IProvider14attachDatabaseINS_18CheckStatusWrapperEEEPNS_11IAttachmentEPT_PKcjPKh (libfbclient.so.2 + 0xa6e20)
                #6  0x00007f222c253621 _ZN8Firebird17IProviderBaseImplIN3Why10DispatcherENS_18CheckStatusWrapperENS_15IPluginBaseImplIS2_S3_NS_7InheritINS_21IReferenceCountedImplIS2_S3_NS5_INS_14IVersionedImplIS2_S3_NS5_INS_9IProviderEEEEEEEEEEEEEE29cloopattachDatabaseDispatcherEPS8_PNS_7IStatusEPKcjPKh (libfbclient.so.2 + 0x53621)
                #7  0x0000000000444256 _ZN8Firebird9IProvider14attachDatabaseINS_18CheckStatusWrapperEEEPNS_11IAttachmentEPT_PKcjPKh (firebird + 0x44256)
                #8  0x0000000000437f7a authenticate (firebird + 0x37f7a)
                #9  0x00000000004474c0 attach_database (firebird + 0x474c0)
                #10 0x00000000004497ab loopThread (firebird + 0x497ab)
                #11 0x0000000000466920 run (firebird + 0x66920)
                #12 0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #13 0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3773070:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771839:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3773367:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x00007f2229fb20f0 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (libEngine13.so + 0x5b20f0)
                #3  0x00007f2229b9442f _ZN3Jrd13BufferControl12cache_writerEPS0_ (libEngine13.so + 0x19442f)
                #4  0x00007f2229b94fc3 _ZN16ThreadFinishSyncIPN3Jrd13BufferControlELPFvS2_E0EE11internalRunEv (libEngine13.so + 0x194fc3)
                #5  0x00007f2229f58f40 run (libEngine13.so + 0x558f40)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3773368:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x00007f2229fb20f0 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (libEngine13.so + 0x5b20f0)
                #3  0x00007f2229ccc1ca _ZN3Jrd8Database17garbage_collectorEPS0_ (libEngine13.so + 0x2cc1ca)
                #4  0x00007f2229ccd9c3 _ZN16ThreadFinishSyncIPN3Jrd8DatabaseELPFvS2_E0EE11internalRunEv (libEngine13.so + 0x2cd9c3)
                #5  0x00007f2229f58f40 run (libEngine13.so + 0x558f40)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3352866:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x00007f2229fb20f0 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (libEngine13.so + 0x5b20f0)
                #3  0x00007f2229ccc1ca _ZN3Jrd8Database17garbage_collectorEPS0_ (libEngine13.so + 0x2cc1ca)
                #4  0x00007f2229ccd9c3 _ZN16ThreadFinishSyncIPN3Jrd8DatabaseELPFvS2_E0EE11internalRunEv (libEngine13.so + 0x2cd9c3)
                #5  0x00007f2229f58f40 run (libEngine13.so + 0x558f40)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3773065:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3768950:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3773069:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771560:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 978401:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x00007f2229fb20f0 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (libEngine13.so + 0x5b20f0)
                #3  0x00007f2229b9442f _ZN3Jrd13BufferControl12cache_writerEPS0_ (libEngine13.so + 0x19442f)
                #4  0x00007f2229b94fc3 _ZN16ThreadFinishSyncIPN3Jrd13BufferControlELPFvS2_E0EE11internalRunEv (libEngine13.so + 0x194fc3)
                #5  0x00007f2229f58f40 run (libEngine13.so + 0x558f40)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 2338780:
                #0  0x00007f222bcefbcf __GI___poll (libc.so.6 + 0x118bcf)
                #1  0x000000000040df92 _ZN6Select6selectEP7timeval (firebird + 0xdf92)
                #2  0x00000000004395bb _Z17SRVR_multi_threadP8rem_portt (firebird + 0x395bb)
                #3  0x0000000000407ddf main (firebird + 0x7ddf)
                #4  0x00007f222bc00d90 __libc_start_call_main (libc.so.6 + 0x29d90)
                #5  0x00007f222bc00e40 __libc_start_main_impl (libc.so.6 + 0x29e40)
                #6  0x000000000040930f _start (firebird + 0x930f)
                
                Stack trace of thread 3771797:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771767:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771796:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3773522:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc6aa41 __pthread_cond_wait_common (libc.so.6 + 0x93a41)
                #2  0x00007f2229f7a72b _ZN8Firebird16SharedMemoryBase9eventWaitEPNS_7event_tEii (libEngine13.so + 0x57a72b)
                #3  0x00007f2229bc91f4 _ZN3Jrd12EventManager14watcher_threadEv (libEngine13.so + 0x1c91f4)
                #4  0x00007f2229bca053 _ZN16ThreadFinishSyncIPN3Jrd12EventManagerELPFvS2_E0EE11internalRunEv (libEngine13.so + 0x1ca053)
                #5  0x00007f2229f58f40 run (libEngine13.so + 0x558f40)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3771831:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73a53 __new_sem_wait_slow64 (libc.so.6 + 0x9ca53)
                #2  0x0000000000490850 _ZN8Firebird19SignalSafeSemaphore8tryEnterEii (firebird + 0x90850)
                #3  0x00000000004310de _ZN6Worker4waitEi (firebird + 0x310de)
                #4  0x000000000044a249 loopThread (firebird + 0x4a249)
                #5  0x0000000000466920 run (firebird + 0x66920)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 3768958:
                #0  0x00007f222bcf5a7b __munmap (libc.so.6 + 0x11ea7b)
                #1  0x00007f2229fae6e9 _ZN8Firebird7MemPool10releaseRawEbPvmb (libEngine13.so + 0x5ae6e9)
                #2  0x00007f2229faef45 _ZN8Firebird16DoubleLinkedList9decrUsageEPNS_13MemMediumHunkEPNS_7MemPoolE (libEngine13.so + 0x5aef45)
                #3  0x00007f2229faed35 _ZN8Firebird11FreeObjectsINS_16DoubleLinkedListENS_12MediumLimitsEE15deallocateBlockEPNS_8MemBlockE (libEngine13.so + 0x5aed35)
                #4  0x00007f2229faf112 _ZN8Firebird11FreeObjectsINS_10LinkedListENS_9LowLimitsEED4Ev (libEngine13.so + 0x5af112)
                #5  0x00007f2229faf659 _ZN8Firebird7MemPoolD4Ev (libEngine13.so + 0x5af659)
                #6  0x00007f2229faf6dc _ZN8Firebird10MemoryPool10deletePoolEPS0_ (libEngine13.so + 0x5af6dc)
                #7  0x00007f2229a9b7e8 _ZN3Jrd8Database10deletePoolEPN8Firebird10MemoryPoolE (libEngine13.so + 0x9b7e8)
                #8  0x00007f2229b92ef3 _ZN3Jrd13BufferControl7destroyEPS0_ (libEngine13.so + 0x192ef3)
                #9  0x00007f2229b930e3 _Z8CCH_finiPN3Jrd9thread_dbE (libEngine13.so + 0x1930e3)
                #10 0x00007f2229c1020a _Z21JRD_shutdown_databasePN3Jrd8DatabaseEj (libEngine13.so + 0x21020a)
                #11 0x00007f2229c21970 purge_attachment (libEngine13.so + 0x221970)
                #12 0x00007f2229c22126 _ZN3Jrd11JAttachment14freeEngineDataEPN8Firebird18CheckStatusWrapperEb (libEngine13.so + 0x222126)
                #13 0x00007f2229c253b2 _ZN3Jrd11JAttachment6detachEPN8Firebird18CheckStatusWrapperE (libEngine13.so + 0x2253b2)
                #14 0x00007f2229c32a5f _ZN8Firebird19IAttachmentBaseImplIN3Jrd11JAttachmentENS_18CheckStatusWrapperENS_21IReferenceCountedImplIS2_S3_NS_7InheritINS_14IVersionedImplIS2_S3_NS5_INS_11IAttachmentEEEEEEEEEE21cloopdetachDispatcherEPS7_PNS_7IStatusE (libEngine13.so + 0x232a5f)
                #15 0x00007f222c288933 _ZN8Firebird11IAttachment6detachINS_18CheckStatusWrapperEEEvPT_ (libfbclient.so.2 + 0x88933)
                #16 0x00007f222c2b914f _ZNKSt8functionIFvvEEclEv (libfbclient.so.2 + 0xb914f)
                #17 0x00007f222c2a0591 _ZN3Why11YAttachment6detachEPN8Firebird18CheckStatusWrapperE (libfbclient.so.2 + 0xa0591)
                #18 0x00007f222c2b931f _ZN8Firebird19IAttachmentBaseImplIN3Why11YAttachmentENS_18CheckStatusWrapperENS_21IReferenceCountedImplIS2_S3_NS_7InheritINS_14IVersionedImplIS2_S3_NS5_INS_11IAttachmentEEEEEEEEEE21cloopdetachDispatcherEPS7_PNS_7IStatusE (libfbclient.so.2 + 0xb931f)
                #19 0x000000000043aac6 _ZN8Firebird11IAttachment6detachINS_18CheckStatusWrapperEEEvPT_ (firebird + 0x3aac6)
                #20 0x00000000004472c6 process_packet (firebird + 0x472c6)
                #21 0x00000000004497ab loopThread (firebird + 0x497ab)
                #22 0x0000000000466920 run (firebird + 0x66920)
                #23 0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #24 0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 2338781:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc73c78 __new_sem_wait_slow64 (libc.so.6 + 0x9cc78)
                #2  0x00007f222c37e572 _ZN8Firebird19SignalSafeSemaphore5enterEv (libfbclient.so.2 + 0x17e572)
                #3  0x00007f222c2a20f5 shutdownThread (libfbclient.so.2 + 0xa20f5)
                #4  0x00007f222c32af70 run (libfbclient.so.2 + 0x12af70)
                #5  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #6  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)
                
                Stack trace of thread 2339014:
                #0  0x00007f222bc68117 __futex_abstimed_wait_common64 (libc.so.6 + 0x91117)
                #1  0x00007f222bc6aa41 __pthread_cond_wait_common (libc.so.6 + 0x93a41)
                #2  0x00007f2229f7a72b _ZN8Firebird16SharedMemoryBase9eventWaitEPNS_7event_tEii (libEngine13.so + 0x57a72b)
                #3  0x00007f2229aede8f clearDeliveryThread (libEngine13.so + 0xede8f)
                #4  0x00007f2229ae6c63 internalRun (libEngine13.so + 0xe6c63)
                #5  0x00007f2229f58f40 run (libEngine13.so + 0x558f40)
                #6  0x00007f222bc6bac3 start_thread (libc.so.6 + 0x94ac3)
                #7  0x00007f222bcfd850 __clone3 (libc.so.6 + 0x126850)

[gerdus]

Happened again on another server, unfortunately the coredump was truncated and does not appear to have the stack trace.
Have adjusted the limits so next crash will have full coredump.
So this is just to notify that crash was not once off event.

[AlexPeshkoff]

On 11/14/24 12:21, Gerdus van Zyl wrote: Happened again on another server, unfortunately the coredump was truncated and does not appear to have the stack trace. Have adjusted the limits so next crash will have full coredump. So this is just to notify that crash was not once off event.

I see. Getting one more core dump can probably help, with single one no idea what happened.

[gerdus]

Ok new coredump, hope it helps.
available at https://infireal.com/github-issues/firebird-crash-2024-11-19/

[AlexPeshkoff]

On 11/19/24 15:15, Gerdus van Zyl wrote: Ok new coredump, hope it helps. available at https://infireal.com/github-issues/firebird-crash-2024-11-19/

I need some libraries from your box: libdl.so.2 libm.so.6 libpthread.so.0 libc.so.6 ld-linux-x86-64.so.2 libthread_db.so.1 Some of them may be symlinks - should say I also need what they point to. A.

[gerdus]

Ok I have uploaded the files: https://infireal.com/github-issues/firebird-crash-2024-11-19/

[gerdus]

New crash, new coredump.
Available at https://infireal.com/github-issues/firebird-crash-2024-11-28/
The so libraries should be the same have not done any updates since last crash.

[AlexPeshkoff]

On 11/28/24 14:02, Gerdus van Zyl wrote: New crash, new coredump. Available at https://infireal.com/github-issues/firebird-crash-2024-11-28/ The so libraries should be the same have not done any updates since last crash.

Exactly like 2 previous. Maybe unexpected question - did you reboot your host during last month? If not - please check date/time of file /tmp/firebird/fb_init. Output of ls -l /tmp/firebird/fb_init is what I need.

[gerdus]

according to uptime command it was restarted 6 days ago.
output of ls -l /tmp/firebird/fb_init

-rw-rw---- 1 firebird firebird 0 Nov 21 21:27 /tmp/firebird/fb_init

[gerdus]

new crash. I don't know if it is same error since in log it is different but is generic error.
Happened during startup of the webapp that is the user of the firebird instance, where it opens 3 databases at a time and runs some sql upgrade scripts.
Before starting web app I stopped and started Firebird using service firebird stop, service firebird start

Mon Dec  9 10:52:26 2024
        /opt/firebird/bin/fbguard: /opt/firebird/bin/firebird terminated

Mon Dec  9 10:52:35 2024
        /opt/firebird/bin/fbguard: guardian starting /opt/firebird/bin/firebird

Mon Dec  9 10:53:00 2024
        /opt/firebird/bin/fbguard: /opt/firebird/bin/firebird terminated abnormally (-1)

Mon Dec  9 10:53:00 2024
        /opt/firebird/bin/fbguard: guardian starting /opt/firebird/bin/firebird

core dump available at https://infireal.com/github-issues/firebird-crash-20241209/