Nominatim import stuck on Recompute word counts

[dbt-lucka]

Hi,

EDIT: It resolved itself. For some reason checking the "currently running psql queries" released something and it continued.

OLD:
I am importing Europe on a 16 core 64 GB machine with 2 TB m.2 ssd.
Since 13:12 today my last log line is:

2023-10-09 13:12:02 2023-10-09 11:12:02: Recompute word counts

Postgresql log:

2023-10-09 09:11:06.895 UTC [60] LOG:  checkpoints are occurring too frequently (22 seconds apart)
2023-10-09 09:11:06.895 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 09:11:24.086 UTC [60] LOG:  checkpoints are occurring too frequently (18 seconds apart)
2023-10-09 09:11:24.086 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 09:11:43.925 UTC [60] LOG:  checkpoints are occurring too frequently (19 seconds apart)
2023-10-09 09:11:43.925 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 09:12:06.556 UTC [60] LOG:  checkpoints are occurring too frequently (23 seconds apart)
2023-10-09 09:12:06.556 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 09:17:44.078 UTC [60] LOG:  checkpoints are occurring too frequently (20 seconds apart)
2023-10-09 09:17:44.078 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 09:18:04.751 UTC [60] LOG:  checkpoints are occurring too frequently (20 seconds apart)
2023-10-09 09:18:04.751 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 09:27:56.217 UTC [60] LOG:  checkpoints are occurring too frequently (26 seconds apart)
2023-10-09 09:27:56.217 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 10:09:47.015 UTC [60] LOG:  checkpoints are occurring too frequently (26 seconds apart)
2023-10-09 10:09:47.015 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 10:34:38.406 UTC [60] LOG:  checkpoints are occurring too frequently (24 seconds apart)
2023-10-09 10:34:38.406 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:01:23.865 UTC [60] LOG:  checkpoints are occurring too frequently (23 seconds apart)
2023-10-09 11:01:23.865 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:02:12.850 UTC [60] LOG:  checkpoints are occurring too frequently (19 seconds apart)
2023-10-09 11:02:12.850 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:02:32.712 UTC [60] LOG:  checkpoints are occurring too frequently (20 seconds apart)
2023-10-09 11:02:32.712 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:02:55.365 UTC [60] LOG:  checkpoints are occurring too frequently (23 seconds apart)
2023-10-09 11:02:55.365 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:03:23.205 UTC [60] LOG:  checkpoints are occurring too frequently (28 seconds apart)
2023-10-09 11:03:23.205 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:03:44.812 UTC [60] LOG:  checkpoints are occurring too frequently (21 seconds apart)
2023-10-09 11:03:44.812 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:04:05.421 UTC [60] LOG:  checkpoints are occurring too frequently (21 seconds apart)
2023-10-09 11:04:05.421 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:04:29.295 UTC [60] LOG:  checkpoints are occurring too frequently (24 seconds apart)
2023-10-09 11:04:29.295 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:04:49.632 UTC [60] LOG:  checkpoints are occurring too frequently (20 seconds apart)
2023-10-09 11:04:49.632 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:05:08.028 UTC [60] LOG:  checkpoints are occurring too frequently (19 seconds apart)
2023-10-09 11:05:08.028 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:06:05.604 UTC [60] LOG:  checkpoints are occurring too frequently (27 seconds apart)
2023-10-09 11:06:05.604 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:06:28.176 UTC [60] LOG:  checkpoints are occurring too frequently (23 seconds apart)
2023-10-09 11:06:28.176 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:06:56.704 UTC [60] LOG:  checkpoints are occurring too frequently (28 seconds apart)
2023-10-09 11:06:56.704 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:07:17.604 UTC [60] LOG:  checkpoints are occurring too frequently (21 seconds apart)
2023-10-09 11:07:17.604 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:07:35.587 UTC [60] LOG:  checkpoints are occurring too frequently (18 seconds apart)
2023-10-09 11:07:35.587 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:07:58.808 UTC [60] LOG:  checkpoints are occurring too frequently (23 seconds apart)
2023-10-09 11:07:58.808 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:08:25.899 UTC [60] LOG:  checkpoints are occurring too frequently (27 seconds apart)
2023-10-09 11:08:25.899 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:08:54.005 UTC [60] LOG:  checkpoints are occurring too frequently (28 seconds apart)
2023-10-09 11:08:54.005 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-10-09 11:09:56.749 UTC [60] LOG:  checkpoints are occurring too frequently (24 seconds apart)
2023-10-09 11:09:56.749 UTC [60] HINT:  Consider increasing the configuration parameter "max_wal_size".

I can see that this query is running for more than 6 hours now:

Is there anything I could do to "rescue" my import? Any idea on why this querry is stuck?
Somewhere else I was reading that 2 hours is normal for this task.

I am still seeing 25% CPU utilization on pgsql buts that's about it.

Thanks

[dbt-lucka]

Update:
It just proceeded. How can this be a coincident?!

2023-10-09 13:10:01 2023-10-09 11:10:01: Create search index for default country names.
2023-10-09 13:12:02 2023-10-09 11:12:02: Recompute word counts
2023-10-09 20:19:57 2023-10-09 18:19:57: Setup website at /nominatim/website
2023-10-09 20:19:58 ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
2023-10-09 20:19:58

[mtmail]

That step really takes a lot of hours and it can't use all available CPU cores. Only waiting helps.

Is there anything I could do to "rescue" my import?

The indexing (where it counts down all the ranks) and all steps that follow can be rerun if needed . The import would've been rescue-able.

[lonvia]

Word counting used to be faster. It looks like something has tripped over and the query planner is doing the wrong thing again. It's on my list to investigate. In the meantime I can only recommend patience.

[dbt-lucka]

It passed eventually and everything works now. Thanks!