Bug 26996: Convert Elasticsearch indexer commit buffer size to integer
authorBjörn Nylén <bjorn.nylen@ub.lu.se>
Wed, 11 Nov 2020 07:35:21 +0000 (08:35 +0100)
committerJonathan Druart <jonathan.druart@bugs.koha-community.org>
Mon, 4 Jan 2021 12:29:55 +0000 (13:29 +0100)
When multithreaded indexing is used, the commit size for children are spread
out resulting in them being of type float. When records are processed and the
commit counter decreased it may then never reach *exactly* 0. This means records
are never commited. This patch makes sure the counter is an integer to avoid the
problem.

To test you must find a set of circumstances that causes the issue. For me:
1. Run: ./rebuild_elasticsearch -v -b -p 2 -c 400
2. Note that only one process is logging "Committing xxx records..."
3. Kill processes.
4. Apply patch.
5. Repeat 1
6. Note that both processes are logging "Committing xxx records..."

Sponsored-by: Lund University Library
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
misc/search_tools/rebuild_elasticsearch.pl

index ff83562..6b9487f 100755 (executable)
@@ -188,6 +188,7 @@ if ($slice_count > 1) {
     }
     # Fudge the commit count a bit to spread out the Elasticsearch commits
     $commit *= 1 + 0.10 * $slice_index;
+    $commit = int( $commit );
     _log(1, "Processing slice @{[$slice_index + 1]} of $slice_count\n");
     $iterator_options{slice} = { index => $slice_index, count => $slice_count };
 }