Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
4e49eb91
Commit
4e49eb91
authored
Jul 07, 2020
by
Dmitry Gruzd
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Fix zero-downtime reindexing docs count issue
parent
af561f52
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
15 additions
and
4 deletions
+15
-4
ee/app/services/elastic/cluster_reindexing_service.rb
ee/app/services/elastic/cluster_reindexing_service.rb
+2
-2
ee/changelogs/unreleased/fix-es-reindexing-documents-count.yml
...angelogs/unreleased/fix-es-reindexing-documents-count.yml
+5
-0
ee/lib/gitlab/elastic/helper.rb
ee/lib/gitlab/elastic/helper.rb
+6
-0
ee/spec/services/elastic/cluster_reindexing_service_spec.rb
ee/spec/services/elastic/cluster_reindexing_service_spec.rb
+2
-2
No files found.
ee/app/services/elastic/cluster_reindexing_service.rb
View file @
4e49eb91
...
...
@@ -62,7 +62,7 @@ module Elastic
index_name
=
elastic_helper
.
create_empty_index
(
with_alias:
false
,
options:
{
settings:
INITIAL_INDEX_OPTIONS
})
# Record documents count
documents_count
=
elastic_helper
.
index_size
.
dig
(
'docs'
,
'count'
)
documents_count
=
elastic_helper
.
documents_count
# Trigger reindex
task_id
=
elastic_helper
.
reindex
(
to:
index_name
)
...
...
@@ -81,7 +81,7 @@ module Elastic
def
save_documents_count!
(
refresh
:)
elastic_helper
.
refresh_index
(
index_name:
current_task
.
index_name_to
)
if
refresh
new_documents_count
=
elastic_helper
.
index_size
(
index_name:
current_task
.
index_name_to
).
dig
(
'docs'
,
'count'
)
new_documents_count
=
elastic_helper
.
documents_count
(
index_name:
current_task
.
index_name_to
)
current_task
.
update!
(
documents_count_target:
new_documents_count
)
end
...
...
ee/changelogs/unreleased/fix-es-reindexing-documents-count.yml
0 → 100644
View file @
4e49eb91
---
title
:
Fix zero-downtime reindexing docs count
merge_request
:
36173
author
:
type
:
fixed
ee/lib/gitlab/elastic/helper.rb
View file @
4e49eb91
...
...
@@ -105,6 +105,12 @@ module Gitlab
client
.
indices
.
stats
[
'indices'
][
index_name
||
target_index_name
][
'total'
]
end
def
documents_count
(
index_name:
nil
)
index
=
index_name
||
target_index_name
client
.
indices
.
stats
.
dig
(
'indices'
,
index
,
'primaries'
,
'docs'
,
'count'
)
end
def
index_size_bytes
index_size
[
'store'
][
'size_in_bytes'
]
end
...
...
ee/spec/services/elastic/cluster_reindexing_service_spec.rb
View file @
4e49eb91
...
...
@@ -57,7 +57,7 @@ RSpec.describe Elastic::ClusterReindexingService, :elastic do
context
'errors are raised'
do
before
do
allow
(
Gitlab
::
Elastic
::
Helper
.
default
).
to
receive
(
:
index_size
).
and_return
(
'docs'
=>
{
'count'
=>
task
.
reload
.
documents_count
*
2
}
)
allow
(
Gitlab
::
Elastic
::
Helper
.
default
).
to
receive
(
:
documents_count
).
with
(
index_name:
task
.
index_name_to
).
and_return
(
task
.
reload
.
documents_count
*
2
)
end
it
'errors if documents count is different'
do
...
...
@@ -75,7 +75,7 @@ RSpec.describe Elastic::ClusterReindexingService, :elastic do
context
'task finishes correctly'
do
before
do
allow
(
Gitlab
::
Elastic
::
Helper
.
default
).
to
receive
(
:
index_size
).
and_return
(
'docs'
=>
{
'count'
=>
task
.
reload
.
documents_count
}
)
allow
(
Gitlab
::
Elastic
::
Helper
.
default
).
to
receive
(
:
documents_count
).
with
(
index_name:
task
.
index_name_to
).
and_return
(
task
.
reload
.
documents_count
)
end
it
'launches all state steps'
do
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment