Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
5f784e2a
Commit
5f784e2a
authored
Sep 04, 2019
by
Douglas Barbosa Alexandre
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Extract a helper method to schedule jobs
parent
54ec4cae
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
58 additions
and
54 deletions
+58
-54
ee/app/workers/concerns/geo/secondary/backfill_worker.rb
ee/app/workers/concerns/geo/secondary/backfill_worker.rb
+16
-0
ee/app/workers/geo/secondary/repository_backfill_worker.rb
ee/app/workers/geo/secondary/repository_backfill_worker.rb
+42
-54
No files found.
ee/app/workers/concerns/geo/secondary/backfill_worker.rb
View file @
5f784e2a
...
...
@@ -30,6 +30,22 @@ module Geo
end
end
def
initialize
@scheduled_jobs
=
[]
@loops
=
0
end
def
perform
(
shard_name
)
@shard_name
=
shard_name
@start_time
=
Time
.
now
.
utc
return
unless
healthy_node?
try_obtain_lease
do
schedule_jobs
end
end
private
def
base_log_data
(
message
)
...
...
ee/app/workers/geo/secondary/repository_backfill_worker.rb
View file @
5f784e2a
...
...
@@ -5,18 +5,35 @@ module Geo
class
RepositoryBackfillWorker
include
Geo
::
Secondary
::
BackfillWorker
def
initialize
@scheduled_jobs
=
[]
private
attr_reader
:scheduled_jobs
def
connection
strong_memoize
(
:connection
)
{
Geo
::
TrackingBase
.
connection
.
raw_connection
}
end
def
perform
(
shard_name
)
@shard_name
=
shard_name
@start_time
=
Time
.
now
.
utc
@loops
=
0
def
max_capacity
# If we don't have a count, that means that for some reason
# RepositorySyncWorker stopped running/updating the cache. We might
# be trying to shut down Geo while this job may still be running.
healthy_count
=
healthy_shard_count
return
0
unless
healthy_count
>
0
capacity_per_shard
=
Gitlab
::
Geo
.
current_node
.
repos_max_capacity
/
healthy_count
return
unless
healthy_node?
[
1
,
capacity_per_shard
.
to_i
].
max
end
try_obtain_lease
do
def
healthy_shard_count
Gitlab
::
ShardHealthCache
.
healthy_shard_count
.
to_i
end
def
over_capacity?
scheduled_jobs
.
size
>=
max_capacity
end
def
schedule_jobs
log_info
(
'Repository backfilling started'
)
reason
=
:unknown
...
...
@@ -53,35 +70,6 @@ module Geo
log_info
(
'Repository backfilling finished'
,
total_loops:
loops
,
duration:
Time
.
now
.
utc
-
start_time
,
reason:
reason
)
end
end
end
private
attr_reader
:scheduled_jobs
def
connection
strong_memoize
(
:connection
)
{
Geo
::
TrackingBase
.
connection
.
raw_connection
}
end
def
max_capacity
# If we don't have a count, that means that for some reason
# RepositorySyncWorker stopped running/updating the cache. We might
# be trying to shut down Geo while this job may still be running.
healthy_count
=
healthy_shard_count
return
0
unless
healthy_count
>
0
capacity_per_shard
=
Gitlab
::
Geo
.
current_node
.
repos_max_capacity
/
healthy_count
[
1
,
capacity_per_shard
.
to_i
].
max
end
def
healthy_shard_count
Gitlab
::
ShardHealthCache
.
healthy_shard_count
.
to_i
end
def
over_capacity?
scheduled_jobs
.
size
>=
max_capacity
end
def
schedule_job
(
project_id
)
job_id
=
Geo
::
ProjectSyncWorker
.
perform_async
(
project_id
,
sync_repository:
true
,
sync_wiki:
true
)
...
...
@@ -94,6 +82,10 @@ module Geo
end
end
def
scheduled_job_ids
scheduled_jobs
.
map
{
|
data
|
data
[
:job_id
]
}
end
def
update_jobs_in_progress
job_ids
=
scheduled_job_ids
return
if
job_ids
.
empty?
...
...
@@ -106,10 +98,6 @@ module Geo
end
end
def
scheduled_job_ids
scheduled_jobs
.
map
{
|
data
|
data
[
:job_id
]
}
end
# rubocop: disable CodeReuse/ActiveRecord
def
projects_ids_unsynced
Geo
::
ProjectUnsyncedFinder
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment