Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Boxiang Sun
gitlab-ce
Commits
0b15570e
Commit
0b15570e
authored
Nov 28, 2017
by
Douwe Maan
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Add ApplicationWorker and make every worker include it
parent
4ca4b0ff
Changes
99
Hide whitespace changes
Inline
Side-by-side
Showing
99 changed files
with
251 additions
and
169 deletions
+251
-169
app/workers/admin_email_worker.rb
app/workers/admin_email_worker.rb
+1
-1
app/workers/authorized_projects_worker.rb
app/workers/authorized_projects_worker.rb
+1
-2
app/workers/background_migration_worker.rb
app/workers/background_migration_worker.rb
+1
-2
app/workers/build_coverage_worker.rb
app/workers/build_coverage_worker.rb
+1
-1
app/workers/build_finished_worker.rb
app/workers/build_finished_worker.rb
+1
-1
app/workers/build_hooks_worker.rb
app/workers/build_hooks_worker.rb
+1
-1
app/workers/build_queue_worker.rb
app/workers/build_queue_worker.rb
+1
-1
app/workers/build_success_worker.rb
app/workers/build_success_worker.rb
+1
-1
app/workers/build_trace_sections_worker.rb
app/workers/build_trace_sections_worker.rb
+1
-1
app/workers/cluster_install_app_worker.rb
app/workers/cluster_install_app_worker.rb
+1
-1
app/workers/cluster_provision_worker.rb
app/workers/cluster_provision_worker.rb
+1
-1
app/workers/cluster_wait_for_app_installation_worker.rb
app/workers/cluster_wait_for_app_installation_worker.rb
+1
-1
app/workers/concerns/application_worker.rb
app/workers/concerns/application_worker.rb
+25
-0
app/workers/concerns/dedicated_sidekiq_queue.rb
app/workers/concerns/dedicated_sidekiq_queue.rb
+0
-9
app/workers/concerns/gitlab/github_import/object_importer.rb
app/workers/concerns/gitlab/github_import/object_importer.rb
+1
-1
app/workers/create_gpg_signature_worker.rb
app/workers/create_gpg_signature_worker.rb
+1
-2
app/workers/create_pipeline_worker.rb
app/workers/create_pipeline_worker.rb
+1
-1
app/workers/delete_merged_branches_worker.rb
app/workers/delete_merged_branches_worker.rb
+1
-2
app/workers/delete_user_worker.rb
app/workers/delete_user_worker.rb
+1
-2
app/workers/email_receiver_worker.rb
app/workers/email_receiver_worker.rb
+1
-2
app/workers/emails_on_push_worker.rb
app/workers/emails_on_push_worker.rb
+1
-2
app/workers/expire_build_artifacts_worker.rb
app/workers/expire_build_artifacts_worker.rb
+1
-1
app/workers/expire_build_instance_artifacts_worker.rb
app/workers/expire_build_instance_artifacts_worker.rb
+1
-2
app/workers/expire_job_cache_worker.rb
app/workers/expire_job_cache_worker.rb
+1
-1
app/workers/expire_pipeline_cache_worker.rb
app/workers/expire_pipeline_cache_worker.rb
+1
-1
app/workers/git_garbage_collect_worker.rb
app/workers/git_garbage_collect_worker.rb
+1
-2
app/workers/gitlab/github_import/advance_stage_worker.rb
app/workers/gitlab/github_import/advance_stage_worker.rb
+1
-1
app/workers/gitlab/github_import/refresh_import_jid_worker.rb
...workers/gitlab/github_import/refresh_import_jid_worker.rb
+1
-1
app/workers/gitlab/github_import/stage/finish_import_worker.rb
...orkers/gitlab/github_import/stage/finish_import_worker.rb
+1
-1
app/workers/gitlab/github_import/stage/import_base_data_worker.rb
...ers/gitlab/github_import/stage/import_base_data_worker.rb
+1
-1
app/workers/gitlab/github_import/stage/import_issues_and_diff_notes_worker.rb
...ithub_import/stage/import_issues_and_diff_notes_worker.rb
+1
-1
app/workers/gitlab/github_import/stage/import_notes_worker.rb
...workers/gitlab/github_import/stage/import_notes_worker.rb
+1
-1
app/workers/gitlab/github_import/stage/import_pull_requests_worker.rb
...gitlab/github_import/stage/import_pull_requests_worker.rb
+1
-1
app/workers/gitlab/github_import/stage/import_repository_worker.rb
...rs/gitlab/github_import/stage/import_repository_worker.rb
+1
-1
app/workers/gitlab_shell_worker.rb
app/workers/gitlab_shell_worker.rb
+1
-2
app/workers/gitlab_usage_ping_worker.rb
app/workers/gitlab_usage_ping_worker.rb
+1
-1
app/workers/group_destroy_worker.rb
app/workers/group_destroy_worker.rb
+1
-2
app/workers/import_export_project_cleanup_worker.rb
app/workers/import_export_project_cleanup_worker.rb
+1
-1
app/workers/invalid_gpg_signature_update_worker.rb
app/workers/invalid_gpg_signature_update_worker.rb
+1
-2
app/workers/irker_worker.rb
app/workers/irker_worker.rb
+1
-2
app/workers/merge_worker.rb
app/workers/merge_worker.rb
+1
-2
app/workers/namespaceless_project_destroy_worker.rb
app/workers/namespaceless_project_destroy_worker.rb
+1
-2
app/workers/new_issue_worker.rb
app/workers/new_issue_worker.rb
+1
-2
app/workers/new_merge_request_worker.rb
app/workers/new_merge_request_worker.rb
+1
-2
app/workers/new_note_worker.rb
app/workers/new_note_worker.rb
+1
-2
app/workers/pages_worker.rb
app/workers/pages_worker.rb
+1
-1
app/workers/pipeline_hooks_worker.rb
app/workers/pipeline_hooks_worker.rb
+1
-1
app/workers/pipeline_metrics_worker.rb
app/workers/pipeline_metrics_worker.rb
+1
-1
app/workers/pipeline_notification_worker.rb
app/workers/pipeline_notification_worker.rb
+1
-1
app/workers/pipeline_process_worker.rb
app/workers/pipeline_process_worker.rb
+1
-1
app/workers/pipeline_schedule_worker.rb
app/workers/pipeline_schedule_worker.rb
+1
-1
app/workers/pipeline_success_worker.rb
app/workers/pipeline_success_worker.rb
+1
-1
app/workers/pipeline_update_worker.rb
app/workers/pipeline_update_worker.rb
+1
-1
app/workers/post_receive.rb
app/workers/post_receive.rb
+1
-2
app/workers/process_commit_worker.rb
app/workers/process_commit_worker.rb
+1
-2
app/workers/project_cache_worker.rb
app/workers/project_cache_worker.rb
+1
-2
app/workers/project_destroy_worker.rb
app/workers/project_destroy_worker.rb
+1
-2
app/workers/project_export_worker.rb
app/workers/project_export_worker.rb
+1
-2
app/workers/project_migrate_hashed_storage_worker.rb
app/workers/project_migrate_hashed_storage_worker.rb
+1
-2
app/workers/project_service_worker.rb
app/workers/project_service_worker.rb
+1
-2
app/workers/propagate_service_template_worker.rb
app/workers/propagate_service_template_worker.rb
+1
-2
app/workers/prune_old_events_worker.rb
app/workers/prune_old_events_worker.rb
+1
-1
app/workers/reactive_caching_worker.rb
app/workers/reactive_caching_worker.rb
+1
-2
app/workers/remove_expired_group_links_worker.rb
app/workers/remove_expired_group_links_worker.rb
+1
-1
app/workers/remove_expired_members_worker.rb
app/workers/remove_expired_members_worker.rb
+1
-1
app/workers/remove_old_web_hook_logs_worker.rb
app/workers/remove_old_web_hook_logs_worker.rb
+1
-1
app/workers/remove_unreferenced_lfs_objects_worker.rb
app/workers/remove_unreferenced_lfs_objects_worker.rb
+1
-1
app/workers/repository_archive_cache_worker.rb
app/workers/repository_archive_cache_worker.rb
+1
-1
app/workers/repository_check/batch_worker.rb
app/workers/repository_check/batch_worker.rb
+1
-1
app/workers/repository_check/clear_worker.rb
app/workers/repository_check/clear_worker.rb
+1
-1
app/workers/repository_check/single_repository_worker.rb
app/workers/repository_check/single_repository_worker.rb
+1
-1
app/workers/repository_fork_worker.rb
app/workers/repository_fork_worker.rb
+1
-2
app/workers/repository_import_worker.rb
app/workers/repository_import_worker.rb
+1
-2
app/workers/requests_profiles_worker.rb
app/workers/requests_profiles_worker.rb
+1
-1
app/workers/schedule_update_user_activity_worker.rb
app/workers/schedule_update_user_activity_worker.rb
+1
-1
app/workers/stage_update_worker.rb
app/workers/stage_update_worker.rb
+1
-1
app/workers/storage_migrator_worker.rb
app/workers/storage_migrator_worker.rb
+1
-2
app/workers/stuck_ci_jobs_worker.rb
app/workers/stuck_ci_jobs_worker.rb
+1
-1
app/workers/stuck_import_jobs_worker.rb
app/workers/stuck_import_jobs_worker.rb
+1
-1
app/workers/stuck_merge_jobs_worker.rb
app/workers/stuck_merge_jobs_worker.rb
+1
-1
app/workers/system_hook_push_worker.rb
app/workers/system_hook_push_worker.rb
+1
-2
app/workers/trending_projects_worker.rb
app/workers/trending_projects_worker.rb
+1
-1
app/workers/update_merge_requests_worker.rb
app/workers/update_merge_requests_worker.rb
+1
-2
app/workers/update_user_activity_worker.rb
app/workers/update_user_activity_worker.rb
+1
-2
app/workers/upload_checksum_worker.rb
app/workers/upload_checksum_worker.rb
+1
-2
app/workers/wait_for_cluster_creation_worker.rb
app/workers/wait_for_cluster_creation_worker.rb
+1
-1
app/workers/web_hook_worker.rb
app/workers/web_hook_worker.rb
+1
-2
config/initializers/sidekiq.rb
config/initializers/sidekiq.rb
+4
-4
doc/development/sidekiq_style_guide.md
doc/development/sidekiq_style_guide.md
+1
-2
lib/gitlab/sidekiq_config.rb
lib/gitlab/sidekiq_config.rb
+50
-0
spec/lib/gitlab/sidekiq_config_spec.rb
spec/lib/gitlab/sidekiq_config_spec.rb
+24
-0
spec/workers/concerns/application_worker_spec.rb
spec/workers/concerns/application_worker_spec.rb
+27
-0
spec/workers/concerns/cluster_queue_spec.rb
spec/workers/concerns/cluster_queue_spec.rb
+5
-1
spec/workers/concerns/cronjob_queue_spec.rb
spec/workers/concerns/cronjob_queue_spec.rb
+5
-1
spec/workers/concerns/gitlab/github_import/object_importer_spec.rb
...ers/concerns/gitlab/github_import/object_importer_spec.rb
+4
-0
spec/workers/concerns/gitlab/github_import/queue_spec.rb
spec/workers/concerns/gitlab/github_import/queue_spec.rb
+5
-1
spec/workers/concerns/pipeline_queue_spec.rb
spec/workers/concerns/pipeline_queue_spec.rb
+5
-1
spec/workers/concerns/repository_check_queue_spec.rb
spec/workers/concerns/repository_check_queue_spec.rb
+5
-1
spec/workers/every_sidekiq_worker_spec.rb
spec/workers/every_sidekiq_worker_spec.rb
+6
-29
No files found.
app/workers/admin_email_worker.rb
View file @
0b15570e
class
AdminEmailWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/authorized_projects_worker.rb
View file @
0b15570e
class
AuthorizedProjectsWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
# Schedules multiple jobs and waits for them to be completed.
def
self
.
bulk_perform_and_wait
(
args_list
)
...
...
app/workers/background_migration_worker.rb
View file @
0b15570e
class
BackgroundMigrationWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
# Enqueues a number of jobs in bulk.
#
...
...
app/workers/build_coverage_worker.rb
View file @
0b15570e
class
BuildCoverageWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
def
perform
(
build_id
)
...
...
app/workers/build_finished_worker.rb
View file @
0b15570e
class
BuildFinishedWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :processing
...
...
app/workers/build_hooks_worker.rb
View file @
0b15570e
class
BuildHooksWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :hooks
...
...
app/workers/build_queue_worker.rb
View file @
0b15570e
class
BuildQueueWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :processing
...
...
app/workers/build_success_worker.rb
View file @
0b15570e
class
BuildSuccessWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :processing
...
...
app/workers/build_trace_sections_worker.rb
View file @
0b15570e
class
BuildTraceSectionsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
def
perform
(
build_id
)
...
...
app/workers/cluster_install_app_worker.rb
View file @
0b15570e
class
ClusterInstallAppWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
ClusterQueue
include
ClusterApplications
...
...
app/workers/cluster_provision_worker.rb
View file @
0b15570e
class
ClusterProvisionWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
ClusterQueue
def
perform
(
cluster_id
)
...
...
app/workers/cluster_wait_for_app_installation_worker.rb
View file @
0b15570e
class
ClusterWaitForAppInstallationWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
ClusterQueue
include
ClusterApplications
...
...
app/workers/concerns/application_worker.rb
0 → 100644
View file @
0b15570e
Sidekiq
::
Worker
.
extend
ActiveSupport
::
Concern
module
ApplicationWorker
extend
ActiveSupport
::
Concern
include
Sidekiq
::
Worker
included
do
sidekiq_options
queue:
base_queue_name
end
module
ClassMethods
def
base_queue_name
name
.
sub
(
/\AGitlab::/
,
''
)
.
sub
(
/Worker\z/
,
''
)
.
underscore
.
tr
(
'/'
,
'_'
)
end
def
queue
get_sidekiq_options
[
'queue'
].
to_s
end
end
end
app/workers/concerns/dedicated_sidekiq_queue.rb
deleted
100644 → 0
View file @
4ca4b0ff
# Concern that sets the queue of a Sidekiq worker based on the worker's class
# name/namespace.
module
DedicatedSidekiqQueue
extend
ActiveSupport
::
Concern
included
do
sidekiq_options
queue:
name
.
sub
(
/Worker\z/
,
''
).
underscore
.
tr
(
'/'
,
'_'
)
end
end
app/workers/concerns/gitlab/github_import/object_importer.rb
View file @
0b15570e
...
...
@@ -8,7 +8,7 @@ module Gitlab
extend
ActiveSupport
::
Concern
included
do
include
Sidekiq
::
Worker
include
Application
Worker
include
GithubImport
::
Queue
include
ReschedulingMethods
include
NotifyUponDeath
...
...
app/workers/create_gpg_signature_worker.rb
View file @
0b15570e
class
CreateGpgSignatureWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
commit_sha
,
project_id
)
project
=
Project
.
find_by
(
id:
project_id
)
...
...
app/workers/create_pipeline_worker.rb
View file @
0b15570e
class
CreatePipelineWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :creation
...
...
app/workers/delete_merged_branches_worker.rb
View file @
0b15570e
class
DeleteMergedBranchesWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
project_id
,
user_id
)
begin
...
...
app/workers/delete_user_worker.rb
View file @
0b15570e
class
DeleteUserWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
current_user_id
,
delete_user_id
,
options
=
{})
delete_user
=
User
.
find
(
delete_user_id
)
...
...
app/workers/email_receiver_worker.rb
View file @
0b15570e
class
EmailReceiverWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
raw
)
return
unless
Gitlab
::
IncomingEmail
.
enabled?
...
...
app/workers/emails_on_push_worker.rb
View file @
0b15570e
class
EmailsOnPushWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
attr_reader
:email
,
:skip_premailer
...
...
app/workers/expire_build_artifacts_worker.rb
View file @
0b15570e
class
ExpireBuildArtifactsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/expire_build_instance_artifacts_worker.rb
View file @
0b15570e
class
ExpireBuildInstanceArtifactsWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
build_id
)
build
=
Ci
::
Build
...
...
app/workers/expire_job_cache_worker.rb
View file @
0b15570e
class
ExpireJobCacheWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :cache
...
...
app/workers/expire_pipeline_cache_worker.rb
View file @
0b15570e
class
ExpirePipelineCacheWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :cache
...
...
app/workers/git_garbage_collect_worker.rb
View file @
0b15570e
class
GitGarbageCollectWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
include
Gitlab
::
CurrentSettings
sidekiq_options
retry:
false
...
...
app/workers/gitlab/github_import/advance_stage_worker.rb
View file @
0b15570e
...
...
@@ -7,7 +7,7 @@ module Gitlab
# been completed this worker will advance the import process to the next
# stage.
class
AdvanceStageWorker
include
Sidekiq
::
Worker
include
Application
Worker
sidekiq_options
queue:
'github_importer_advance_stage'
,
dead:
false
...
...
app/workers/gitlab/github_import/refresh_import_jid_worker.rb
View file @
0b15570e
...
...
@@ -3,7 +3,7 @@
module
Gitlab
module
GithubImport
class
RefreshImportJidWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
GithubImport
::
Queue
# The interval to schedule new instances of this job at.
...
...
app/workers/gitlab/github_import/stage/finish_import_worker.rb
View file @
0b15570e
...
...
@@ -4,7 +4,7 @@ module Gitlab
module
GithubImport
module
Stage
class
FinishImportWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
GithubImport
::
Queue
include
StageMethods
...
...
app/workers/gitlab/github_import/stage/import_base_data_worker.rb
View file @
0b15570e
...
...
@@ -4,7 +4,7 @@ module Gitlab
module
GithubImport
module
Stage
class
ImportBaseDataWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
GithubImport
::
Queue
include
StageMethods
...
...
app/workers/gitlab/github_import/stage/import_issues_and_diff_notes_worker.rb
View file @
0b15570e
...
...
@@ -4,7 +4,7 @@ module Gitlab
module
GithubImport
module
Stage
class
ImportIssuesAndDiffNotesWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
GithubImport
::
Queue
include
StageMethods
...
...
app/workers/gitlab/github_import/stage/import_notes_worker.rb
View file @
0b15570e
...
...
@@ -4,7 +4,7 @@ module Gitlab
module
GithubImport
module
Stage
class
ImportNotesWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
GithubImport
::
Queue
include
StageMethods
...
...
app/workers/gitlab/github_import/stage/import_pull_requests_worker.rb
View file @
0b15570e
...
...
@@ -4,7 +4,7 @@ module Gitlab
module
GithubImport
module
Stage
class
ImportPullRequestsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
GithubImport
::
Queue
include
StageMethods
...
...
app/workers/gitlab/github_import/stage/import_repository_worker.rb
View file @
0b15570e
...
...
@@ -4,7 +4,7 @@ module Gitlab
module
GithubImport
module
Stage
class
ImportRepositoryWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
GithubImport
::
Queue
include
StageMethods
...
...
app/workers/gitlab_shell_worker.rb
View file @
0b15570e
class
GitlabShellWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
Gitlab
::
ShellAdapter
include
DedicatedSidekiqQueue
def
perform
(
action
,
*
arg
)
gitlab_shell
.
__send__
(
action
,
*
arg
)
# rubocop:disable GitlabSecurity/PublicSend
...
...
app/workers/gitlab_usage_ping_worker.rb
View file @
0b15570e
class
GitlabUsagePingWorker
LEASE_TIMEOUT
=
86400
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/group_destroy_worker.rb
View file @
0b15570e
class
GroupDestroyWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
include
ExceptionBacktrace
def
perform
(
group_id
,
user_id
)
...
...
app/workers/import_export_project_cleanup_worker.rb
View file @
0b15570e
class
ImportExportProjectCleanupWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/invalid_gpg_signature_update_worker.rb
View file @
0b15570e
class
InvalidGpgSignatureUpdateWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
gpg_key_id
)
gpg_key
=
GpgKey
.
find_by
(
id:
gpg_key_id
)
...
...
app/workers/irker_worker.rb
View file @
0b15570e
...
...
@@ -2,8 +2,7 @@ require 'json'
require
'socket'
class
IrkerWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
project_id
,
chans
,
colors
,
push_data
,
settings
)
project
=
Project
.
find
(
project_id
)
...
...
app/workers/merge_worker.rb
View file @
0b15570e
class
MergeWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
merge_request_id
,
current_user_id
,
params
)
params
=
params
.
with_indifferent_access
...
...
app/workers/namespaceless_project_destroy_worker.rb
View file @
0b15570e
...
...
@@ -5,8 +5,7 @@
# The worker will reject doing anything for projects that *do* have a
# namespace. For those use ProjectDestroyWorker instead.
class
NamespacelessProjectDestroyWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
include
ExceptionBacktrace
def
self
.
bulk_perform_async
(
args_list
)
...
...
app/workers/new_issue_worker.rb
View file @
0b15570e
class
NewIssueWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
include
NewIssuable
def
perform
(
issue_id
,
user_id
)
...
...
app/workers/new_merge_request_worker.rb
View file @
0b15570e
class
NewMergeRequestWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
include
NewIssuable
def
perform
(
merge_request_id
,
user_id
)
...
...
app/workers/new_note_worker.rb
View file @
0b15570e
class
NewNoteWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
# Keep extra parameter to preserve backwards compatibility with
# old `NewNoteWorker` jobs (can remove later)
...
...
app/workers/pages_worker.rb
View file @
0b15570e
class
PagesWorker
include
Sidekiq
::
Worker
include
Application
Worker
sidekiq_options
queue: :pages
,
retry:
false
...
...
app/workers/pipeline_hooks_worker.rb
View file @
0b15570e
class
PipelineHooksWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :hooks
...
...
app/workers/pipeline_metrics_worker.rb
View file @
0b15570e
class
PipelineMetricsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
def
perform
(
pipeline_id
)
...
...
app/workers/pipeline_notification_worker.rb
View file @
0b15570e
class
PipelineNotificationWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
def
perform
(
pipeline_id
,
recipients
=
nil
)
...
...
app/workers/pipeline_process_worker.rb
View file @
0b15570e
class
PipelineProcessWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :processing
...
...
app/workers/pipeline_schedule_worker.rb
View file @
0b15570e
class
PipelineScheduleWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/pipeline_success_worker.rb
View file @
0b15570e
class
PipelineSuccessWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :processing
...
...
app/workers/pipeline_update_worker.rb
View file @
0b15570e
class
PipelineUpdateWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :processing
...
...
app/workers/post_receive.rb
View file @
0b15570e
class
PostReceive
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
gl_repository
,
identifier
,
changes
)
project
,
is_wiki
=
Gitlab
::
GlRepository
.
parse
(
gl_repository
)
...
...
app/workers/process_commit_worker.rb
View file @
0b15570e
...
...
@@ -5,8 +5,7 @@
# Consider using an extra worker if you need to add any extra (and potentially
# slow) processing of commits.
class
ProcessCommitWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
# project_id - The ID of the project this commit belongs to.
# user_id - The ID of the user that pushed the commit.
...
...
app/workers/project_cache_worker.rb
View file @
0b15570e
# Worker for updating any project specific caches.
class
ProjectCacheWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
LEASE_TIMEOUT
=
15
.
minutes
.
to_i
...
...
app/workers/project_destroy_worker.rb
View file @
0b15570e
class
ProjectDestroyWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
include
ExceptionBacktrace
def
perform
(
project_id
,
user_id
,
params
)
...
...
app/workers/project_export_worker.rb
View file @
0b15570e
class
ProjectExportWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
include
ExceptionBacktrace
sidekiq_options
retry:
3
...
...
app/workers/project_migrate_hashed_storage_worker.rb
View file @
0b15570e
class
ProjectMigrateHashedStorageWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
LEASE_TIMEOUT
=
30
.
seconds
.
to_i
...
...
app/workers/project_service_worker.rb
View file @
0b15570e
class
ProjectServiceWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
sidekiq_options
dead:
false
...
...
app/workers/propagate_service_template_worker.rb
View file @
0b15570e
# Worker for updating any project specific caches.
class
PropagateServiceTemplateWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
LEASE_TIMEOUT
=
4
.
hours
.
to_i
...
...
app/workers/prune_old_events_worker.rb
View file @
0b15570e
class
PruneOldEventsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/reactive_caching_worker.rb
View file @
0b15570e
class
ReactiveCachingWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
class_name
,
id
,
*
args
)
klass
=
begin
...
...
app/workers/remove_expired_group_links_worker.rb
View file @
0b15570e
class
RemoveExpiredGroupLinksWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/remove_expired_members_worker.rb
View file @
0b15570e
class
RemoveExpiredMembersWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/remove_old_web_hook_logs_worker.rb
View file @
0b15570e
class
RemoveOldWebHookLogsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
WEB_HOOK_LOG_LIFETIME
=
2
.
days
...
...
app/workers/remove_unreferenced_lfs_objects_worker.rb
View file @
0b15570e
class
RemoveUnreferencedLfsObjectsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/repository_archive_cache_worker.rb
View file @
0b15570e
class
RepositoryArchiveCacheWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/repository_check/batch_worker.rb
View file @
0b15570e
module
RepositoryCheck
class
BatchWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
RUN_TIME
=
3600
...
...
app/workers/repository_check/clear_worker.rb
View file @
0b15570e
module
RepositoryCheck
class
ClearWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
RepositoryCheckQueue
def
perform
...
...
app/workers/repository_check/single_repository_worker.rb
View file @
0b15570e
module
RepositoryCheck
class
SingleRepositoryWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
RepositoryCheckQueue
def
perform
(
project_id
)
...
...
app/workers/repository_fork_worker.rb
View file @
0b15570e
class
RepositoryForkWorker
ForkError
=
Class
.
new
(
StandardError
)
include
Sidekiq
::
Worker
include
Application
Worker
include
Gitlab
::
ShellAdapter
include
DedicatedSidekiqQueue
include
ProjectStartImport
sidekiq_options
status_expiration:
StuckImportJobsWorker
::
IMPORT_JOBS_EXPIRATION
...
...
app/workers/repository_import_worker.rb
View file @
0b15570e
class
RepositoryImportWorker
ImportError
=
Class
.
new
(
StandardError
)
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
include
ExceptionBacktrace
include
ProjectStartImport
...
...
app/workers/requests_profiles_worker.rb
View file @
0b15570e
class
RequestsProfilesWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/schedule_update_user_activity_worker.rb
View file @
0b15570e
class
ScheduleUpdateUserActivityWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
(
batch_size
=
500
)
...
...
app/workers/stage_update_worker.rb
View file @
0b15570e
class
StageUpdateWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
PipelineQueue
enqueue_in
group: :processing
...
...
app/workers/storage_migrator_worker.rb
View file @
0b15570e
class
StorageMigratorWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
BATCH_SIZE
=
100
...
...
app/workers/stuck_ci_jobs_worker.rb
View file @
0b15570e
class
StuckCiJobsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
EXCLUSIVE_LEASE_KEY
=
'stuck_ci_builds_worker_lease'
.
freeze
...
...
app/workers/stuck_import_jobs_worker.rb
View file @
0b15570e
class
StuckImportJobsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
IMPORT_JOBS_EXPIRATION
=
15
.
hours
.
to_i
...
...
app/workers/stuck_merge_jobs_worker.rb
View file @
0b15570e
class
StuckMergeJobsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/system_hook_push_worker.rb
View file @
0b15570e
class
SystemHookPushWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
push_data
,
hook_id
)
SystemHooksService
.
new
.
execute_hooks
(
push_data
,
hook_id
)
...
...
app/workers/trending_projects_worker.rb
View file @
0b15570e
class
TrendingProjectsWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
CronjobQueue
def
perform
...
...
app/workers/update_merge_requests_worker.rb
View file @
0b15570e
class
UpdateMergeRequestsWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
LOG_TIME_THRESHOLD
=
90
# seconds
...
...
app/workers/update_user_activity_worker.rb
View file @
0b15570e
class
UpdateUserActivityWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
pairs
)
pairs
=
cast_data
(
pairs
)
...
...
app/workers/upload_checksum_worker.rb
View file @
0b15570e
class
UploadChecksumWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
def
perform
(
upload_id
)
upload
=
Upload
.
find
(
upload_id
)
...
...
app/workers/wait_for_cluster_creation_worker.rb
View file @
0b15570e
class
WaitForClusterCreationWorker
include
Sidekiq
::
Worker
include
Application
Worker
include
ClusterQueue
def
perform
(
cluster_id
)
...
...
app/workers/web_hook_worker.rb
View file @
0b15570e
class
WebHookWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
sidekiq_options
retry:
4
,
dead:
false
...
...
config/initializers/sidekiq.rb
View file @
0b15570e
...
...
@@ -64,13 +64,13 @@ end
# The Sidekiq client API always adds the queue to the Sidekiq queue
# list, but mail_room and gitlab-shell do not. This is only necessary
# for monitoring.
config
=
YAML
.
load_file
(
Rails
.
root
.
join
(
'config'
,
'sidekiq_queues.yml'
).
to_s
)
begin
queues
=
Gitlab
::
SidekiqConfig
.
worker_queues
Sidekiq
.
redis
do
|
conn
|
conn
.
pipelined
do
config
[
:queues
]
.
each
do
|
queue
|
conn
.
sadd
(
'queues'
,
queue
[
0
]
)
queues
.
each
do
|
queue
|
conn
.
sadd
(
'queues'
,
queue
)
end
end
end
...
...
doc/development/sidekiq_style_guide.md
View file @
0b15570e
...
...
@@ -18,8 +18,7 @@ include the `DedicatedSidekiqQueue` concern as follows:
```
ruby
class
ProcessSomethingWorker
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
end
```
...
...
lib/gitlab/sidekiq_config.rb
0 → 100644
View file @
0b15570e
require
'yaml'
module
Gitlab
module
SidekiqConfig
def
self
.
redis_queues
@redis_queues
||=
Sidekiq
::
Queue
.
all
.
map
(
&
:name
)
end
# This method is called by `bin/sidekiq-cluster` in EE, which runs outside
# of bundler/Rails context, so we cannot use any gem or Rails methods.
def
self
.
config_queues
(
rails_path
=
Rails
.
root
.
to_s
)
@config_queues
||=
begin
config
=
YAML
.
load_file
(
File
.
join
(
rails_path
,
'config'
,
'sidekiq_queues.yml'
))
config
[
:queues
].
map
(
&
:first
)
end
end
def
self
.
cron_workers
@cron_workers
||=
Settings
.
cron_jobs
.
map
{
|
job_name
,
options
|
options
[
'job_class'
].
constantize
}
end
def
self
.
workers
@workers
||=
find_workers
(
Rails
.
root
.
join
(
'app'
,
'workers'
))
end
def
self
.
default_queues
[
ActionMailer
::
DeliveryJob
.
queue_name
,
'default'
]
end
def
self
.
worker_queues
@worker_queues
||=
(
workers
.
map
(
&
:queue
)
+
default_queues
).
uniq
end
def
self
.
find_workers
(
root
)
concerns
=
root
.
join
(
'concerns'
).
to_s
workers
=
Dir
[
root
.
join
(
'**'
,
'*.rb'
)]
.
reject
{
|
path
|
path
.
start_with?
(
concerns
)
}
workers
.
map!
do
|
path
|
ns
=
Pathname
.
new
(
path
).
relative_path_from
(
root
).
to_s
.
gsub
(
'.rb'
,
''
)
ns
.
camelize
.
constantize
end
# Skip concerns
workers
.
select
{
|
w
|
w
<
Sidekiq
::
Worker
}
end
end
end
spec/lib/gitlab/sidekiq_config_spec.rb
0 → 100644
View file @
0b15570e
require
'rails_helper'
describe
Gitlab
::
SidekiqConfig
do
describe
'.workers'
do
it
'includes all workers'
do
workers
=
described_class
.
workers
expect
(
workers
).
to
include
(
PostReceive
)
expect
(
workers
).
to
include
(
MergeWorker
)
end
end
describe
'.worker_queues'
do
it
'includes all queues'
do
queues
=
described_class
.
worker_queues
expect
(
queues
).
to
include
(
'post_receive'
)
expect
(
queues
).
to
include
(
'merge'
)
expect
(
queues
).
to
include
(
'cronjob'
)
expect
(
queues
).
to
include
(
'mailers'
)
expect
(
queues
).
to
include
(
'default'
)
end
end
end
spec/workers/concerns/
dedicated_sidekiq_queue
_spec.rb
→
spec/workers/concerns/
application_worker
_spec.rb
View file @
0b15570e
require
'spec_helper'
describe
DedicatedSidekiqQueue
do
describe
ApplicationWorker
do
let
(
:worker
)
do
Class
.
new
do
def
self
.
name
'Foo::Bar::DummyWorker'
'
Gitlab::
Foo::Bar::DummyWorker'
end
include
Sidekiq
::
Worker
include
DedicatedSidekiqQueue
include
ApplicationWorker
end
end
describe
'
queue name
s'
do
describe
'
Sidekiq option
s'
do
it
'sets the queue name based on the class name'
do
expect
(
worker
.
sidekiq_options
[
'queue'
]).
to
eq
(
'foo_bar_dummy'
)
end
end
describe
'.queue'
do
it
'returns the queue name'
do
worker
.
sidekiq_options
queue: :some_queue
expect
(
worker
.
queue
).
to
eq
(
'some_queue'
)
end
end
end
spec/workers/concerns/cluster_queue_spec.rb
View file @
0b15570e
...
...
@@ -3,7 +3,11 @@ require 'spec_helper'
describe
ClusterQueue
do
let
(
:worker
)
do
Class
.
new
do
include
Sidekiq
::
Worker
def
self
.
name
'DummyWorker'
end
include
ApplicationWorker
include
ClusterQueue
end
end
...
...
spec/workers/concerns/cronjob_queue_spec.rb
View file @
0b15570e
...
...
@@ -3,7 +3,11 @@ require 'spec_helper'
describe
CronjobQueue
do
let
(
:worker
)
do
Class
.
new
do
include
Sidekiq
::
Worker
def
self
.
name
'DummyWorker'
end
include
ApplicationWorker
include
CronjobQueue
end
end
...
...
spec/workers/concerns/gitlab/github_import/object_importer_spec.rb
View file @
0b15570e
...
...
@@ -3,6 +3,10 @@ require 'spec_helper'
describe
Gitlab
::
GithubImport
::
ObjectImporter
do
let
(
:worker
)
do
Class
.
new
do
def
self
.
name
'DummyWorker'
end
include
(
Gitlab
::
GithubImport
::
ObjectImporter
)
def
counter_name
...
...
spec/workers/concerns/gitlab/github_import/queue_spec.rb
View file @
0b15570e
...
...
@@ -3,7 +3,11 @@ require 'spec_helper'
describe
Gitlab
::
GithubImport
::
Queue
do
it
'sets the Sidekiq options for the worker'
do
worker
=
Class
.
new
do
include
Sidekiq
::
Worker
def
self
.
name
'DummyWorker'
end
include
ApplicationWorker
include
Gitlab
::
GithubImport
::
Queue
end
...
...
spec/workers/concerns/pipeline_queue_spec.rb
View file @
0b15570e
...
...
@@ -3,7 +3,11 @@ require 'spec_helper'
describe
PipelineQueue
do
let
(
:worker
)
do
Class
.
new
do
include
Sidekiq
::
Worker
def
self
.
name
'DummyWorker'
end
include
ApplicationWorker
include
PipelineQueue
end
end
...
...
spec/workers/concerns/repository_check_queue_spec.rb
View file @
0b15570e
...
...
@@ -3,7 +3,11 @@ require 'spec_helper'
describe
RepositoryCheckQueue
do
let
(
:worker
)
do
Class
.
new
do
include
Sidekiq
::
Worker
def
self
.
name
'DummyWorker'
end
include
ApplicationWorker
include
RepositoryCheckQueue
end
end
...
...
spec/workers/every_sidekiq_worker_spec.rb
View file @
0b15570e
require
'spec_helper'
describe
'Every Sidekiq worker'
do
let
(
:workers
)
do
root
=
Rails
.
root
.
join
(
'app'
,
'workers'
)
concerns
=
root
.
join
(
'concerns'
).
to_s
workers
=
Dir
[
root
.
join
(
'**'
,
'*.rb'
)]
.
reject
{
|
path
|
path
.
start_with?
(
concerns
)
}
workers
.
map
do
|
path
|
ns
=
Pathname
.
new
(
path
).
relative_path_from
(
root
).
to_s
.
gsub
(
'.rb'
,
''
)
ns
.
camelize
.
constantize
end
it
'includes ApplicationWorker'
do
expect
(
Gitlab
::
SidekiqConfig
.
workers
).
to
all
(
include
(
ApplicationWorker
))
end
it
'does not use the default queue'
do
workers
.
each
do
|
worker
|
expect
(
worker
.
sidekiq_options
[
'queue'
].
to_s
).
not_to
eq
(
'default'
)
end
expect
(
Gitlab
::
SidekiqConfig
.
workers
.
map
(
&
:queue
)).
not_to
include
(
'default'
)
end
it
'uses the cronjob queue when the worker runs as a cronjob'
do
cron_workers
=
Settings
.
cron_jobs
.
map
{
|
job_name
,
options
|
options
[
'job_class'
].
constantize
}
.
to_set
workers
.
each
do
|
worker
|
next
unless
cron_workers
.
include?
(
worker
)
expect
(
worker
.
sidekiq_options
[
'queue'
].
to_s
).
to
eq
(
'cronjob'
)
end
expect
(
Gitlab
::
SidekiqConfig
.
cron_workers
.
map
(
&
:queue
)).
to
all
(
eq
(
'cronjob'
))
end
it
'defines the queue in the Sidekiq configuration file'
do
config
=
YAML
.
load_file
(
Rails
.
root
.
join
(
'config'
,
'sidekiq_queues.yml'
).
to_s
)
queue_names
=
config
[
:queues
].
map
{
|
(
queue
,
_
)
|
queue
}.
to_set
config_queue_names
=
Gitlab
::
SidekiqConfig
.
config_queues
.
to_set
workers
.
each
do
|
worker
|
expect
(
queue_names
).
to
include
(
worker
.
sidekiq_options
[
'queue'
].
to_s
)
end
expect
(
Gitlab
::
SidekiqConfig
.
worker_queues
).
to
all
(
be_in
(
config_queue_names
))
end
end
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment