Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
7a19879d
Commit
7a19879d
authored
5 years ago
by
Hossein Pursultani
Committed by
Robert Speicher
5 years ago
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Move sidekiq-cluster to ee/bin
parent
c5c6e639
No related merge requests found
Changes
5
Hide whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
20 additions
and
14 deletions
+20
-14
doc/administration/operations/extra_sidekiq_processes.md
doc/administration/operations/extra_sidekiq_processes.md
+7
-7
ee/bin/sidekiq-cluster
ee/bin/sidekiq-cluster
+4
-3
ee/changelogs/unreleased/6565-move-sidekiq-cluster.yml
ee/changelogs/unreleased/6565-move-sidekiq-cluster.yml
+5
-0
ee/spec/bin/sidekiq_cluster_spec.rb
ee/spec/bin/sidekiq_cluster_spec.rb
+2
-2
lib/gitlab/sidekiq_config.rb
lib/gitlab/sidekiq_config.rb
+2
-2
No files found.
doc/administration/operations/extra_sidekiq_processes.md
View file @
7a19879d
...
@@ -93,11 +93,11 @@ This tells the additional processes how often to check for enqueued jobs.
...
@@ -93,11 +93,11 @@ This tells the additional processes how often to check for enqueued jobs.
## Starting extra processes via command line
## Starting extra processes via command line
Starting extra Sidekiq processes can be done using the command
Starting extra Sidekiq processes can be done using the command
`/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster`
. This command
`/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster`
. This command
takes arguments using the following syntax:
takes arguments using the following syntax:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster
[
QUEUE,QUEUE,...]
[
QUEUE, ...]
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster
[
QUEUE,QUEUE,...]
[
QUEUE, ...]
```
```
Each separate argument denotes a group of queues that have to be processed by a
Each separate argument denotes a group of queues that have to be processed by a
...
@@ -115,14 +115,14 @@ For example, say you want to start 2 extra processes: one to process the
...
@@ -115,14 +115,14 @@ For example, say you want to start 2 extra processes: one to process the
done as follows:
done as follows:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster process_commit post_receive
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster process_commit post_receive
```
```
If you instead want to start one process processing both queues you'd use the
If you instead want to start one process processing both queues you'd use the
following syntax:
following syntax:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster process_commit,post_receive
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster process_commit,post_receive
```
```
If you want to have one Sidekiq process process the "process_commit" and
If you want to have one Sidekiq process process the "process_commit" and
...
@@ -130,7 +130,7 @@ If you want to have one Sidekiq process process the "process_commit" and
...
@@ -130,7 +130,7 @@ If you want to have one Sidekiq process process the "process_commit" and
you'd use the following:
you'd use the following:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster process_commit,post_receive gitlab_shell
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster process_commit,post_receive gitlab_shell
```
```
### Monitoring
### Monitoring
...
@@ -162,7 +162,7 @@ file is written, but this can be changed by passing the `--pidfile` option to
...
@@ -162,7 +162,7 @@ file is written, but this can be changed by passing the `--pidfile` option to
`sidekiq-cluster`
. For example:
`sidekiq-cluster`
. For example:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster
--pidfile
/var/run/gitlab/sidekiq_cluster.pid process_commit
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster
--pidfile
/var/run/gitlab/sidekiq_cluster.pid process_commit
```
```
Keep in mind that the PID file will contain the PID of the
`sidekiq-cluster`
Keep in mind that the PID file will contain the PID of the
`sidekiq-cluster`
...
@@ -199,7 +199,7 @@ one thread per queue up to a maximum of 50. If you wish to change the cap, use
...
@@ -199,7 +199,7 @@ one thread per queue up to a maximum of 50. If you wish to change the cap, use
the
`-m N`
option. For example, this would cap the maximum number of threads to 1:
the
`-m N`
option. For example, this would cap the maximum number of threads to 1:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster process_commit,post_receive
-m
1
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster process_commit,post_receive
-m
1
```
```
For each queue group, the concurrency factor will be set to min(number of
For each queue group, the concurrency factor will be set to min(number of
...
...
This diff is collapsed.
Click to expand it.
bin/sidekiq-cluster
→
ee/
bin/sidekiq-cluster
View file @
7a19879d
#!/usr/bin/env ruby
#!/usr/bin/env ruby
# frozen_string_literal: true
require
'optparse'
require
'optparse'
require_relative
'../lib/gitlab/sidekiq_config'
require_relative
'../
../
lib/gitlab/sidekiq_config'
require_relative
'../
ee/
lib/gitlab/sidekiq_cluster'
require_relative
'../lib/gitlab/sidekiq_cluster'
require_relative
'../
ee/
lib/gitlab/sidekiq_cluster/cli'
require_relative
'../lib/gitlab/sidekiq_cluster/cli'
Thread
.
abort_on_exception
=
true
Thread
.
abort_on_exception
=
true
...
...
This diff is collapsed.
Click to expand it.
ee/changelogs/unreleased/6565-move-sidekiq-cluster.yml
0 → 100644
View file @
7a19879d
---
title
:
Move sidekiq-cluster to ee/bin
merge_request
:
11001
author
:
type
:
other
This diff is collapsed.
Click to expand it.
ee/spec/bin/sidekiq_cluster_spec.rb
View file @
7a19879d
require
'spec_helper'
require
'spec_helper'
describe
'bin/sidekiq-cluster'
do
describe
'
ee/
bin/sidekiq-cluster'
do
it
'runs successfully'
,
:aggregate_failures
do
it
'runs successfully'
,
:aggregate_failures
do
cmd
=
%w[bin/sidekiq-cluster --dryrun --negate cronjob]
cmd
=
%w[
ee/
bin/sidekiq-cluster --dryrun --negate cronjob]
output
,
status
=
Gitlab
::
Popen
.
popen
(
cmd
,
Rails
.
root
.
to_s
)
output
,
status
=
Gitlab
::
Popen
.
popen
(
cmd
,
Rails
.
root
.
to_s
)
...
...
This diff is collapsed.
Click to expand it.
lib/gitlab/sidekiq_config.rb
View file @
7a19879d
...
@@ -7,7 +7,7 @@ module Gitlab
...
@@ -7,7 +7,7 @@ module Gitlab
module
SidekiqConfig
module
SidekiqConfig
QUEUE_CONFIG_PATHS
=
%w[app/workers/all_queues.yml ee/app/workers/all_queues.yml]
.
freeze
QUEUE_CONFIG_PATHS
=
%w[app/workers/all_queues.yml ee/app/workers/all_queues.yml]
.
freeze
# This method is called by `bin/sidekiq-cluster` in EE, which runs outside
# This method is called by `
ee/
bin/sidekiq-cluster` in EE, which runs outside
# of bundler/Rails context, so we cannot use any gem or Rails methods.
# of bundler/Rails context, so we cannot use any gem or Rails methods.
def
self
.
worker_queues
(
rails_path
=
Rails
.
root
.
to_s
)
def
self
.
worker_queues
(
rails_path
=
Rails
.
root
.
to_s
)
@worker_queues
||=
{}
@worker_queues
||=
{}
...
@@ -19,7 +19,7 @@ module Gitlab
...
@@ -19,7 +19,7 @@ module Gitlab
end
end
end
end
# This method is called by `bin/sidekiq-cluster` in EE, which runs outside
# This method is called by `
ee/
bin/sidekiq-cluster` in EE, which runs outside
# of bundler/Rails context, so we cannot use any gem or Rails methods.
# of bundler/Rails context, so we cannot use any gem or Rails methods.
def
self
.
expand_queues
(
queues
,
all_queues
=
self
.
worker_queues
)
def
self
.
expand_queues
(
queues
,
all_queues
=
self
.
worker_queues
)
return
[]
if
queues
.
empty?
return
[]
if
queues
.
empty?
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment