Commit 423f3b3c authored by Sean McGivern's avatar Sean McGivern Committed by Mike Lewis

Prefer memory bound to CPU bound if a worker is both

parent 9dfd885b
...@@ -167,7 +167,7 @@ Most workers tend to spend most of their time blocked, wait on network responses ...@@ -167,7 +167,7 @@ Most workers tend to spend most of their time blocked, wait on network responses
from other services such as Redis, Postgres and Gitaly. Since Sidekiq is a from other services such as Redis, Postgres and Gitaly. Since Sidekiq is a
multithreaded environment, these jobs can be scheduled with high concurrency. multithreaded environment, these jobs can be scheduled with high concurrency.
Some workers, however, spend large amounts of time _on-cpu_ running logic in Some workers, however, spend large amounts of time _on-CPU_ running logic in
Ruby. Ruby MRI does not support true multithreading - it relies on the Ruby. Ruby MRI does not support true multithreading - it relies on the
[GIL](https://thoughtbot.com/blog/untangling-ruby-threads#the-global-interpreter-lock) [GIL](https://thoughtbot.com/blog/untangling-ruby-threads#the-global-interpreter-lock)
to greatly simplify application development by only allowing one section of Ruby to greatly simplify application development by only allowing one section of Ruby
...@@ -187,13 +187,17 @@ performance. ...@@ -187,13 +187,17 @@ performance.
Likewise, if a worker uses large amounts of memory, we can run these on a Likewise, if a worker uses large amounts of memory, we can run these on a
bespoke low concurrency, high memory fleet. bespoke low concurrency, high memory fleet.
Note that Memory-bound workers create heavy GC workloads, with pauses of Note that memory-bound workers create heavy GC workloads, with pauses of
10-50ms. This will have an impact on the latency requirements for the 10-50ms. This will have an impact on the latency requirements for the
worker. For this reason, `memory` bound, `latency_sensitive` jobs are not worker. For this reason, `memory` bound, `latency_sensitive` jobs are not
permitted and will fail CI. In general, `memory` bound workers are permitted and will fail CI. In general, `memory` bound workers are
discouraged, and alternative approaches to processing the work should be discouraged, and alternative approaches to processing the work should be
considered. considered.
If a worker needs large amounts of both memory and CPU time, it should be marked as
memory-bound, due to the above restrction on latency-sensitive memory-bound
workers.
## Declaring a Job as CPU-bound ## Declaring a Job as CPU-bound
This example shows how to declare a job as being CPU-bound. This example shows how to declare a job as being CPU-bound.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment