Commit 389e6f58 authored by Andrew Newdigate's avatar Andrew Newdigate Committed by Stan Hu

Improves the performance of the Redis set cache

Testing has shown that treating membership optimistically reduces the
number of Redis calls substantially.

`include?` will only perform an EXISTS if SISMEMBER returns negative.

`fetch` will only perform an EXISTS if there are no values.

`read` uses SSCAN to avoid Redis lockup on huge sets
parent 13113802
...@@ -58,11 +58,16 @@ module Gitlab ...@@ -58,11 +58,16 @@ module Gitlab
# wrong answer. We handle that by querying the full list - which fills # wrong answer. We handle that by querying the full list - which fills
# the cache - and using it directly to answer the question. # the cache - and using it directly to answer the question.
define_method("#{name}_include?") do |value| define_method("#{name}_include?") do |value|
if strong_memoized?(name) || !redis_set_cache.exist?(name) return __send__(name).include?(value) if strong_memoized?(name) # rubocop:disable GitlabSecurity/PublicSend
return __send__(name).include?(value) # rubocop:disable GitlabSecurity/PublicSend
end # If the member exists in the set, return as such early.
return true if redis_set_cache.include?(name, value)
# If it did not, make sure the collection exists.
# If the collection exists, then item does not.
return false if redis_set_cache.exist?(name)
redis_set_cache.include?(name, value) __send__(name).include?(value) # rubocop:disable GitlabSecurity/PublicSend
end end
end end
......
...@@ -25,7 +25,7 @@ module Gitlab ...@@ -25,7 +25,7 @@ module Gitlab
end end
def read(key) def read(key)
with { |redis| redis.smembers(cache_key(key)) } with { |redis| redis.sscan_each(cache_key(key)).to_a }
end end
def write(key, value) def write(key, value)
...@@ -47,11 +47,10 @@ module Gitlab ...@@ -47,11 +47,10 @@ module Gitlab
end end
def fetch(key, &block) def fetch(key, &block)
if exist?(key) result = read(key)
read(key) return result unless result.empty?
else
write(key, yield) write(key, yield)
end
end end
def include?(key, value) def include?(key, value)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment