Commit 324b3bba authored by Stan Hu's avatar Stan Hu

Memoize GitLab logger to reduce open file descriptors

We see that in gitlab-org/gitlab-ee#3664 that if we log a lot of
data in Sidekiq workers, the number of open file descriptors
reaches over 1000. To avoid this, we can memoize the logger per
thread via RequestStore.

Closes gitlab-org/gitlab-ee#3664
parent 82446a2b
---
title: Memoize GitLab logger to reduce open file descriptors
merge_request:
author:
type: fixed
...@@ -13,7 +13,7 @@ module Gitlab ...@@ -13,7 +13,7 @@ module Gitlab
end end
def self.read_latest def self.read_latest
path = Rails.root.join("log", file_name) path = self.full_log_path
return [] unless File.readable?(path) return [] unless File.readable?(path)
...@@ -22,7 +22,15 @@ module Gitlab ...@@ -22,7 +22,15 @@ module Gitlab
end end
def self.build def self.build
new(Rails.root.join("log", file_name)) RequestStore[self.cache_key] ||= new(self.full_log_path)
end
def self.full_log_path
Rails.root.join("log", file_name)
end
def self.cache_key
'logger:'.freeze + self.full_log_path.to_s
end end
end end
end end
require 'spec_helper'
describe Gitlab::AppLogger, :request_store do
subject { described_class }
it 'builds a logger once' do
expect(::Logger).to receive(:new).and_call_original
subject.info('hello world')
subject.error('hello again')
end
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment