Commit e0fda460 authored by Kerri Miller's avatar Kerri Miller Committed by Bob Van Landuyt

Move batching to oids array from AR query

Since oids can be an unbounded array, extremely large sets of oids can
cause query parsing problems in PG, leading to timeoutes, clogging logs,
etc.
parent e99ddf60
......@@ -23,7 +23,9 @@ module Projects
all_existing_objects = []
iterations = 0
LfsObject.where(oid: oids).each_batch(of: BATCH_SIZE) do |existent_lfs_objects|
oids.each_slice(BATCH_SIZE) do |oids_batch|
existent_lfs_objects = LfsObject.where(oid: oids_batch)
next unless existent_lfs_objects.any?
iterations += 1
......
......@@ -48,5 +48,15 @@ describe Projects::LfsPointers::LfsLinkService do
expect(project.all_lfs_objects.count).to eq 9
expect(linked.size).to eq 7
end
it 'only queries for the batch that will be processed', :aggregate_failures do
stub_const("#{described_class}::BATCH_SIZE", 1)
oids = %w(one two)
expect(LfsObject).to receive(:where).with(oid: %w(one)).once.and_call_original
expect(LfsObject).to receive(:where).with(oid: %w(two)).once.and_call_original
subject.execute(oids)
end
end
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment