Significantly cut memory usage and SQL queries when reloading diffs

By preloading certain models with the diff, we can eliminate many N+1
queries. For a push to the staging GitLab.com www-gitlab-com repository,
this eliminates over 3000 SQL queries and appears to bring down the RSS
usage by several gigabytes.

Relates to https://gitlab.com/gitlab-org/gitlab-ce/issues/49703

app/services/merge_requests/reload_diffs_service.rb

@@ -36,7 +36,10 @@ module MergeRequests
       # Remove cache for all diffs on this MR. Do not use the association on the
       # model, as that will interfere with other actions happening when
       # reloading the diff.
-      MergeRequestDiff.where(merge_request: merge_request).each do |merge_request_diff|
+      MergeRequestDiff
+        .where(merge_request: merge_request)
+        .preload(:merge_request_diff_files, merge_request: :target_project)
+        .find_each do |merge_request_diff|
         next if merge_request_diff == new_diff
 
         cacheable_collection(merge_request_diff).clear_cache

changelogs/unreleased/sh-optimize-reload-diffs-service.yml

+---
+title: Significantly cut memory usage and SQL queries when reloading diffs
+merge_request: 22725
+author:
+type: performance

spec/services/merge_requests/reload_diffs_service_spec.rb

@@ -60,6 +60,17 @@ describe MergeRequests::ReloadDiffsService, :use_clean_rails_memory_store_cachin
 
         subject.execute
       end
+
+      it 'avoids N+1 queries', :request_store do
+        current_user
+        merge_request
+
+        control_count = ActiveRecord::QueryRecorder.new do
+          subject.execute
+        end.count
+
+        expect { subject.execute }.not_to exceed_query_limit(control_count)
+      end
     end
   end
 end