Commit 86c77ebb authored by Nick Thomas's avatar Nick Thomas

Refuse to perform an LFS clean on projects that are fork roots

When a project is at the root of a fork network, an intersection of
technical debt means that all child forks have a claim on the LFS
objects owned by the fork parent. This makes it dangerous to remove
any LFS references in the fork root without checking all the repos
for all the children, which quickly mounts up so as to be unmanageable.

For now, just skip projects like this.

If a fork is an internal or leaf node in the fork graph, they may have
some LFS objects of their own. Those references are safe to prune.
parent fa5ac55c
---
title: Refuse to perform an LFS clean on projects that are fork roots
merge_request: 41703
author:
type: fixed
......@@ -10,6 +10,10 @@ DANGER: **Danger:**
Do not run this within 12 hours of a GitLab upgrade. This is to ensure that all background migrations
have finished, which otherwise may lead to data loss.
CAUTION: **WARNING:**
Removing LFS files from a project with forks is currently unsafe. The rake task
will refuse to run on projects with forks.
When you remove LFS files from a repository's history, they become orphaned and continue to consume
disk space. With this Rake task, you can remove invalid references from the database, which
will allow garbage collection of LFS files.
......
......@@ -17,6 +17,14 @@ module Gitlab
end
def run!
# If this project is an LFS storage project (e.g. is the root of a fork
# network), what it is safe to remove depends on the sum of its forks.
# For now, skip cleaning up LFS for this complicated case
if project.forks_count > 0 && project.lfs_storage_project == project
log_info("Skipping orphan LFS check for #{project.name_with_namespace} as it is a fork root")
return
end
log_info("Looking for orphan LFS files for project #{project.name_with_namespace}")
remove_orphan_references
......
......@@ -3,6 +3,8 @@
require 'spec_helper'
RSpec.describe Gitlab::Cleanup::OrphanLfsFileReferences do
include ProjectForksHelper
let(:null_logger) { Logger.new('/dev/null') }
let(:project) { create(:project, :repository, lfs_enabled: true) }
let(:lfs_object) { create(:lfs_object) }
......@@ -85,4 +87,42 @@ RSpec.describe Gitlab::Cleanup::OrphanLfsFileReferences do
.to receive(:get_all_lfs_pointers)
.and_return(oids.map { |oid| OpenStruct.new(lfs_oid: oid) })
end
context 'LFS for forked projects' do
let!(:fork_root) { create(:project, :repository, lfs_enabled: true) }
let!(:fork_internal) { fork_project(fork_root, nil, repository: true) }
let!(:fork_leaf) { fork_project(fork_internal, nil, repository: true) }
let(:dry_run) { true }
context 'root node' do
let(:project) { fork_root }
it 'skips cleanup' do
expect(service).not_to receive(:remove_orphan_references)
service.run!
end
end
context 'internal node' do
let(:project) { fork_internal }
it 'runs cleanup' do
expect(service).to receive(:remove_orphan_references)
service.run!
end
end
context 'leaf node' do
let(:project) { fork_leaf }
it 'runs cleanup' do
expect(service).to receive(:remove_orphan_references)
service.run!
end
end
end
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment