Commit 960be8a1 authored by Evan Read's avatar Evan Read

Merge branch 'revise-reduce-repo-size-docs' into 'master'

Revise reduce repo size docs

See merge request gitlab-org/gitlab!35707
parents 66cd7d8f 14174667
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
GitLab provides Rake tasks for cleaning up GitLab instances. GitLab provides Rake tasks for cleaning up GitLab instances.
## Remove unreferenced LFS files from filesystem ## Remove unreferenced LFS files
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36628) in GitLab 12.10. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36628) in GitLab 12.10.
...@@ -42,7 +42,7 @@ Note that this Rake task only removes the references to LFS files. Unreferenced ...@@ -42,7 +42,7 @@ Note that this Rake task only removes the references to LFS files. Unreferenced
later (once a day). If you need to garbage collect them immediately, run later (once a day). If you need to garbage collect them immediately, run
`rake gitlab:cleanup:orphan_lfs_files` described below. `rake gitlab:cleanup:orphan_lfs_files` described below.
## Remove unreferenced LFS files ### Remove unreferenced LFS files immediately
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36628) in GitLab 12.10. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36628) in GitLab 12.10.
......
...@@ -25,6 +25,11 @@ Rewriting repository history is a destructive operation. Make sure to backup you ...@@ -25,6 +25,11 @@ Rewriting repository history is a destructive operation. Make sure to backup you
you begin. The best way back up a repository is to you begin. The best way back up a repository is to
[export the project](../settings/import_export.md#exporting-a-project-and-its-data). [export the project](../settings/import_export.md#exporting-a-project-and-its-data).
NOTE: **Note:**
Git LFS files can only be removed by an Administrator using a
[Rake task](../../../raketasks/cleanup.md). Removal of this limitation
[is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/223621).
## Purge files from repository history ## Purge files from repository history
To make cloning your project faster, rewrite branches and tags to remove unwanted files. To make cloning your project faster, rewrite branches and tags to remove unwanted files.
...@@ -40,12 +45,25 @@ To make cloning your project faster, rewrite branches and tags to remove unwante ...@@ -40,12 +45,25 @@ To make cloning your project faster, rewrite branches and tags to remove unwante
1. Using `git filter-repo`, purge any files from the history of your repository. 1. Using `git filter-repo`, purge any files from the history of your repository.
To purge all large files, the `--strip-blobs-bigger-than` option can be used: To purge large files, the `--strip-blobs-bigger-than` option can be used:
```shell ```shell
git filter-repo --strip-blobs-bigger-than 10M git filter-repo --strip-blobs-bigger-than 10M
``` ```
To purge large files stored using Git LFS, the `--blob--callback` option can
be used. The example below, uses the callback to read the file size from the
Git LFS pointer, and removes files larger than 10MB.
```shell
git filter-repo --blob-callback '
if blob.data.startswith(b"version https://git-lfs.github.com/spec/v1"):
size_in_bytes = int.from_bytes(blob.data[124:], byteorder="big")
if size_in_bytes > 10*1000:
blob.skip()
'
```
To purge specific large files by path, the `--path` and `--invert-paths` options can be combined: To purge specific large files by path, the `--path` and `--invert-paths` options can be combined:
```shell ```shell
...@@ -80,6 +98,12 @@ To make cloning your project faster, rewrite branches and tags to remove unwante ...@@ -80,6 +98,12 @@ To make cloning your project faster, rewrite branches and tags to remove unwante
[Protected tags](../protected_tags.md) will cause this to fail. To proceed, you must remove tag [Protected tags](../protected_tags.md) will cause this to fail. To proceed, you must remove tag
protection, push, and then re-enable protected tags. protection, push, and then re-enable protected tags.
1. Manually run [project housekeeping](../../../administration/housekeeping.md#manual-housekeeping)
NOTE: **Note**
Project statistics are cached for performance. You may need to wait 5-10 minutes
to see a reduction in storage utilization.
## Purge files from GitLab storage ## Purge files from GitLab storage
To reduce the size of your repository in GitLab, you must remove GitLab internal references to To reduce the size of your repository in GitLab, you must remove GitLab internal references to
...@@ -176,6 +200,7 @@ You will receive an email once it has completed. ...@@ -176,6 +200,7 @@ You will receive an email once it has completed.
When using repository cleanup, note: When using repository cleanup, note:
- Project statistics are cached. You may need to wait 5-10 minutes to see a reduction in storage utilization.
- Housekeeping prunes loose objects older than 2 weeks. This means objects added in the last 2 weeks - Housekeeping prunes loose objects older than 2 weeks. This means objects added in the last 2 weeks
will not be removed immediately. If you have access to the will not be removed immediately. If you have access to the
[Gitaly](../../../administration/gitaly/index.md) server, you may run `git gc --prune=now` to [Gitaly](../../../administration/gitaly/index.md) server, you may run `git gc --prune=now` to
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment