Commit 31d6a1f8 authored by Evan Read's avatar Evan Read

Merge branch 'product-2645-workaround-to-import-too-large-repo' into 'master'

Import troubleshooting: Document workaround for a too large repo

See merge request gitlab-org/gitlab!67593
parents bb1c4569 04a51300
...@@ -49,6 +49,7 @@ ...@@ -49,6 +49,7 @@
"Geo", "Geo",
"Git LFS", "Git LFS",
"git-annex", "git-annex",
"git-sizer",
"Git", "Git",
"Gitaly", "Gitaly",
"GitHub", "GitHub",
......
...@@ -174,18 +174,18 @@ the operation you want to perform in each commit. To do so, you need to edit ...@@ -174,18 +174,18 @@ the operation you want to perform in each commit. To do so, you need to edit
the commits in your terminal's text editor. the commits in your terminal's text editor.
For example, if you're using [Vim](https://www.vim.org/) as the text editor in For example, if you're using [Vim](https://www.vim.org/) as the text editor in
a macOS's `ZSH` shell, and you want to **squash** all the three commits a macOS's `ZSH` shell, and you want to `squash` or `fixup` all the three commits
(join them into one): (join them into one):
1. Press <!-- vale gitlab.FirstPerson = NO --> <kbd>i</kbd> <!-- vale gitlab.FirstPerson = YES --> 1. Press <!-- vale gitlab.FirstPerson = NO --> <kbd>i</kbd> <!-- vale gitlab.FirstPerson = YES -->
on your keyboard to switch to Vim's editing mode. on your keyboard to switch to Vim's editing mode.
1. Navigate with your keyboard arrows to edit the **second** commit keyword 1. Navigate with your keyboard arrows to edit the **second** commit keyword
from `pick` to `squash` (or `s`). Do the same to the **third** commit. from `pick` to `squash` or `fixup` (or `s` or `f`). Do the same to the **third** commit.
The first commit should be left **unchanged** (`pick`) as we want to squash The first commit should be left **unchanged** (`pick`) as we want to squash
the second and third into the first. the second and third into the first.
1. Press <kbd>Escape</kbd> to leave the editing mode. 1. Press <kbd>Escape</kbd> to leave the editing mode.
1. Type `:wq` to "write" (save) and "quit". 1. Type `:wq` to "write" (save) and "quit".
1. Git outputs the commit message so you have a chance to edit it: 1. When squashing, Git outputs the commit message so you have a chance to edit it:
- All lines starting with `#` are ignored and not included in the commit - All lines starting with `#` are ignored and not included in the commit
message. Everything else is included. message. Everything else is included.
- To leave it as it is, type `:wq`. To edit the commit message: switch to the - To leave it as it is, type `:wq`. To edit the commit message: switch to the
......
...@@ -13,6 +13,8 @@ Git repositories become larger over time. When large files are added to a Git re ...@@ -13,6 +13,8 @@ Git repositories become larger over time. When large files are added to a Git re
- They take up a large amount of storage space on the server. - They take up a large amount of storage space on the server.
- Git repository storage limits [can be reached](#storage-limits). - Git repository storage limits [can be reached](#storage-limits).
Such problems can be detected with [git-sizer](https://github.com/github/git-sizer#getting-started).
Rewriting a repository can remove unwanted history to make the repository smaller. Rewriting a repository can remove unwanted history to make the repository smaller.
We **recommend [`git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/README.md)** We **recommend [`git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/README.md)**
over [`git filter-branch`](https://git-scm.com/docs/git-filter-branch) and over [`git filter-branch`](https://git-scm.com/docs/git-filter-branch) and
......
...@@ -210,3 +210,59 @@ To help avoid abuse, by default, users are rate limited to: ...@@ -210,3 +210,59 @@ To help avoid abuse, by default, users are rate limited to:
| Import | 6 projects per minute | | Import | 6 projects per minute |
GitLab.com may have [different settings](../../gitlab_com/index.md#importexport) from the defaults. GitLab.com may have [different settings](../../gitlab_com/index.md#importexport) from the defaults.
## Troubleshooting
### Import workaround for large repositories
[Maximum import size limitations](#importing-the-project)
can prevent an import from being successful.
If changing the import limits is not possible,
the following local workflow can be used to temporarily
reduce the repository size for another import attempt.
1. Create a temporary working directory from the export:
```shell
EXPORT=<filename-without-extension>
mkdir "$EXPORT"
tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/
cd "$EXPORT"/
git clone project.bundle
# Prevent interference with recreating an importable file later
mv project.bundle ../"$EXPORT"-original.bundle
mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz
```
1. To reduce the repository size,
[identify and remove large files](../repository/reducing_the_repo_size_using_git.md)
or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase)
to reduce the number of commits.
```shell
# Reduce the .git/objects/pack/ file size
cd project
git reflog expire --expire=now --all
git gc --prune=now --aggressive
# Prepare recreating an importable file
git bundle create ../project.bundle <default-branch-name>
cd ..
mv project/ ../"$EXPORT"-project
cd ..
# Recreate an importable file
tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ .
```
1. Import this new, smaller file into GitLab.
1. In a full clone of the original repository,
use `git remote set-url origin <new-url> && git push --force --all`
to complete the import.
1. Update the imported repository's
[branch protection rules](../protected_branches.md) and
its [default branch](../repository/branches/default.md), and
delete the temporary, `smaller-…` branch, and
the local, temporary data.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment