Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
32d2a76b
Commit
32d2a76b
authored
Sep 18, 2018
by
Balasankar "Balu" C
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Add documentation regarding rake task
parent
c4668c38
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
41 additions
and
3 deletions
+41
-3
doc/administration/raketasks/uploads/migrate.md
doc/administration/raketasks/uploads/migrate.md
+34
-3
doc/development/file_storage.md
doc/development/file_storage.md
+7
-0
No files found.
doc/administration/raketasks/uploads/migrate.md
View file @
32d2a76b
...
@@ -7,10 +7,32 @@ After [configuring the object storage](../../uploads.md#using-object-storage) fo
...
@@ -7,10 +7,32 @@ After [configuring the object storage](../../uploads.md#using-object-storage) fo
>**Note:**
>**Note:**
All of the processing will be done in a background worker and requires
**no downtime**
.
All of the processing will be done in a background worker and requires
**no downtime**
.
This tasks uses 3 parameters to find uploads to migrate.
### All-in-one rake task
GitLab provides a wrapper rake task that migrates all uploaded files - avatars,
logos, attachments, favicon, etc. - to object storage in one go. Under the hood,
it invokes individual rake tasks to migrate files falling under each of this
category one by one. The specifications of these individual rake tasks are
described in the next section.
**Omnibus Installation**
```
bash
gitlab-rake
"gitlab:uploads:migrate:all"
```
**Source Installation**
```
bash
sudo
RAILS_ENV
=
production
-u
git
-H
bundle
exec
rake gitlab:uploads:migrate:all
```
### Individual rake tasks
>**Note:**
>**Note:**
These parameters are mainly internal to GitLab's structure, you may want to refer to the task list instead below.
If you already ran the rake task mentioned above, no need to run these individual rake tasks as that has been done automatically.
The rake task uses 3 parameters to find uploads to migrate.
Parameter | Type | Description
Parameter | Type | Description
--------- | ---- | -----------
--------- | ---- | -----------
...
@@ -18,6 +40,9 @@ Parameter | Type | Description
...
@@ -18,6 +40,9 @@ Parameter | Type | Description
`model_class`
| string | Type of the model to migrate from
`model_class`
| string | Type of the model to migrate from
`mount_point`
| string/symbol | Name of the model's column on which the uploader is mounted on.
`mount_point`
| string/symbol | Name of the model's column on which the uploader is mounted on.
>**Note:**
These parameters are mainly internal to GitLab's structure, you may want to refer to the task list instead below.
This task also accepts some environment variables which you can use to override
This task also accepts some environment variables which you can use to override
certain values:
certain values:
...
@@ -25,7 +50,7 @@ Variable | Type | Description
...
@@ -25,7 +50,7 @@ Variable | Type | Description
-------- | ---- | -----------
-------- | ---- | -----------
`BATCH`
| integer | Specifies the size of the batch. Defaults to 200.
`BATCH`
| integer | Specifies the size of the batch. Defaults to 200.
**
Omnibus Installation**
**Omnibus Installation**
```
bash
```
bash
# gitlab-rake gitlab:uploads:migrate[uploader_class, model_class, mount_point]
# gitlab-rake gitlab:uploads:migrate[uploader_class, model_class, mount_point]
...
@@ -40,6 +65,9 @@ gitlab-rake "gitlab:uploads:migrate[AttachmentUploader, Note, :attachment]"
...
@@ -40,6 +65,9 @@ gitlab-rake "gitlab:uploads:migrate[AttachmentUploader, Note, :attachment]"
gitlab-rake
"gitlab:uploads:migrate[AttachmentUploader, Appearance, :logo]"
gitlab-rake
"gitlab:uploads:migrate[AttachmentUploader, Appearance, :logo]"
gitlab-rake
"gitlab:uploads:migrate[AttachmentUploader, Appearance, :header_logo]"
gitlab-rake
"gitlab:uploads:migrate[AttachmentUploader, Appearance, :header_logo]"
# Favicon
gitlab-rake
"gitlab:uploads:migrate[FaviconUploader, Appearance, :favicon]"
# Markdown
# Markdown
gitlab-rake
"gitlab:uploads:migrate[FileUploader, Project]"
gitlab-rake
"gitlab:uploads:migrate[FileUploader, Project]"
gitlab-rake
"gitlab:uploads:migrate[PersonalFileUploader, Snippet]"
gitlab-rake
"gitlab:uploads:migrate[PersonalFileUploader, Snippet]"
...
@@ -65,6 +93,9 @@ sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AttachmentUploader, Note
...
@@ -65,6 +93,9 @@ sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AttachmentUploader, Note
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[AttachmentUploader, Appearance, :logo]"
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[AttachmentUploader, Appearance, :logo]"
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[AttachmentUploader, Appearance, :header_logo]"
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[AttachmentUploader, Appearance, :header_logo]"
# Favicon
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[FaviconUploader, Appearance, :favicon]"
# Markdown
# Markdown
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[FileUploader, Project]"
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[FileUploader, Project]"
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[PersonalFileUploader, Snippet]"
sudo
-u
git
-H
bundle
exec
rake
"gitlab:uploads:migrate[PersonalFileUploader, Snippet]"
...
...
doc/development/file_storage.md
View file @
32d2a76b
...
@@ -45,6 +45,11 @@ In the case of Issues/MR/Notes Markdown attachments, there is a different approa
...
@@ -45,6 +45,11 @@ In the case of Issues/MR/Notes Markdown attachments, there is a different approa
instead of basing the path into a mutable variable
`:project_path_with_namespace`
, it's possible to use the
instead of basing the path into a mutable variable
`:project_path_with_namespace`
, it's possible to use the
hash of the project ID instead, if project migrates to the new approach (introduced in 10.2).
hash of the project ID instead, if project migrates to the new approach (introduced in 10.2).
> Note: We provide an [all-in-one rake task] to migrate all uploads to object
> storage in one go. If a new Uploader class or model type is introduced, make
> sure you add a rake task invocation corresponding to it to the [category
> list].
### Path segments
### Path segments
Files are stored at multiple locations and use different path schemes.
Files are stored at multiple locations and use different path schemes.
...
@@ -137,3 +142,5 @@ end
...
@@ -137,3 +142,5 @@ end
[
CarrierWave
]:
https://github.com/carrierwaveuploader/carrierwave
[
CarrierWave
]:
https://github.com/carrierwaveuploader/carrierwave
[
Hashed Storage
]:
../administration/repository_storage_types.md
[
Hashed Storage
]:
../administration/repository_storage_types.md
[
all-in-one rake task
]:
../administration/raketasks/uploads/migrate.md
[
category list
]:
../../lib/tasks/gitlab/uploads/migrate.rake#L6-16
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment