Skip to content

G
gitlab-ce
Commit 9621ace1 authored by GitLab Bot's avatar GitLab Bot
Browse files
Options

Automatic merge of gitlab-org/gitlab-ce master

parents 69860f22 13ace389
Hide whitespace changes
Inline Side-by-side
Showing with 354 additions and 147 deletions
doc/administration/auth/google_secure_ldap.md 0 → 100644
View file @ 9621ace1
# Google Secure LDAP **[CORE ONLY]**
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/46391) in GitLab 11.9.
[Google Cloud Identity](https://cloud.google.com/identity/) provides a Secure
LDAP service that can be configured with GitLab for authentication and group sync.
Secure LDAP requires a slightly different configuration than standard LDAP servers.
The steps below cover:
- Configuring the Secure LDAP Client in the Google Admin console.
- Required GitLab configuration.
## Configuring Google LDAP client
1. Navigate to https://admin.google.com and sign in as a GSuite domain administrator.
1. Go to **Apps > LDAP > Add Client**.
1. Provide an `LDAP client name` and an optional `Description`. Any descriptive
values are acceptable. For example, the name could be 'GitLab' and the
description could be 'GitLab LDAP Client'. Click the **Continue** button.
![Add LDAP Client Step 1](img/google_secure_ldap_add_step_1.png)
1. Set **Access Permission** according to your needs. You must choose either
'Entire domain (GitLab)' or 'Selected organizational units' for both 'Verify user
credentials' and 'Read user information'. Select 'Add LDAP Client'
TIP: **Tip:** If you plan to use GitLab [LDAP Group Sync](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)
, turn on 'Read group information'.
![Add LDAP Client Step 2](img/google_secure_ldap_add_step_2.png)
1. Download the generated certificate. This is required for GitLab to
communicate with the Google Secure LDAP service. Save the downloaded certificates
for later use. After downloading, click the **Continue to Client Details** button.
1. Expand the **Service Status** section and turn the LDAP client 'ON for everyone'.
After selecting 'Save', click on the 'Service Status' bar again to collapse
and return to the rest of the settings.
1. Expand the **Authentication** section and choose 'Generate New Credentials'.
Copy/note these credentials for later use. After selecting 'Close', click
on the 'Authentication' bar again to collapse and return to the rest of the settings.
Now the Google Secure LDAP Client configuration is finished. The screenshot below
shows an example of the final settings. Continue on to configure GitLab.
![LDAP Client Settings](img/google_secure_ldap_client_settings.png)
## Configuring GitLab
Edit GitLab configuration, inserting the access credentials and certificate
obtained earlier.
The following are the configuration keys that need to be modified using the
values obtained during the LDAP client configuration earlier:
- `bind_dn`: The access credentials username
- `password`: The access credentials password
- `cert`: The `.crt` file text from the downloaded certificate bundle
- `key`: The `.key` file text from the downloaded certificate bundle
**For Omnibus installations**
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below
main: # 'main' is the GitLab 'provider ID' of this LDAP server
label: 'Google Secure LDAP'
host: 'ldap.google.com'
port: 636
uid: 'uid'
bind_dn: 'DizzyHorse'
password: 'd6V5H8nhMUW9AuDP25abXeLd'
encryption: 'simple_tls'
verify_certificates: true
tls_options:
cert: |
-----BEGIN CERTIFICATE-----
MIIDbDCCAlSgAwIBAgIGAWlzxiIfMA0GCSqGSIb3DQEBCwUAMHcxFDASBgNVBAoTC0dvb2dsZSBJ
bmMuMRYwFAYDVQQHEw1Nb3VudGFpbiBWaWV3MRQwEgYDVQQDEwtMREFQIENsaWVudDEPMA0GA1UE
CxMGR1N1aXRlMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTAeFw0xOTAzMTIyMTE5
MThaFw0yMjAzMTEyMTE5MThaMHcxFDASBgNVBAoTC0dvb2dsZSBJbmMuMRYwFAYDVQQHEw1Nb3Vu
dGFpbiBWaWV3MRQwEgYDVQQDEwtMREFQIENsaWVudDEPMA0GA1UECxMGR1N1aXRlMQswCQYDVQQG
EwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
ALOTy4aC38dyjESk6N8fRsKk8DN23ZX/GaNFL5OUmmA1KWzrvVC881OzNdtGm3vNOIxr9clteEG/
tQwsmsJvQT5U+GkBt+tGKF/zm7zueHUYqTP7Pg5pxAnAei90qkIRFi17ulObyRHPYv1BbCt8pxNB
4fG/gAXkFbCNxwh1eiQXXRTfruasCZ4/mHfX7MVm8JmWU9uAVIOLW+DSWOFhrDQduJdGBXJOyC2r
Gqoeg9+tkBmNH/jjxpnEkFW8q7io9DdOUqqNgoidA1h9vpKTs3084sy2DOgUvKN9uXWx14uxIyYU
Y1DnDy0wczcsuRt7l+EgtCEgpsLiLJQbKW+JS1UCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAf60J
yazhbHkDKIH2gFxfm7QLhhnqsmafvl4WP7JqZt0u0KdnvbDPfokdkM87yfbKJU1MTI86M36wEC+1
P6bzklKz7kXbzAD4GggksAzxsEE64OWHC+Y64Tkxq2NiZTw/76POkcg9StiIXjG0ZcebHub9+Ux/
rTncip92nDuvgEM7lbPFKRIS/YMhLCk09B/U0F6XLsf1yYjyf5miUTDikPkov23b/YGfpc8kh6hq
1kqdi6a1cYPP34eAhtRhMqcZU9qezpJF6s9EeN/3YFfKzLODFSsVToBRAdZgGHzj//SAtLyQTD4n
KCSvK1UmaMxNaZyTHg8JnMf0ZuRpv26iSg==
-----END CERTIFICATE-----
key: |
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCzk8uGgt/HcoxEpOjfH0bCpPAz
dt2V/xmjRS+TlJpgNSls671QvPNTszXbRpt7zTiMa/XJbXhBv7UMLJrCb0E+VPhpAbfrRihf85u8
7nh1GKkz+z4OacQJwHovdKpCERYte7pTm8kRz2L9QWwrfKcTQeHxv4AF5BWwjccIdXokF10U367m
rAmeP5h31+zFZvCZllPbgFSDi1vg0ljhYaw0HbiXRgVyTsgtqxqqHoPfrZAZjR/448aZxJBVvKu4
qPQ3TlKqjYKInQNYfb6Sk7N9POLMtgzoFLyjfbl1sdeLsSMmFGNQ5w8tMHM3LLkbe5fhILQhIKbC
4iyUGylviUtVAgMBAAECggEAIPb0CQy0RJoX+q/lGbRVmnyJpYDf+115WNnl+mrwjdGkeZyqw4v0
BPzkWYzUFP1esJRO6buBNFybQRFdFW0z5lvVv/zzRKq71aVUBPInxaMRyHuJ8D5lIL8nDtgVOwyE
7DOGyDtURUMzMjdUwoTe7K+O6QBU4X/1pVPZYgmissYSMmt68LiP8k0p601F4+r5xOi/QEy44aVp
aOJZBUOisKB8BmUXZqmQ4Cy05vU9Xi1rLyzkn9s7fxnZ+JO6Sd1r0Thm1mE0yuPgxkDBh/b4f3/2
GsQNKKKCiij/6TfkjnBi8ZvWR44LnKpu760g/K7psVNrKwqJG6C/8RAcgISWQQKBgQDop7BaKGhK
1QMJJ/vnlyYFTucfGLn6bM//pzTys5Gop0tpcfX/Hf6a6Dd+zBhmC3tBmhr80XOX/PiyAIbc0lOI
31rafZuD/oVx5mlIySWX35EqS14LXmdVs/5vOhsInNgNiE+EPFf1L9YZgG/zA7OUBmqtTeYIPDVC
7ViJcydItQKBgQDFmK0H0IA6W4opGQo+zQKhefooqZ+RDk9IIZMPOAtnvOM7y3rSVrfsSjzYVuMS
w/RP/vs7rwhaZejnCZ8/7uIqwg4sdUBRzZYR3PRNFeheW+BPZvb+2keRCGzOs7xkbF1mu54qtYTa
HZGZj1OsD83AoMwVLcdLDgO1kw32dkS8IQKBgFRdgoifAHqqVah7VFB9se7Y1tyi5cXWsXI+Wufr
j9U9nQ4GojK52LqpnH4hWnOelDqMvF6TQTyLIk/B+yWWK26Ft/dk9wDdSdystd8L+dLh4k0Y+Whb
+lLMq2YABw+PeJUnqdYE38xsZVHoDjBsVjFGRmbDybeQxauYT7PACy3FAoGBAK2+k9bdNQMbXp7I
j8OszHVkJdz/WXlY1cmdDAxDwXOUGVKIlxTAf7TbiijILZ5gg0Cb+hj+zR9/oI0WXtr+mAv02jWp
W8cSOLS4TnBBpTLjIpdu+BwbnvYeLF6MmEjNKEufCXKQbaLEgTQ/XNlchBSuzwSIXkbWqdhM1+gx
EjtBAoGARAdMIiDMPWIIZg3nNnFebbmtBP0qiBsYohQZ+6i/8s/vautEHBEN6Q0brIU/goo+nTHc
t9VaOkzjCmAJSLPUanuBC8pdYgLu5J20NXUZLD9AE/2bBT3OpezKcdYeI2jqoc1qlWHlNtVtdqQ2
AcZSFJQjdg5BTyvdEDhaYUKGdRw=
-----END PRIVATE KEY-----
EOS
```
1. Save the file and [reconfigure] GitLab for the changes to take effect.
---
**For installations from source**
1. Edit `config/gitlab.yml`:
```yaml
ldap:
enabled: true
servers:
main: # 'main' is the GitLab 'provider ID' of this LDAP server
label: 'Google Secure LDAP'
host: 'ldap.google.com'
port: 636
uid: 'uid'
bind_dn: 'DizzyHorse'
password: 'd6V5H8nhMUW9AuDP25abXeLd'
encryption: 'simple_tls'
verify_certificates: true
tls_options:
cert: |
-----BEGIN CERTIFICATE-----
MIIDbDCCAlSgAwIBAgIGAWlzxiIfMA0GCSqGSIb3DQEBCwUAMHcxFDASBgNVBAoTC0dvb2dsZSBJ
bmMuMRYwFAYDVQQHEw1Nb3VudGFpbiBWaWV3MRQwEgYDVQQDEwtMREFQIENsaWVudDEPMA0GA1UE
CxMGR1N1aXRlMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTAeFw0xOTAzMTIyMTE5
MThaFw0yMjAzMTEyMTE5MThaMHcxFDASBgNVBAoTC0dvb2dsZSBJbmMuMRYwFAYDVQQHEw1Nb3Vu
dGFpbiBWaWV3MRQwEgYDVQQDEwtMREFQIENsaWVudDEPMA0GA1UECxMGR1N1aXRlMQswCQYDVQQG
EwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
ALOTy4aC38dyjESk6N8fRsKk8DN23ZX/GaNFL5OUmmA1KWzrvVC881OzNdtGm3vNOIxr9clteEG/
tQwsmsJvQT5U+GkBt+tGKF/zm7zueHUYqTP7Pg5pxAnAei90qkIRFi17ulObyRHPYv1BbCt8pxNB
4fG/gAXkFbCNxwh1eiQXXRTfruasCZ4/mHfX7MVm8JmWU9uAVIOLW+DSWOFhrDQduJdGBXJOyC2r
Gqoeg9+tkBmNH/jjxpnEkFW8q7io9DdOUqqNgoidA1h9vpKTs3084sy2DOgUvKN9uXWx14uxIyYU
Y1DnDy0wczcsuRt7l+EgtCEgpsLiLJQbKW+JS1UCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAf60J
yazhbHkDKIH2gFxfm7QLhhnqsmafvl4WP7JqZt0u0KdnvbDPfokdkM87yfbKJU1MTI86M36wEC+1
P6bzklKz7kXbzAD4GggksAzxsEE64OWHC+Y64Tkxq2NiZTw/76POkcg9StiIXjG0ZcebHub9+Ux/
rTncip92nDuvgEM7lbPFKRIS/YMhLCk09B/U0F6XLsf1yYjyf5miUTDikPkov23b/YGfpc8kh6hq
1kqdi6a1cYPP34eAhtRhMqcZU9qezpJF6s9EeN/3YFfKzLODFSsVToBRAdZgGHzj//SAtLyQTD4n
KCSvK1UmaMxNaZyTHg8JnMf0ZuRpv26iSg==
-----END CERTIFICATE-----
key: |
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCzk8uGgt/HcoxEpOjfH0bCpPAz
dt2V/xmjRS+TlJpgNSls671QvPNTszXbRpt7zTiMa/XJbXhBv7UMLJrCb0E+VPhpAbfrRihf85u8
7nh1GKkz+z4OacQJwHovdKpCERYte7pTm8kRz2L9QWwrfKcTQeHxv4AF5BWwjccIdXokF10U367m
rAmeP5h31+zFZvCZllPbgFSDi1vg0ljhYaw0HbiXRgVyTsgtqxqqHoPfrZAZjR/448aZxJBVvKu4
qPQ3TlKqjYKInQNYfb6Sk7N9POLMtgzoFLyjfbl1sdeLsSMmFGNQ5w8tMHM3LLkbe5fhILQhIKbC
4iyUGylviUtVAgMBAAECggEAIPb0CQy0RJoX+q/lGbRVmnyJpYDf+115WNnl+mrwjdGkeZyqw4v0
BPzkWYzUFP1esJRO6buBNFybQRFdFW0z5lvVv/zzRKq71aVUBPInxaMRyHuJ8D5lIL8nDtgVOwyE
7DOGyDtURUMzMjdUwoTe7K+O6QBU4X/1pVPZYgmissYSMmt68LiP8k0p601F4+r5xOi/QEy44aVp
aOJZBUOisKB8BmUXZqmQ4Cy05vU9Xi1rLyzkn9s7fxnZ+JO6Sd1r0Thm1mE0yuPgxkDBh/b4f3/2
GsQNKKKCiij/6TfkjnBi8ZvWR44LnKpu760g/K7psVNrKwqJG6C/8RAcgISWQQKBgQDop7BaKGhK
1QMJJ/vnlyYFTucfGLn6bM//pzTys5Gop0tpcfX/Hf6a6Dd+zBhmC3tBmhr80XOX/PiyAIbc0lOI
31rafZuD/oVx5mlIySWX35EqS14LXmdVs/5vOhsInNgNiE+EPFf1L9YZgG/zA7OUBmqtTeYIPDVC
7ViJcydItQKBgQDFmK0H0IA6W4opGQo+zQKhefooqZ+RDk9IIZMPOAtnvOM7y3rSVrfsSjzYVuMS
w/RP/vs7rwhaZejnCZ8/7uIqwg4sdUBRzZYR3PRNFeheW+BPZvb+2keRCGzOs7xkbF1mu54qtYTa
HZGZj1OsD83AoMwVLcdLDgO1kw32dkS8IQKBgFRdgoifAHqqVah7VFB9se7Y1tyi5cXWsXI+Wufr
j9U9nQ4GojK52LqpnH4hWnOelDqMvF6TQTyLIk/B+yWWK26Ft/dk9wDdSdystd8L+dLh4k0Y+Whb
+lLMq2YABw+PeJUnqdYE38xsZVHoDjBsVjFGRmbDybeQxauYT7PACy3FAoGBAK2+k9bdNQMbXp7I
j8OszHVkJdz/WXlY1cmdDAxDwXOUGVKIlxTAf7TbiijILZ5gg0Cb+hj+zR9/oI0WXtr+mAv02jWp
W8cSOLS4TnBBpTLjIpdu+BwbnvYeLF6MmEjNKEufCXKQbaLEgTQ/XNlchBSuzwSIXkbWqdhM1+gx
EjtBAoGARAdMIiDMPWIIZg3nNnFebbmtBP0qiBsYohQZ+6i/8s/vautEHBEN6Q0brIU/goo+nTHc
t9VaOkzjCmAJSLPUanuBC8pdYgLu5J20NXUZLD9AE/2bBT3OpezKcdYeI2jqoc1qlWHlNtVtdqQ2
AcZSFJQjdg5BTyvdEDhaYUKGdRw=
-----END PRIVATE KEY-----
```
1. Save the file and [restart] GitLab for the changes to take effect.
[reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure
[restart]: ../restart_gitlab.md#installations-from-source
doc/administration/auth/ldap.md
View file @ 9621ace1
...@@ -48,6 +48,14 @@ LDAP-enabled users can always authenticate with Git using their GitLab username ...@@ -48,6 +48,14 @@ LDAP-enabled users can always authenticate with Git using their GitLab username
or email and LDAP password, even if password authentication for Git is disabled or email and LDAP password, even if password authentication for Git is disabled
in the application settings. in the application settings.
## Google Secure LDAP **[CORE ONLY]**
> Introduced in GitLab 11.9.
[Google Cloud Identity](https://cloud.google.com/identity/) provides a Secure
LDAP service that can be configured with GitLab for authentication and group sync.
See [Google Secure LDAP](google_secure_ldap.md) for detailed configuration instructions.
## Configuration ## Configuration
NOTE: **Note**: NOTE: **Note**:
......
doc/update/mysql_to_postgresql.md
View file @ 9621ace1
--- ---
last_updated: 2018-02-07 last_updated: 2019-03-27
--- ---
# Migrating from MySQL to PostgreSQL # Migrating from MySQL to PostgreSQL
> **Note:** This guide assumes you have a working GitLab instance with This guide documents how to take a working GitLab instance that uses MySQL and
> MySQL and want to migrate to bundled PostgreSQL database. migrate it to a PostgreSQL database.
## Omnibus installation ## Requirements
### Prerequisites [pgloader](http://pgloader.io) 3.4.1+ is required.
First, we'll need to enable the bundled PostgreSQL database with up-to-date You can install it directly from your distribution, for example in
schema. Next, we'll use [pgloader](http://pgloader.io) to migrate the data Debian/Ubuntu:
from the old MySQL database to the new PostgreSQL one.
Here's what you'll need to have installed: 1. Search for the version:
- pgloader 3.4.1+ ```bash
- Omnibus GitLab apt-cache madison pgloader
- MySQL ```
### Enable bundled PostgreSQL database 1. If the version is 3.4.1+, install it with:
```bash
sudo apt-get install pgloader
```
If your distribution's version is too old, use PostgreSQL's repository:
```bash
# Add repository
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
# Add key
sudo apt-get install wget ca-certificates
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
# Install package
sudo apt-get update
sudo apt-get install pgloader
```
For other distributions, follow the instructions in PostrgreSQL's
[download page](https://www.postgresql.org/download/) to add their repository
and then install `pgloader`.
## Omnibus GitLab installations
For [Omnibus GitLab packages](https://about.gitlab.com/install/), you'll first
need to enable the bundled PostgreSQL:
1. Stop GitLab: 1. Stop GitLab:
``` bash ```bash
sudo gitlab-ctl stop sudo gitlab-ctl stop
``` ```
...@@ -40,39 +67,34 @@ Here's what you'll need to have installed: ...@@ -40,39 +67,34 @@ Here's what you'll need to have installed:
and alike. You could just comment all of them out so that we'll just use and alike. You could just comment all of them out so that we'll just use
the defaults. the defaults.
1. [Reconfigure GitLab] for the changes to take effect: 1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure)
for the changes to take effect.
``` bash
sudo gitlab-ctl reconfigure
```
1. Start Unicorn and PostgreSQL so that we can prepare the schema: 1. Start Unicorn and PostgreSQL so that we can prepare the schema:
``` bash ```bash
sudo gitlab-ctl start unicorn sudo gitlab-ctl start unicorn
sudo gitlab-ctl start postgresql sudo gitlab-ctl start postgresql
``` ```
1. Run the following commands to prepare the schema: 1. Run the following commands to prepare the schema:
``` bash ```bash
sudo gitlab-rake db:create db:migrate sudo gitlab-rake db:create db:migrate
``` ```
1. Stop Unicorn to prevent other database access from interfering with the loading of data: 1. Stop Unicorn to prevent other database access from interfering with the loading of data:
``` bash ```bash
sudo gitlab-ctl stop unicorn sudo gitlab-ctl stop unicorn
``` ```
After these steps, you'll have a fresh PostgreSQL database with up-to-date schema. After these steps, you'll have a fresh PostgreSQL database with up-to-date schema.
### Migrate data from MySQL to PostgreSQL Next, we'll use `pgloader` to migrate the data from the old MySQL database to the
new PostgreSQL one:
Now, you can use pgloader to migrate the data from MySQL to PostgreSQL:
1. Save the following snippet in a `commands.load` file, and edit with your 1. Save the following snippet in a `commands.load` file, and edit with your
database `username`, `password` and `host`: MySQL database `username`, `password` and `host`:
``` ```
LOAD DATABASE LOAD DATABASE
...@@ -90,7 +112,7 @@ Now, you can use pgloader to migrate the data from MySQL to PostgreSQL: ...@@ -90,7 +112,7 @@ Now, you can use pgloader to migrate the data from MySQL to PostgreSQL:
1. Start the migration: 1. Start the migration:
``` bash ```bash
sudo -u gitlab-psql pgloader commands.load sudo -u gitlab-psql pgloader commands.load
``` ```
...@@ -117,170 +139,140 @@ Now, you can use pgloader to migrate the data from MySQL to PostgreSQL: ...@@ -117,170 +139,140 @@ Now, you can use pgloader to migrate the data from MySQL to PostgreSQL:
Total import time 1894 1894 0 12.497s Total import time 1894 1894 0 12.497s
``` ```
If there is no output for more than 30 minutes, it's possible pgloader encountered an error. See If there is no output for more than 30 minutes, it's possible `pgloader` encountered an error. See
the [troubleshooting guide](#Troubleshooting) for more details. the [troubleshooting guide](#troubleshooting) for more details.
1. Start GitLab: 1. Start GitLab:
``` bash ```bash
sudo gitlab-ctl start sudo gitlab-ctl start
``` ```
Now, you can verify that everything worked by visiting GitLab. You can now verify that everything works as expected by visiting GitLab.
### Troubleshooting
#### Permissions
Note that the PostgreSQL user that you use for the above MUST have **superuser** privileges. Otherwise, you may see
a similar message to the following:
```
debugger invoked on a CL-POSTGRES-ERROR:INSUFFICIENT-PRIVILEGE in thread
#<THREAD "lparallel" RUNNING {10078A3513}>:
Database error 42501: permission denied: "RI_ConstraintTrigger_a_20937" is a system trigger
QUERY: ALTER TABLE ci_builds DISABLE TRIGGER ALL;
2017-08-23T00:36:56.782000Z ERROR Database error 42501: permission denied: "RI_ConstraintTrigger_c_20864" is a system trigger
QUERY: ALTER TABLE approver_groups DISABLE TRIGGER ALL;
```
#### Experiencing 500 errors after the migration
If you experience 500 errors after the migration, try to clear the cache:
``` bash
sudo gitlab-rake cache:clear
```
[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
## Source installation
### Prerequisites ## Source installations
#### Install PostgreSQL and create database For installations from source that use MySQL, you'll first need to
[install PostgreSQL and create a database](../install/installation.md#6-database).
See [installation guide](../install/installation.md#6-database). After the database is created, go on with the following steps:
#### Install [pgloader](http://pgloader.io) 3.4.1+
Install directly from your distro:
``` bash
sudo apt-get install pgloader
```
If this version is too old, use PostgreSQL's repository:
``` bash
# add repository
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
# add key
sudo apt-get install wget ca-certificates
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
# install package
sudo apt-get update
sudo apt-get install pgloader
```
### Enable bundled PostgreSQL database
1. Stop GitLab: 1. Stop GitLab:
``` bash ```bash
sudo service gitlab stop sudo service gitlab stop
``` ```
1. Switch database from MySQL to PostgreSQL 1. Switch database from MySQL to PostgreSQL
``` bash ```bash
cd /home/git/gitlab cd /home/git/gitlab
sudo -u git mv config/database.yml config/database.yml.bak sudo -u git mv config/database.yml config/database.yml.bak
sudo -u git cp config/database.yml.postgresql config/database.yml sudo -u git cp config/database.yml.postgresql config/database.yml
sudo -u git -H chmod o-rwx config/database.yml sudo -u git -H chmod o-rwx config/database.yml
``` ```
1. Install Gems related to Postgresql 1. Install Gems related to Postgresql
``` bash ```bash
sudo -u git -H rm .bundle/config sudo -u git -H rm .bundle/config
sudo -u git -H bundle install --deployment --without development test mysql aws kerberos sudo -u git -H bundle install --deployment --without development test mysql aws kerberos
``` ```
1. Run the following commands to prepare the schema: 1. Run the following commands to prepare the schema:
``` bash ```bash
sudo -u git -H bundle exec rake db:create db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake db:create db:migrate RAILS_ENV=production
``` ```
After these steps, you'll have a fresh PostgreSQL database with up-to-date schema. After these steps, you'll have a fresh PostgreSQL database with up-to-date schema.
### Migrate data from MySQL to PostgreSQL Next, we'll use `pgloader` to migrate the data from the old MySQL database to the
new PostgreSQL one:
Now, you can use pgloader to migrate the data from MySQL to PostgreSQL:
1. Save the following snippet in a `commands.load` file, and edit with your 1. Save the following snippet in a `commands.load` file, and edit with your
MySQL `username`, `password` and `host`: MySQL `username`, `password` and `host`:
``` ```
LOAD DATABASE LOAD DATABASE
FROM mysql://username:password@host/gitlabhq_production FROM mysql://username:password@host/gitlabhq_production
INTO postgresql://postgres@unix://var/run/postgresql:/gitlabhq_production INTO postgresql://postgres@unix://var/run/postgresql:/gitlabhq_production
WITH include no drop, truncate, disable triggers, create no tables, WITH include no drop, truncate, disable triggers, create no tables,
create no indexes, preserve index names, no foreign keys, create no indexes, preserve index names, no foreign keys,
data only data only
ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public' ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public'
; ;
``` ```
1. Start the migration: 1. Start the migration:
``` bash ```bash
sudo -u postgres pgloader commands.load sudo -u postgres pgloader commands.load
``` ```
1. Once the migration finishes, you should see a summary table that looks like 1. Once the migration finishes, you should see a summary table that looks like
the following: the following:
``` ```
table name read imported errors total time table name read imported errors total time
----------------------------------------------- --------- --------- --------- -------------- ----------------------------------------------- --------- --------- --------- --------------
fetch meta data 119 119 0 0.388s fetch meta data 119 119 0 0.388s
Truncate 119 119 0 1.134s Truncate 119 119 0 1.134s
----------------------------------------------- --------- --------- --------- -------------- ----------------------------------------------- --------- --------- --------- --------------
public.abuse_reports 0 0 0 0.490s public.abuse_reports 0 0 0 0.490s
public.appearances 0 0 0 0.488s public.appearances 0 0 0 0.488s
. .
. .
. .
public.web_hook_logs 0 0 0 1.080s public.web_hook_logs 0 0 0 1.080s
----------------------------------------------- --------- --------- --------- -------------- ----------------------------------------------- --------- --------- --------- --------------
COPY Threads Completion 4 4 0 2.008s COPY Threads Completion 4 4 0 2.008s
Reset Sequences 113 113 0 0.304s Reset Sequences 113 113 0 0.304s
Install Comments 0 0 0 0.000s Install Comments 0 0 0 0.000s
----------------------------------------------- --------- --------- --------- -------------- ----------------------------------------------- --------- --------- --------- --------------
Total import time 1894 1894 0 12.497s Total import time 1894 1894 0 12.497s
``` ```
If there is no output for more than 30 minutes, it's possible pgloader encountered an error. See If there is no output for more than 30 minutes, it's possible `pgloader` encountered an error. See
the [troubleshooting guide](#Troubleshooting) for more details. the [troubleshooting guide](#troubleshooting) for more details.
1. Start GitLab: 1. Start GitLab:
``` bash ```bash
sudo service gitlab start sudo service gitlab start
``` ```
You can now verify that everything works as expected by visiting GitLab.
## Troubleshooting
Sometimes, you might encounter some errors during or after the migration.
Now, you can verify that everything worked by visiting GitLab. ### Database error permission denied
### Troubleshooting The PostgreSQL user that you use for the migration MUST have **superuser** privileges.
Otherwise, you may see a similar message to the following:
#### Experiencing 500 errors after the migration ```
debugger invoked on a CL-POSTGRES-ERROR:INSUFFICIENT-PRIVILEGE in thread
#<THREAD "lparallel" RUNNING {10078A3513}>:
Database error 42501: permission denied: "RI_ConstraintTrigger_a_20937" is a system trigger
QUERY: ALTER TABLE ci_builds DISABLE TRIGGER ALL;
2017-08-23T00:36:56.782000Z ERROR Database error 42501: permission denied: "RI_ConstraintTrigger_c_20864" is a system trigger
QUERY: ALTER TABLE approver_groups DISABLE TRIGGER ALL;
```
### Experiencing 500 errors after the migration
If you experience 500 errors after the migration, try to clear the cache: If you experience 500 errors after the migration, try to clear the cache:
``` bash ```bash
# Omnibus GitLab
sudo gitlab-rake cache:clear
# Installations from source
sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production
``` ```
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7