Skip to content

G
gitlab-ce
Commit a8e6ebd9 authored by Achilleas Pipinellis's avatar Achilleas Pipinellis
Browse files
Options

Split Geo node configuration for Omnibus/source 

[ci skip]
parent 54abfccb
Hide whitespace changes
Inline Side-by-side
Showing with 32 additions and 142 deletions
doc/gitlab-geo/configuration.md
View file @ a8e6ebd9
# GitLab Geo configuration
>**Note:**
This is the documentation for the Omnibus GitLab packages. For installations
from source, follow the [**GitLab Geo nodes configuration for installations
from source**](configuration_source.md) guide.
This is the final step you need to follow in order to setup a Geo node.
You are encouraged to first read through all the steps before executing them
in your testing/production environment.
## Setting up GitLab
>**Notes:**
- Don't setup any custom authentication in the secondary nodes, this will be
- **Do not** setup any custom authentication in the secondary nodes, this will be
handled by the primary node.
- Do not add anything in the secondaries Geo nodes admin area
- **Do not** add anything in the secondaries Geo nodes admin area
(**Admin Area ➔ Geo Nodes**). This is handled solely by the primary node.
After having installed GitLab Enterprise Edition in the instance that will serve
......@@ -56,9 +64,6 @@ logins opened on all nodes as we will be moving back and forth.
```
# Omnibus GitLab installations
sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
# Installations from source
sudo -u git cat /home/git/.ssh/id_rsa.pub
```
1. Visit the primary node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`) in
......@@ -95,9 +100,6 @@ logins opened on all nodes as we will be moving back and forth.
```
# Omnibus GitLab installations
cat /var/opt/gitlab/.ssh/known_hosts
# Installations from source
cat /home/git/.ssh/known_hosts
```
### Step 3. Copying the database encryption key
......@@ -117,9 +119,6 @@ sensitive data in the database. Any secondary node must have the
```
# Omnibus GitLab installations
cat /etc/gitlab/gitlab-secrets.json | grep db_key_base
# Installations from source
cat /home/git/gitlab/config/secrets.yml | grep db_key_base
```
1. SSH into the **secondary** node and login as root:
......@@ -134,9 +133,6 @@ sensitive data in the database. Any secondary node must have the
```
# Omnibus GitLab installations
editor /etc/gitlab/gitlab-secrets.json
# Installations from source
editor /home/git/gitlab/config/secrets.yml
```
1. Save and close the file.
......@@ -163,9 +159,6 @@ sensitive data in the database. Any secondary node must have the
```
# Omnibus installations
sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
# Installations from source
sudo -u git cat /home/git/.ssh/id_rsa.pub
```
1. Visit the **primary** node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`)
......@@ -216,10 +209,6 @@ connection between the servers.
# For Omnibus installations
rsync -guavrP root@1.2.3.4:/var/opt/gitlab/git-data/repositories/ /var/opt/gitlab/git-data/repositories/
gitlab-ctl reconfigure # to fix directory permissions
# For installations from source
rsync -guavrP root@1.2.3.4:/home/git/repositories/ /home/git/repositories/
chmod ug+rwX,o-rwx /home/git/repositories
```
If this step is not followed, the secondary node will eventually clone and
......@@ -241,9 +230,6 @@ run:
```
# For Omnibus installations
gitlab-rake gitlab:shell:setup
# For source installations
sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production
```
This will enable `git` operations to authorize against your existing users.
......@@ -293,6 +279,11 @@ Host example.com # The FQDN of the primary Geo node
## Troubleshooting
>**Note:**
This list is an attempt to document all the moving parts that can go wrong.
We are working into getting all this steps verified automatically in a
rake task in the future.
Setting up Geo requires careful attention to details and sometimes it's easy to
miss a step. Here is a checklist of questions you should ask to try to detect
where you have to fix (all commands and path locations are for Omnibus installs):
......@@ -345,10 +336,3 @@ where you have to fix (all commands and path locations are for Omnibus installs)
- Can secondary nodes communicate with primary node by HTTP/HTTPS/SSH ports?
- Can secondary nodes execute a successful git clone using git user's own
SSH Key to primary node repository?
>**Note:**
This list is an attempt to document all the moving parts that can go wrong.
We are working into getting all this steps verified automatically in a
rake task in the future.
[ssh-pair]: #create-ssh-key-pairs-for-new-geo-nodes
doc/gitlab-geo/configuration_source.md
View file @ a8e6ebd9
# GitLab Geo configuration
>**Note:**
This is the documentation for installations from source. For installations
using the Omnibus GitLab packages, follow the
[**Omnibus GitLab Geo nodes configuration**](configuration.md) guide.
This is the final step you need to follow in order to setup a Geo node.
You are encouraged to first read through all the steps before executing them
in your testing/production environment.
## Setting up GitLab
>**Notes:**
......@@ -11,8 +19,8 @@ This is the final step you need to follow in order to setup a Geo node.
(**Admin Area ➔ Geo Nodes**). This is handled solely by the primary node.
After having installed GitLab Enterprise Edition in the instance that will serve
as a Geo node and set up the [database replication](database.md), the next steps
can be summed up to:
as a Geo node and set up the [database replication](database_source.md), the
next steps can be summed up to:
1. Configure the primary node
1. Replicate some required configurations between the primary and the secondaries
......@@ -54,9 +62,6 @@ logins opened on all nodes as we will be moving back and forth.
1. Get the contents of `id_rsa.pub` for the git user:
```
# Omnibus GitLab installations
sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
# Installations from source
sudo -u git cat /home/git/.ssh/id_rsa.pub
```
......@@ -93,9 +98,6 @@ logins opened on all nodes as we will be moving back and forth.
1. Verify that the fingerprint was added by checking `known_hosts`:
```
# Omnibus GitLab installations
cat /var/opt/gitlab/.ssh/known_hosts
# Installations from source
cat /home/git/.ssh/known_hosts
```
......@@ -115,9 +117,6 @@ sensitive data in the database. Any secondary node must have the
1. Find the value of `db_key_base` and copy it:
```
# Omnibus GitLab installations
cat /etc/gitlab/gitlab-secrets.json | grep db_key_base
# Installations from source
cat /home/git/gitlab/config/secrets.yml | grep db_key_base
```
......@@ -132,9 +131,6 @@ sensitive data in the database. Any secondary node must have the
previous step:
```
# Omnibus GitLab installations
editor /etc/gitlab/gitlab-secrets.json
# Installations from source
editor /home/git/gitlab/config/secrets.yml
```
......@@ -161,9 +157,6 @@ sensitive data in the database. Any secondary node must have the
1. Get the contents of `id_rsa.pub` the was just created:
```
# Omnibus installations
sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
# Installations from source
sudo -u git cat /home/git/.ssh/id_rsa.pub
```
......@@ -186,7 +179,7 @@ The two most obvious issues that replication can have here are:
1. Instance to instance notification not working. In that case, it can be
something of the following:
- You are using a custom certificate or custom CA (see the
[Troubleshooting](#troubleshooting) section)
[Troubleshooting](configuration.md#troubleshooting) section)
- Instance is firewalled (check your firewall rules)
### Step 5. Replicating the repositories data
......@@ -195,7 +188,7 @@ Getting a new secondary Geo node up and running, will also require the
repositories directory to be synced from the primary node.
With GitLab **8.14** you can start the syncing process by clicking the
"Backfill all repositories" button on `Admin > Geo Nodes` screen.
"Backfill all repositories" button on `Admin > Geo Nodes` screen.
On previous versions, you can use `rsync` for that:
......@@ -213,11 +206,7 @@ connection between the servers.
to start the sync:
```bash
# For Omnibus installations
rsync -guavrP root@1.2.3.4:/var/opt/gitlab/git-data/repositories/ /var/opt/gitlab/git-data/repositories/
gitlab-ctl reconfigure # to fix directory permissions
# For installations from source
# Installations from source
rsync -guavrP root@1.2.3.4:/home/git/repositories/ /home/git/repositories/
chmod ug+rwX,o-rwx /home/git/repositories
```
......@@ -239,10 +228,7 @@ On the **secondary** node where the database is [already replicated](./database.
run:
```
# For Omnibus installations
gitlab-rake gitlab:shell:setup
# For source installations
# Installations from source
sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production
```
......@@ -257,7 +243,7 @@ with the same credentials as used in the primary. Visit the secondary node's
correctly identified as a secondary Geo node and if Geo is enabled.
If your installation isn't working properly, check the
[troubleshooting](#troubleshooting) section.
[troubleshooting](configuration.md#troubleshooting) section.
## Adding another secondary Geo node
......@@ -267,88 +253,8 @@ Just omit the first step that sets up the primary node.
## Additional information for the SSH key pairs
When adding a new Geo node, you must provide an SSH public key of the user that
your GitLab instance runs on (unless changed, should be the user `git`). This
user will act as a "normal user" who fetches from the primary Geo node.
If for any reason you generate the key using a different name from the default
`id_rsa`, or you want to generate an extra key only for the repository
synchronization feature, you can do so, but you have to create/modify your
`~/.ssh/config` (for the `git` user).
This is an example on how to change the default key for all remote hosts:
```bash
Host * # Match all remote hosts
IdentityFile ~/.ssh/mycustom.key # The location of your private key
```
This is how to change it for an specific host:
```bash
Host example.com # The FQDN of the primary Geo node
HostName example.com # The FQDN of the primary Geo node
IdentityFile ~/.ssh/mycustom.key # The location of your private key
```
Read [Additional information for the SSH key pairs](configuration.md#additional-information-for-the-ssh-key-pairs).
## Troubleshooting
Setting up Geo requires careful attention to details and sometimes it's easy to
miss a step. Here is a checklist of questions you should ask to try to detect
where you have to fix (all commands and path locations are for Omnibus installs):
- Is Postgres replication working?
- Are my nodes pointing to the correct database instance?
- You should make sure your primary Geo node points to the instance with
writing permissions.
- Any secondary nodes should point only to read-only instances.
- Can Geo detect my current node correctly?
- Geo uses your defined node from `Admin ➔ Geo` screen, and tries to match
with the value defined in `/etc/gitlab/gitlab.rb` configuration file.
The relevant line looks like: `external_url "http://gitlab.example.com"`.
- To check if node on current machine is correctly detected type:
```
sudo gitlab-rails runner "puts Gitlab::Geo.current_node.inspect"
```
and expect something like:
```
#<GeoNode id: 2, schema: "https", host: "gitlab.example.com", port: 443, relative_url_root: "", primary: false, ...>
```
- By running the command above, `primary` should be `true` when executed in
the primary node, and `false` on any secondary
- Did I define the correct SSH Key for the node?
- You must create an SSH Key for `git` user
- This key is the one you have to inform at `Admin > Geo`
- Can I SSH from secondary to primary node using `git` user account?
- This is the most obvious cause of problems with repository replication issues.
If you haven't added the primary node's key to `known_hosts`, you will end up with
a lot of failed sidekiq jobs with an error similar to:
```
Gitlab::Shell::Error: Host key verification failed. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
```
An easy way to fix is by logging in as the `git` user in the secondary node and run:
```
# remove old entries to your primary gitlab in known_hosts
ssh-keyscan -R your-primary-gitlab.example.com
# add a new entry in known_hosts
ssh-keyscan -t rsa your-primary-gitlab.example.com >> ~/.ssh/known_hosts
```
- Can primary node communicate with secondary node by HTTP/HTTPS ports?
- Can secondary nodes communicate with primary node by HTTP/HTTPS/SSH ports?
- Can secondary nodes execute a successful git clone using git user's own
SSH Key to primary node repository?
>**Note:**
This list is an attempt to document all the moving parts that can go wrong.
We are working into getting all this steps verified automatically in a
rake task in the future.
[ssh-pair]: #create-ssh-key-pairs-for-new-geo-nodes
Read the [Omnibus troubleshooting section](configuration.md#troubleshooting).
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7