Split Geo node configuration for Omnibus/source

[ci skip]

Split Geo node configuration for Omnibus/source
[ci skip]
a8e6ebd9 · Achilleas Pipinellis · 54abfccb · a8e6ebd9 · a8e6ebd9
Commit a8e6ebd9 authored Mar 16, 2017 by Achilleas Pipinellis
Show whitespace changes
Inline Side-by-side

Showing with 32 additions and 142 deletions

doc/gitlab-geo/configuration.md doc/gitlab-geo/configuration.md +15 -31

doc/gitlab-geo/configuration_source.md doc/gitlab-geo/configuration_source.md +17 -111

No files found.
--- a/doc/gitlab-geo/configuration.md
+++ b/doc/gitlab-geo/configuration.md
 # GitLab Geo configuration
+>**Note:**
+This is the documentation for the Omnibus GitLab packages. For installations
+from source, follow the [**GitLab Geo nodes configuration for installations
+from source**](configuration_source.md) guide.
 This is the final step you need to follow in order to setup a Geo node.
+You are encouraged to first read through all the steps before executing them
+in your testing/production environment.
 ## Setting up GitLab
 >**Notes:**
- Don't setup any custom authentication in the secondary nodes, this will be
+- **Do not** setup any custom authentication in the secondary nodes, this will be
  handled by the primary node.
- Do not add anything in the secondaries Geo nodes admin area
+- **Do not** add anything in the secondaries Geo nodes admin area
  (**Admin Area ➔ Geo Nodes**). This is handled solely by the primary node.
 After having installed GitLab Enterprise Edition in the instance that will serve
@@ -56,9 +64,6 @@ logins opened on all nodes as we will be moving back and forth.
    ```
    # Omnibus GitLab installations
    sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
-    # Installations from source
-    sudo -u git cat /home/git/.ssh/id_rsa.pub
    ```
 1. Visit the primary node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`) in
@@ -95,9 +100,6 @@ logins opened on all nodes as we will be moving back and forth.
    ```
    # Omnibus GitLab installations
    cat /var/opt/gitlab/.ssh/known_hosts
-    # Installations from source
-    cat /home/git/.ssh/known_hosts
    ```
 ### Step 3. Copying the database encryption key
@@ -117,9 +119,6 @@ sensitive data in the database. Any secondary node must have the
     ```
     # Omnibus GitLab installations
     cat /etc/gitlab/gitlab-secrets.json | grep db_key_base
-     # Installations from source
-     cat /home/git/gitlab/config/secrets.yml | grep db_key_base
     ```
 1. SSH into the **secondary** node and login as root:
@@ -134,9 +133,6 @@ sensitive data in the database. Any secondary node must have the
     ```
     # Omnibus GitLab installations
     editor /etc/gitlab/gitlab-secrets.json
-     # Installations from source
-     editor /home/git/gitlab/config/secrets.yml
     ```
 1. Save and close the file.
@@ -163,9 +159,6 @@ sensitive data in the database. Any secondary node must have the
    ```
    # Omnibus installations
    sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
-    # Installations from source
-    sudo -u git cat /home/git/.ssh/id_rsa.pub
    ```
 1. Visit the **primary** node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`)
@@ -216,10 +209,6 @@ connection between the servers.
    # For Omnibus installations
    rsync -guavrP root@1.2.3.4:/var/opt/gitlab/git-data/repositories/ /var/opt/gitlab/git-data/repositories/
    gitlab-ctl reconfigure # to fix directory permissions
-    # For installations from source
-    rsync -guavrP root@1.2.3.4:/home/git/repositories/ /home/git/repositories/
-    chmod ug+rwX,o-rwx /home/git/repositories
    ```
 If this step is not followed, the secondary node will eventually clone and
@@ -241,9 +230,6 @@ run:
 ```
 # For Omnibus installations
 gitlab-rake gitlab:shell:setup
-# For source installations
-sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production
 ```
 This will enable `git` operations to authorize against your existing users.
@@ -293,6 +279,11 @@ Host example.com                    # The FQDN of the primary Geo node
 ## Troubleshooting
+>**Note:**
+This list is an attempt to document all the moving parts that can go wrong.
+We are working into getting all this steps verified automatically in a
+rake task in the future.
 Setting up Geo requires careful attention to details and sometimes it's easy to
 miss a step. Here is a checklist of questions you should ask to try to detect
 where you have to fix (all commands and path locations are for Omnibus installs):
@@ -345,10 +336,3 @@ where you have to fix (all commands and path locations are for Omnibus installs)
 - Can secondary nodes communicate with primary node by HTTP/HTTPS/SSH ports?
 - Can secondary nodes execute a successful git clone using git user's own
  SSH Key to primary node repository?
->**Note:**
-This list is an attempt to document all the moving parts that can go wrong.
-We are working into getting all this steps verified automatically in a
-rake task in the future.
-[ssh-pair]: #create-ssh-key-pairs-for-new-geo-nodes
--- a/doc/gitlab-geo/configuration_source.md
+++ b/doc/gitlab-geo/configuration_source.md
 # GitLab Geo configuration
+>**Note:**
+This is the documentation for installations from source. For installations
+using the Omnibus GitLab packages, follow the
+[**Omnibus GitLab Geo nodes configuration**](configuration.md) guide.
 This is the final step you need to follow in order to setup a Geo node.
+You are encouraged to first read through all the steps before executing them
+in your testing/production environment.
 ## Setting up GitLab
 >**Notes:**
@@ -11,8 +19,8 @@ This is the final step you need to follow in order to setup a Geo node.
  (**Admin Area ➔ Geo Nodes**). This is handled solely by the primary node.
 After having installed GitLab Enterprise Edition in the instance that will serve
-as a Geo node and set up the [database replication](database.md), the next steps
+as a Geo node and set up the [database replication](database_source.md), the
-can be summed up to:
+next steps can be summed up to:
 1. Configure the primary node
 1. Replicate some required configurations between the primary and the secondaries
@@ -54,9 +62,6 @@ logins opened on all nodes as we will be moving back and forth.
 1. Get the contents of `id_rsa.pub` for the git user:
    ```
-    # Omnibus GitLab installations
-    sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
    # Installations from source
    sudo -u git cat /home/git/.ssh/id_rsa.pub
    ```
@@ -93,9 +98,6 @@ logins opened on all nodes as we will be moving back and forth.
 1. Verify that the fingerprint was added by checking `known_hosts`:
    ```
-    # Omnibus GitLab installations
-    cat /var/opt/gitlab/.ssh/known_hosts
    # Installations from source
    cat /home/git/.ssh/known_hosts
    ```
@@ -115,9 +117,6 @@ sensitive data in the database. Any secondary node must have the
 1. Find the value of `db_key_base` and copy it:
     ```
-     # Omnibus GitLab installations
-     cat /etc/gitlab/gitlab-secrets.json | grep db_key_base
     # Installations from source
     cat /home/git/gitlab/config/secrets.yml | grep db_key_base
     ```
@@ -132,9 +131,6 @@ sensitive data in the database. Any secondary node must have the
   previous step:
     ```
-     # Omnibus GitLab installations
-     editor /etc/gitlab/gitlab-secrets.json
     # Installations from source
     editor /home/git/gitlab/config/secrets.yml
     ```
@@ -161,9 +157,6 @@ sensitive data in the database. Any secondary node must have the
 1. Get the contents of `id_rsa.pub` the was just created:
    ```
-    # Omnibus installations
-    sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
    # Installations from source
    sudo -u git cat /home/git/.ssh/id_rsa.pub
    ```
@@ -186,7 +179,7 @@ The two most obvious issues that replication can have here are:
 1. Instance to instance notification not working. In that case, it can be
   something of the following:
     - You are using a custom certificate or custom CA (see the
-       [Troubleshooting](#troubleshooting) section)
+       [Troubleshooting](configuration.md#troubleshooting) section)
     - Instance is firewalled (check your firewall rules)
 ### Step 5. Replicating the repositories data
@@ -213,11 +206,7 @@ connection between the servers.
   to start the sync:
    ```bash
-    # For Omnibus installations
+    # Installations from source
-    rsync -guavrP root@1.2.3.4:/var/opt/gitlab/git-data/repositories/ /var/opt/gitlab/git-data/repositories/
-    gitlab-ctl reconfigure # to fix directory permissions
-    # For installations from source
    rsync -guavrP root@1.2.3.4:/home/git/repositories/ /home/git/repositories/
    chmod ug+rwX,o-rwx /home/git/repositories
    ```
@@ -239,10 +228,7 @@ On the **secondary** node where the database is [already replicated](./database.
 run:
 ```
-# For Omnibus installations
+# Installations from source
-gitlab-rake gitlab:shell:setup
-# For source installations
 sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production
 ```
@@ -257,7 +243,7 @@ with the same credentials as used in the primary. Visit the secondary node's
 correctly identified as a secondary Geo node and if Geo is enabled.
 If your installation isn't working properly, check the
-[troubleshooting](#troubleshooting) section.
+[troubleshooting](configuration.md#troubleshooting) section.
 ## Adding another secondary Geo node
@@ -267,88 +253,8 @@ Just omit the first step that sets up the primary node.
 ## Additional information for the SSH key pairs
-When adding a new Geo node, you must provide an SSH public key of the user that
+Read [Additional information for the SSH key pairs](configuration.md#additional-information-for-the-ssh-key-pairs).
-your GitLab instance runs on (unless changed, should be the user `git`). This
-user will act as a "normal user" who fetches from the primary Geo node.
-If for any reason you generate the key using a different name from the default
-`id_rsa`, or you want to generate an extra key only for the repository
-synchronization feature, you can do so, but you have to create/modify your
-`~/.ssh/config` (for the `git` user).
-This is an example on how to change the default key for all remote hosts:
-```bash
-Host *                              # Match all remote hosts
-  IdentityFile ~/.ssh/mycustom.key  # The location of your private key
-```
-This is how to change it for an specific host:
-```bash
-Host example.com                    # The FQDN of the primary Geo node
-  HostName example.com              # The FQDN of the primary Geo node
-  IdentityFile ~/.ssh/mycustom.key  # The location of your private key
-```
 ## Troubleshooting
-Setting up Geo requires careful attention to details and sometimes it's easy to
+Read the [Omnibus troubleshooting section](configuration.md#troubleshooting).
-miss a step. Here is a checklist of questions you should ask to try to detect
-where you have to fix (all commands and path locations are for Omnibus installs):
- Is Postgres replication working?
- Are my nodes pointing to the correct database instance?
-    - You should make sure your primary Geo node points to the instance with
-      writing permissions.
-    - Any secondary nodes should point only to read-only instances.
- Can Geo detect my current node correctly?
-    - Geo uses your defined node from `Admin ➔ Geo` screen, and tries to match
-      with the value defined in `/etc/gitlab/gitlab.rb` configuration file.
-      The relevant line looks like: `external_url "http://gitlab.example.com"`.
-    - To check if node on current machine is correctly detected type:
-        ```
-        sudo gitlab-rails runner "puts Gitlab::Geo.current_node.inspect"
-        ```
-        and expect something like:
-        ```
-        #<GeoNode id: 2, schema: "https", host: "gitlab.example.com", port: 443, relative_url_root: "", primary: false, ...>
-        ```
-    - By running the command above, `primary` should be `true` when executed in
-      the primary node, and `false` on any secondary
- Did I define the correct SSH Key for the node?
-    - You must create an SSH Key for `git` user
-    - This key is the one you have to inform at `Admin > Geo`
- Can I SSH from secondary to primary node using `git` user account?
-    - This is the most obvious cause of problems with repository replication issues.
-      If you haven't added the primary node's key to `known_hosts`, you will end up with
-      a lot of failed sidekiq jobs with an error similar to:
-        ```
-        Gitlab::Shell::Error: Host key verification failed. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
-        ```
-        An easy way to fix is by logging in as the `git` user in the secondary node and run:
-        ```
-        # remove old entries to your primary gitlab in known_hosts
-        ssh-keyscan -R your-primary-gitlab.example.com
-        # add a new entry in known_hosts
-        ssh-keyscan -t rsa your-primary-gitlab.example.com >> ~/.ssh/known_hosts
-        ```
- Can primary node communicate with secondary node by HTTP/HTTPS ports?
- Can secondary nodes communicate with primary node by HTTP/HTTPS/SSH ports?
- Can secondary nodes execute a successful git clone using git user's own
-  SSH Key to primary node repository?
->**Note:**
-This list is an attempt to document all the moving parts that can go wrong.
-We are working into getting all this steps verified automatically in a
-rake task in the future.
-[ssh-pair]: #create-ssh-key-pairs-for-new-geo-nodes