@@ -5,27 +5,27 @@ info: To determine the technical writer assigned to the Stage/Group associated w
...
@@ -5,27 +5,27 @@ info: To determine the technical writer assigned to the Stage/Group associated w
type:howto
type:howto
---
---
# Bring a demoted primary node back online **(PREMIUM SELF)**
# Bring a demoted primary site back online **(PREMIUM SELF)**
After a failover, it is possible to fail back to the demoted **primary**node to
After a failover, it is possible to fail back to the demoted **primary**site to
restore your original configuration. This process consists of two steps:
restore your original configuration. This process consists of two steps:
1. Making the old **primary**node a **secondary** node.
1. Making the old **primary**site a **secondary** site.
1. Promoting a **secondary**node to a **primary** node.
1. Promoting a **secondary**site to a **primary** site.
WARNING:
WARNING:
If you have any doubts about the consistency of the data on this node, we recommend setting it up from scratch.
If you have any doubts about the consistency of the data on this site, we recommend setting it up from scratch.
## Configure the former **primary** node to be a **secondary** node
## Configure the former **primary** site to be a **secondary** site
Since the former **primary**node will be out of sync with the current **primary** node, the first step is to bring the former **primary** node up to date. Note, deletion of data stored on disk like
Since the former **primary**site will be out of sync with the current **primary** site, the first step is to bring the former **primary** site up to date. Note, deletion of data stored on disk like
repositories and uploads will not be replayed when bringing the former **primary**node back
repositories and uploads will not be replayed when bringing the former **primary**site back
into sync, which may result in increased disk usage.
into sync, which may result in increased disk usage.
Alternatively, you can [set up a new **secondary** GitLab instance](../setup/index.md) to avoid this.
Alternatively, you can [set up a new **secondary** GitLab instance](../setup/index.md) to avoid this.
To bring the former **primary**node up to date:
To bring the former **primary**site up to date:
1. SSH into the former **primary**node that has fallen behind.
1. SSH into the former **primary**site that has fallen behind.
1. Make sure all the services are up:
1. Make sure all the services are up:
```shell
```shell
...
@@ -33,36 +33,36 @@ To bring the former **primary** node up to date:
...
@@ -33,36 +33,36 @@ To bring the former **primary** node up to date:
```
```
NOTE:
NOTE:
If you [disabled the **primary** node permanently](index.md#step-2-permanently-disable-the-primary-node),
If you [disabled the **primary** site permanently](index.md#step-2-permanently-disable-the-primary-site),
you need to undo those steps now. For Debian/Ubuntu you just need to run
you need to undo those steps now. For Debian/Ubuntu you just need to run
`sudo systemctl enable gitlab-runsvdir`. For CentOS 6, you need to install
`sudo systemctl enable gitlab-runsvdir`. For CentOS 6, you need to install
the GitLab instance from scratch and set it up as a **secondary**node by
the GitLab instance from scratch and set it up as a **secondary**site by
following [Setup instructions](../setup/index.md). In this case, you don't need to follow the next step.
following [Setup instructions](../setup/index.md). In this case, you don't need to follow the next step.
NOTE:
NOTE:
If you [changed the DNS records](index.md#step-4-optional-updating-the-primary-domain-dns-record)
If you [changed the DNS records](index.md#step-4-optional-updating-the-primary-domain-dns-record)
for this node during disaster recovery procedure you may need to [block
for this site during disaster recovery procedure you may need to [block
all the writes to this node](planned_failover.md#prevent-updates-to-the-primary-node)
all the writes to this site](planned_failover.md#prevent-updates-to-the-primary-node)
during this procedure.
during this procedure.
1.[Set up database replication](../setup/database.md). In this case, the **secondary**node
1.[Set up database replication](../setup/database.md). In this case, the **secondary**site
refers to the former **primary**node.
refers to the former **primary**site.
1. If [PgBouncer](../../postgresql/pgbouncer.md) was enabled on the **current secondary**node
1. If [PgBouncer](../../postgresql/pgbouncer.md) was enabled on the **current secondary**site
(when it was a primary node) disable it by editing `/etc/gitlab/gitlab.rb`
(when it was a primary site) disable it by editing `/etc/gitlab/gitlab.rb`
and running `sudo gitlab-ctl reconfigure`.
and running `sudo gitlab-ctl reconfigure`.
1. You can then set up database replication on the **secondary**node.
1. You can then set up database replication on the **secondary**site.
If you have lost your original **primary**node, follow the
If you have lost your original **primary**site, follow the
[setup instructions](../setup/index.md) to set up a new **secondary**node.
[setup instructions](../setup/index.md) to set up a new **secondary**site.
## Promote the **secondary** node to **primary** node
## Promote the **secondary** site to **primary** site
When the initial replication is complete and the **primary**node and **secondary** node are
When the initial replication is complete and the **primary**site and **secondary** site are
closely in sync, you can do a [planned failover](planned_failover.md).
closely in sync, you can do a [planned failover](planned_failover.md).
## Restore the **secondary** node
## Restore the **secondary** site
If your objective is to have two nodes again, you need to bring your **secondary**
If your objective is to have two sites again, you need to bring your **secondary**
node back online as well by repeating the first step
site back online as well by repeating the first step
([configure the former **primary** node to be a **secondary** node](#configure-the-former-primary-node-to-be-a-secondary-node))
([configure the former **primary** site to be a **secondary** site](#configure-the-former-primary-site-to-be-a-secondary-site))
@@ -683,7 +683,7 @@ when promoting a secondary to a primary node with strategies to resolve them.
...
@@ -683,7 +683,7 @@ when promoting a secondary to a primary node with strategies to resolve them.
### Message: ActiveRecord::RecordInvalid: Validation failed: Name has already been taken
### Message: ActiveRecord::RecordInvalid: Validation failed: Name has already been taken
When [promoting a **secondary** node](../disaster_recovery/index.md#step-3-promoting-a-secondary-node),
When [promoting a **secondary** site](../disaster_recovery/index.md#step-3-promoting-a-secondary-site),
you might encounter the following error:
you might encounter the following error:
```plaintext
```plaintext
...
@@ -751,7 +751,7 @@ This can be fixed in the database.
...
@@ -751,7 +751,7 @@ This can be fixed in the database.
### Message: ``NoMethodError: undefined method `secondary?' for nil:NilClass``
### Message: ``NoMethodError: undefined method `secondary?' for nil:NilClass``
When [promoting a **secondary** node](../disaster_recovery/index.md#step-3-promoting-a-secondary-node),
When [promoting a **secondary** site](../disaster_recovery/index.md#step-3-promoting-a-secondary-site),
you might encounter the following error:
you might encounter the following error:
```plaintext
```plaintext
...
@@ -767,13 +767,13 @@ Tasks: TOP => geo:set_secondary_as_primary
...
@@ -767,13 +767,13 @@ Tasks: TOP => geo:set_secondary_as_primary
(See full trace by running task with --trace)
(See full trace by running task with --trace)
```
```
This command is intended to be executed on a secondary node only, and this error
This command is intended to be executed on a secondary site only, and this error
is displayed if you attempt to run this command on a primary node.
is displayed if you attempt to run this command on a primary site.
### Message: `sudo: gitlab-pg-ctl: command not found`
### Message: `sudo: gitlab-pg-ctl: command not found`
When
When
[promoting a **secondary** node with multiple servers](../disaster_recovery/index.md#promoting-a-secondary-node-with-multiple-servers),
[promoting a **secondary** site with multiple nodes](../disaster_recovery/index.md#promoting-a-secondary-site-with-multiple-nodes-running-gitlab-144-and-earlier),
you need to run the `gitlab-pg-ctl` command to promote the PostgreSQL
you need to run the `gitlab-pg-ctl` command to promote the PostgreSQL