Skip to content

G
gitlab-ce
Commit f83af0f4 authored by Marcel Amirault's avatar Marcel Amirault
Browse files
Options

Merge branch 'eread/change-reference-links-to-inline' into 'master' 

Change reference-style links to inline style

See merge request gitlab-org/gitlab!28489
parents c0d092b9 abbab31d
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 141 additions and 234 deletions
doc/administration/geo/disaster_recovery/background_verification.md
View file @ f83af0f4
......@@ -18,7 +18,7 @@ If verification succeeds on the **primary** node but fails on the **secondary**
this indicates that the object was corrupted during the replication process.
Geo actively try to correct verification failures marking the repository to
be resynced with a back-off period. If you want to reset the verification for
these failures, so you should follow [these instructions][reset-verification].
these failures, so you should follow [these instructions](background_verification.md#reset-verification-for-projects-where-verification-has-failed).
If verification is lagging significantly behind replication, consider giving
the node more time before scheduling a planned failover.
......@@ -172,8 +172,10 @@ If the **primary** and **secondary** nodes have a checksum verification mismatch
Automatic background verification doesn't cover attachments, LFS objects,
job artifacts, and user uploads in file storage. You can keep track of the
progress to include them in [ee-1430]. For now, you can verify their integrity
manually by following [these instructions][foreground-verification] on both
progress to include them in [Geo: Verify all replicated data](https://gitlab.com/groups/gitlab-org/-/epics/1430).
For now, you can verify their integrity
manually by following [these instructions](../../raketasks/check.md) on both
nodes, and comparing the output between them.
In GitLab EE 12.1, Geo calculates checksums for attachments, LFS objects, and
......@@ -184,7 +186,3 @@ been synced before GitLab EE 12.1.
Data in object storage is **not verified**, as the object store is responsible
for ensuring the integrity of the data.
[reset-verification]: background_verification.md#reset-verification-for-projects-where-verification-has-failed
[foreground-verification]: ../../raketasks/check.md
[ee-1430]: https://gitlab.com/groups/gitlab-org/-/epics/1430
doc/administration/geo/disaster_recovery/bring_primary_back.md
View file @ f83af0f4
......@@ -14,7 +14,7 @@ If you have any doubts about the consistency of the data on this node, we recomm
Since the former **primary** node will be out of sync with the current **primary** node, the first step is to bring the former **primary** node up to date. Note, deletion of data stored on disk like
repositories and uploads will not be replayed when bringing the former **primary** node back
into sync, which may result in increased disk usage.
Alternatively, you can [set up a new **secondary** GitLab instance][setup-geo] to avoid this.
Alternatively, you can [set up a new **secondary** GitLab instance](../replication/index.md#setup-instructions) to avoid this.
To bring the former **primary** node up to date:
......@@ -25,28 +25,28 @@ To bring the former **primary** node up to date:
sudo gitlab-ctl start
```
NOTE: **Note:** If you [disabled the **primary** node permanently][disaster-recovery-disable-primary],
NOTE: **Note:** If you [disabled the **primary** node permanently](index.md#step-2-permanently-disable-the-primary-node),
you need to undo those steps now. For Debian/Ubuntu you just need to run
`sudo systemctl enable gitlab-runsvdir`. For CentOS 6, you need to install
the GitLab instance from scratch and set it up as a **secondary** node by
following [Setup instructions][setup-geo]. In this case, you don't need to follow the next step.
following [Setup instructions](../replication/index.md#setup-instructions). In this case, you don't need to follow the next step.
NOTE: **Note:** If you [changed the DNS records](index.md#step-4-optional-updating-the-primary-domain-dns-record)
for this node during disaster recovery procedure you may need to [block
all the writes to this node](planned_failover.md#prevent-updates-to-the-primary-node)
during this procedure.
1. [Setup database replication][database-replication]. Note that in this
1. [Setup database replication](../replication/database.md). Note that in this
case, **primary** node refers to the current **primary** node, and **secondary** node refers to the
former **primary** node.
If you have lost your original **primary** node, follow the
[setup instructions][setup-geo] to set up a new **secondary** node.
[setup instructions](../replication/index.md#setup-instructions) to set up a new **secondary** node.
## Promote the **secondary** node to **primary** node
When the initial replication is complete and the **primary** node and **secondary** node are
closely in sync, you can do a [planned failover].
closely in sync, you can do a [planned failover](planned_failover.md).
## Restore the **secondary** node
......@@ -54,8 +54,3 @@ If your objective is to have two nodes again, you need to bring your **secondary
node back online as well by repeating the first step
([configure the former **primary** node to be a **secondary** node](#configure-the-former-primary-node-to-be-a-secondary-node))
for the **secondary** node.
[setup-geo]: ../replication/index.md#setup-instructions
[database-replication]: ../replication/database.md
[disaster-recovery-disable-primary]: index.md#step-2-permanently-disable-the-primary-node
[planned failover]: planned_failover.md
doc/administration/geo/disaster_recovery/index.md
View file @ f83af0f4
......@@ -4,11 +4,11 @@ Geo replicates your database, your Git repositories, and few other assets.
We will support and replicate more data in the future, that will enable you to
failover with minimal effort, in a disaster situation.
See [Geo current limitations][geo-limitations] for more information.
See [Geo current limitations](../replication/index.md#current-limitations) for more information.
CAUTION: **Warning:**
Disaster recovery for multi-secondary configurations is in **Alpha**.
For the latest updates, check the multi-secondary [Disaster Recovery epic][gitlab-org&65].
For the latest updates, check the multi-secondary [Disaster Recovery epic](https://gitlab.com/groups/gitlab-org/-/epics/65).
## Promoting a **secondary** Geo node in single-secondary configurations
......@@ -22,7 +22,7 @@ immediately after following these instructions.
### Step 1. Allow replication to finish if possible
If the **secondary** node is still replicating data from the **primary** node, follow
[the planned failover docs][planned-failover] as closely as possible in
[the planned failover docs](planned_failover.md) as closely as possible in
order to avoid unnecessary data loss.
### Step 2. Permanently disable the **primary** node
......@@ -235,7 +235,7 @@ secondary domain, like changing Git remotes and API URLs.
Promoting a **secondary** node to **primary** node using the process above does not enable
Geo on the new **primary** node.
To bring a new **secondary** node online, follow the [Geo setup instructions][setup-geo].
To bring a new **secondary** node online, follow the [Geo setup instructions](../replication/index.md#setup-instructions).
### Step 6. (Optional) Removing the secondary's tracking database
......@@ -284,7 +284,7 @@ and after that you also need two extra steps.
gitlab_rails['auto_migrate'] = false
```
(For more details about these settings you can read [Configure the primary server][configure-the-primary-server])
(For more details about these settings you can read [Configure the primary server](../replication/database.md#step-1-configure-the-primary-server))
1. Save the file and reconfigure GitLab for the database listen changes and
the replication slot changes to be applied.
......@@ -317,7 +317,7 @@ and after that you also need two extra steps.
### Step 2. Initiate the replication process
Now we need to make each **secondary** node listen to changes on the new **primary** node. To do that you need
to [initiate the replication process][initiate-the-replication-process] again but this time
to [initiate the replication process](../replication/database.md#step-3-initiate-the-replication-process) again but this time
for another **primary** node. All the old replication settings will be overwritten.
## Troubleshooting
......@@ -332,15 +332,6 @@ after a failover.
If you still have access to the old **primary** node, you can follow the
instructions in the
[Upgrading to GitLab 10.5][updating-geo]
[Upgrading to GitLab 10.5](../replication/version_specific_updates.md#updating-to-gitlab-105)
section to resolve the error. Otherwise, the secret is lost and you'll need to
[reset two-factor authentication for all users][sec-tfa].
[gitlab-org&65]: https://gitlab.com/groups/gitlab-org/-/epics/65
[geo-limitations]: ../replication/index.md#current-limitations
[planned-failover]: planned_failover.md
[setup-geo]: ../replication/index.md#setup-instructions
[updating-geo]: ../replication/version_specific_updates.md#updating-to-gitlab-105
[sec-tfa]: ../../../security/two_factor_authentication.md#disabling-2fa-for-everyone
[initiate-the-replication-process]: ../replication/database.html#step-3-initiate-the-replication-process
[configure-the-primary-server]: ../replication/database.html#step-1-configure-the-primary-server
[reset two-factor authentication for all users](../../../security/two_factor_authentication.md#disabling-2fa-for-everyone).
doc/administration/geo/disaster_recovery/planned_failover.md
View file @ f83af0f4
......@@ -12,7 +12,7 @@ length of this window is determined by your replication capacity - once the
data loss.
This document assumes you already have a fully configured, working Geo setup.
Please read it and the [Disaster Recovery][disaster-recovery] failover
Please read it and the [Disaster Recovery](index.md) failover
documentation in full before proceeding. Planned failover is a major operation,
and if performed incorrectly, there is a high risk of data loss. Consider
rehearsing the procedure until you are comfortable with the necessary steps and
......@@ -20,7 +20,7 @@ have a high degree of confidence in being able to perform them accurately.
## Not all data is automatically replicated
If you are using any GitLab features that Geo [doesn't support][limitations],
If you are using any GitLab features that Geo [doesn't support](../replication/index.md#current-limitations),
you must make separate provisions to ensure that the **secondary** node has an
up-to-date copy of any data associated with that feature. This may extend the
required scheduled maintenance period significantly.
......@@ -32,7 +32,7 @@ final transfer inside the maintenance window) will then transfer only the
*changes* between the **primary** node and the **secondary** nodes.
Repository-centric strategies for using `rsync` effectively can be found in the
[moving repositories][moving-repositories] documentation; these strategies can
[moving repositories](../../operations/moving_repositories.md) documentation; these strategies can
be adapted for use with any other file-based data, such as GitLab Pages (to
be found in `/var/opt/gitlab/gitlab-rails/shared/pages` if using Omnibus).
......@@ -44,12 +44,12 @@ will go smoothly.
### Object storage
If you have a large GitLab installation or cannot tolerate downtime, consider
[migrating to Object Storage][os-conf] **before** scheduling a planned failover.
[migrating to Object Storage](../replication/object_storage.md) **before** scheduling a planned failover.
Doing so reduces both the length of the maintenance window, and the risk of data
loss as a result of a poorly executed planned failover.
In GitLab 12.4, you can optionally allow GitLab to manage replication of Object Storage for
**secondary** nodes. For more information, see [Object Storage replication][os-conf].
**secondary** nodes. For more information, see [Object Storage replication](../replication/object_storage.md).
### Review the configuration of each **secondary** node
......@@ -113,7 +113,7 @@ or removing references to the missing data.
### Verify the integrity of replicated data
This [content was moved to another location][background-verification].
This [content was moved to another location](background_verification.md).
### Notify users of scheduled maintenance
......@@ -126,7 +126,7 @@ will take to finish syncing. An example message would be:
## Prevent updates to the **primary** node
Until a [read-only mode][ce-19739] is implemented, updates must be prevented
Until a [read-only mode](https://gitlab.com/gitlab-org/gitlab-foss/issues/19739) is implemented, updates must be prevented
from happening manually. Note that your **secondary** node still needs read-only
access to the **primary** node during the maintenance window.
......@@ -186,7 +186,7 @@ access to the **primary** node during the maintenance window.
1. On the **secondary** node, navigate to **{admin}** **Admin Area >** **{monitor}** **Monitoring > Background Jobs > Queues**
and wait for all the `geo` queues to drop to 0 queued and 0 running jobs.
1. On the **secondary** node, use [these instructions][foreground-verification]
1. On the **secondary** node, use [these instructions](../../raketasks/check.md)
to verify the integrity of CI artifacts, LFS objects, and uploads in file
storage.
......@@ -195,24 +195,12 @@ At this point, your **secondary** node will contain an up-to-date copy of everyt
## Promote the **secondary** node
Finally, follow the [Disaster Recovery docs][disaster-recovery] to promote the
Finally, follow the [Disaster Recovery docs](index.md) to promote the
**secondary** node to a **primary** node. This process will cause a brief outage on the **secondary** node, and users may need to log in again.
Once it is completed, the maintenance window is over! Your new **primary** node will now
begin to diverge from the old one. If problems do arise at this point, failing
back to the old **primary** node [is possible][bring-primary-back], but likely to result
back to the old **primary** node [is possible](bring_primary_back.md), but likely to result
in the loss of any data uploaded to the new **primary** in the meantime.
Don't forget to remove the broadcast message after failover is complete.
[bring-primary-back]: bring_primary_back.md
[ce-19739]: https://gitlab.com/gitlab-org/gitlab-foss/issues/19739
[container-registry]: ../replication/container_registry.md
[disaster-recovery]: index.md
[ee-4930]: https://gitlab.com/gitlab-org/gitlab/issues/4930
[ee-5064]: https://gitlab.com/gitlab-org/gitlab/issues/5064
[foreground-verification]: ../../raketasks/check.md
[background-verification]: background_verification.md
[limitations]: ../replication/index.md#current-limitations
[moving-repositories]: ../../operations/moving_repositories.md
[os-conf]: ../replication/object_storage.md
doc/administration/geo/replication/configuration.md
View file @ f83af0f4
......@@ -5,7 +5,7 @@
NOTE: **Note:**
This is the final step in setting up a **secondary** Geo node. Stages of the
setup process must be completed in the documented order.
Before attempting the steps in this stage, [complete all prior stages][setup-geo-omnibus].
Before attempting the steps in this stage, [complete all prior stages](index.md#using-omnibus-gitlab).
The basic steps of configuring a **secondary** node are to:
......@@ -77,7 +77,7 @@ they must be manually replicated to the **secondary** node.
GitLab integrates with the system-installed SSH daemon, designating a user
(typically named `git`) through which all access requests are handled.
In a [Disaster Recovery] situation, GitLab system
In a [Disaster Recovery](../disaster_recovery/index.md) situation, GitLab system
administrators will promote a **secondary** node to the **primary** node. DNS records for the
**primary** domain should also be updated to point to the new **primary** node
(previously a **secondary** node). Doing so will avoid the need to update Git remotes and API URLs.
......@@ -242,7 +242,7 @@ You can safely skip this step if your **primary** node uses a CA-issued HTTPS ce
If your **primary** node is using a self-signed certificate for *HTTPS* support, you will
need to add that certificate to the **secondary** node's trust store. Retrieve the
certificate from the **primary** node and follow
[these instructions][omnibus-ssl]
[these instructions](https://docs.gitlab.com/omnibus/settings/ssl.html)
on the **secondary** node.
### Step 6. Enable Git access over HTTP/HTTPS
......@@ -283,7 +283,7 @@ Please note that disabling a **secondary** node will stop the synchronization pr
Please note that if `git_data_dirs` is customized on the **primary** node for multiple
repository shards you must duplicate the same configuration on each **secondary** node.
Point your users to the ["Using a Geo Server" guide][using-geo].
Point your users to the ["Using a Geo Server" guide](using_a_geo_server.md).
Currently, this is what is synced:
......@@ -334,10 +334,3 @@ See the [updating the Geo nodes document](updating_the_geo_nodes.md).
## Troubleshooting
See the [troubleshooting document](troubleshooting.md).
[setup-geo-omnibus]: index.md#using-omnibus-gitlab
[Hashed Storage]: ../../repository_storage_types.md
[Disaster Recovery]: ../disaster_recovery/index.md
[gitlab-com/infrastructure#2821]: https://gitlab.com/gitlab-com/infrastructure/issues/2821
[omnibus-ssl]: https://docs.gitlab.com/omnibus/settings/ssl.html
[using-geo]: using_a_geo_server.md
doc/administration/geo/replication/database.md
View file @ f83af0f4
......@@ -8,7 +8,7 @@ configuration steps. In this case,
NOTE: **Note:**
The stages of the setup process must be completed in the documented order.
Before attempting the steps in this stage, [complete all prior stages][toc].
Before attempting the steps in this stage, [complete all prior stages](index.md#using-omnibus-gitlab).
This document describes the minimal steps you have to take in order to
replicate your **primary** GitLab database to a **secondary** node's database. You may
......@@ -27,7 +27,7 @@ NOTE: **Note:**
In database documentation, you may see "**primary**" being referenced as "master"
and "**secondary**" as either "slave" or "standby" server (read-only).
We recommend using [PostgreSQL replication slots][replication-slots-article]
We recommend using [PostgreSQL replication slots](https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75)
to ensure that the **primary** node retains all the data necessary for the **secondary** nodes to
recover. See below for more details.
......@@ -97,7 +97,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
gitlab_rails['db_password'] = '<your_password_here>'
```
1. Omnibus GitLab already has a [replication user]
1. Omnibus GitLab already has a [replication user](https://wiki.postgresql.org/wiki/Streaming_Replication)
called `gitlab_replicator`. You must set the password for this user manually.
You will be prompted to enter a password:
......@@ -280,7 +280,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
NOTE: **Note:**
This step is important so we don't try to execute anything before the node is fully configured.
1. [Check TCP connectivity][rake-maintenance] to the **primary** node's PostgreSQL server:
1. [Check TCP connectivity](../../raketasks/maintenance.md) to the **primary** node's PostgreSQL server:
```shell
gitlab-rake gitlab:tcp_check[<primary_node_ip>,5432]
......@@ -508,8 +508,3 @@ work:
## Troubleshooting
Read the [troubleshooting document](troubleshooting.md).
[replication-slots-article]: https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75
[replication user]:https://wiki.postgresql.org/wiki/Streaming_Replication
[toc]: index.md#using-omnibus-gitlab
[rake-maintenance]: ../../raketasks/maintenance.md
doc/administration/geo/replication/high_availability.md
View file @ f83af0f4
......@@ -8,7 +8,7 @@ described, it is possible to adapt these instructions to your needs.
![Geo HA Diagram](../../high_availability/img/geo-ha-diagram.png)
_[diagram source - GitLab employees only][diagram-source]_
_[diagram source - GitLab employees only](https://docs.google.com/drawings/d/1z0VlizKiLNXVVVaERFwgsIOuEgjcUqDTWPdQYsE7Z4c/edit)_
The topology above assumes that the **primary** and **secondary** Geo clusters
are located in two separate locations, on their own virtual network
......@@ -81,7 +81,7 @@ The following steps enable a GitLab cluster to serve as the **primary** node.
gitlab_rails['auto_migrate'] = false
```
After making these changes, [reconfigure GitLab][gitlab-reconfigure] so the changes take effect.
After making these changes, [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) so the changes take effect.
NOTE: **Note:** PostgreSQL and Redis should have already been disabled on the
application servers, and connections from the application servers to those
......@@ -193,7 +193,7 @@ the **primary** database. Use the following as a guide.
geo_logcursor['enable'] = false
```
After making these changes, [reconfigure GitLab][gitlab-reconfigure] so the changes take effect.
After making these changes, [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) so the changes take effect.
If using an external PostgreSQL instance, refer also to
[Geo with external PostgreSQL instances](external_database.md).
......@@ -264,7 +264,7 @@ Configure the tracking database.
unicorn['enable'] = false
```
After making these changes, [reconfigure GitLab][gitlab-reconfigure] so the changes take effect.
After making these changes, [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) so the changes take effect.
If using an external PostgreSQL instance, refer also to
[Geo with external PostgreSQL instances](external_database.md).
......@@ -342,7 +342,7 @@ servers connect to the databases.
NOTE: **Note:**
Make sure that current node IP is listed in `postgresql['md5_auth_cidr_addresses']` setting of your remote database.
After making these changes [Reconfigure GitLab][gitlab-reconfigure] so the changes take effect.
After making these changes [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) so the changes take effect.
On the secondary the following GitLab frontend services will be enabled:
......@@ -458,6 +458,3 @@ application servers above, with some changes to run only the `sidekiq` service:
`sidekiq['enable'] = false`.
These servers do not need to be attached to the load balancer.
[diagram-source]: https://docs.google.com/drawings/d/1z0VlizKiLNXVVVaERFwgsIOuEgjcUqDTWPdQYsE7Z4c/edit
[gitlab-reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
doc/administration/geo/replication/troubleshooting.md
View file @ f83af0f4
......@@ -261,7 +261,7 @@ default to 1. You may need to increase this value if you have more
Be sure to restart PostgreSQL for this to take
effect. See the [PostgreSQL replication
setup][database-pg-replication] guide for more details.
setup](database.md#postgresql-replication) guide for more details.
### Message: `FATAL: could not start WAL streaming: ERROR: replication slot "geo_secondary_my_domain_com" does not exist`?
......@@ -273,7 +273,7 @@ process](database.md) on the **secondary** node .
### Message: "Command exceeded allowed execution time" when setting up replication?
This may happen while [initiating the replication process][database-start-replication] on the **secondary** node,
This may happen while [initiating the replication process](database.md#step-3-initiate-the-replication-process) on the **secondary** node,
and indicates that your initial dataset is too large to be replicated in the default timeout (30 minutes).
Re-run `gitlab-ctl replicate-geo-database`, but include a larger value for
......@@ -767,9 +767,6 @@ reload of the FDW schema. To manually reload the FDW schema:
SELECT * FROM gitlab_secondary.projects limit 1;
```
[database-start-replication]: database.md#step-3-initiate-the-replication-process
[database-pg-replication]: database.md#postgresql-replication
### "Geo database has an outdated FDW remote schema" error
GitLab can error with a `Geo database has an outdated FDW remote schema` message.
......
doc/administration/geo/replication/using_a_geo_server.md
View file @ f83af0f4
......@@ -2,7 +2,7 @@
# Using a Geo Server **(PREMIUM ONLY)**
After you set up the [database replication and configure the Geo nodes][req], use your closest GitLab node as you would a normal standalone GitLab instance.
After you set up the [database replication and configure the Geo nodes](index.md#setup-instructions), use your closest GitLab node as you would a normal standalone GitLab instance.
Pushing directly to a **secondary** node (for both HTTP, SSH including Git LFS) was [introduced](https://about.gitlab.com/releases/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
......@@ -18,5 +18,3 @@ remote: ssh://git@primary.geo/user/repo.git
remote:
Everything up-to-date
```
[req]: index.md#setup-instructions
doc/administration/high_availability/database.md
View file @ f83af0f4
......@@ -83,7 +83,7 @@ deploy the bundled PostgreSQL.
NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3
1. [Reconfigure GitLab] for the changes to take effect.
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. Note the PostgreSQL node's IP address or hostname, port, and
plain text password. These will be necessary when configuring the GitLab
application servers later.
......@@ -338,7 +338,7 @@ When installing the GitLab package, do not supply `EXTERNAL_URL` value.
repmgr['master_on_initialization'] = false
```
1. [Reconfigure GitLab] for the changes to take effect.
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. [Enable Monitoring](#enable-monitoring)
> Please note:
......@@ -487,7 +487,7 @@ attributes set, but the following need to be set.
gitlab_rails['auto_migrate'] = false
```
1. [Reconfigure GitLab] for the changes to take effect.
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
##### Application node post-configuration
......@@ -554,7 +554,7 @@ consul['configuration'] = {
consul['monitoring_service_discovery'] = true
```
[Reconfigure Omnibus GitLab][reconfigure GitLab] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
##### Example recommended setup for PgBouncer servers
......@@ -584,7 +584,7 @@ consul['configuration'] = {
consul['monitoring_service_discovery'] = true
```
[Reconfigure Omnibus GitLab][reconfigure GitLab] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
##### Internal load balancer setup
......@@ -625,7 +625,7 @@ consul['configuration'] = {
consul['monitoring_service_discovery'] = true
```
[Reconfigure Omnibus GitLab][reconfigure GitLab] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
###### Secondary nodes
......@@ -638,7 +638,7 @@ configuration:
repmgr['master_on_initialization'] = false
```
[Reconfigure Omnibus GitLab][reconfigure GitLab] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
##### Example recommended setup for application server
......@@ -673,7 +673,7 @@ consul['configuration'] = {
}
```
[Reconfigure Omnibus GitLab][reconfigure GitLab] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
##### Example recommended setup manual steps
......@@ -778,7 +778,7 @@ consul['configuration'] = {
}
```
[Reconfigure Omnibus GitLab][reconfigure GitLab] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
###### Secondary nodes
......@@ -826,7 +826,7 @@ consul['configuration'] = {
}
```
[Reconfigure Omnibus GitLab][reconfigure GitLab] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
##### Example minimal setup manual steps
......@@ -1005,7 +1005,7 @@ the previous section:
1. On each PgBouncer node, edit `/etc/gitlab/gitlab.rb`:
1. Ensure `gitlab_rails['db_password']` is set to the plaintext password for
the `gitlab` database user
1. [Reconfigure GitLab] for the changes to take effect
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect
## Enable Monitoring
......@@ -1047,7 +1047,7 @@ steps to fix the problem:
1. On the master database node, connect to the database prompt - `gitlab-psql -d template1`
1. Delete the `gitlab-consul` user - `DROP USER "gitlab-consul";`
1. Exit the database prompt - `\q`
1. [Reconfigure GitLab] and the user will be re-added with the proper permissions.
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) and the user will be re-added with the proper permissions.
1. Change to the `gitlab-consul` user - `su - gitlab-consul`
1. Try the check command again - `gitlab-ctl repmgr-check-master`.
......@@ -1079,7 +1079,7 @@ To fix the problem, add the IP address to `/etc/gitlab/gitlab.rb`.
postgresql['trust_auth_cidr_addresses'] = %w(123.123.123.123/32 <other_cidrs>)
```
[Reconfigure GitLab] for the changes to take effect.
[Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
### Issues with other components
......@@ -1101,5 +1101,3 @@ Read more on high-availability configuration:
1. [Configure the GitLab application servers](gitlab.md)
1. [Configure the load balancers](load_balancer.md)
1. [Manage the bundled Consul cluster](consul.md)
[reconfigure GitLab]: ../restart_gitlab.md#omnibus-gitlab-reconfigure
doc/administration/high_availability/nfs.md
View file @ f83af0f4
......@@ -229,7 +229,7 @@ following are the 4 locations need to be shared:
Other GitLab directories should not be shared between nodes. They contain
node-specific files and GitLab code that does not need to be shared. To ship
logs to a central location consider using remote syslog. GitLab Omnibus packages
provide configuration for [UDP log shipping][udp-log-shipping].
provide configuration for [UDP log shipping](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only).
Having multiple NFS mounts will require manually making sure the data directories
are empty before attempting a restore. Read more about the
......@@ -244,8 +244,6 @@ Read more on high-availability configuration:
1. [Configure the GitLab application servers](gitlab.md)
1. [Configure the load balancers](load_balancer.md)
[udp-log-shipping]: https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only "UDP log shipping"
<!-- ## Troubleshooting
Include any troubleshooting steps that you can foresee. If you know beforehand what issues
......
doc/administration/high_availability/redis.md
View file @ f83af0f4
......@@ -74,7 +74,7 @@ Omnibus:
gitlab_rails['enable'] = false
```
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1. [Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. Note the Redis node's IP address or hostname, port, and
Redis password. These will be necessary when configuring the GitLab
application servers later.
......@@ -88,13 +88,13 @@ Continue configuration of other components by going back to the
### High Availability with GitLab Omnibus **(PREMIUM ONLY)**
> Experimental Redis Sentinel support was [introduced in GitLab 8.11][ce-1877].
> Experimental Redis Sentinel support was [introduced in GitLab 8.11](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/1877).
Starting with 8.14, Redis Sentinel is no longer experimental.
If you've used it with versions `< 8.14` before, please check the updated
documentation here.
High Availability with [Redis] is possible using a **Master** x **Slave**
topology with a [Redis Sentinel][sentinel] service to watch and automatically
High Availability with [Redis](https://redis.io/) is possible using a **Master** x **Slave**
topology with a [Redis Sentinel](https://redis.io/topics/sentinel) service to watch and automatically
start the failover procedure.
You can choose to install and manage Redis and Sentinel yourself, use
......@@ -107,7 +107,7 @@ Omnibus GitLab packages.
> [Redis Security](https://redis.io/topics/security) documentation for more
> information. We recommend using a combination of a Redis password and tight
> firewall rules to secure your Redis service.
> - You are highly encouraged to read the [Redis Sentinel][sentinel] documentation
> - You are highly encouraged to read the [Redis Sentinel](https://redis.io/topics/sentinel) documentation
> before configuring Redis HA with GitLab to fully understand the topology and
> architecture.
> - This is the documentation for the Omnibus GitLab packages. For installations
......@@ -296,7 +296,7 @@ multiple ways to configure Redis HA. Omnibus GitLab packages have Redis and/or
Redis Sentinel bundled with them so you only need to focus on configuration.
Pick the one that suits your needs.
- [Installations from source][source]: You need to install Redis and Sentinel
- [Installations from source](../../install/installation.md): You need to install Redis and Sentinel
yourself. Use the [Redis HA installation from source](redis_source.md)
documentation.
- [Omnibus GitLab **Community Edition** (CE) package](https://about.gitlab.com/install/?version=ce): Redis is bundled, so you
......@@ -341,7 +341,7 @@ The prerequisites for a HA Redis setup are the following:
change the default ones).
1. The server that hosts the GitLab application must be able to access the
Redis nodes.
1. Protect the nodes from access from external networks ([Internet][it]), using
1. Protect the nodes from access from external networks ([Internet](https://gitlab.com/gitlab-org/gitlab-foss/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png)), using
firewall.
### Step 1. Configuring the master Redis instance
......@@ -381,7 +381,7 @@ The prerequisites for a HA Redis setup are the following:
gitlab_rails['auto_migrate'] = false
```
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1. [Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
> Note: You can specify multiple roles like sentinel and Redis as:
> `roles ['redis_sentinel_role', 'redis_master_role']`. Read more about high
......@@ -429,7 +429,7 @@ The prerequisites for a HA Redis setup are the following:
sudo touch /etc/gitlab/skip-auto-reconfigure
```
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1. [Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. Go through the steps again for all the other slave nodes.
> Note: You can specify multiple roles like sentinel and Redis as:
......@@ -561,7 +561,7 @@ multiple machines with the Sentinel daemon.
Only the primary GitLab application server should handle migrations.
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1. [Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. Go through the steps again for all the other Sentinel nodes.
### Step 4. Configuring the GitLab application
......@@ -598,7 +598,7 @@ which ideally should not have Redis or Sentinels on it for a HA setup.
]
```
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1. [Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
## Switching from an existing single-machine installation to Redis HA
......@@ -677,7 +677,7 @@ sentinel['quorum'] = 2
# sentinel['failover_timeout'] = 60000
```
[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
### Example configuration for Redis slave 1 and Sentinel 2
......@@ -699,7 +699,7 @@ sentinel['quorum'] = 2
# sentinel['failover_timeout'] = 60000
```
[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
### Example configuration for Redis slave 2 and Sentinel 3
......@@ -721,7 +721,7 @@ sentinel['quorum'] = 2
# sentinel['failover_timeout'] = 60000
```
[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
### Example configuration for the GitLab application
......@@ -737,7 +737,7 @@ gitlab_rails['redis_sentinels'] = [
]
```
[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
## Enable Monitoring
......@@ -862,7 +862,7 @@ mailroom['enable'] = false
redis['master'] = false
```
You can find the relevant attributes defined in [`gitlab_rails.rb`][omnifile].
You can find the relevant attributes defined in [`gitlab_rails.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/libraries/gitlab_rails.rb).
## Troubleshooting
......@@ -929,7 +929,7 @@ repl_backlog_histlen:0
If you get an error like: `Redis::CannotConnectError: No sentinels available.`,
there may be something wrong with your configuration files or it can be related
to [this issue][gh-531].
to [this issue](https://github.com/redis/redis-rb/issues/531).
You must make sure you are defining the same value in `redis['master_name']`
and `redis['master_pasword']` as you defined for your sentinel node.
......@@ -1001,14 +1001,3 @@ Read more on High Availability:
1. [Configure NFS](nfs.md)
1. [Configure the GitLab application servers](gitlab.md)
1. [Configure the load balancers](load_balancer.md)
[ce-1877]: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/1877
[restart]: ../restart_gitlab.md#installations-from-source
[reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure
[gh-531]: https://github.com/redis/redis-rb/issues/531
[gh-534]: https://github.com/redis/redis-rb/issues/534
[redis]: https://redis.io/
[sentinel]: https://redis.io/topics/sentinel
[omnifile]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/libraries/gitlab_rails.rb
[source]: ../../install/installation.md
[it]: https://gitlab.com/gitlab-org/gitlab-foss/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png
doc/administration/high_availability/redis_source.md
View file @ f83af0f4
......@@ -182,7 +182,7 @@ which ideally should not have Redis or Sentinels in the same machine for a HA
setup:
1. Edit `/home/git/gitlab/config/resque.yml` following the example in
[resque.yml.example][resque], and uncomment the Sentinel lines, pointing to
[resque.yml.example](https://gitlab.com/gitlab-org/gitlab/blob/master/config/resque.yml.example), and uncomment the Sentinel lines, pointing to
the correct server credentials:
```yaml
......@@ -201,7 +201,7 @@ setup:
port: 26379 # point to sentinel, not to redis port
```
1. [Restart GitLab][restart] for the changes to take effect.
1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect.
## Example of minimal configuration with 1 master, 2 slaves and 3 Sentinels
......@@ -211,7 +211,7 @@ to each other using these IPs.
In a real world usage, you would also set up firewall rules to prevent
unauthorized access from other machines, and block traffic from the
outside ([Internet][it]).
outside ([Internet](https://gitlab.com/gitlab-org/gitlab-foss/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png)).
For this example, **Sentinel 1** will be configured in the same machine as the
**Redis Master**, **Sentinel 2** and **Sentinel 3** in the same machines as the
......@@ -326,7 +326,7 @@ or a failover promotes a different **Master** node.
port: 26379 # point to sentinel, not to redis port
```
1. [Restart GitLab][restart] for the changes to take effect.
1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect.
## Troubleshooting
......@@ -336,7 +336,7 @@ the things that are specific to a source installation.
If you get an error in GitLab like `Redis::CannotConnectError: No sentinels available.`,
there may be something wrong with your configuration files or it can be related
to [this upstream issue][gh-531].
to [this upstream issue](https://github.com/redis/redis-rb/issues/531).
You must make sure that `resque.yml` and `sentinel.conf` are configured correctly,
otherwise `redis-rb` will not work properly.
......@@ -369,9 +369,3 @@ production:
```
When in doubt, please read [Redis Sentinel documentation](https://redis.io/topics/sentinel).
[gh-531]: https://github.com/redis/redis-rb/issues/531
[downloads]: https://about.gitlab.com/downloads
[restart]: ../restart_gitlab.md#installations-from-source
[it]: https://gitlab.com/gitlab-org/gitlab-foss/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png
[resque]: https://gitlab.com/gitlab-org/gitlab/blob/master/config/resque.yml.example
doc/api/graphql/index.md
View file @ f83af0f4
# GraphQL API
> - [Introduced][ce-19008] in GitLab 11.0 (enabled by feature flag `graphql`).
> - [Always enabled](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30444)
in GitLab 12.1.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19008) in GitLab 11.0 (enabled by feature flag `graphql`).
> - [Always enabled](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30444) in GitLab 12.1.
## Getting Started
......@@ -87,6 +86,3 @@ Machine-readable versions are also available:
- [JSON format](reference/gitlab_schema.json)
- [IDL format](reference/gitlab_schema.graphql)
[ce-19008]: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19008
[features-api]: ../features.md
doc/development/feature_flags/development.md
View file @ f83af0f4
......@@ -36,9 +36,9 @@ In the rare case that you need the feature flag to be on automatically, use
Feature.enabled?(:feature_flag, project, default_enabled: true)
```
The [`Project#feature_available?`][project-fa],
[`Namespace#feature_available?`][namespace-fa] (EE), and
[`License.feature_available?`][license-fa] (EE) methods all implicitly check for
The [`Project#feature_available?`](https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/app/models/project_feature.rb#L63-68),
[`Namespace#feature_available?`](https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/ee/namespace.rb#L71-85) (EE), and
[`License.feature_available?`](https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/license.rb#L293-300) (EE) methods all implicitly check for
a by default enabled feature flag with the same name as the provided argument.
For example if a feature is license-gated, there's no need to add an additional
......@@ -49,10 +49,6 @@ feature flag once the feature has reached general availability.
You'd still want to use an explicit `Feature.enabled?` check if your new feature
isn't gated by a License or Plan.
[project-fa]: https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/app/models/project_feature.rb#L63-68
[namespace-fa]: https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/ee/namespace.rb#L71-85
[license-fa]: https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/license.rb#L293-300
**An important side-effect of the implicit feature flags mentioned above is that
unless the feature is explicitly disabled or limited to a percentage of users,
the feature flag check will default to `true`.**
......
doc/development/gitaly.md
View file @ f83af0f4
......@@ -5,13 +5,17 @@ Workhorse and GitLab-Shell.
## Deep Dive
In May 2019, Bob Van Landuyt hosted a [Deep Dive] on GitLab's [Gitaly project] and how to contribute to it as a Ruby developer, to share his domain specific knowledge with anyone who may work in this part of the code base in the future. You can find the [recording on YouTube], and the slides on [Google Slides] and in [PDF]. Everything covered in this deep dive was accurate as of GitLab 11.11, and while specific details may have changed since then, it should still serve as a good introduction.
In May 2019, Bob Van Landuyt hosted a [Deep Dive](https://gitlab.com/gitlab-org/create-stage/issues/1)
on GitLab's [Gitaly project](https://gitlab.com/gitlab-org/gitaly) and how to contribute to it as a
Ruby developer, to share his domain specific knowledge with anyone who may work in this part of the
code base in the future.
[Deep Dive]: https://gitlab.com/gitlab-org/create-stage/issues/1
[Gitaly project]: https://gitlab.com/gitlab-org/gitaly
[recording on YouTube]: https://www.youtube.com/watch?v=BmlEWFS8ORo
[Google Slides]: https://docs.google.com/presentation/d/1VgRbiYih9ODhcPnL8dS0W98EwFYpJ7GXMPpX-1TM6YE/edit
[PDF]: https://gitlab.com/gitlab-org/create-stage/uploads/a4fdb1026278bda5c1c5bb574379cf80/Create_Deep_Dive__Gitaly_for_Create_Ruby_Devs.pdf
You can find the [recording on YouTube](https://www.youtube.com/watch?v=BmlEWFS8ORo), and the slides
on [Google Slides](https://docs.google.com/presentation/d/1VgRbiYih9ODhcPnL8dS0W98EwFYpJ7GXMPpX-1TM6YE/edit)
and in [PDF](https://gitlab.com/gitlab-org/create-stage/uploads/a4fdb1026278bda5c1c5bb574379cf80/Create_Deep_Dive__Gitaly_for_Create_Ruby_Devs.pdf).
Everything covered in this deep dive was accurate as of GitLab 11.11, and while specific details may
have changed since then, it should still serve as a good introduction.
## Beginner's guide
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7