Commit 899c290f authored by Marcia Ramos's avatar Marcia Ramos

Merge branch 'docs/update-health-check' into 'master'

Update health check docs

See merge request !12794
parents aab51dbf 1322042b
# IP whitelist
> Introduced in GitLab 9.4.
GitLab provides some [monitoring endpoints] that provide health check information
when probed.
To control access to those endpoints via IP whitelisting, you can add single
hosts or use IP ranges:
**For Omnibus installations**
1. Open `/etc/gitlab/gitlab.rb` and add or uncomment the following:
```ruby
gitlab_rails['monitoring_whitelist'] = ['127.0.0.0/8', '192.168.0.1']
```
1. Save the file and [reconfigure] GitLab for the changes to take effect.
---
**For installations from source**
1. Edit `config/gitlab.yml`:
```yaml
monitoring:
# by default only local IPs are allowed to access monitoring resources
ip_whitelist:
- 127.0.0.0/8
- 192.168.0.1
```
1. Save the file and [restart] GitLab for the changes to take effect.
[reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure
[restart]: ../restart_gitlab.md#installations-from-source
[monitoring endpoints]: ../../user/admin_area/monitoring/health_check.md
# GitLab Prometheus metrics # GitLab Prometheus metrics
>**Note:** >**Note:**
Available since [Omnibus GitLab 9.3][29118]. Currently experimental. For installations from source Available since [Omnibus GitLab 9.3][29118]. Currently experimental. For
you'll have to configure it yourself. installations from source you'll have to configure it yourself.
GitLab monitors its own internal service metrics, and makes them available at the `/-/metrics` endpoint. Unlike other [Prometheus] exporters, this endpoint requires authentication as it is available on the same URL and port as user traffic.
To enable the GitLab Prometheus metrics: To enable the GitLab Prometheus metrics:
...@@ -15,9 +13,14 @@ To enable the GitLab Prometheus metrics: ...@@ -15,9 +13,14 @@ To enable the GitLab Prometheus metrics:
## Collecting the metrics ## Collecting the metrics
Since the metrics endpoint is available on the same host and port as other traffic, it requires authentication. The token and URL to access is displayed on the [Health Check][health-check] page. GitLab monitors its own internal service metrics, and makes them available at the
`/-/metrics` endpoint. Unlike other [Prometheus] exporters, in order to access
it, the client IP needs to be [included in a whitelist][whitelist].
Currently the embedded Prometheus server is not automatically configured to collect metrics from this endpoint. We recommend setting up another Prometheus server, because the embedded server configuration is overwritten one every reconfigure of GitLab. In the future this will not be required. Currently the embedded Prometheus server is not automatically configured to
collect metrics from this endpoint. We recommend setting up another Prometheus
server, because the embedded server configuration is overwritten once every
[reconfigure of GitLab][reconfigure]. In the future this will not be required.
## Metrics available ## Metrics available
...@@ -47,4 +50,5 @@ In this experimental phase, only a few metrics are available: ...@@ -47,4 +50,5 @@ In this experimental phase, only a few metrics are available:
[29118]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29118 [29118]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29118
[Prometheus]: https://prometheus.io [Prometheus]: https://prometheus.io
[restart]: ../../restart_gitlab.md#omnibus-gitlab-restart [restart]: ../../restart_gitlab.md#omnibus-gitlab-restart
[health-check]: ../../../user/admin_area/monitoring/health_check.md [whitelist]: ../ip_whitelist.md
[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior) be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior)
section. section.
- [Access token](#access-token) has been deprecated in GitLab 9.4 - [Access token](#access-token) has been deprecated in GitLab 9.4
in favor of [IP Whitelist](#ip-whitelist) in favor of [IP whitelist](#ip-whitelist)
GitLab provides liveness and readiness probes to indicate service health and GitLab provides liveness and readiness probes to indicate service health and
reachability to required services. These probes report on the status of the reachability to required services. These probes report on the status of the
...@@ -14,109 +14,101 @@ database connection, Redis connection, and access to the filesystem. These ...@@ -14,109 +14,101 @@ database connection, Redis connection, and access to the filesystem. These
endpoints [can be provided to schedulers like Kubernetes][kubernetes] to hold endpoints [can be provided to schedulers like Kubernetes][kubernetes] to hold
traffic until the system is ready or restart the container as needed. traffic until the system is ready or restart the container as needed.
## IP Whitelist ## IP whitelist
To access monitoring resources the client IP needs to be included in the whitelist. To access monitoring resources, the client IP needs to be included in a whitelist.
To add or remove hosts or IP ranges from the list you can edit `gitlab.rb` or `gitlab.yml`.
Example whitelist configuration: [Read how to add IPs to a whitelist for the monitoring endpoints.][admin].
```yaml
monitoring:
ip_whitelist:
- 127.0.0.0/8 # by default only local IPs are allowed to access monitoring resources
```
## Access Token (Deprecated) ## Using the endpoint
An access token needs to be provided while accessing the probe endpoints. The current With default whitelist settings, the probes can be accessed from localhost:
accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check**
(`admin/health_check`) page of your GitLab instance.
![access token](img/health_check_token.png) - `http://localhost/-/readiness`
- `http://localhost/-/liveness`
The access token can be passed as a URL parameter: which will then provide a report of system health in JSON format.
Readiness example output:
``` ```
https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN {
"queues_check" : {
"status" : "ok"
},
"redis_check" : {
"status" : "ok"
},
"shared_state_check" : {
"status" : "ok"
},
"fs_shards_check" : {
"labels" : {
"shard" : "default"
},
"status" : "ok"
},
"db_check" : {
"status" : "ok"
},
"cache_check" : {
"status" : "ok"
}
}
``` ```
which will then provide a report of system health in JSON format: Liveness example output:
``` ```
{ {
"db_check": { "fs_shards_check" : {
"status": "ok" "status" : "ok"
}, },
"redis_check": { "cache_check" : {
"status": "ok" "status" : "ok"
}, },
"fs_shards_check": { "db_check" : {
"status": "ok", "status" : "ok"
"labels": { },
"shard": "default" "redis_check" : {
} "status" : "ok"
},
"queues_check" : {
"status" : "ok"
},
"shared_state_check" : {
"status" : "ok"
} }
} }
``` ```
## Using the Endpoint
With default whitelist settings, the probes can be accessed from localhost:
- `http://localhost/-/readiness`
- `http://localhost/-/liveness`
## Status ## Status
On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
will return a valid successful HTTP status code, and a `success` message. will return a valid successful HTTP status code, and a `success` message.
## Old behavior ## Access token (Deprecated)
>**Notes:**
- Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1.
- The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and will
be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior)
section.
GitLab provides a health check endpoint for uptime monitoring on the `health_check` web
endpoint. The health check reports on the overall system status based on the status of
the database connection, the state of the database migrations, and the ability to write
and access the cache. This endpoint can be provided to uptime monitoring services like
[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health].
Once you have the [access token](#access-token) or your client IP is [whitelisted](#ip-whitelist), >**Note:**
health information can be retrieved as plain text, JSON, or XML using the `health_check` endpoint: Access token has been deprecated in GitLab 9.4
in favor of [IP whitelist](#ip-whitelist)
- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN` An access token needs to be provided while accessing the probe endpoints. The current
- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN` accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check**
- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN` (`admin/health_check`) page of your GitLab instance.
You can also ask for the status of specific services:
- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN`
For example, the JSON output of the following health check:
```bash ![access token](img/health_check_token.png)
curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
```
would be like: The access token can be passed as a URL parameter:
``` ```
{"healthy":true,"message":"success"} https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN
``` ```
On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
will return a valid successful HTTP status code, and a `success` message. Ideally your
uptime monitoring should look for the success message.
[ce-10416]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10416 [ce-10416]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10416
[ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888 [ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888
[pingdom]: https://www.pingdom.com [pingdom]: https://www.pingdom.com
[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html [nagios-health]: https://nagios-plugins.org/doc/man/check_http.html
[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring [newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring
[kubernetes]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/ [kubernetes]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
[admin]: ../../../administration/monitoring/ip_whitelist.md
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment