@@ -25,10 +25,11 @@ GitLab supports two authentication methods:
### Authentication against a local database with X.509 certificates
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/726) in
[GitLab Premium](https://about.gitlab.com/pricing/) 11.6 as an experimental
feature. Smartcard authentication against local databases may change or be
removed completely in future releases.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/726) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.6 as an experimental feature.
CAUTION: **Caution:**
Smartcard authentication against local databases may change or be removed completely in future
releases.
Smartcards with X.509 certificates can be used to authenticate with GitLab.
@@ -146,7 +146,7 @@ Each service in the package comes with a set of [default ports](https://docs.git
- Application servers connect to either PgBouncer directly via its [default port](https://docs.gitlab.com/omnibus/package-information/defaults.html#pgbouncer) or via a configured Internal Load Balancer (TCP) that serves multiple PgBouncers.
- PgBouncer connects to the primary database servers [PostgreSQL default port](https://docs.gitlab.com/omnibus/package-information/defaults.html#postgresql)
- Repmgr connects to the database servers [PostgreSQL default port](https://docs.gitlab.com/omnibus/package-information/defaults.html#postgresql)
- Postgres secondaries connect to the primary database servers [PostgreSQL default port](https://docs.gitlab.com/omnibus/package-information/defaults.html#postgresql)
- PostgreSQL secondaries connect to the primary database servers [PostgreSQL default port](https://docs.gitlab.com/omnibus/package-information/defaults.html#postgresql)
- Consul servers and agents connect to each others [Consul default ports](https://docs.gitlab.com/omnibus/package-information/defaults.html#consul)
#### Required information
...
...
@@ -899,7 +899,7 @@ after it has been restored to service.
The node exporter allows you to measure various machine resources, such as
memory, disk, and CPU utilization.
[➔ Read more about the node exporter.](node_exporter.md)
[Read more about the node exporter](node_exporter.md).
### Redis exporter
The Redis exporter allows you to measure various Redis metrics.
[➔ Read more about the Redis exporter.](redis_exporter.md)
[Read more about the Redis exporter](redis_exporter.md).
### Postgres exporter
### PostgreSQL exporter
The Postgres exporter allows you to measure various PostgreSQL metrics.
The PostgreSQL exporter allows you to measure various PostgreSQL metrics.
[➔ Read more about the Postgres exporter.](postgres_exporter.md)
[Read more about the PostgreSQL exporter](postgres_exporter.md).
### PgBouncer exporter
The PgBouncer exporter allows you to measure various PgBouncer metrics.
[➔ Read more about the PgBouncer exporter.](pgbouncer_exporter.md)
[Read more about the PgBouncer exporter](pgbouncer_exporter.md).
### Registry exporter
The Registry exporter allows you to measure various Registry metrics.
[➔ Read more about the Registry exporter.](registry_exporter.md)
[Read more about the Registry exporter](registry_exporter.md).
### GitLab exporter
The GitLab exporter allows you to measure various GitLab metrics, pulled from Redis and the database.
[➔ Read more about the GitLab exporter.](gitlab_exporter.md)
[Read more about the GitLab exporter](gitlab_exporter.md).
## Configuring Prometheus to monitor Kubernetes
> Introduced in GitLab 9.0.
> Pod monitoring introduced in GitLab 9.4.
> - Introduced in GitLab 9.0.
> - Pod monitoring introduced in GitLab 9.4.
If your GitLab server is running within Kubernetes, Prometheus will collect metrics from the Nodes and [annotated Pods](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config) in the cluster, including performance data on each container. This is particularly helpful if your CI/CD environments run in the same cluster, as you can use the [Prometheus project integration][prometheus integration] to monitor them.
When adding a custom domain, users will be required to prove they own it by
adding a GitLab-controlled verification code to the DNS records for that domain.
If your userbase is private or otherwise trusted, you can disable the
If your userbase is private or otherwise trusted, you can disable the
verification requirement. Navigate to **Admin Area > Settings > Preferences** and
uncheck **Require users to prove ownership of custom domains** in the **Pages** section.
This setting is enabled by default.
...
...
@@ -358,7 +358,7 @@ For Omnibus, normally this would be fixed by [installing a custom CA in GitLab O
but a [bug](https://gitlab.com/gitlab-org/gitlab/issues/25411) is currently preventing
that method from working. Use the following workaround:
1. Append your GitLab server TLS/SSL certficate to `/opt/gitlab/embedded/ssl/certs/cacert.pem` where `gitlab-domain-example.com` is your GitLab application URL
1. Append your GitLab server TLS/SSL certificate to `/opt/gitlab/embedded/ssl/certs/cacert.pem` where `gitlab-domain-example.com` is your GitLab application URL
-**Lucene**: A full-text search library written in Java.
-**Near Realtime (NRT)**: Refers to the slight latency from the time to index a
-**Near real time (NRT)**: Refers to the slight latency from the time to index a
document to the time when it becomes searchable.
-**Cluster**: A collection of one or more nodes that work together to hold all
the data, providing indexing and search capabilities.
...
...
@@ -271,7 +271,7 @@ Generally speaking, ensure:
- The Elasticsearch server have enough RAM and CPU cores.
- That sharding **is** being used.
Going into some more detail here, if Elasticsearch is running on the same server as GitLab, resource contention is **very** likely to occur. Ideally, Elasticsearch, which requires ample resources, should be running on its own server (maybe coupled with logstash and kibana).
Going into some more detail here, if Elasticsearch is running on the same server as GitLab, resource contention is **very** likely to occur. Ideally, Elasticsearch, which requires ample resources, should be running on its own server (maybe coupled with Logstash and Kibana).
When it comes to Elasticsearch, RAM is the key resource. Elasticsearch themselves recommend:
With the command above, you create a Runner that uses the [ruby:2.6](https://hub.docker.com/_/ruby) image and uses a [postgres](https://hub.docker.com/_/postgres) database.
With the command above, you create a Runner that uses the [ruby:2.6](https://hub.docker.com/_/ruby) image and uses a [PostgreSQL](https://hub.docker.com/_/postgres) database.
To access the PostgreSQL database, connect to `host: postgres` as user `postgres` with no password.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9186) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.0.
> [Squash and merge](../../../../user/project/merge_requests/squash_and_merge.md) support [introduced](https://gitlab.com/gitlab-org/gitlab/issues/13001) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.6.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9186) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.0.
> - [Squash and merge](../../../../user/project/merge_requests/squash_and_merge.md) support [introduced](https://gitlab.com/gitlab-org/gitlab/issues/13001) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.6.
[Pipelines for merged results](../index.md#pipelines-for-merged-results-premium) introduces
running a build on the result of the merged code prior to merging, as a way to keep master green.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20390) in
GitLab 11.2. Requires GitLab Runner 11.2 and above.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20390) in GitLab 11.2. Requires GitLab Runner 11.2 and above.
The `reports` keyword is used for collecting test reports, code quality reports, and security reports from jobs.
It also exposes these reports in GitLab's UI (merge requests, pipeline views, and security dashboards).
...
...
@@ -2235,8 +2234,7 @@ If you also want the ability to browse the report output files, include the
##### `artifacts:reports:junit`
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20390) in
GitLab 11.2. Requires GitLab Runner 11.2 and above.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20390) in GitLab 11.2. Requires GitLab Runner 11.2 and above.
The `junit` report collects [JUnit XML files](https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_14.1.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html)
as artifacts. Although JUnit was originally developed in Java, there are many
...
...
@@ -2288,8 +2286,7 @@ There are a couple of limitations on top of the [original dotenv rules](https://
##### `artifacts:reports:cobertura`
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/3708) in GitLab 12.9.
> Requires [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 and above.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/3708) in GitLab 12.9. Requires [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 and above.
The `cobertura` report collects [Cobertura coverage XML files](../../user/project/merge_requests/test_coverage_visualization.md).
The collected Cobertura coverage reports will be uploaded to GitLab as an artifact
...
...
@@ -3620,9 +3617,11 @@ Learn more about [variables and their priority][variables].
#### Git strategy
> Introduced in GitLab 8.9 as an experimental feature. May change or be removed
> completely in future releases. `GIT_STRATEGY=none` requires GitLab Runner
> v1.7+.
> - Introduced in GitLab 8.9 as an experimental feature.
[Postgres-exporter](https://github.com/wrouesnel/postgres_exporter) is the community provided Prometheus exporter that will deliver data about Postgres to Prometheus for use in Grafana Dashboards.
[`postgres_exporter`](https://github.com/wrouesnel/postgres_exporter) is the community provided Prometheus exporter that will deliver data about PostgreSQL to Prometheus for use in Grafana Dashboards.
#### Prometheus
...
...
@@ -486,7 +486,7 @@ When making a request to an HTTP Endpoint (think `/users/sign_in`) the request w
- NGINX - Acts as our first line reverse proxy.
- GitLab Workhorse - This determines if it needs to go to the Rails application or somewhere else to reduce load on Unicorn.
- Unicorn - Since this is a web request, and it needs to access the application it will go to Unicorn.
- Postgres/Gitaly/Redis - Depending on the type of request, it may hit these services to store or retrieve data.
- PostgreSQL/Gitaly/Redis - Depending on the type of request, it may hit these services to store or retrieve data.
@@ -707,7 +707,13 @@ For more information about the available options, run:
### `#database-lab`
Another tool GitLab employees can use is a chatbot powered by [Joe](https://gitlab.com/postgres-ai/joe) which uses [Database Lab](https://gitlab.com/postgres-ai/database-lab) to instantly provide developers with their own clone of the production database. Joe is available in the [`#database-lab`](https://gitlab.slack.com/archives/CLJMDRD8C) channel on Slack.
Another tool GitLab employees can use is a chatbot powered by [Joe](https://gitlab.com/postgres-ai/joe)
which uses [Database Lab](https://gitlab.com/postgres-ai/database-lab) to instantly provide developers
with their own clone of the production database.
Joe is available in the
[`#database-lab`](https://gitlab.slack.com/archives/CLJMDRD8C) channel on Slack.
Unlike chatops, it gives you a way to execute DDL statements (like creating indexes and tables) and get query plan not only for `SELECT` but also `UPDATE` and `DELETE`.
For example, in order to test new index you can do the following:
@@ -904,7 +904,7 @@ for the changes to take effect.
If you'd like to connect to a Redis server on a non-standard port or a different host, you can configure its connection string via the `config/resque.yml` file.
```
```yaml
# example
production:
url:redis://redis.example.tld:6379
...
...
@@ -912,7 +912,7 @@ production:
If you want to connect the Redis server via socket, use the "unix:" URL scheme and the path to the Redis socket file in the `config/resque.yml` file.
```
```yaml
# example
production:
url:unix:/path/to/redis/socket
...
...
@@ -920,7 +920,7 @@ production:
Also, you can use environment variables in the `config/resque.yml` file:
```
```yaml
# example
production:
url:<%= ENV.fetch('GITLAB_REDIS_URL') %>
...
...
@@ -930,7 +930,7 @@ production:
If you are running SSH on a non-standard port, you must change the GitLab user's SSH config.
```
```plaintext
# Add to /home/git/.ssh/config
host localhost # Give your setup a name (here: override localhost)
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/109 "Elasticsearch Merge Request") in GitLab [Starter](https://about.gitlab.com/pricing/) 8.4. Support
> for [Amazon Elasticsearch](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html) was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1305) in GitLab
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/109 "Elasticsearch Merge Request") in GitLab [Starter](https://about.gitlab.com/pricing/) 8.4.
> - Support for [Amazon Elasticsearch](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html) was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1305) in GitLab [Starter](https://about.gitlab.com/pricing/) 9.0.
This document describes how to set up Elasticsearch with GitLab. Once enabled,
you'll have the benefit of fast search response times and the advantage of two
...
...
@@ -563,7 +562,7 @@ Here are some common pitfalls and how to overcome them:
If you enabled Elasticsearch before GitLab 8.12 and have not rebuilt indexes you will get
in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3672) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4.
## Overview
...
...
@@ -176,12 +175,12 @@ using environment variables.
| `CLAIR_OUTPUT` | Severity level threshold. Vulnerabilities with severity level higher than or equal to this threshold will be outputted. Supported levels are `Unknown`, `Negligible`, `Low`, `Medium`, `High`, `Critical` and `Defcon1`. | `Unknown` |
| `REGISTRY_INSECURE` | Allow [Klar](https://github.com/optiopay/klar) to access insecure registries (HTTP only). Should only be set to `true` when testing the image locally. | `"false"` |
| `DOCKER_INSECURE` | Allow [Klar](https://github.com/optiopay/klar) to access secure Docker registries using HTTPS with bad (or self-signed) SSL certificates. | `"false"` |
| `CLAIR_VULNERABILITIES_DB_URL` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [Postgres server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. | `clair-vulnerabilities-db` |
| `CLAIR_DB_CONNECTION_STRING` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [Postgres server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` |
| `CLAIR_VULNERABILITIES_DB_URL` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. | `clair-vulnerabilities-db` |
| `CLAIR_DB_CONNECTION_STRING` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` |
| `CI_APPLICATION_REPOSITORY` | Docker repository URL for the image to be scanned. | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` |
| `CI_APPLICATION_TAG` | Docker respository tag for the image to be scanned. | `$CI_COMMIT_SHA` |
| `CLAIR_DB_IMAGE` | The Docker image name and tag for the [Postgres server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise air-gapped installation. | `arminc/clair-db:latest` |
| `CLAIR_DB_IMAGE_TAG` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [Postgres server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | `latest` |
| `CLAIR_DB_IMAGE` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise air-gapped installation. | `arminc/clair-db:latest` |
| `CLAIR_DB_IMAGE_TAG` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | `latest` |
| `DOCKERFILE_PATH` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner will look for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | `Dockerfile` |
| `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs that you want to trust. | "" |
@@ -158,8 +158,7 @@ It's possible to generate diagrams and flowcharts from text in GitLab using [Mer
#### Mermaid
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/15107) in
GitLab 10.3.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/15107) in GitLab 10.3.
Visit the [official page](https://mermaidjs.github.io/) for more details. If you're new to using Mermaid or need help identifying issues in your Mermaid code, the [Mermaid Live Editor](https://mermaid-js.github.io/mermaid-live-editor/) is a helpful tool for creating and resolving issues within Mermaid diagrams.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/28996) in GitLab 12.1. For versions earlier than GitLab 12.1, see the [manual Let's Encrypt instructions](../lets_encrypt_for_gitlab_pages.md).
NOTE: **Note:**
This feature is in **beta** and may still have bugs. See all the related issues linked from this [issue's description](https://gitlab.com/gitlab-org/gitlab-foss/issues/28996) for more information.
The GitLab Pages integration with Let's Encrypt (LE) allows you
include_examples'management project is not required'
end
shared_examples'creates a management project'do
let(:project_params)do
{
name: "#{cluster.name} Cluster Management",
description: 'This project is automatically generated and will be used to manage your Kubernetes cluster. [More information](/help/user/clusters/management_project)',