Skip to content

G
gitlab-ce
Commit 10455f18 authored by Evan Read's avatar Evan Read
Browse files
Options

Merge branch 'selhorn-docs-docker-using-5' into 'master' 

Edited for style and clarity

See merge request gitlab-org/gitlab!58175
parents 139b332c fde4f287
Hide whitespace changes
Inline Side-by-side
Showing with 92 additions and 99 deletions
doc/ci/docker/using_docker_build.md
View file @ 10455f18
...@@ -623,7 +623,7 @@ fails, because the file is mounted as read-only. Do not change it from ...@@ -623,7 +623,7 @@ fails, because the file is mounted as read-only. Do not change it from
read-only, because problems occur. read-only, because problems occur.
Here is an example of `/opt/.docker/config.json` that follows the Here is an example of `/opt/.docker/config.json` that follows the
[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determining-your-docker_auth_config-data) [`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data)
documentation: documentation:
```json ```json
...@@ -683,7 +683,7 @@ to include the file. ...@@ -683,7 +683,7 @@ to include the file.
### Option 3: Use `DOCKER_AUTH_CONFIG` ### Option 3: Use `DOCKER_AUTH_CONFIG`
If you already have If you already have
[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determining-your-docker_auth_config-data) [`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data)
defined, you can use the variable and save it in defined, you can use the variable and save it in
`~/.docker/config.json`. `~/.docker/config.json`.
......
doc/ci/docker/using_docker_images.md
View file @ 10455f18
...@@ -9,23 +9,22 @@ type: concepts, howto ...@@ -9,23 +9,22 @@ type: concepts, howto
You can run your CI/CD jobs in separate, isolated Docker containers. You can run your CI/CD jobs in separate, isolated Docker containers.
When you run a Docker container on your local machine, it acts as a reproducible build environment. If you run Docker on your local machine, you can run tests in the container,
You can run tests in the container, instead of testing on a dedicated CI/CD server. rather than testing on a dedicated CI/CD server.
To run CI/CD jobs in a Docker container, you need to: To run CI/CD jobs in a Docker container, you need to:
- Register a runner that uses the Docker executor. Then all jobs run in a Docker container. 1. Register a runner so that all jobs run in Docker containers. Do this by choosing the Docker executor during registration.
- Specify an image in your `.gitlab-ci.yml` file. The runner creates a container from this image 1. Specify which container to run the jobs in. Do this by specifying an image in your `.gitlab-ci.yml` file.
and runs the jobs in it. 1. Optional. Run other services, like MySQL, in containers. Do this by specifying [services](../services/index.md)
- Optional. Specify other images in your `.gitlab-ci.yml` file. These containers are known as in your `.gitlab-ci.yml` file.
["services"](../services/index.md) and you can use them to run services like MySQL separately.
## Register a runner that uses the Docker executor ## Register a runner that uses the Docker executor
To use GitLab Runner with Docker you need to [register a runner](https://docs.gitlab.com/runner/register/) To use GitLab Runner with Docker you need to [register a runner](https://docs.gitlab.com/runner/register/)
that uses the Docker executor. that uses the Docker executor.
In this example, we first set up a temporary template to supply the services: This example shows how to set up a temporary template to supply services:
```shell ```shell
cat > /tmp/test-config.template.toml << EOF cat > /tmp/test-config.template.toml << EOF
...@@ -57,20 +56,20 @@ accessible during the build process. ...@@ -57,20 +56,20 @@ accessible during the build process.
## What is an image ## What is an image
The `image` keyword is the name of the Docker image the Docker executor The `image` keyword is the name of the Docker image the Docker executor
runs to perform the CI tasks. uses to run CI/CD jobs.
By default, the executor pulls images only from [Docker Hub](https://hub.docker.com/). By default, the executor pulls images from [Docker Hub](https://hub.docker.com/).
However, you can configure the location in the `gitlab-runner/config.toml` file. For example, However, you can configure the registry location in the `gitlab-runner/config.toml` file.
you can set the [Docker pull policy](https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work) For example, you can set the [Docker pull policy](https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work)
to use local images. to use local images.
For more information about images and Docker Hub, read For more information about images and Docker Hub, see
the [Docker Fundamentals](https://docs.docker.com/engine/understanding-docker/) documentation. the [Docker Fundamentals](https://docs.docker.com/engine/understanding-docker/) documentation.
## Define `image` in the `.gitlab-ci.yml` file ## Define `image` in the `.gitlab-ci.yml` file
You can define an image that's used for all jobs, and a list of You can define an image that's used for all jobs, and a list of
services that you want to use during build time: services that you want to use during runtime:
```yaml ```yaml
default: default:
...@@ -97,58 +96,57 @@ The image name must be in one of the following formats: ...@@ -97,58 +96,57 @@ The image name must be in one of the following formats:
> Introduced in GitLab and GitLab Runner 9.4. > Introduced in GitLab and GitLab Runner 9.4.
When configuring the `image` or `services` entries, you can use a string or a map as You can use a string or a map for the `image` or `services` entries:
options:
- When using a string as an option, it must be the full name of the image to use - Strings must include the full image name
(including the Registry part if you want to download the image from a Registry (including the registry, if you want to download the image from a registry
other than Docker Hub). other than Docker Hub).
- When using a map as an option, then it must contain at least the `name` - Maps must contain at least the `name` option,
option, which is the same name of the image as used for the string setting. which is the same image name as used for the string setting.
For example, the following two definitions are equal: For example, the following two definitions are equal:
1. Using a string as an option to `image` and `services`: - A string for `image` and `services`:
```yaml ```yaml
image: "registry.example.com/my/image:latest" image: "registry.example.com/my/image:latest"
services: services:
- postgresql:9.4 - postgresql:9.4
- redis:latest - redis:latest
``` ```
1. Using a map as an option to `image` and `services`. The use of `image:name` is - A map for `image` and `services`. The `image:name` is
required: required:
```yaml ```yaml
image: image:
name: "registry.example.com/my/image:latest" name: "registry.example.com/my/image:latest"
services: services:
- name: postgresql:9.4 - name: postgresql:9.4
- name: redis:latest - name: redis:latest
``` ```
### Available settings for `image` ### Available settings for `image`
> Introduced in GitLab and GitLab Runner 9.4. > Introduced in GitLab and GitLab Runner 9.4.
| Setting | Required | GitLab version | Description | | Setting | Required | Description |
|------------|----------|----------------| ----------- | |------------|----------| ----------- |
| `name` | yes, when used with any other option | 9.4 |Full name of the image to use. It should contain the Registry part if needed. | | `name` | Yes, when used with any other option. | Full name of the image. It should contain the registry part if needed. |
| `entrypoint` | no | 9.4 |Command or script to execute as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. | | `entrypoint` | No. | Command or script to execute as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. |
### Overriding the entrypoint of an image ### Overriding the entrypoint of an image
> Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended configuration options](../docker/using_docker_images.md#extended-docker-configuration-options). > Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended configuration options](../docker/using_docker_images.md#extended-docker-configuration-options).
Before showing the available entrypoint override methods, let's describe Before explaining the available entrypoint override methods, let's describe
how the runner starts. It uses a Docker image for the containers used in the how the runner starts. It uses a Docker image for the containers used in the
CI/CD jobs: CI/CD jobs:
1. The runner starts a Docker container using the defined entrypoint (default 1. The runner starts a Docker container using the defined entrypoint. The default
from `Dockerfile` that may be overridden in `.gitlab-ci.yml`) from `Dockerfile` that may be overridden in the `.gitlab-ci.yml` file.
1. The runner attaches itself to a running container. 1. The runner attaches itself to a running container.
1. The runner prepares a script (the combination of 1. The runner prepares a script (the combination of
[`before_script`](../yaml/README.md#before_script), [`before_script`](../yaml/README.md#before_script),
...@@ -157,14 +155,13 @@ CI/CD jobs: ...@@ -157,14 +155,13 @@ CI/CD jobs:
1. The runner sends the script to the container's shell `stdin` and receives the 1. The runner sends the script to the container's shell `stdin` and receives the
output. output.
To override the entrypoint of a Docker image, you should To override the entrypoint of a Docker image,
define an empty `entrypoint` in `.gitlab-ci.yml`, so the runner does not start define an empty `entrypoint` in the `.gitlab-ci.yml` file, so the runner does not start
a useless shell layer. However, that does not work for all Docker versions, and a useless shell layer. However, that does not work for all Docker versions.
you should check which one your runner is using:
- _If Docker 17.06 or later is used,_ the `entrypoint` can be set to an empty value. - For Docker 17.06 and later, the `entrypoint` can be set to an empty value.
- _If Docker 17.03 or previous versions are used,_ the `entrypoint` can be set to - For Docker 17.03 and earlier, the `entrypoint` can be set to
`/bin/sh -c`, `/bin/bash -c` or an equivalent shell available in the image. `/bin/sh -c`, `/bin/bash -c`, or an equivalent shell available in the image.
The syntax of `image:entrypoint` is similar to [Dockerfile's `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint). The syntax of `image:entrypoint` is similar to [Dockerfile's `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint).
...@@ -184,7 +181,7 @@ With the extended Docker configuration options, instead of: ...@@ -184,7 +181,7 @@ With the extended Docker configuration options, instead of:
You can now define an `entrypoint` in the `.gitlab-ci.yml` file. You can now define an `entrypoint` in the `.gitlab-ci.yml` file.
**For Docker 17.06+:** **For Docker 17.06 and later:**
```yaml ```yaml
image: image:
...@@ -192,7 +189,7 @@ image: ...@@ -192,7 +189,7 @@ image:
entrypoint: [""] entrypoint: [""]
``` ```
**For Docker =< 17.03:** **For Docker 17.03 and earlier:**
```yaml ```yaml
image: image:
...@@ -217,40 +214,38 @@ that runner. ...@@ -217,40 +214,38 @@ that runner.
To access private container registries, the GitLab Runner process can use: To access private container registries, the GitLab Runner process can use:
- [Statically defined credentials](#using-statically-defined-credentials). That is, a username and password for a specific registry. - [Statically defined credentials](#use-statically-defined-credentials). That is, a username and password for a specific registry.
- [Credentials Store](#use-a-credentials-store). For more information, read [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credentials-store). - [Credentials Store](#use-a-credentials-store). For more information, see [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credentials-store).
- [Credential Helpers](#use-credential-helpers). For more information, read [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers). - [Credential Helpers](#use-credential-helpers). For more information, see [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers).
To define which should be used, the GitLab Runner process reads the configuration in the following order: To define which option should be used, the runner process reads the configuration in this order:
- `DOCKER_AUTH_CONFIG` variable provided as either: - A `DOCKER_AUTH_CONFIG` variable provided as either:
- A [CI/CD variable](../variables/README.md) in `.gitlab-ci.yml`. - A [CI/CD variable](../variables/README.md) in the `.gitlab-ci.yml` file.
- A project's variables stored on the projects **Settings > CI/CD** page. - A project's variables stored on the project's **Settings > CI/CD** page.
- `DOCKER_AUTH_CONFIG` variable provided as environment variable in `config.toml` of the runner. - A `DOCKER_AUTH_CONFIG` variable provided as environment variable in the runner's `config.toml` file.
- `config.json` file placed in `$HOME/.docker` directory of the user running GitLab Runner process. - A `config.json` file in `$HOME/.docker` directory of the user running the process.
If the `--user` flag is provided to run the GitLab Runner child processes as unprivileged user, If the `--user` flag is provided to run the child processes as unprivileged user,
the home directory of the main GitLab Runner process user is used. the home directory of the main runner process user is used.
GitLab Runner reads this configuration **only** from `config.toml` and ignores it if The runner reads this configuration **only** from the `config.toml` file and ignores it if
it's provided as an environment variable. This is because GitLab Runner uses **only** it's provided as a CI/CD variable. This is because the runner uses **only**
`config.toml` configuration and does not interpolate **any** environment variables at `config.toml` configuration and does not interpolate **any** CI/CD variables at
runtime. runtime.
### Requirements and limitations ### Requirements and limitations
- This feature requires GitLab Runner **1.8** or higher.
- For GitLab Runner versions **>= 0.6, <1.8** there was a partial
support for using private registries, which required manual configuration
of credentials on runner's host. We recommend to upgrade your runner to
at least version **1.8** if you want to use private registries.
- Available for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html) - Available for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html)
in GitLab Runner 13.1 and later. in GitLab Runner 13.1 and later.
- [Credentials Store](#use-a-credentials-store) and [Credential Helpers](#use-credential-helpers) require binaries to be added to the GitLab Runner's `$PATH`, and require access to do so. Therefore, these features are not available on shared runners, or any other runner where the user does not have access to the environment where the runner is installed. - [Credentials Store](#use-a-credentials-store) and [Credential Helpers](#use-credential-helpers)
require binaries to be added to the GitLab Runner `$PATH`, and require access to do so. Therefore,
these features are not available on shared runners, or any other runner where the user does not
have access to the environment where the runner is installed.
### Using statically-defined credentials ### Use statically-defined credentials
There are two approaches that you can take in order to access a There are two approaches that you can take to access a
private registry. Both require setting the environment variable private registry. Both require setting the CI/CD variable
`DOCKER_AUTH_CONFIG` with appropriate authentication information. `DOCKER_AUTH_CONFIG` with appropriate authentication information.
1. Per-job: To configure one job to access a private registry, add 1. Per-job: To configure one job to access a private registry, add
...@@ -261,7 +256,7 @@ private registry. Both require setting the environment variable ...@@ -261,7 +256,7 @@ private registry. Both require setting the environment variable
See below for examples of each. See below for examples of each.
#### Determining your `DOCKER_AUTH_CONFIG` data #### Determine your `DOCKER_AUTH_CONFIG` data
As an example, let's assume you want to use the `registry.example.com:5000/private/image:latest` As an example, let's assume you want to use the `registry.example.com:5000/private/image:latest`
image. This image is private and requires you to sign in to a private container image. This image is private and requires you to sign in to a private container
...@@ -292,7 +287,7 @@ Use one of the following methods to determine the value of `DOCKER_AUTH_CONFIG`: ...@@ -292,7 +287,7 @@ Use one of the following methods to determine the value of `DOCKER_AUTH_CONFIG`:
docker logout registry.example.com:5000 docker logout registry.example.com:5000
``` ```
- In some setups, it's possible that Docker client uses the available system key - In some setups, it's possible the Docker client uses the available system key
store to store the result of `docker login`. In that case, it's impossible to store to store the result of `docker login`. In that case, it's impossible to
read `~/.docker/config.json`, so you must prepare the required read `~/.docker/config.json`, so you must prepare the required
base64-encoded version of `${username}:${password}` and create the Docker base64-encoded version of `${username}:${password}` and create the Docker
...@@ -318,7 +313,7 @@ Use one of the following methods to determine the value of `DOCKER_AUTH_CONFIG`: ...@@ -318,7 +313,7 @@ Use one of the following methods to determine the value of `DOCKER_AUTH_CONFIG`:
} }
``` ```
#### Configuring a job #### Configure a job
To configure a single job with access for `registry.example.com:5000`, To configure a single job with access for `registry.example.com:5000`,
follow these steps: follow these steps:
...@@ -337,7 +332,7 @@ follow these steps: ...@@ -337,7 +332,7 @@ follow these steps:
``` ```
1. You can now use any private image from `registry.example.com:5000` defined in 1. You can now use any private image from `registry.example.com:5000` defined in
`image` and/or `services` in your `.gitlab-ci.yml` file: `image` or `services` in your `.gitlab-ci.yml` file:
```yaml ```yaml
image: registry.example.com:5000/namespace/image:tag image: registry.example.com:5000/namespace/image:tag
...@@ -351,19 +346,19 @@ registries to the `"auths"` hash as described above. ...@@ -351,19 +346,19 @@ registries to the `"auths"` hash as described above.
The full `hostname:port` combination is required everywhere The full `hostname:port` combination is required everywhere
for the runner to match the `DOCKER_AUTH_CONFIG`. For example, if for the runner to match the `DOCKER_AUTH_CONFIG`. For example, if
`registry.example.com:5000/namespace/image:tag` is specified in `.gitlab-ci.yml`, `registry.example.com:5000/namespace/image:tag` is specified in the `.gitlab-ci.yml` file,
then the `DOCKER_AUTH_CONFIG` must also specify `registry.example.com:5000`. then the `DOCKER_AUTH_CONFIG` must also specify `registry.example.com:5000`.
Specifying only `registry.example.com` does not work. Specifying only `registry.example.com` does not work.
### Configuring a runner ### Configuring a runner
If you have many pipelines that access the same registry, it is If you have many pipelines that access the same registry, you should
probably better to set up registry access at the runner level. This set up registry access at the runner level. This
allows pipeline authors to have access to a private registry just by allows pipeline authors to have access to a private registry just by
running a job on the appropriate runner. It also makes registry running a job on the appropriate runner. It also helps simplify registry
changes and credential rotations much simpler. changes and credential rotations.
Of course this means that any job on that runner can access the This means that any job on that runner can access the
registry with the same privilege, even across projects. If you need to registry with the same privilege, even across projects. If you need to
control access to the registry, you need to be sure to control control access to the registry, you need to be sure to control
access to the runner. access to the runner.
...@@ -388,10 +383,10 @@ To add `DOCKER_AUTH_CONFIG` to a runner: ...@@ -388,10 +383,10 @@ To add `DOCKER_AUTH_CONFIG` to a runner:
### Use a Credentials Store ### Use a Credentials Store
To configure a credentials store: To configure a Credentials Store:
1. To use a credentials store, you need an external helper program to interact with a specific keychain or external store. 1. To use a Credentials Store, you need an external helper program to interact with a specific keychain or external store.
Make sure the helper program is available in GitLab Runner `$PATH`. Make sure the helper program is available in the GitLab Runner `$PATH`.
1. Make GitLab Runner use it. There are two ways to accomplish this. Either: 1. Make GitLab Runner use it. There are two ways to accomplish this. Either:
...@@ -416,14 +411,14 @@ pulling from Docker Hub fails. Docker daemon tries to use the same credentials f ...@@ -416,14 +411,14 @@ pulling from Docker Hub fails. Docker daemon tries to use the same credentials f
### Use Credential Helpers ### Use Credential Helpers
> Support for using Credential Helpers was added in GitLab Runner 12.0 > Introduced in GitLab Runner 12.0.
As an example, let's assume that you want to use the `aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest` As an example, let's assume that you want to use the `aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest`
image. This image is private and requires you to log in into a private container registry. image. This image is private and requires you to log in into a private container registry.
To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow these steps: To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow these steps:
1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`. 1. Make sure `docker-credential-ecr-login` is available in the GitLab Runner `$PATH`.
1. Have any of the following [AWS credentials setup](https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials). 1. Have any of the following [AWS credentials setup](https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials).
Make sure that GitLab Runner can access the credentials. Make sure that GitLab Runner can access the credentials.
1. Make GitLab Runner use it. There are two ways to accomplish this. Either: 1. Make GitLab Runner use it. There are two ways to accomplish this. Either:
...@@ -440,9 +435,9 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th ...@@ -440,9 +435,9 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
} }
``` ```
This configures Docker to use the credential helper for a specific registry. This configures Docker to use the Credential Helper for a specific registry.
or Instead, you can configure Docker to use the Credential Helper for all Amazon Elastic Container Registry (ECR) registries:
```json ```json
{ {
...@@ -450,10 +445,8 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th ...@@ -450,10 +445,8 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
} }
``` ```
This configures Docker to use the credential helper for all Amazon Elastic Container Registry (ECR) registries.
- Or, if you're running self-managed runners, - Or, if you're running self-managed runners,
add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. add the previous JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`.
GitLab Runner reads this configuration file and uses the needed helper for this GitLab Runner reads this configuration file and uses the needed helper for this
specific repository. specific repository.
...@@ -464,8 +457,8 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th ...@@ -464,8 +457,8 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest
``` ```
In the example above, GitLab Runner looks at `aws_account_id.dkr.ecr.region.amazonaws.com` for the In the example, GitLab Runner looks at `aws_account_id.dkr.ecr.region.amazonaws.com` for the
image `private/image:latest`. image `private/image:latest`.
You can add configuration for as many registries as you want, adding more You can add configuration for as many registries as you want, adding more
registries to the `"credHelpers"` hash as described above. registries to the `"credHelpers"` hash.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7