Skip to content

G
gitlab-ce
Commit 275af535 authored by Marcel Amirault's avatar Marcel Amirault
Browse files
Options

Merge branch 'selhorn-includes-edits' into 'master' 

Updated include keyword to meet standards

See merge request gitlab-org/gitlab!73872
parents 0dca037a d0f39aca
Show whitespace changes
Inline Side-by-side
Showing with 175 additions and 167 deletions
doc/ci/yaml/includes.md
View file @ 275af535
...@@ -216,3 +216,101 @@ default: ...@@ -216,3 +216,101 @@ default:
after_script: after_script:
- echo "Job complete." - echo "Job complete."
``` ```
## Use variables with `include`
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/284883) in GitLab 13.8.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/294294) in GitLab 13.9.
> - [Support for project, group, and instance variables added](https://gitlab.com/gitlab-org/gitlab/-/issues/219065) in GitLab 14.2.
> - [Support for pipeline variables added](https://gitlab.com/gitlab-org/gitlab/-/issues/337633) in GitLab 14.5.
In `include` sections in your `.gitlab-ci.yml` file, you can use:
- [Project variables](../variables/index.md#add-a-cicd-variable-to-a-project)
- [Group variables](../variables/index.md#add-a-cicd-variable-to-a-group)
- [Instance variables](../variables/index.md#add-a-cicd-variable-to-an-instance)
- Project [predefined variables](../variables/predefined_variables.md)
- In GitLab 14.2 and later, the `$CI_COMMIT_REF_NAME` [predefined variable](../variables/predefined_variables.md).
When used in `include`, the `CI_COMMIT_REF_NAME` variable returns the full
ref path, like `refs/heads/branch-name`. In `include:rules`, you might need to use
`if: $CI_COMMIT_REF_NAME =~ /main/` (not `== main`). This behavior is resolved in GitLab 14.5.
In GitLab 14.5 and later, you can also use:
- [Trigger variables](../triggers/index.md#making-use-of-trigger-variables).
- [Scheduled pipeline variables](../pipelines/schedules.md#using-variables).
- [Manual pipeline run variables](../variables/index.md#override-a-variable-when-running-a-pipeline-manually).
- Pipeline [predefined variables](../variables/predefined_variables.md).
YAML files are parsed before the pipeline is created, so the following pipeline predefined variables
are **not** available:
- `CI_PIPELINE_ID`
- `CI_PIPELINE_URL`
- `CI_PIPELINE_IID`
- `CI_PIPELINE_CREATED_AT`
For example:
```yaml
include:
project: '$CI_PROJECT_PATH'
file: '.compliance-gitlab-ci.yml'
```
For an example of how you can include these predefined variables, and the variables' impact on CI/CD jobs,
see this [CI/CD variable demo](https://youtu.be/4XR8gw3Pkos).
## Use `rules` with `include`
> - Introduced in GitLab 14.2 [with a flag](../../administration/feature_flags.md) named `ci_include_rules`. Disabled by default.
> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.3.
> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) GitLab 14.3.
> - [Feature flag `ci_include_rules` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.4.
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.4.
You can use [`rules`](index.md#rules) with `include` to conditionally include other configuration files.
You can only use [`if` rules](index.md#rulesif) in `include`, and only with [certain variables](#use-variables-with-include).
`rules` keywords such as `changes` and `exists` are not supported.
```yaml
include:
- local: builds.yml
rules:
- if: '$INCLUDE_BUILDS == "true"'
- local: deploys.yml
rules:
- if: $CI_COMMIT_BRANCH == "main"
test:
stage: test
script: exit 0
```
## Use `include:local` with wildcard file paths
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25921) in GitLab 13.11.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/327315) in GitLab 14.2.
You can use wildcard paths (`*` and `**`) with `include:local`.
Example:
```yaml
include: 'configs/*.yml'
```
When the pipeline runs, GitLab:
- Adds all `.yml` files in the `configs` directory into the pipeline configuration.
- Does not add `.yml` files in subfolders of the `configs` directory. To allow this,
add the following configuration:
```yaml
# This matches all `.yml` files in `configs` and any subfolder in it.
include: 'configs/**.yml'
# This matches all `.yml` files only in subfolders of `configs`.
include: 'configs/**/*.yml'
```
doc/ci/yaml/index.md
View file @ 275af535
...@@ -36,7 +36,6 @@ The keywords available for jobs are: ...@@ -36,7 +36,6 @@ The keywords available for jobs are:
| [`except`](#only--except) | Control when jobs are not created. | | [`except`](#only--except) | Control when jobs are not created. |
| [`extends`](#extends) | Configuration entries that this job inherits from. | | [`extends`](#extends) | Configuration entries that this job inherits from. |
| [`image`](#image) | Use Docker images. | | [`image`](#image) | Use Docker images. |
| [`include`](#include) | Include external YAML files. |
| [`inherit`](#inherit) | Select which global defaults all jobs inherit. | | [`inherit`](#inherit) | Select which global defaults all jobs inherit. |
| [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. | | [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. |
| [`needs`](#needs) | Execute jobs earlier than the stage ordering. | | [`needs`](#needs) | Execute jobs earlier than the stage ordering. |
...@@ -382,126 +381,53 @@ does not block triggered pipelines. ...@@ -382,126 +381,53 @@ does not block triggered pipelines.
> [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/42861) to GitLab Free in 11.4. > [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/42861) to GitLab Free in 11.4.
Use `include` to include external YAML files in your CI/CD configuration. Use `include` to include external YAML files in your CI/CD configuration.
You can break down one long `.gitlab-ci.yml` file into multiple files to increase readability, You can split one long `.gitlab-ci.yml` file into multiple files to increase readability,
or reduce duplication of the same configuration in multiple places. or reduce duplication of the same configuration in multiple places.
You can also store template files in a central repository and `include` them in projects. You can also store template files in a central repository and include them in projects.
`include` requires the external YAML file to have the extensions `.yml` or `.yaml`,
otherwise the external file is not included.
You can't use [YAML anchors](yaml_specific_features.md#anchors) across different YAML files sourced by `include`.
You can only refer to anchors in the same file. To reuse configuration from different
YAML files, use [`!reference` tags](yaml_specific_features.md#reference-tags) or the [`extends` keyword](#extends).
`include` supports the following inclusion methods:
| Keyword | Method |
|:--------------------------------|:------------------------------------------------------------------|
| [`local`](#includelocal) | Include a file from the local project repository. |
| [`file`](#includefile) | Include a file from a different project repository. |
| [`remote`](#includeremote) | Include a file from a remote URL. Must be publicly accessible. |
| [`template`](#includetemplate) | Include templates that are provided by GitLab. |
When the pipeline starts, the `.gitlab-ci.yml` file configuration included by all methods is evaluated.
The configuration is a snapshot in time and persists in the database. GitLab does not reflect any changes to
the referenced `.gitlab-ci.yml` file configuration until the next pipeline starts.
The `include` files are: The `include` files are:
- Deep merged with those in the `.gitlab-ci.yml` file. - Merged with those in the `.gitlab-ci.yml` file.
- Always evaluated first and merged with the content of the `.gitlab-ci.yml` file, - Always evaluated first and then merged with the content of the `.gitlab-ci.yml` file,
regardless of the position of the `include` keyword. regardless of the position of the `include` keyword.
NOTE: You can [nest](includes.md#use-nested-includes) up to 100 includes, but you can't have duplicate includes.
Use merging to customize and override included CI/CD configurations with local In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/28212),
configurations. Local configurations in the `.gitlab-ci.yml` file override included configurations. the time limit to resolve all files is 30 seconds.
#### Variables with `include`
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/284883) in GitLab 13.8.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/294294) in GitLab 13.9.
> - [Support for project, group, and instance variables added](https://gitlab.com/gitlab-org/gitlab/-/issues/219065) in GitLab 14.2.
> - [Support for pipeline variables added](https://gitlab.com/gitlab-org/gitlab/-/issues/337633) in GitLab 14.5.
In `include` sections in your `.gitlab-ci.yml` file, you can use:
- [Project variables](../variables/index.md#add-a-cicd-variable-to-a-project) **Keyword type**: Global keyword.
- [Group variables](../variables/index.md#add-a-cicd-variable-to-a-group)
- [Instance variables](../variables/index.md#add-a-cicd-variable-to-an-instance)
- Project [predefined variables](../variables/predefined_variables.md)
- In GitLab 14.2 and later, the `$CI_COMMIT_REF_NAME` [predefined variable](../variables/predefined_variables.md).
When used in `include`, the `CI_COMMIT_REF_NAME` variable returns the full **Possible inputs**: The `include` subkeys:
ref path, like `refs/heads/branch-name`. In `include:rules`, you might need to use
`if: $CI_COMMIT_REF_NAME =~ /main/` (not `== main`). This behavior is resolved in GitLab 14.5.
In GitLab 14.5 and later, you can also use: - [`include:local`](#includelocal)
- [`include:file`](#includefile)
- [Trigger variables](../triggers/index.md#making-use-of-trigger-variables). - [`include:remote`](#includeremote)
- [Scheduled pipeline variables](../pipelines/schedules.md#using-variables). - [`include:template`](#includetemplate)
- [Manual pipeline run variables](../variables/index.md#override-a-variable-when-running-a-pipeline-manually).
- Pipeline [predefined variables](../variables/predefined_variables.md).
YAML files are parsed before the pipeline is created, so the following pipeline predefined variables
are **not** available:
- `CI_PIPELINE_ID`
- `CI_PIPELINE_URL`
- `CI_PIPELINE_IID`
- `CI_PIPELINE_CREATED_AT`
For example:
```yaml
include:
project: '$CI_PROJECT_PATH'
file: '.compliance-gitlab-ci.yml'
```
For an example of how you can include these predefined variables, and the variables' impact on CI/CD jobs, **Additional details**:
see this [CI/CD variable demo](https://youtu.be/4XR8gw3Pkos).
#### `rules` with `include` - Use merging to customize and override included CI/CD configurations with local
- You can override included configuration by having the same job name or global keyword
in the `.gitlab-ci.yml` file. The two configurations are merged together, and the
configuration in the `.gitlab-ci.yml` file takes precedence over the included configuration.
> - Introduced in GitLab 14.2 [with a flag](../../administration/feature_flags.md) named `ci_include_rules`. Disabled by default. **Related topics**:
> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.3.
> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) GitLab 14.3.
> - [Feature flag `ci_include_rules` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.4.
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.4.
You can use [`rules`](#rules) with `include` to conditionally include other configuration files. - [Use variables with `include`](includes.md#use-variables-with-include).
You can only use [`if` rules](#rulesif) in `include`, and only with [certain variables](#variables-with-include). - [Use `rules` with `include`](includes.md#use-rules-with-include).
`rules` keywords such as `changes` and `exists` are not supported.
```yaml
include:
- local: builds.yml
rules:
- if: '$INCLUDE_BUILDS == "true"'
- local: deploys.yml
rules:
- if: $CI_COMMIT_BRANCH == "main"
test:
stage: test
script: exit 0
```
#### `include:local` #### `include:local`
Use `include:local` to include a file that is in the same repository as the `.gitlab-ci.yml` file. Use `include:local` to include a file that is in the same repository as the `.gitlab-ci.yml` file.
Use a full path relative to the root directory (`/`). Use `include:local` instead of symbolic links.
If you use `include:local`, make sure that both the `.gitlab-ci.yml` file and the local file **Keyword type**: Global keyword.
are on the same branch.
You can't include local files through Git submodules paths. **Possible inputs**: A full path relative to the root directory (`/`).
The YAML file must have the extension `.yml` or `.yaml`. Wildcard paths (`*` and `**`) are supported.
All [nested includes](#nested-includes) are executed in the scope of the same project, **Example of `include:local`**:
so it's possible to use local, project, remote, or template includes.
Example:
```yaml ```yaml
include: include:
...@@ -514,44 +440,31 @@ You can also use shorter syntax to define the path: ...@@ -514,44 +440,31 @@ You can also use shorter syntax to define the path:
include: '.gitlab-ci-production.yml' include: '.gitlab-ci-production.yml'
``` ```
Use local includes instead of symbolic links. **Additional details**:
##### `include:local` with wildcard file paths
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25921) in GitLab 13.11.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/327315) in GitLab 14.2.
You can use wildcard paths (`*` and `**`) with `include:local`.
Example:
```yaml
include: 'configs/*.yml'
```
When the pipeline runs, GitLab:
- Adds all `.yml` files in the `configs` directory into the pipeline configuration. - The `.gitlab-ci.yml` file and the local file must be on the same branch.
- Does not add `.yml` files in subfolders of the `configs` directory. To allow this, - You can't include local files through Git submodules paths.
add the following configuration: - All [nested includes](includes.md#use-nested-includes) are executed in the scope of the same project,
so you can use local, project, remote, or template includes.
```yaml **Related topics**:
# This matches all `.yml` files in `configs` and any subfolder in it.
include: 'configs/**.yml'
# This matches all `.yml` files only in subfolders of `configs`. - [Use `include:local` with wildcard file paths](includes.md#use-includelocal-with-wildcard-file-paths).
include: 'configs/**/*.yml'
```
#### `include:file` #### `include:file`
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53903) in GitLab 11.7. > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53903) in GitLab 11.7.
> - Including multiple files from the same project [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26793) in GitLab 13.6. [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/271560) in GitLab 13.8.
To include files from another private project on the same GitLab instance, To include files from another private project on the same GitLab instance,
use `include:file`. You can use `include:file` in combination with `include:project` only. use `include:file`. You can use `include:file` in combination with `include:project` only.
Use a full path, relative to the root directory (`/`).
For example: **Keyword type**: Global keyword.
**Possible inputs**: A full path, relative to the root directory (`/`).
The YAML file must have the extension `.yml` or `.yaml`.
**Example of `include:file`**:
```yaml ```yaml
include: include:
...@@ -576,17 +489,6 @@ include: ...@@ -576,17 +489,6 @@ include:
file: '/templates/.gitlab-ci-template.yml' file: '/templates/.gitlab-ci-template.yml'
``` ```
All [nested includes](#nested-includes) are executed in the scope of the target project.
You can use local (relative to target project), project, remote, or template includes.
NOTE:
When including a YAML file from another private project, the user running the pipeline must be a member of both projects and have the appropriate permissions to run pipelines. A `not found or access denied` error may be shown if the user does not have access to any of the included files.
##### Multiple files from a project
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26793) in GitLab 13.6.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/271560) in GitLab 13.8.
You can include multiple files from the same project: You can include multiple files from the same project:
```yaml ```yaml
...@@ -598,33 +500,54 @@ include: ...@@ -598,33 +500,54 @@ include:
- '/templates/.tests.yml' - '/templates/.tests.yml'
``` ```
**Additional details**:
- All [nested includes](includes.md#use-nested-includes) are executed in the scope of the target project.
You can use `local` (relative to the target project), `project`, `remote`, or `template` includes.
- When the pipeline starts, the `.gitlab-ci.yml` file configuration included by all methods is evaluated.
The configuration is a snapshot in time and persists in the database. GitLab does not reflect any changes to
the referenced `.gitlab-ci.yml` file configuration until the next pipeline starts.
- When you include a YAML file from another private project, the user running the pipeline
must be a member of both projects and have the appropriate permissions to run pipelines.
A `not found or access denied` error may be displayed if the user does not have access to any of the included files.
#### `include:remote` #### `include:remote`
Use `include:remote` with a full URL to include a file from a different location. Use `include:remote` with a full URL to include a file from a different location.
The remote file must be publicly accessible by an HTTP/HTTPS `GET` request, because
authentication in the remote URL is not supported. For example: **Keyword type**: Global keyword.
**Possible inputs**: A public URL accessible by an HTTP/HTTPS `GET` request.
Authentication with the remote URL is not supported.
The YAML file must have the extension `.yml` or `.yaml`.
**Example of `include:remote`**:
```yaml ```yaml
include: include:
- remote: 'https://gitlab.com/example-project/-/raw/main/.gitlab-ci.yml' - remote: 'https://gitlab.com/example-project/-/raw/main/.gitlab-ci.yml'
``` ```
All [nested includes](#nested-includes) execute without context as a public user, **Additional details**:
so you can only `include` public projects or templates.
NOTE: - All [nested includes](includes.md#use-nested-includes) execute without context as a public user,
Be careful when including a remote CI/CD configuration file. No pipelines or notifications so you can only include public projects or templates.
trigger when external CI/CD configuration files change. From a security perspective, - Be careful when including a remote CI/CD configuration file. No pipelines or notifications
this is similar to pulling a third party dependency. trigger when external CI/CD configuration files change. From a security perspective,
this is similar to pulling a third party dependency.
#### `include:template` #### `include:template`
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53445) in GitLab 11.7. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53445) in GitLab 11.7.
Use `include:template` to include `.gitlab-ci.yml` templates that are Use `include:template` to include [`.gitlab-ci.yml` templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates).
[shipped with GitLab](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates).
For example: **Keyword type**: Global keyword.
**Possible inputs**: [`.gitlab-ci.yml` templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates).
**Example of `include:template`**:
```yaml ```yaml
# File sourced from the GitLab template collection # File sourced from the GitLab template collection
...@@ -640,23 +563,10 @@ include: ...@@ -640,23 +563,10 @@ include:
- template: Auto-DevOps.gitlab-ci.yml - template: Auto-DevOps.gitlab-ci.yml
``` ```
All [nested includes](#nested-includes) are executed only with the permission of the user, **Additional details**:
so it's possible to use project, remote or template includes.
#### Nested includes
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/56836) in GitLab 11.9.
Use nested includes to compose a set of includes.
You can have up to 100 includes, but you can't have duplicate includes.
In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/28212), the time limit
to resolve all files is 30 seconds.
#### Additional `includes` examples
View [additional `includes` examples](includes.md). - All [nested includes](includes.md#use-nested-includes) are executed only with the permission of the user,
so it's possible to use `project`, `remote`, or `template` includes.
## Keyword details ## Keyword details
...@@ -1341,7 +1251,7 @@ job: ...@@ -1341,7 +1251,7 @@ job:
all rules. You can't mix `when` at the job-level with `when` in rules. all rules. You can't mix `when` at the job-level with `when` in rules.
- Unlike variables in [`script`](../variables/index.md#use-cicd-variables-in-job-scripts) - Unlike variables in [`script`](../variables/index.md#use-cicd-variables-in-job-scripts)
sections, variables in rules expressions are always formatted as `$VARIABLE`. sections, variables in rules expressions are always formatted as `$VARIABLE`.
- You can use `rules:if` with `include` to [conditionally include other configuration files](#rules-with-include). - You can use `rules:if` with `include` to [conditionally include other configuration files](includes.md#use-rules-with-include).
**Related topics**: **Related topics**:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7