Commit f82a4fd6 authored by Rémy Coutable's avatar Rémy Coutable

Update the pipelines documentation with the latest changes

Signed-off-by: default avatarRémy Coutable <remy@rymai.me>
parent f949ae7f
......@@ -64,180 +64,62 @@ Most of the jobs [extend from a few CI definitions](../ci/yaml/README.md#extends
that are scoped to a single
[configuration parameter](../ci/yaml/README.md#configuration-parameters).
These common definitions are:
- `.default-tags`: Ensures a job has the `gitlab-org` tag to ensure it's using
our dedicated runners.
- `.default-retry`: Allows a job to [retry](../ci/yaml/README.md#retry) upon `unknown_failure`, `api_failure`,
`runner_system_failure`, `job_execution_timeout`, or `stuck_or_timeout_failure`.
- `.default-before_script`: Allows a job to use a default `before_script` definition
suitable for Ruby/Rails tasks that may need a database running (e.g. tests).
- `.default-cache`: Allows a job to use a default `cache` definition suitable for
Ruby/Rails and frontend tasks.
- `.default-only`: Restricts the cases where a job is created. This currently
includes `master`, `/^[\d-]+-stable(-ee)?$/` (stable branches),
`/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), `merge_requests`, `tags`.
Note that jobs won't be created for branches with this default configuration.
- `.only:variables-canonical-dot-com`: Only creates a job if the project is
located under <https://gitlab.com/gitlab-org>.
- `.only:variables_refs-canonical-dot-com-schedules`: Same as
`.only:variables-canonical-dot-com` but add the condition that pipeline is scheduled.
- `.except:refs-deploy`: Don't create a job if the `ref` is an auto-deploy branch.
- `.except:refs-master-tags-stable-deploy`: Don't create a job if the `ref` is one of:
- `master`
- a tag
- a stable branch
- an auto-deploy branch
- `.only:kubernetes`: Only creates a job if a Kubernetes integration is enabled
on the project.
- `.only-review`: This extends from:
- `.only:variables-canonical-dot-com`
- `.only:kubernetes`
- `.except:refs-master-tags-stable-deploy`
- `.only-review-schedules`: This extends from:
- `.only:variables_refs-canonical-dot-com-schedules`
- `.only:kubernetes`
- `.except:refs-deploy`
- `.use-pg9`: Allows a job to use the `postgres:9.6` and `redis:alpine` services.
- `.use-pg10`: Allows a job to use the `postgres:10.9` and `redis:alpine` services.
- `.use-pg9-ee`: Same as `.use-pg9` but also use the
`docker.elastic.co/elasticsearch/elasticsearch:5.6.12` services.
- `.use-pg10-ee`: Same as `.use-pg10` but also use the
`docker.elastic.co/elasticsearch/elasticsearch:5.6.12` services.
- `.only-ee`: Only creates a job for the `gitlab` or `gitlab-ee` project.
- `.only-ee-as-if-foss`: Same as `.only-ee` but simulate the FOSS project by
setting the `FOSS_ONLY='1'` environment variable.
## Changes detection
If a job extends from `.default-only` (and most of the jobs should), it can restrict
the cases where it should be created
[based on the changes](../ci/yaml/README.md#onlychangesexceptchanges)
from a commit or MR by extending from the following CI definitions:
- `.only:changes-code`: Allows a job to only be created upon code-related changes.
- `.only:changes-qa`: Allows a job to only be created upon QA-related changes.
- `.only:changes-docs`: Allows a job to only be created upon docs-related changes.
- `.only:changes-graphql`: Allows a job to only be created upon GraphQL-related changes.
- `.only:changes-code-backstage`: Allows a job to only be created upon code-related or backstage-related (e.g. Danger, RuboCop, specs) changes.
- `.only:changes-code-qa`: Allows a job to only be created upon code-related or QA-related changes.
- `.only:changes-code-backstage-qa`: Allows a job to only be created upon code-related, backstage-related (e.g. Danger, RuboCop, specs) or QA-related changes.
**See <https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml>
for the list of exact patterns.**
## Rules conditions and changes patterns
We're making use of the [`rules` keyword](https://docs.gitlab.com/ee/ci/yaml/#rules) but we're currently
duplicating the `if` conditions and `changes` patterns lists since they cannot be shared across
`include`d files as we do with `extends`.
**If you update an `if` condition or `changes`
patterns list, make sure to mass-update those across all the CI config files (i.e. `.gitlab/ci/*.yml`).**
### Canonical/security namespace merge requests only
This condition limits jobs creation to merge requests under the `gitlab-org/` top-level group
on GitLab.com only (i.e. this won't run for `master`, stable or auto-deploy branches).
This is similar to the `.only:variables-canonical-dot-com` + `only:refs: [merge_requests]`
CI definitions.
The definition for `if-canonical-dot-com-gitlab-org-groups-merge-request` can be
seen in <https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/docs.gitlab-ci.yml>.
### Canonical/security namespace tags only
This condition limits jobs creation to tags under the `gitlab-org/` top-level group
on GitLab.com only.
This is similar to the `.only:variables-canonical-dot-com` + `only:refs: [tags]` CI definition:
The definition for `if-canonical-dot-com-gitlab-org-groups-tag` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/cng.gitlab-ci.yml>.
### Canonical namespace `master` only
This condition limits jobs creation to `master` pipelines for the `gitlab-org` top-level group
on GitLab.com only.
This is similar to the `.only:variables-canonical-dot-com` + `only:refs: [master]` CI definition:
The definition for `if-canonical-dot-com-gitlab-org-group-master-refs` can be
seen in <https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/pages.gitlab-ci.yml>.
### Canonical namespace schedules only
This condition limits jobs creation to scheduled pipelines for the `gitlab-org` top-level group
on GitLab.com only.
This is similar to the `.only:variables-canonical-dot-com` + `only:refs: [schedules]` CI definition:
The definition for `if-canonical-dot-com-gitlab-org-group-schedule` can be seen
in <https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa.gitlab-ci.yml>.
### Not canonical/security namespace
This condition matches if the project isn't in the canonical/security namespace.
Useful to **not** create a job if the project is a fork, or in other words, when
a job should only run in the canonical projects.
The definition for `if-not-canonical-namespace` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
### Not EE
This condition matches if the project isn't EE. Useful to **not** create a job if
the project is GitLab, or in other words, when a job should only run in the GitLab
FOSS project.
The definition for `if-not-ee` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
### Default refs only
This condition is the equivalent of `.default-only`.
The definition for `if-default-refs` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
### `master` refs only
This condition is the equivalent of `only:refs: [master]`.
The definition for `if-master-refs` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
### Code changes patterns
Similar patterns as for `.only:changes-code`:
The definition for `code-patterns` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa.gitlab-ci.yml>.
### QA changes patterns
Similar patterns as for `.only:changes-qa`:
The definition for `qa-patterns` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa.gitlab-ci.yml>.
### Docs changes patterns
Similar patterns as for `.only:changes-docs`:
The definition for `docs-patterns` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/docs.gitlab-ci.yml>.
### Code and QA changes patterns
Similar patterns as for `.only:changes-code-qa`:
The definition for `code-qa-patterns` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/review.gitlab-ci.yml>.
### Code, backstage and QA changes patterns
Similar patterns as for `.only:changes-code-backstage-qa`:
The definition for `code-backstage-qa-patterns` can be seen in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
| Job definitions | Description |
|------------------|-------------|
| `.default-tags` | Ensures a job has the `gitlab-org` tag to ensure it's using our dedicated runners. |
| `.default-retry` | Allows a job to [retry](../ci/yaml/README.md#retry) upon `unknown_failure`, `api_failure`, `runner_system_failure`, `job_execution_timeout`, or `stuck_or_timeout_failure`. |
| `.default-before_script` | Allows a job to use a default `before_script` definition suitable for Ruby/Rails tasks that may need a database running (e.g. tests). |
| `.default-cache` | Allows a job to use a default `cache` definition suitable for Ruby/Rails and frontend tasks. |
| `.use-pg9` | Allows a job to use the `postgres:9.6.17` and `redis:alpine` services. |
| `.use-pg10` | Allows a job to use the `postgres:10.12` and `redis:alpine` services. |
| `.use-pg9-ee` | Same as `.use-pg9` but also use the `docker.elastic.co/elasticsearch/elasticsearch:6.4.2` services. |
| `.use-pg10-ee` | Same as `.use-pg10` but also use the `docker.elastic.co/elasticsearch/elasticsearch:6.4.2` services. |
| `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` environment variable. |
## `rules`, `if:` conditions and `changes:` patterns
We're using the [`rules` keyword](../ci/yaml/README.md#rules) extensively.
All `rules` definitions are defined in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml>,
then included in individual jobs via [`extends`](../ci/yaml/README.md#extends).
The `rules` definitions are composed of `if:` conditions and `changes:` patterns,
which are also defined in
<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml>
and included in `rules` definitions via [YAML anchors](../ci/yaml/README.md#anchors)
### `if:` conditions
| `if:` conditions | Description | Notes |
|------------------|-------------|-------|
| `if-not-canonical-namespace` | Matches if the project isn't in the canonical (`gitlab-org/`) or security (`gitlab-org/security`) namespace. | Use to create a job for forks (by using `when: on_success\|manual`), or **not** create a job for forks (by using `when: never`). |
| `if-not-ee` | Matches if the project isn't EE (i.e. project name isn't `gitlab` or `gitlab-ee`). | Use to create a job only in the FOSS project (by using `when: on_success|manual`), or **not** create a job if the project is EE (by using `when: never`). |
| `if-not-foss` | Matches if the project isn't FOSS (i.e. project name isn't `gitlab-foss`, `gitlab-ce`, or `gitlabhq`). | Use to create a job only in the EE project (by using `when: on_success|manual`), or **not** create a job if the project is FOSS (by using `when: never`). |
| `if-default-refs` | Matches if the pipeline is for `master`, `/^[\d-]+-stable(-ee)?$/` (stable branches), `/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), merge requests, and tags. | Note that jobs won't be created for branches with this default configuration. |
| `if-master-refs` | Matches if the current branch is `master`. | |
| `if-master-or-tag` | Matches if the pipeline is for the `master` branch or for a tag. | |
| `if-merge-request` | Matches if the pipeline is for a merge request. | |
| `if-dot-com-gitlab-org-schedule` | Limits jobs creation to scheduled pipelines for the `gitlab-org` group on GitLab.com. | |
| `if-dot-com-gitlab-org-master` | Limits jobs creation to the `master` branch for the `gitlab-org` group on GitLab.com. | |
| `if-dot-com-gitlab-org-merge-request` | Limits jobs creation to merge requests for the `gitlab-org` group on GitLab.com. | |
| `if-dot-com-gitlab-org-and-security-tag` | Limits job creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | |
| `if-dot-com-gitlab-org-and-security-merge-request` | Limit jobs creation to merge requests for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | |
| `if-dot-com-ee-schedule` | Limits jobs to scheduled pipelines for the `gitlab-org/gitlab` project on GitLab.com. | |
| `if-cache-credentials-schedule` | Limits jobs to scheduled pipelines with the `$CI_REPO_CACHE_CREDENTIALS` variable set. | |
### `changes:` patterns
| `changes:` patterns | Description |
|------------------------------|--------------------------------------------------------------------------|
| `yaml-patterns` | Only create job for YAML-related changes. |
| `docs-patterns` | Only create job for docs-related changes. |
| `backstage-patterns` | Only create job for backstage-related changes. |
| `code-patterns` | Only create job for code-related changes. |
| `qa-patterns` | Only create job for QA-related changes. |
| `code-backstage-patterns` | Combination of `code-patterns` and `backstage-patterns`. |
| `code-qa-patterns` | Combination of `code-patterns` and `qa-patterns`. |
| `code-backstage-qa-patterns` | Combination of `code-patterns`, `backstage-patterns`, and `qa-patterns`. |
## Directed acyclic graph
......@@ -255,11 +137,11 @@ graph RL;
G[review-deploy];
G2["schedule:review-deploy<br/>(master only)"];
I["karma, jest, webpack-dev-server, static-analysis"];
I2["karma-foss, jest-foss<br/>(EE default refs only)"];
I2["karma-as-if-foss, jest-as-if-foss<br/>(EE default refs only)"];
J["compile-assets pull-push-cache<br/>(master only)"];
J2["compile-assets pull-push-cache foss<br/>(EE master only)"];
J2["compile-assets pull-push-cache as-if-foss<br/>(EE master only)"];
K[compile-assets pull-cache];
K2["compile-assets pull-cache foss<br/>(EE default refs only)"];
K2["compile-assets pull-cache as-if-foss<br/>(EE default refs only)"];
M[coverage];
N["pages (master only)"];
Q[package-and-qa];
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment