Skip to content

G
gitlab-ce
Commit 1233228f authored by Marcel Amirault's avatar Marcel Amirault
Browse files
Options

Merge branch 'selhorn-pipelines' into 'master' 

Reordered and streamlined pipeline topic

See merge request gitlab-org/gitlab!29096
parents eec7f7eb 804fc781
Show whitespace changes
Inline Side-by-side
Showing with 315 additions and 323 deletions
doc/README.md
View file @ 1233228f
...@@ -238,7 +238,7 @@ The following documentation relates to the DevOps **Verify** stage: ...@@ -238,7 +238,7 @@ The following documentation relates to the DevOps **Verify** stage:
| [GitLab CI/CD](ci/README.md) | Explore the features and capabilities of Continuous Integration with GitLab. | | [GitLab CI/CD](ci/README.md) | Explore the features and capabilities of Continuous Integration with GitLab. |
| [JUnit test reports](ci/junit_test_reports.md) | Display JUnit test reports on merge requests. | | [JUnit test reports](ci/junit_test_reports.md) | Display JUnit test reports on merge requests. |
| [Multi-project pipelines](ci/multi_project_pipelines.md) **(PREMIUM)** | Visualize entire pipelines that span multiple projects, including all cross-project inter-dependencies. | | [Multi-project pipelines](ci/multi_project_pipelines.md) **(PREMIUM)** | Visualize entire pipelines that span multiple projects, including all cross-project inter-dependencies. |
| [Pipeline Graphs](ci/pipelines/index.md#visualizing-pipelines) | Visualize builds. | | [Pipeline Graphs](ci/pipelines/index.md#visualize-pipelines) | Visualize builds. |
| [Review Apps](ci/review_apps/index.md) | Preview changes to your application right from a merge request. | | [Review Apps](ci/review_apps/index.md) | Preview changes to your application right from a merge request. |
<div align="right"> <div align="right">
......
doc/ci/jenkins/index.md
View file @ 1233228f
...@@ -50,7 +50,7 @@ There are some high level differences between the products worth mentioning: ...@@ -50,7 +50,7 @@ There are some high level differences between the products worth mentioning:
- on push - on push
- on [schedule](../pipelines/schedules.md) - on [schedule](../pipelines/schedules.md)
- from the [GitLab UI](../pipelines/index.md#manually-executing-pipelines) - from the [GitLab UI](../pipelines/index.md#run-a-pipeline-manually)
- by [API call](../triggers/README.md) - by [API call](../triggers/README.md)
- by [webhook](../triggers/README.md#triggering-a-pipeline-from-a-webhook) - by [webhook](../triggers/README.md#triggering-a-pipeline-from-a-webhook)
- by [ChatOps](../chatops/README.md) - by [ChatOps](../chatops/README.md)
......
doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md
View file @ 1233228f
...@@ -116,7 +116,7 @@ unexpected timing. For example, when a source or target branch is advanced. ...@@ -116,7 +116,7 @@ unexpected timing. For example, when a source or target branch is advanced.
In this case, the pipeline fails because of `fatal: reference is not a tree:` error, In this case, the pipeline fails because of `fatal: reference is not a tree:` error,
which indicates that the checkout-SHA is not found in the merge ref. which indicates that the checkout-SHA is not found in the merge ref.
This behavior was improved at GitLab 12.4 by introducing [Persistent pipeline refs](../../pipelines/index.md#persistent-pipeline-refs). This behavior was improved at GitLab 12.4 by introducing [Persistent pipeline refs](../../pipelines/index.md#troubleshooting-fatal-reference-is-not-a-tree).
You should be able to create pipelines at any timings without concerning the error. You should be able to create pipelines at any timings without concerning the error.
## Using Merge Trains **(PREMIUM)** ## Using Merge Trains **(PREMIUM)**
......
doc/ci/multi_project_pipelines.md
View file @ 1233228f
...@@ -38,7 +38,7 @@ With Multi-Project Pipelines you can visualize the entire pipeline, including al ...@@ -38,7 +38,7 @@ With Multi-Project Pipelines you can visualize the entire pipeline, including al
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/2121) in [GitLab Premium 9.3](https://about.gitlab.com/releases/2017/06/22/gitlab-9-3-released/#multi-project-pipeline-graphs). > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/2121) in [GitLab Premium 9.3](https://about.gitlab.com/releases/2017/06/22/gitlab-9-3-released/#multi-project-pipeline-graphs).
When you configure GitLab CI/CD for your project, you can visualize the stages of your When you configure GitLab CI/CD for your project, you can visualize the stages of your
[jobs](pipelines/index.md#configuring-pipelines) on a [pipeline graph](pipelines/index.md#visualizing-pipelines). [jobs](pipelines/index.md#configure-a-pipeline) on a [pipeline graph](pipelines/index.md#visualize-pipelines).
![Multi-project pipeline graph](img/multi_project_pipeline_graph.png) ![Multi-project pipeline graph](img/multi_project_pipeline_graph.png)
......
doc/ci/pipelines/index.md
View file @ 1233228f
...@@ -3,183 +3,193 @@ disqus_identifier: 'https://docs.gitlab.com/ee/ci/pipelines.html' ...@@ -3,183 +3,193 @@ disqus_identifier: 'https://docs.gitlab.com/ee/ci/pipelines.html'
type: reference type: reference
--- ---
# Creating and using CI/CD pipelines # CI/CD pipelines
> Introduced in GitLab 8.8. > Introduced in GitLab 8.8.
NOTE: **Tip:** NOTE: **Tip:**
Watch our Watch the
["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) ["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/)
webcast to see a comprehensive demo of GitLab CI/CD pipeline. webcast to see a comprehensive demo of a GitLab CI/CD pipeline.
Pipelines are the top-level component of continuous integration, delivery, and deployment. Pipelines are the top-level component of continuous integration, delivery, and deployment.
Pipelines comprise: Pipelines comprise:
- Jobs that define what to run. For example, code compilation or test runs. - Jobs, which define *what* to do. For example, jobs that compile or test code.
- Stages that define when and how to run. For example, that tests run only after code compilation. - Stages, which define *when* to run the jobs. For example, stages that run tests after stages that compile the code.
Multiple jobs in the same stage are executed by [Runners](../runners/README.md) in parallel, if there are enough concurrent [Runners](../runners/README.md). Jobs are executed by [Runners](../runners/README.md). Multiple jobs in the same stage are executed in parallel,
if there are enough concurrent runners.
If all the jobs in a stage: If all the jobs in a stage:
- Succeed, the pipeline moves on to the next stage. - Succeed, the pipeline moves on to the next stage.
- Fail, the next stage is not (usually) executed and the pipeline ends early. - Fail, the next stage is not (usually) executed and the pipeline ends early.
In general, pipelines are executed automatically and require no intervention once created. However, there are
also times when you can manually interact with a pipeline.
A typical pipeline might consist of four stages, executed in the following order:
- A `build` stage, with a job called `compile`.
- A `test` stage, with two jobs called `test1` and `test2`.
- A `staging` stage, with a job called `deploy-to-stage`.
- A `production` stage, with a job called `deploy-to-prod`.
NOTE: **Note:** NOTE: **Note:**
If you have a [mirrored repository that GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter), If you have a [mirrored repository that GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter),
you may need to enable pipeline triggering in your project's you may need to enable pipeline triggering in your project's
**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**. **Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**.
## Simple pipeline example ## Types of pipelines
As an example, imagine a pipeline consisting of four stages, executed in the following order: Pipelines can be configured in many different ways:
- `build`, with a job called `compile`. - [Multi-project pipelines](../multi_project_pipelines.md) combine pipelines for different projects together.
- `test`, with two jobs called `test` and `test2`. - [Parent-Child pipelines](../parent_child_pipelines.md) break down complex pipelines
- `staging`, with a job called `deploy-to-stage`. into one parent pipeline that can trigger multiple child sub-pipelines, which all
- `production`, with a job called `deploy-to-prod`. run in the same project and with the same SHA.
- [Pipelines for Merge Requests](../merge_request_pipelines/index.md) run for merge
requests only (rather than for every commit).
- [Pipelines for Merged Results](../merge_request_pipelines/pipelines_for_merged_results/index.md)
are merge request pipelines that act as though the changes from the source branch have
already been merged into the target branch.
- [Merge Trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md)
use pipelines for merged results to queue merges one after the other.
## Visualizing pipelines ## Configure a pipeline
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5742) in GitLab 8.11. Pipelines and their component jobs and stages are defined in the CI/CD pipeline configuration file for each project.
Pipelines can be complex structures with many sequential and parallel jobs. - Jobs are the [basic configuration](../yaml/README.md#introduction) component.
- Stages are defined by using the [`stages`](../yaml/README.md#stages) keyword.
To make it easier to understand the flow of a pipeline, GitLab has pipeline graphs for viewing pipelines
and their statuses.
Pipeline graphs can be displayed in two different ways, depending on the page you For a list of configuration options in the CI pipeline file, see the [GitLab CI/CD Pipeline Configuration Reference](../yaml/README.md).
access the graph from.
NOTE: **Note:** You can also configure specific aspects of your pipelines through the GitLab UI. For example:
GitLab capitalizes the stages' names when shown in the pipeline graphs (below).
### Regular pipeline graphs - [Pipeline settings](settings.md) for each project.
- [Pipeline schedules](schedules.md).
- [Custom CI/CD variables](../variables/README.md#creating-a-custom-environment-variable).
Regular pipeline graphs show the names of the jobs of each stage. Regular pipeline graphs can ### View pipelines
be found when you are on a [single pipeline page](#accessing-pipelines). For example:
![Pipelines example](img/pipelines.png) You can find the current and historical pipeline runs under your project's
**CI/CD > Pipelines** page. You can also access pipelines for a merge request by navigating
to its **Pipelines** tab.
### Pipeline mini graphs ![Pipelines index page](img/pipelines_index.png)
Pipeline mini graphs take less space and can tell you at a Clicking a pipeline will bring you to the **Pipeline Details** page and show
quick glance if all jobs passed or something failed. The pipeline mini graph can the jobs that were run for that pipeline. From here you can cancel a running pipeline,
be found when you navigate to: retry jobs on a failed pipeline, or [delete a pipeline](#delete-a-pipeline).
- The pipelines index page. [Starting in GitLab 12.3](https://gitlab.com/gitlab-org/gitlab-foss/issues/50499), a link to the
- A single commit page. latest pipeline for the last commit of a given branch is available at `/project/pipelines/[branch]/latest`.
- A merge request page. Also, `/project/pipelines/latest` will redirect you to the latest pipeline for the last commit
on the project's default branch.
Pipeline mini graphs allow you to see all related jobs for a single commit and the net result ### Run a pipeline manually
of each stage of your pipeline. This allows you to quickly see what failed and
fix it.
Stages in pipeline mini graphs are collapsible. Hover your mouse over them and click to expand their jobs. Pipelines can be manually executed, with predefined or manually-specified [variables](../variables/README.md).
| Mini graph | Mini graph expanded | You might do this if the results of a pipeline (for example, a code build) are required outside the normal
|:-------------------------------------------------------------|:---------------------------------------------------------------| operation of the pipeline.
| ![Pipelines mini graph](img/pipelines_mini_graph_simple.png) | ![Pipelines mini graph extended](img/pipelines_mini_graph.png) |
### Job ordering in pipeline graphs To execute a pipeline manually:
Job ordering depends on the type of pipeline graph. For [regular pipeline graphs](#regular-pipeline-graphs), jobs are sorted by name. 1. Navigate to your project's **CI/CD > Pipelines**.
1. Click on the **Run Pipeline** button.
1. On the **Run Pipeline** page:
1. Select the branch to run the pipeline for in the **Create for** field.
1. Enter any [environment variables](../variables/README.md) required for the pipeline run.
1. Click the **Create pipeline** button.
For [pipeline mini graphs](#pipeline-mini-graphs) ([introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/9760) The pipeline will execute the jobs as configured.
in GitLab 9.0), jobs are sorted by severity and then by name.
The order of severity is: ### Run a pipeline by using a URL query string
- failed > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24146) in GitLab 12.5.
- warning
- pending
- running
- manual
- scheduled
- canceled
- success
- skipped
- created
For example:
![Pipeline mini graph sorting](img/pipelines_mini_graph_sorting.png) You can use a query string to pre-populate the **Run Pipeline** page. For example, the query string
`.../pipelines/new?ref=my_branch&var[foo]=bar&file_var[file_foo]=file_bar` will pre-populate the
**Run Pipeline** page with:
### Expanding and collapsing job log sections - **Run for** field: `my_branch`.
- **Variables** section:
- Variable:
- Key: `foo`
- Value: `bar`
- File:
- Key: `file_foo`
- Value: `file_bar`
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/14664) in GitLab 12.0. The format of the `pipelines/new` URL is:
Job logs are divided into sections that can be collapsed or expanded. Each section will display ```plaintext
the duration. .../pipelines/new?ref=<branch>&var[<variable_key>]=<value>&file_var[<file_key>]=<value>
```
In the following example: The following parameters are supported:
- Two sections are collapsed and can be expanded. - `ref`: specify the branch to populate the **Run for** field with.
- Three sections are expanded and can be collapsed. - `var`: specify a `Variable` variable.
- `file_var`: specify a `File` variable.
![Collapsible sections](img/collapsible_log_v12_6.png) For each `var` or `file_var`, a key and value are required.
#### Custom collapsible sections ### Add manual interaction to your pipeline
You can create collapsible sections in job logs by manually outputting special codes > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7931) in GitLab 8.15.
that GitLab will use to determine what sections to collapse:
- Section start marker: `section_start:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` + `TEXT_OF_SECTION_HEADER` Manual actions, configured using the [`when:manual`](../yaml/README.md#whenmanual) parameter,
- Section end marker: `section_end:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` allow you to require manual interaction before moving forward in the pipeline.
You must add these codes to the script section of the CI configuration. For example, You can do this straight from the pipeline graph. Just click the play button
using `echo`: to execute that particular job.
```yaml For example, your pipeline might start automatically, but it requires manual action to
job1: [deploy to production](../environments.md#configuring-manual-deployments). In the example below, the `production`
script: stage has a job with a manual action.
- echo -e "section_start:`date +%s`:my_first_section\r\e[0KHeader of the 1st collapsible section"
- echo 'this line should be hidden when collapsed'
- echo -e "section_end:`date +%s`:my_first_section\r\e[0K"
```
In the example above: ![Pipelines example](img/pipelines.png)
- `date +%s`: The Unix timestamp (for example `1560896352`). #### Start multiple manual actions in a stage
- `my_first_section`: The name given to the section.
- `\r\e[0K`: Prevents the section markers from displaying in the rendered (colored)
job log, but they are displayed in the raw job log. To see them, in the top right
of the job log, click **{doc-text}** (**Show complete raw**).
- `\r`: carriage return.
- `\e[0K`: clear line ANSI escape code.
Sample raw job log: > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27188) in GitLab 11.11.
```plaintext Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button.
section_start:1560896352:my_first_section\r\e[0KHeader of the 1st collapsible section Once the user clicks this button, each individual manual action will be triggered and refreshed
this line should be hidden when collapsed to an updated status.
section_end:1560896353:my_first_section\r\e[0K
```
### Pipeline success and duration charts This functionality is only available:
> - Introduced in GitLab 3.1.1 as Commit Stats, and later renamed to Pipeline Charts. - For users with at least Developer access.
> - [Renamed](https://gitlab.com/gitlab-org/gitlab/issues/38318) to CI / CD Analytics in GitLab 12.8. - If the stage contains [manual actions](#add-manual-interaction-to-your-pipeline).
GitLab tracks the history of your pipeline successes and failures, as well as how long each pipeline ran. To view this information, go to **Analytics > CI / CD Analytics**. ### Delete a pipeline
View successful pipelines: > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24851) in GitLab 12.7.
![Successful pipelines](img/pipelines_success_chart.png) Users with [owner permissions](../../user/permissions.md) in a project can delete a pipeline
by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details**
page, then using the **Delete** button.
View pipeline duration history: ![Pipeline Delete Button](img/pipeline-delete.png)
![Pipeline duration](img/pipelines_duration_chart.png) CAUTION: **Warning:**
Deleting a pipeline will expire all pipeline caches, and delete all related objects,
such as builds, logs, artifacts, and triggers. **This action cannot be undone.**
## Pipeline quotas ### Pipeline quotas
Each user has a personal pipeline quota that tracks the usage of shared runners in all personal projects. Each user has a personal pipeline quota that tracks the usage of shared runners in all personal projects.
Each group has a [usage quota](../../subscriptions/index.md#ci-pipeline-minutes) that tracks the usage of shared runners for all projects created within the group. Each group has a [usage quota](../../subscriptions/index.md#ci-pipeline-minutes) that tracks the usage of shared runners for all projects created within the group.
When a pipeline is triggered, regardless of who triggered it, the pipeline quota for the project owner's [namespace](../../user/group/index.md#namespaces) is used. In this case, the namespace can be the user or group that owns the project. When a pipeline is triggered, regardless of who triggered it, the pipeline quota for the project owner's [namespace](../../user/group/index.md#namespaces) is used. In this case, the namespace can be the user or group that owns the project.
### How pipeline duration is calculated #### How pipeline duration is calculated
Total running time for a given pipeline excludes retries and pending Total running time for a given pipeline excludes retries and pending
(queued) time. (queued) time.
...@@ -216,245 +226,285 @@ The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time ...@@ -216,245 +226,285 @@ The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time
(4 - 1) + (7 - 6) => 4 (4 - 1) + (7 - 6) => 4
``` ```
## Configuring pipelines ### Pipeline security on protected branches
Pipelines, and their component jobs and stages, are defined in the [`.gitlab-ci.yml`](../yaml/README.md) file for each project. A strict security model is enforced when pipelines are executed on
[protected branches](../../user/project/protected_branches.md).
In particular: The following actions are allowed on protected branches only if the user is
[allowed to merge or push](../../user/project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings)
on that specific branch:
- Jobs are the [basic configuration](../yaml/README.md#introduction) component. - Run manual pipelines (using the [Web UI](#run-a-pipeline-manually) or [pipelines API](#pipelines-api)).
- Stages are defined using the [`stages`](../yaml/README.md#stages) keyword. - Run scheduled pipelines.
- Run pipelines using triggers.
- Trigger manual actions on existing pipelines.
- Retry or cancel existing jobs (using the Web UI or pipelines API).
For all available configuration options, see the [GitLab CI/CD Pipeline Configuration Reference](../yaml/README.md). **Variables** marked as **protected** are accessible only to jobs that
run on protected branches, preventing untrusted users getting unintended access to
sensitive information like deployment credentials and tokens.
### Settings and schedules **Runners** marked as **protected** can run jobs only on protected
branches, preventing untrusted code from executing on the protected runner and
preserving deployment keys and other credentials from being unintentionally
accessed. In order to ensure that jobs intended to be executed on protected
runners will not use regular runners, they must be tagged accordingly.
In addition to configuring jobs through `.gitlab-ci.yml`, additional configuration options are available ## View jobs in a pipeline
through the GitLab UI:
- Pipeline settings for each project. For more information, see [Pipeline settings](settings.md). When you access a pipeline, you can see the related jobs for that pipeline.
- Schedules for pipelines. For more information, see [Pipeline schedules](schedules.md).
### Grouping jobs Clicking an individual job will show you its job log, and allow you to:
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/6242) in GitLab 8.12. - Cancel the job.
- Retry the job.
- Erase the job log.
If you have many similar jobs, your [pipeline graph](#visualizing-pipelines) becomes long and hard ### See why a job failed
to read.
For that reason, similar jobs can automatically be grouped together. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17782) in GitLab 10.7.
If the job names are formatted in certain ways, they will be collapsed into
a single group in regular pipeline graphs (not the mini graphs).
You'll know when a pipeline has grouped jobs if you don't see the retry or When a pipeline fails or is allowed to fail, there are several places where you
cancel button inside them. Hovering over them will show the number of grouped can find the reason:
jobs. Click to expand them.
![Grouped pipelines](img/pipelines_grouped.png) - In the [pipeline graph](#visualize-pipelines), on the pipeline detail view.
- In the pipeline widgets, in the merge requests and commit pages.
- In the job views, in the global and detailed views of a job.
#### Configuring grouping In each place, if you hover over the failed job you can see the reason it failed.
In the pipeline [configuration file](../yaml/README.md), job names must include two numbers separated with one of ![Pipeline detail](img/job_failure_reason.png)
the following (you can even use them interchangeably):
- A space. In [GitLab 10.8](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17814) and later,
- A slash (`/`). you can also see the reason it failed on the Job detail page.
- A colon (`:`).
NOTE: **Note:** ### The order of jobs in a pipeline
More specifically, it uses [this](https://gitlab.com/gitlab-org/gitlab/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99) regular expression: `\d+[\s:\/\\]+\d+\s*`.
#### How grouping works The order of jobs in a pipeline depends on the type of pipeline graph.
The jobs will be ordered by comparing those two numbers from left to right. You - For [regular pipeline graphs](#regular-pipeline-graphs), jobs are sorted by name.
usually want the first to be the index and the second the total. - For [pipeline mini graphs](#pipeline-mini-graphs), jobs are sorted by severity and then by name.
For example, the following jobs will be grouped under a job named `test`: The order of severity is:
- `test 0 3` - failed
- `test 1 3` - warning
- `test 2 3` - pending
- running
- manual
- scheduled
- canceled
- success
- skipped
- created
The following jobs will be grouped under a job named `test ruby`: For example:
- `test 1:2 ruby` ![Pipeline mini graph sorting](img/pipelines_mini_graph_sorting.png)
- `test 2:2 ruby`
The following jobs will be grouped under a job named `test ruby` as well: ### Group jobs in a pipeline
- `1/3 test ruby` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/6242) in GitLab 8.12.
- `2/3 test ruby`
- `3/3 test ruby`
### Pipelines for merge requests If you have many similar jobs, your [pipeline graph](#visualize-pipelines) becomes long and hard
to read.
GitLab supports configuring pipelines that run only for merge requests. For more information, see You can automatically group similar jobs together. If the job names are formatted in a certain way,
[Pipelines for merge requests](../merge_request_pipelines/index.md). they will be collapsed into a single group in regular pipeline graphs (not the mini graphs).
### Badges You'll know when a pipeline has grouped jobs if you don't see the retry or
cancel button inside them. Hovering over them will show the number of grouped
jobs. Click to expand them.
Pipeline status and test coverage report badges are available and configurable for each project. ![Grouped pipelines](img/pipelines_grouped.png)
For information on adding pipeline badges to projects, see [Pipeline badges](settings.md#pipeline-badges). To create a group of jobs, in the [CI/CD pipeline configuration file](../yaml/README.md),
separate each job name with a number and one of the following:
## Multi-project pipelines - A slash (`/`), for example, `test 1/3`, `test 2/3`, `test 3/3`.
- A colon (`:`), for example, `test 1:3`, `test 2:3`, `test 3:3`.
- A space, for example `test 0 3`, `test 1 3`, `test 2 3`.
Pipelines for different projects can be combined together into [Multi-project pipelines](../multi_project_pipelines.md). You can use these symbols interchangeably.
[Multi-project pipeline graphs](../multi_project_pipelines.md#multi-project-pipeline-visualization-premium) help For example, these three jobs will be in a group named `build ruby`:
you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)**
## Parent-child pipelines ```yaml
build ruby 1/3:
stage: build
script:
- echo "ruby1"
Complex pipelines can be broken down into one parent pipeline that can trigger build ruby 2/3:
multiple child sub-pipelines, which all run in the same project and with the same SHA. stage: build
script:
- echo "ruby2"
For more information, see [Parent-Child pipelines](../parent_child_pipelines.md). build ruby 3/3:
stage: build
script:
- echo "ruby3"
```
## Working with pipelines In the pipeline, the result is a group named `build ruby` with three jobs:
In general, pipelines are executed automatically and require no intervention once created. ![Job group](img/job_group_v12_10.png)
However, there are instances where you'll need to interact with pipelines. These are documented below. The jobs will be ordered by comparing the numbers from left to right. You
usually want the first number to be the index and the second number to be the total.
### Manually executing pipelines [This regular expression](https://gitlab.com/gitlab-org/gitlab/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99)
evaluates the job names: `\d+[\s:\/\\]+\d+\s*`.
Pipelines can be manually executed, with predefined or manually-specified [variables](../variables/README.md). ### Specifying variables when running manual jobs
You might do this if the results of a pipeline (for example, a code build) is required outside the normal > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30485) in GitLab 12.2.
operation of the pipeline.
To execute a pipeline manually: When running manual jobs you can supply additional job specific variables.
1. Navigate to your project's **CI/CD > Pipelines**. You can do this from the job page of the manual job you want to run with
1. Click on the **Run Pipeline** button. additional variables.
1. On the **Run Pipeline** page:
1. Select the branch to run the pipeline for in the **Create for** field.
1. Enter any [environment variables](../variables/README.md) required for the pipeline run.
1. Click the **Create pipeline** button.
The pipeline will execute the jobs as configured. This is useful when you want to alter the execution of a job by using
environment variables.
#### Using a query string ![Manual job variables](img/manual_job_variables.png)
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24146) in GitLab 12.5. ### Delay a job
Variables on the **Run Pipeline** page can be pre-populated by passing variable keys and values > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/21767) in GitLab 11.4.
in a query string appended to the `pipelines/new` URL. The format is:
```plaintext When you do not want to run a job immediately, you can use the [`when:delayed`](../yaml/README.md#whendelayed) parameter to
.../pipelines/new?ref=<branch>&var[<variable_key>]=<value>&file_var[<file_key>]=<value> delay a job's execution for a certain period.
```
The following parameters are supported: This is especially useful for timed incremental rollout where new code is rolled out gradually.
- `ref`: specify the branch to populate the **Run for** field with. For example, if you start rolling out new code and:
- `var`: specify a `Variable` variable.
- `file_var`: specify a `File` variable.
For each `var` or `file_var`, a key and value are required. - Users do not experience trouble, GitLab can automatically complete the deployment from 0% to 100%.
- Users experience trouble with the new code, you can stop the timed incremental rollout by canceling the pipeline
and [rolling](../environments.md#retrying-and-rolling-back) back to the last stable version.
For example, the query string ![Pipelines example](img/pipeline_incremental_rollout.png)
`.../pipelines/new?ref=my_branch&var[foo]=bar&file_var[file_foo]=file_bar` will pre-populate the
**Run Pipeline** page as follows:
- **Run for** field: `my_branch`. ### Expand and collapse job log sections
- **Variables** section:
- Variable:
- Key: `foo`
- Value: `bar`
- File:
- Key: `file_foo`
- Value: `file_bar`
### Accessing pipelines > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/14664) in GitLab 12.0.
You can find the current and historical pipeline runs under your project's Job logs are divided into sections that can be collapsed or expanded. Each section will display
**CI/CD > Pipelines** page. You can also access pipelines for a merge request by navigating the duration.
to its **Pipelines** tab.
![Pipelines index page](img/pipelines_index.png) In the following example:
Clicking on a pipeline will bring you to the **Pipeline Details** page and show - Two sections are collapsed and can be expanded.
the jobs that were run for that pipeline. From here you can cancel a running pipeline, - Three sections are expanded and can be collapsed.
retry jobs on a failed pipeline, or [delete a pipeline](#deleting-a-single-pipeline).
### Accessing individual jobs ![Collapsible sections](img/collapsible_log_v12_6.png)
When you access a pipeline, you can see the related jobs for that pipeline. #### Custom collapsible sections
Clicking on an individual job will show you its job log, and allow you to: You can create collapsible sections in job logs by manually outputting special codes
that GitLab will use to determine what sections to collapse:
- Cancel the job. - Section start marker: `section_start:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` + `TEXT_OF_SECTION_HEADER`
- Retry the job. - Section end marker: `section_end:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K`
- Erase the job log.
### Seeing the failure reason for jobs You must add these codes to the script section of the CI configuration. For example,
using `echo`:
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17782) in GitLab 10.7. ```yaml
job1:
script:
- echo -e "section_start:`date +%s`:my_first_section\r\e[0KHeader of the 1st collapsible section"
- echo 'this line should be hidden when collapsed'
- echo -e "section_end:`date +%s`:my_first_section\r\e[0K"
```
When a pipeline fails or is allowed to fail, there are several places where you In the example above:
can quickly check the reason it failed:
- In the pipeline graph, on the pipeline detail view. - `date +%s`: The Unix timestamp (for example `1560896352`).
- In the pipeline widgets, in the merge requests and commit pages. - `my_first_section`: The name given to the section.
- In the job views, in the global and detailed views of a job. - `\r\e[0K`: Prevents the section markers from displaying in the rendered (colored)
job log, but they are displayed in the raw job log. To see them, in the top right
of the job log, click **{doc-text}** (**Show complete raw**).
- `\r`: carriage return.
- `\e[0K`: clear line ANSI escape code.
In each place, if you hover over the failed job you can see the reason it failed. Sample raw job log:
![Pipeline detail](img/job_failure_reason.png) ```plaintext
section_start:1560896352:my_first_section\r\e[0KHeader of the 1st collapsible section
this line should be hidden when collapsed
section_end:1560896353:my_first_section\r\e[0K
```
From [GitLab 10.8](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17814), ## Visualize pipelines
you can also see the reason it failed on the Job detail page.
### Manual actions from pipeline graphs > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5742) in GitLab 8.11.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7931) in GitLab 8.15. Pipelines can be complex structures with many sequential and parallel jobs.
Manual actions, configured using the [`when:manual`](../yaml/README.md#whenmanual) parameter, To make it easier to understand the flow of a pipeline, GitLab has pipeline graphs for viewing pipelines
allow you to require manual interaction before moving forward in the pipeline. and their statuses.
You can do this straight from the pipeline graph. Just click on the play button Pipeline graphs can be displayed in two different ways, depending on the page you
to execute that particular job. access the graph from.
For example, your pipeline start automatically, but require manual action to NOTE: **Note:**
[deploy to production](../environments.md#configuring-manual-deployments). In the example below, the `production` GitLab capitalizes the stages' names in the pipeline graphs.
stage has a job with a manual action.
### Regular pipeline graphs
Regular pipeline graphs show the names of the jobs in each stage. Regular pipeline graphs can
be found when you are on a [single pipeline page](#view-pipelines). For example:
![Pipelines example](img/pipelines.png) ![Pipelines example](img/pipelines.png)
### Specifying variables when running manual jobs [Multi-project pipeline graphs](../multi_project_pipelines.md#multi-project-pipeline-visualization-premium) help
you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30485) in GitLab 12.2. ### Pipeline mini graphs
When running manual jobs you can supply additional job specific variables. Pipeline mini graphs take less space and can tell you at a
quick glance if all jobs passed or something failed. The pipeline mini graph can
be found when you navigate to:
You can do this from the job page of the manual job you want to run with - The pipelines index page.
additional variables. - A single commit page.
- A merge request page.
This is useful when you want to alter the execution of a job by using Pipeline mini graphs allow you to see all related jobs for a single commit and the net result
environment variables. of each stage of your pipeline. This allows you to quickly see what failed and
fix it.
![Manual job variables](img/manual_job_variables.png) Stages in pipeline mini graphs are collapsible. Hover your mouse over them and click to expand their jobs.
### Delay a job in a pipeline graph | Mini graph | Mini graph expanded |
|:-------------------------------------------------------------|:---------------------------------------------------------------|
| ![Pipelines mini graph](img/pipelines_mini_graph_simple.png) | ![Pipelines mini graph extended](img/pipelines_mini_graph.png) |
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/21767) in GitLab 11.4. ### Pipeline success and duration charts
When you do not want to run a job immediately, you can use the [`when:delayed`](../yaml/README.md#whendelayed) parameter to > - Introduced in GitLab 3.1.1 as Commit Stats, and later renamed to Pipeline Charts.
delay a job's execution for a certain period. > - [Renamed](https://gitlab.com/gitlab-org/gitlab/issues/38318) to CI / CD Analytics in GitLab 12.8.
This is especially useful for timed incremental rollout where new code is rolled out gradually. GitLab tracks the history of your pipeline successes and failures, as well as how long each pipeline ran. To view this information, go to **Analytics > CI / CD Analytics**.
For example, if you start rolling out new code and: View successful pipelines:
- Users do not experience trouble, GitLab can automatically complete the deployment from 0% to 100%. ![Successful pipelines](img/pipelines_success_chart.png)
- Users experience trouble with the new code, you can stop the timed incremental rollout by canceling the pipeline
and [rolling](../environments.md#retrying-and-rolling-back) back to the last stable version.
![Pipelines example](img/pipeline_incremental_rollout.png) View pipeline duration history:
![Pipeline duration](img/pipelines_duration_chart.png)
### Pipeline badges
Pipeline status and test coverage report badges are available and configurable for each project.
For information on adding pipeline badges to projects, see [Pipeline badges](settings.md#pipeline-badges).
### Using the API ## Pipelines API
GitLab provides API endpoints to: GitLab provides API endpoints to:
...@@ -464,65 +514,7 @@ GitLab provides API endpoints to: ...@@ -464,65 +514,7 @@ GitLab provides API endpoints to:
- [Triggering pipelines through the API](../triggers/README.md). - [Triggering pipelines through the API](../triggers/README.md).
- [Pipeline triggers API](../../api/pipeline_triggers.md). - [Pipeline triggers API](../../api/pipeline_triggers.md).
### Start multiple manual actions in a stage ## Troubleshooting `fatal: reference is not a tree:`
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27188) in GitLab 11.11.
Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button.
Once the user clicks this button, each individual manual action will be triggered and refreshed
to an updated status.
This functionality is only available:
- For users with at least Developer access.
- If the stage contains [manual actions](#manual-actions-from-pipeline-graphs).
### Deleting a single pipeline
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24851) in GitLab 12.7.
Users with [owner permissions](../../user/permissions.md) in a project can delete a pipeline
by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details**
page, then using the **Delete** button.
![Pipeline Delete Button](img/pipeline-delete.png)
CAUTION: **Warning:**
Deleting a pipeline will expire all pipeline caches, and delete all related objects,
such as builds, logs, artifacts, and triggers. **This action cannot be undone.**
## Most Recent Pipeline
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/50499) in GitLab 12.3.
There's a link to the latest pipeline for the last commit of a given branch at `/project/pipelines/[branch]/latest`. Also, `/project/pipelines/latest` will redirect you to the latest pipeline for the last commit on the project's default branch.
## Security on protected branches
A strict security model is enforced when pipelines are executed on
[protected branches](../../user/project/protected_branches.md).
The following actions are allowed on protected branches only if the user is
[allowed to merge or push](../../user/project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings)
on that specific branch:
- Run manual pipelines (using the [Web UI](#manually-executing-pipelines) or pipelines API).
- Run scheduled pipelines.
- Run pipelines using triggers.
- Trigger manual actions on existing pipelines.
- Retry or cancel existing jobs (using the Web UI or pipelines API).
**Variables** marked as **protected** are accessible only to jobs that
run on protected branches, preventing untrusted users getting unintended access to
sensitive information like deployment credentials and tokens.
**Runners** marked as **protected** can run jobs only on protected
branches, avoiding untrusted code to be executed on the protected runner and
preserving deployment keys and other credentials from being unintentionally
accessed. In order to ensure that jobs intended to be executed on protected
runners will not use regular runners, they must be tagged accordingly.
## Persistent pipeline refs
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17043) in GitLab 12.4. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17043) in GitLab 12.4.
......
doc/ci/variables/README.md
View file @ 1233228f
...@@ -465,7 +465,7 @@ limitations with the current Auto DevOps scripting environment. ...@@ -465,7 +465,7 @@ limitations with the current Auto DevOps scripting environment.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/44059) in GitLab 10.8. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/44059) in GitLab 10.8.
[Manually triggered pipelines](../pipelines/index.md#manually-executing-pipelines) allow you to override the value of a current variable. [Manually triggered pipelines](../pipelines/index.md#run-a-pipeline-manually) allow you to override the value of a current variable.
For instance, suppose you added a For instance, suppose you added a
[custom variable `$TEST`](#creating-a-custom-environment-variable) [custom variable `$TEST`](#creating-a-custom-environment-variable)
...@@ -621,7 +621,7 @@ variables that were set, etc. ...@@ -621,7 +621,7 @@ variables that were set, etc.
Before enabling this, you should ensure jobs are visible to Before enabling this, you should ensure jobs are visible to
[team members only](../../user/permissions.md#project-features). You should [team members only](../../user/permissions.md#project-features). You should
also [erase](../pipelines/index.md#accessing-individual-jobs) all generated job logs also [erase](../pipelines/index.md#view-jobs-in-a-pipeline) all generated job logs
before making them visible again. before making them visible again.
To enable debug logs (traces), set the `CI_DEBUG_TRACE` variable to `true`: To enable debug logs (traces), set the `CI_DEBUG_TRACE` variable to `true`:
......
doc/topics/autodevops/customize.md
View file @ 1233228f
...@@ -521,7 +521,7 @@ increasing the rollout up to 100%. ...@@ -521,7 +521,7 @@ increasing the rollout up to 100%.
If `INCREMENTAL_ROLLOUT_MODE` is set to `manual` in your project, then instead If `INCREMENTAL_ROLLOUT_MODE` is set to `manual` in your project, then instead
of the standard `production` job, 4 different of the standard `production` job, 4 different
[manual jobs](../../ci/pipelines/index.md#manual-actions-from-pipeline-graphs) [manual jobs](../../ci/pipelines/index.md#add-manual-interaction-to-your-pipeline)
will be created: will be created:
1. `rollout 10%` 1. `rollout 10%`
......
doc/user/permissions.md
View file @ 1233228f
...@@ -425,7 +425,7 @@ read through the documentation on the [new CI/CD permissions model](project/new_ ...@@ -425,7 +425,7 @@ read through the documentation on the [new CI/CD permissions model](project/new_
The permission to merge or push to protected branches is used to define if a user can The permission to merge or push to protected branches is used to define if a user can
run CI/CD pipelines and execute actions on jobs that are related to those branches. run CI/CD pipelines and execute actions on jobs that are related to those branches.
See [Security on protected branches](../ci/pipelines/index.md#security-on-protected-branches) See [Security on protected branches](../ci/pipelines/index.md#pipeline-security-on-protected-branches)
for details about the pipelines security model. for details about the pipelines security model.
## LDAP users permissions ## LDAP users permissions
......
doc/user/project/index.md
View file @ 1233228f
...@@ -70,7 +70,7 @@ When you create a project in GitLab, you'll have access to a large number of ...@@ -70,7 +70,7 @@ When you create a project in GitLab, you'll have access to a large number of
your GitLab CI/CD pipelines from the UI your GitLab CI/CD pipelines from the UI
- [Scheduled Pipelines](../../ci/pipelines/schedules.md): Schedule a pipeline - [Scheduled Pipelines](../../ci/pipelines/schedules.md): Schedule a pipeline
to start at a chosen time to start at a chosen time
- [Pipeline Graphs](../../ci/pipelines/index.md#visualizing-pipelines): View your - [Pipeline Graphs](../../ci/pipelines/index.md#visualize-pipelines): View your
entire pipeline from the UI entire pipeline from the UI
- [Job artifacts](../../ci/pipelines/job_artifacts.md): Define, - [Job artifacts](../../ci/pipelines/job_artifacts.md): Define,
browse, and download job artifacts browse, and download job artifacts
......
doc/user/project/merge_requests/index.md
View file @ 1233228f
...@@ -101,7 +101,7 @@ or link to useful information directly in the merge request page: ...@@ -101,7 +101,7 @@ or link to useful information directly in the merge request page:
| [Metrics Reports](../../../ci/metrics_reports.md) **(PREMIUM)** | Display the Metrics Report on the merge request so that it's fast and easy to identify changes to important metrics. | | [Metrics Reports](../../../ci/metrics_reports.md) **(PREMIUM)** | Display the Metrics Report on the merge request so that it's fast and easy to identify changes to important metrics. |
| [Multi-Project pipelines](../../../ci/multi_project_pipelines.md) **(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. | | [Multi-Project pipelines](../../../ci/multi_project_pipelines.md) **(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. |
| [Pipelines for merge requests](../../../ci/merge_request_pipelines/index.md) | Customize a specific pipeline structure for merge requests in order to speed the cycle up by running only important jobs. | | [Pipelines for merge requests](../../../ci/merge_request_pipelines/index.md) | Customize a specific pipeline structure for merge requests in order to speed the cycle up by running only important jobs. |
| [Pipeline Graphs](../../../ci/pipelines/index.md#visualizing-pipelines) | View the status of pipelines within the merge request, including the deployment process. | | [Pipeline Graphs](../../../ci/pipelines/index.md#visualize-pipelines) | View the status of pipelines within the merge request, including the deployment process. |
| [Test Coverage visualization](test_coverage_visualization.md) | See test coverage results for merge requests, within the file diff. | | [Test Coverage visualization](test_coverage_visualization.md) | See test coverage results for merge requests, within the file diff. |
### Security Reports **(ULTIMATE)** ### Security Reports **(ULTIMATE)**
......
doc/user/project/protected_branches.md
View file @ 1233228f
...@@ -187,7 +187,7 @@ Additionally, direct pushes to the protected branch are denied if a rule is matc ...@@ -187,7 +187,7 @@ Additionally, direct pushes to the protected branch are denied if a rule is matc
The permission to merge or push to protected branches is used to define if a user can The permission to merge or push to protected branches is used to define if a user can
run CI/CD pipelines and execute actions on jobs that are related to those branches. run CI/CD pipelines and execute actions on jobs that are related to those branches.
See [Security on protected branches](../../ci/pipelines/index.md#security-on-protected-branches) See [Security on protected branches](../../ci/pipelines/index.md#pipeline-security-on-protected-branches)
for details about the pipelines security model. for details about the pipelines security model.
## Changelog ## Changelog
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7