@@ -118,14 +118,14 @@ This section describes the earlier configuration format.
For source installations the following settings are nested under `artifacts:` and then `object_store:`. On Omnibus GitLab installs they are prefixed by `artifacts_object_store_`.
| `remote_directory`| The bucket name where Artifacts will be stored| |
| `direct_upload`| Set to `true` to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. | `false` |
| `background_upload` | Set to `false` to disable automatic upload. Option may be removed once upload is direct to S3 | `true` |
| `proxy_download`| Set to `true` to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` |
| `connection`| Various connection options described below | |
| `remote_directory` | | The bucket name where Artifacts are stored |
| `direct_upload` | `false` | Set to `true` to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. |
| `background_upload` | `true` | Set to `false` to disable automatic upload. Option may be removed once upload is direct to S3 |
| `proxy_download` | `false` | Set to `true` to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data. |
| `connection` | | Various connection options described below |
#### Connection settings
...
...
@@ -336,7 +336,7 @@ To migrate back to local storage:
If [`artifacts:expire_in`](../ci/yaml/README.md#artifactsexpire_in) is used to set
an expiry for the artifacts, they are marked for deletion right after that date passes.
Otherwise, they will expire per the [default artifacts expiration setting](../user/admin_area/settings/continuous_integration.md).
Otherwise, they expire per the [default artifacts expiration setting](../user/admin_area/settings/continuous_integration.md).
Artifacts are cleaned up by the `expire_build_artifacts_worker` cron job which Sidekiq
runs every hour at 50 minutes (`50 * * * *`).
...
...
@@ -367,7 +367,7 @@ steps below.
1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect.
If the `expire` directive is not set explicitly in your pipeline, artifacts will expire per the
If the `expire` directive is not set explicitly in your pipeline, artifacts expire per the
default artifacts expiration setting, which you can find in the [CI/CD Admin settings](../user/admin_area/settings/continuous_integration.md).
## Validation for dependencies
...
...
@@ -444,7 +444,7 @@ reasons are:
- The number of jobs run, and hence artifacts generated, is higher than expected.
- Job logs are larger than expected, and have accumulated over time.
In these and other cases, you'll need to identify the projects most responsible
In these and other cases, identify the projects most responsible
for disk space usage, figure out what types of artifacts are using the most
space, and in some cases, manually delete job artifacts to reclaim disk space.
...
...
@@ -508,7 +508,7 @@ If you need to manually remove job artifacts associated with multiple jobs while
1. Delete job artifacts older than a specific date:
NOTE: **Note:**
This step will also erase artifacts that users have chosen to
This step also erases artifacts that users have chosen to
| `id`| integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user |
| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID |
| `description`| string | no | The description of pipeline schedule |
| `ref`| string | no | The branch/tag name will be triggered |
| `cron`| string | no | The cron (e.g. `0 1 * * *`) ([Cron syntax](https://en.wikipedia.org/wiki/Cron)) |
| `cron_timezone`| string | no | The timezone supported by `ActiveSupport::TimeZone` (e.g. `Pacific Time (US & Canada)`) or `TZInfo::Timezone` (e.g. `America/Los_Angeles`) |
| `active`| boolean | no | The activation of pipeline schedule. If false is set, the pipeline schedule will deactivated initially. |
| `description` | string | no | The description of the pipeline schedule. |
| `ref` | string | no | The branch or tag name that is triggered. |
| `cron` | string | no | The [cron](https://en.wikipedia.org/wiki/Cron) schedule, for example: `0 1 * * *`. |
| `cron_timezone` | string | no | The timezone supported by `ActiveSupport::TimeZone` (for example `Pacific Time (US & Canada)`), or `TZInfo::Timezone` (for example `America/Los_Angeles`). |
| `active` | boolean | no | The activation of pipeline schedule. If false is set, the pipeline schedule is initially deactivated. |
```shell
curl --request PUT --header"PRIVATE-TOKEN: <your_access_token>"--formcron="0 2 * * *""https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13"
GitLab provides a lot of great reporting tools for [merge requests](../user/project/merge_requests/index.md) - [Unit test reports](unit_test_reports.md), [code quality](../user/project/merge_requests/code_quality.md), performance tests, etc. While JUnit is a great open framework for tests that "pass" or "fail", it is also important to see other types of metrics from a given change.
You can configure your job to use custom Metrics Reports, and GitLab will display a report on the merge request so that it's easier and faster to identify changes without having to check the entire log.
You can configure your job to use custom Metrics Reports, and GitLab displays a report on the merge request so that it's easier and faster to identify changes without having to check the entire log.
In the Merge Request Widget, multi-project pipeline mini-graphs are displayed,
and when hovering or tapping (on touchscreen devices) they will expand and be shown adjacent to each other.
and when hovering or tapping (on touchscreen devices) they expand and are shown adjacent to each other.
...
...
@@ -97,16 +97,16 @@ staging:
trigger:my/deployment
```
In the example above, as soon as `rspec` job succeeds in the `test` stage,
the `staging` bridge job is going to be started. The initial status of this
job will be `pending`. GitLab will create a downstream pipeline in the
`my/deployment` project and, as soon as the pipeline gets created, the
`staging` job will succeed. `my/deployment` is a full path to that project.
In the example above, as soon as the `rspec` job succeeds in the `test` stage,
the `staging` bridge job is started. The initial status of this
job is `pending`. GitLab then creates a downstream pipeline in the
`my/deployment` project and, as soon as the pipeline is created, the
`staging` job succeeds. `my/deployment` is a full path to that project.
The user that created the upstream pipeline needs to have access rights to the
downstream project (`my/deployment` in this case). If a downstream project can
not be found, or a user does not have access rights to create pipeline there,
the `staging` job is going to be marked as _failed_.
downstream project (`my/deployment` in this case). If a downstream project is
not found, or a user does not have access rights to create a pipeline there,
the `staging` job is marked as _failed_.
When using:
...
...
@@ -117,21 +117,18 @@ When using:
-[`only/except`](yaml/README.md#onlyexcept-basic) to control job behavior, use the
`pipelines` keyword.
CAUTION: **Caution:**
In the example, `staging` will be marked as succeeded as soon as a downstream pipeline
In the example, `staging` is marked as successful as soon as a downstream pipeline
gets created. If you want to display the downstream pipeline's status instead, see
[Mirroring status from triggered pipeline](#mirroring-status-from-triggered-pipeline).
NOTE: **Note:**
Bridge jobs do not support every configuration entry that a user can use
in the case of regular jobs. Bridge jobs will not be picked by a runner,
so there is no point in adding support for `script`, for example. If a user
tries to use unsupported configuration syntax, YAML validation will fail upon
pipeline creation.
Bridge jobs [do not support every configuration keyword](#limitations) that can be used
with other jobs. If a user tries to use unsupported configuration keywords, YAML
validation fails on pipeline creation.
### Specifying a downstream pipeline branch
It is possible to specify a branch name that a downstream pipeline will use:
It is possible to specify a branch name that a downstream pipeline uses:
```yaml
rspec:
...
...
@@ -152,7 +149,7 @@ Use:
[From GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/10126), variable expansion is
supported.
GitLab will use a commit that is currently on the HEAD of the branch when
GitLab uses a commit that is on the head of the branch when
creating a downstream pipeline.
NOTE: **Note:**
...
...
@@ -181,10 +178,10 @@ staging:
trigger:my/deployment
```
The `ENVIRONMENT` variable will be passed to every job defined in a downstream
pipeline. It will be available as an environment variable when GitLab Runner picks a job.
The `ENVIRONMENT` variable is passed to every job defined in a downstream
pipeline. It is available as an environment variable when GitLab Runner picks a job.
In the following configuration, the `MY_VARIABLE` variable will be passed to the downstream pipeline
In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline
that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream`
job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline.
...
...
@@ -210,7 +207,7 @@ downstream-job:
```
In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the
upstream pipeline will be passed to the `downstream-job` job, and will be available
upstream pipeline is passed to the `downstream-job` job, and is available
within the context of all downstream builds.
Upstream pipelines take precedence over downstream ones. If there are two
...
...
@@ -289,9 +286,9 @@ upstream_bridge:
### Limitations
Because bridge jobs are a little different to regular jobs, it is not
possible to use exactly the same configuration syntax here, as one would
normally do when defining a regular job that will be picked by a runner.
Bridge jobs are a little different from regular jobs. It is not
possible to use exactly the same configuration syntax as when defining regular jobs
that are picked by a runner.
Some features are not implemented yet. For example, support for environments.
...
...
@@ -318,7 +315,7 @@ tag in a different project:
1. Click subscribe.
Any pipelines that complete successfully for new tags in the subscribed project
will now trigger a pipeline on the current project's default branch. The maximum
now trigger a pipeline on the current project's default branch. The maximum
number of upstream pipeline subscriptions is 2 by default, for both the upstream and
downstream projects. This [application limit](../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project) can be changed on self-managed instances by a GitLab administrator.
@@ -32,8 +32,8 @@ On the left side we have the events that can trigger a pipeline based on various
- When GitHub integration is used with [external pull requests](../../ci/ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests).
- When an upstream pipeline contains a [bridge job](../../ci/yaml/README.md#trigger) which triggers a downstream pipeline.
Triggering any of these events will invoke the [`CreatePipelineService`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/ci/create_pipeline_service.rb)
which takes as input event data and the user triggering it, then will attempt to create a pipeline.
Triggering any of these events invokes the [`CreatePipelineService`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/ci/create_pipeline_service.rb)
which takes as input event data and the user triggering it, then attempts to create a pipeline.
The `CreatePipelineService` relies heavily on the [`YAML Processor`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/yaml_processor.rb)
component, which is responsible for taking in a YAML blob as input and returns the abstract data structure of a
...
...
@@ -65,20 +65,20 @@ the `Runner API Gateway`.
We can register, delete, and verify runners, which also causes read/write queries to the database. After a runner is connected,
it keeps asking for the next job to execute. This invokes the [`RegisterJobService`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/services/ci/register_job_service.rb)
which will pick the next job and assign it to the runner. At this point the job will transition to a
which picks the next job and assigns it to the runner. At this point the job transitions to a
`running` state, which again triggers `ProcessPipelineService` due to the status change.
For more details read [Job scheduling](#job-scheduling)).
While a job is being executed, the runner sends logs back to the server as well any possible artifacts
that need to be stored. Also, a job may depend on artifacts from previous jobs in order to run. In this
case the runner will download them using a dedicated API endpoint.
case the runner downloads them using a dedicated API endpoint.
Artifacts are stored in object storage, while metadata is kept in the database. An important example of artifacts
are reports (JUnit, SAST, DAST, etc.) which are parsed and rendered in the merge request.
Job status transitions are not all automated. A user may run [manual jobs](../../ci/yaml/README.md#whenmanual), cancel a pipeline, retry
specific failed jobs or the entire pipeline. Anything that
causes a job to change status will trigger`ProcessPipelineService`, as it's responsible for
causes a job to change status triggers`ProcessPipelineService`, as it's responsible for
tracking the status of the entire pipeline.
A special type of job is the [bridge job](../../ci/yaml/README.md#trigger) which is executed server-side
...
...
@@ -90,7 +90,7 @@ from the `CreatePipelineService` every time a downstream pipeline is triggered.
When a Pipeline is created all its jobs are created at once for all stages, with an initial state of `created`. This makes it possible to visualize the full content of a pipeline.
A job with the `created` state won't be seen by the runner yet. To make it possible to assign a job to a runner, the job must transition first into the `pending` state, which can happen if:
A job with the `created` state isn't seen by the runner yet. To make it possible to assign a job to a runner, the job must transition first into the `pending` state, which can happen if:
1. The job is created in the very first stage of the pipeline.
1. The job required a manual start and it has been triggered.
...
...
@@ -135,7 +135,7 @@ There are 3 top level queries that this service uses to gather the majority of t
This list of jobs is then filtered further by matching tags between job and runner tags.
NOTE: **Note:**
If a job contains tags, the runner will not pick the job if it does not match **all** the tags.
If a job contains tags, the runner doesn't pick the job if it does not match **all** the tags.
The runner may have more tags than defined for the job, but not vice-versa.
Finally if the runner can only pick jobs that are tagged, all untagged jobs are filtered out.
