Merge branch 'docs-ci-variables-cleanup-2' into 'master'

Revamp second half of variable docs

See merge request gitlab-org/gitlab!58742

doc/ci/chatops/index.md

@@ -57,7 +57,7 @@ functions available. Consider these best practices when creating ChatOps jobs:
   in the project, the job runs. The job itself can use existing
   [CI/CD variables](../variables/README.md#predefined-cicd-variables) like
   `GITLAB_USER_ID` to perform additional rights validation, but
-  these variables can be [overridden](../variables/README.md#priority-of-cicd-variables).
+  these variables can be [overridden](../variables/README.md#cicd-variable-precedence).
 
 ### Controlling the ChatOps reply
 

doc/ci/environments/index.md

@@ -690,7 +690,7 @@ with `review/` would have that variable available.
 
 Some GitLab features can behave differently for each environment.
 For example, you can
-[create a secret variable to be injected only into a production environment](../variables/README.md#limit-the-environment-scopes-of-cicd-variables).
+[create a project CI/CD variable to be injected only into a production environment](../variables/README.md#limit-the-environment-scope-of-a-cicd-variable).
 
 In most cases, these features use the _environment specs_ mechanism, which offers
 an efficient way to implement scoping in each environment group.

doc/ci/multi_project_pipelines.md

@@ -220,7 +220,7 @@ the ones defined in the upstream project take precedence.
 
 #### With variable inheritance
 
-You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](variables/README.md#inherit-cicd-variables) and [cross project artifact downloads](yaml/README.md#cross-project-artifact-downloads-with-needs).
+You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](variables/README.md#pass-an-environment-variable-to-another-job) and [cross project artifact downloads](yaml/README.md#cross-project-artifact-downloads-with-needs).
 
 In the upstream pipeline:
 

doc/ci/triggers/README.md

@@ -255,7 +255,7 @@ curl --request POST \
   "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline"
 ```
 
-Trigger variables have the [highest priority](../variables/README.md#priority-of-cicd-variables)
+Trigger variables have the [highest priority](../variables/README.md#cicd-variable-precedence)
 of all types of variables.
 
 ## Using cron to trigger nightly pipelines

doc/ci/variables/README.md

@@ -66,6 +66,11 @@ or have them [prefilled in manual pipelines](../pipelines/index.md#prefill-varia
 
 There are two types of variables: [`File` or `Variable`](#cicd-variable-types).
 
+Variable names are limited by the [shell the runner uses](https://docs.gitlab.com/runner/shells/index.html)
+to execute scripts. Each shell has its own set of reserved variable names.
+
+Make sure each variable is defined for the [scope you want to use it in](where_variables_can_be_used.md).
+
 ### Create a custom CI/CD variable in the `.gitlab-ci.yml` file
 
 To create a custom variable in the [`.gitlab-ci.yml`](../yaml/README.md#variables) file,
@@ -467,16 +472,23 @@ export GITLAB_USER_ID="42"
 ...
 ```
 
-## Inherit CI/CD variables
+## Pass an environment variable to another job
 
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22638) in GitLab 13.0 behind a disabled [feature flag](../../administration/feature_flags.md): `ci_dependency_variables`.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22638) in GitLab 13.0.
 > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217834) in GitLab 13.1.
 
-You can inherit CI/CD variables from dependent jobs.
+You can pass environment variables from one job to another job in a later stage.
+These variables cannot be used as CI/CD variables to configure a pipeline, but
+they can be used in job scripts.
 
-This feature makes use of the [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) report feature.
+1. In the job script, save the variable as a `.env` file.
+1. Save the `.env` file as an [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv)
+artifact.
+1. Set a job in a later stage to receive the artifact by using the [`dependencies`](../yaml/README.md#dependencies)
+   or the [`needs`](../yaml/README.md#artifact-downloads-with-needs) keywords.
+1. The later job can then [use the variable in scripts](#use-cicd-variables-in-job-scripts).
 
-Example with [`dependencies`](../yaml/README.md#dependencies) keyword.
+For example, with the [`dependencies`](../yaml/README.md#dependencies) keyword:
 
 ```yaml
 build:
@@ -490,12 +502,12 @@ build:
 deploy:
   stage: deploy
   script:
-    - echo $BUILD_VERSION  # => hello
+    - echo $BUILD_VERSION  # Output is: 'hello'
   dependencies:
     - build
 ```
 
-Example with the [`needs`](../yaml/README.md#artifact-downloads-with-needs) keyword:
+For example, with the [`needs`](../yaml/README.md#artifact-downloads-with-needs) keyword:
 
 ```yaml
 build:
@@ -509,131 +521,113 @@ build:
 deploy:
   stage: deploy
   script:
-    - echo $BUILD_VERSION  # => hello
+    - echo $BUILD_VERSION  # Output is: 'hello'
   needs:
     - job: build
       artifacts: true
 ```
 
-## Priority of CI/CD variables
+## CI/CD variable precedence
 
-Variables of different types can take precedence over other
-variables, depending on where they are defined.
+You can use CI/CD variables with the same name in different places, but the values
+can overwrite each other. The type of variable and where they are defined determines
+which variables take precedence.
 
 The order of precedence for variables is (from highest to lowest):
 
-1. [Trigger variables](../triggers/README.md#making-use-of-trigger-variables), [scheduled pipeline variables](../pipelines/schedules.md#using-variables),
-   and [manual pipeline run variables](#override-a-variable-by-manually-running-a-pipeline).
-1. Project-level [variables](#custom-cicd-variables) or [protected variables](#protect-a-cicd-variable).
-1. Group-level [variables](#group-cicd-variables) or [protected variables](#protect-a-cicd-variable).
-1. Instance-level [variables](#instance-cicd-variables) or [protected variables](#protect-a-cicd-variable).
-1. [Inherited CI/CD variables](#inherit-cicd-variables).
-1. YAML-defined [job-level variables](../yaml/README.md#variables).
-1. YAML-defined [global variables](../yaml/README.md#variables).
+1. [Trigger variables](../triggers/README.md#making-use-of-trigger-variables),
+   [scheduled pipeline variables](../pipelines/schedules.md#using-variables),
+   and [manual pipeline run variables](#override-a-variable-when-running-a-pipeline-manually).
+1. Project [variables](#custom-cicd-variables).
+1. Group [variables](#group-cicd-variables).
+1. Instance [variables](#instance-cicd-variables).
+1. [Inherited variables](#pass-an-environment-variable-to-another-job).
+1. Variables defined in jobs in the `.gitlab-ci.yml` file.
+1. Variables defined outside of jobs (globally) in the `.gitlab-ci.yml` file.
 1. [Deployment variables](#deployment-variables).
-1. [Predefined CI/CD variables](predefined_variables.md).
-
-For example, if you define:
-
-- `API_TOKEN=secure` as a project variable.
-- `API_TOKEN=yaml` in your `.gitlab-ci.yml`.
-
-`API_TOKEN` takes the value `secure` as the project
-variables take precedence over those defined in `.gitlab-ci.yml`.
-
-## Unsupported variables
-
-Variable names are limited by the underlying shell used to execute scripts (see [available shells](https://docs.gitlab.com/runner/shells/index.html).
-Each shell has its own unique set of reserved variable names.
-Keep in mind the [scope of CI/CD variables](where_variables_can_be_used.md) to ensure a variable is defined in the scope in which you wish to use it.
-
-## Where variables can be used
+1. [Predefined variables](predefined_variables.md).
 
-[This section](where_variables_can_be_used.md) describes where and how the different types of variables can be used.
+In the following example, when the script in `job1` executes, the value of `API_TOKEN` is `secure`.
+Variables defined in jobs have a higher precedence than variables defined globally.
 
-## Advanced use
-
-### Limit the environment scopes of CI/CD variables
-
-You can limit the environment scope of a variable by
-[defining which environments](../environments/index.md) it can be available for.
-
-To learn more about scoping environments, see [Scoping environments with specs](../environments/index.md#scoping-environments-with-specs).
+```yaml
+variables:
+  API_TOKEN: "default"
 
-### Deployment variables
+job1:
+  variables:
+    API_TOKEN: "secure"
+  script:
+    - echo "The variable value is $API_TOKEN"
+```
 
-[Integrations](../../user/project/integrations/overview.md) that are
-responsible for deployment configuration may define their own variables that
-are set in the build environment. These variables are only defined for
-[deployment jobs](../environments/index.md). Please consult the documentation of
-the integrations that you are using to learn which variables they define.
+## Override a defined CI/CD variable
 
-An example integration that defines deployment variables is the
-[Kubernetes integration](../../user/project/clusters/index.md#deployment-variables).
+You can override the value of a variable when you:
 
-### Auto DevOps environment variables
+1. [Run a pipeline manually](#override-a-variable-when-running-a-pipeline-manually) in the UI.
+1. Create a pipeline by using [the API](../../api/pipelines.md#create-a-new-pipeline).
+1. Run a job manually in the UI.
+1. Use [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd).
+1. Trigger a pipeline by using [the API](../triggers/README.md#making-use-of-trigger-variables).
+1. Pass variables to a [downstream pipeline](../multi_project_pipelines.md#passing-cicd-variables-to-a-downstream-pipeline).
 
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49056) in GitLab 11.7.
+The pipeline variables declared in these events take [priority over other variables](#cicd-variable-precedence).
 
-You can configure [Auto DevOps](../../topics/autodevops/index.md) to
-pass CI/CD variables to the running application by prefixing the key of the
-variable with `K8S_SECRET_`.
+### Override a variable when running a pipeline manually
 
-These [prefixed variables](../../topics/autodevops/customize.md#application-secret-variables) are
-then available as environment variables on the running application
-container.
+You can override the value of a CI/CD variable when you
+[run a pipeline manually](../pipelines/index.md#run-a-pipeline-manually).
 
-WARNING:
-Variables with multi-line values are not supported due to
-limitations with the Auto DevOps scripting environment.
+1. Go to your project's **CI/CD > Pipelines** and select **Run pipeline**.
+1. Choose the branch you want to run the pipeline for.
+1. Input the variable and its value in the UI.
 
-### When you can override variables
+### Restrict who can override variables
 
-You can override the value of a variable when:
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/295234) in GitLab 13.8.
 
-1. [Manually running](#override-a-variable-by-manually-running-a-pipeline) pipelines in the UI.
-1. Manually creating pipelines [via API](../../api/pipelines.md#create-a-new-pipeline).
-1. Manually playing a job via the UI.
-1. Using [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd).
-1. Manually triggering pipelines with [the API](../triggers/README.md#making-use-of-trigger-variables).
-1. Passing variables to a [downstream pipeline](../multi_project_pipelines.md#passing-cicd-variables-to-a-downstream-pipeline).
+You can grant permission to override variables to [maintainers](../../user/permissions.md#project-features) only. When other users try to run a pipeline
+with overridden variables, they receive the `Insufficient permissions to set pipeline variables`
+error message.
 
-These pipeline variables declared in these events take [priority over other variables](#priority-of-cicd-variables).
+If you [store your CI/CD configurations in a different repository](../../ci/pipelines/settings.md#custom-cicd-configuration-path),
+use this setting for control over the environment the pipeline runs in.
 
-#### Restrict who can override variables
+You can enable this feature by using [the projects API](../../api/projects.md#edit-project)
+to enable the `restrict_user_defined_variables` setting. The setting is `disabled` by default.
 
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/295234) in GitLab 13.8.
+## Limit the environment scope of a CI/CD variable
 
-To allow only users with Maintainer role to set these variables, you can use
-[the API](../../api/projects.md#edit-project) to enable the project setting `restrict_user_defined_variables`.
-When a user without Maintainer role tries to run a pipeline with overridden
-variables, an `Insufficient permissions to set pipeline variables` error occurs.
+You can limit the environment scope of a variable by
+[defining which environments](../environments/index.md) it can be available for.
 
-The setting is `disabled` by default.
+To learn more about scoping environments, see [Scoping environments with specs](../environments/index.md#scoping-environments-with-specs).
 
-If you [store your CI/CD configurations in a different repository](../../ci/pipelines/settings.md#custom-cicd-configuration-path),
-use this setting for strict control over all aspects of the environment
-the pipeline runs in.
+## Deployment variables
 
-#### Override a variable by manually running a pipeline
+Integrations that are responsible for deployment configuration can define their own
+variables that are set in the build environment. These variables are only defined
+for [deployment jobs](../environments/index.md).
 
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44059) in GitLab 10.8.
+For example, the [Kubernetes integration](../../user/project/clusters/index.md#deployment-variables)
+defines deployment variables that you can use with the integration.
 
-You can override the value of a current variable by
-[running a pipeline manually](../pipelines/index.md#run-a-pipeline-manually).
+The [documentation for each integration](../../user/project/integrations/overview.md)
+explains if the integration has any deployment variables available.
 
-For instance, suppose you added a custom variable named `$TEST`
-and you want to override it in a manual pipeline.
+## Auto DevOps environment variables
 
-Go to your project's **CI/CD > Pipelines** and select **Run pipeline**.
-Choose the branch you want to run the pipeline for, then add a variable and its value in the UI:
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49056) in GitLab 11.7.
 
-![Override variable value](img/override_variable_manual_pipeline.png)
+You can configure [Auto DevOps](../../topics/autodevops/index.md) to pass CI/CD variables
+to a running application.
 
-The runner overrides the value previously set and uses the custom
-value for this specific pipeline.
+To make a CI/CD variable available as an environment variable in the running application's container,
+[prefix the variable key](../../topics/autodevops/customize.md#application-secret-variables)
+with `K8S_SECRET_`.
 
-![Manually overridden variable output](img/override_value_via_manual_pipeline_output.png)
+CI/CD variables with multi-line values are not supported.
 
 ## CI/CD variable expressions
 
@@ -828,44 +822,22 @@ testvariable:
 > Introduced in GitLab Runner 1.7.
 
 WARNING:
-Enabling debug tracing can have severe security implications. The
-output **will** contain the content of all your variables and any other
-secrets! The output **will** be uploaded to the GitLab server and made visible
-in job logs!
-
-By default, the runner hides most of the details of what it is doing when
-processing a job. This behavior keeps job logs short, and prevents secrets
-from being leaked into the log unless your script writes them to the screen.
-
-If a job isn't working as expected, this can make the problem difficult to
-investigate; in these cases, you can enable debug tracing in `.gitlab-ci.yml`.
-Available on GitLab Runner v1.7+, this feature enables the shell's execution log. This results in a verbose job log listing all commands that were run, variables that were set, and so on.
-
-Before enabling this, you should ensure jobs are visible to
-[team members only](../../user/permissions.md#project-features). You should
-also [erase](../jobs/index.md#view-jobs-in-a-pipeline) all generated job logs
-before making them visible again.
-
-### Restricted access to debug logging
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213159) in GitLab 13.7.
-> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292661) in GitLab 13.8.
+Debug logging can be a serious security risk. The output contains the content of
+all variables and other secrets available to the job. The output is uploaded to the
+GitLab server and visible in job logs.
 
-With restricted access to debug logging, only users with
-[developer or higher permissions](../../user/permissions.md#project-members-permissions)
-can view job logs when debug logging is enabled with a variable in:
+You can use debug logging to help troubleshoot problems with pipeline configuration
+or job scripts. Debug logging exposes job execution details that are usually hidden
+by the runner and makes job logs more verbose. It also exposes all variables and secrets
+available to the job.
 
-- The [`.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file).
-- The CI/CD variables set in the GitLab UI.
-
-WARNING:
-If you add `CI_DEBUG_TRACE` as a local variable to your runners, debug logs are visible
-to all users with access to job logs. The permission levels are not checked by Runner,
-so you should make use of the variable in GitLab only.
+Before you enable debug logging, make sure only [team members](../../user/permissions.md#project-features)
+can view job logs. You should also [delete job logs](../jobs/index.md#view-jobs-in-a-pipeline)
+with debug output before you make logs public again.
 
 ### Enable Debug logging
 
-To enable debug logs (traces), set the `CI_DEBUG_TRACE` variable to `true`:
+To enable debug logging (tracing), set the `CI_DEBUG_TRACE` variable to `true`:
 
 ```yaml
 job_name:
@@ -873,11 +845,10 @@ job_name:
     CI_DEBUG_TRACE: "true"
 ```
 
-Example truncated output with `CI_DEBUG_TRACE` set to `true`:
+Example output (truncated):
 
 ```shell
 ...
-
 export CI_SERVER_TLS_CA_FILE="/builds/gitlab-examples/ci-debug-trace.tmp/CI_SERVER_TLS_CA_FILE"
 if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then
   echo $'\''\x1b[32;1mFetching changes...\x1b[0;m'\''
@@ -982,10 +953,26 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then
 ++ CI_COMMIT_REF_NAME=master
 ++ export CI_COMMIT_REF_SLUG=master
 ++ CI_COMMIT_REF_SLUG=master
-
 ...
 ```
 
+### Restrict access to debug logging
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213159) in GitLab 13.7.
+> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292661) in GitLab 13.8.
+
+You can restrict access to debug logging. When restricted, only users with
+[developer or higher permissions](../../user/permissions.md#project-members-permissions)
+can view job logs when debug logging is enabled with a variable in:
+
+- The [`.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file).
+- The CI/CD variables set in the GitLab UI.
+
+WARNING:
+If you add `CI_DEBUG_TRACE` as a local variable to runners, debug logs generate and are visible
+to all users with access to job logs. The permission levels are not checked by the runner,
+so you should only use the variable in GitLab itself.
+
 ## Video walkthrough of a working example
 
 The [Managing the Complex Configuration Data Management Monster Using GitLab](https://www.youtube.com/watch?v=v4ZOJ96hAck)

doc/ci/yaml/README.md

@@ -4636,7 +4636,7 @@ meaning it applies to all jobs. If you define a variable in a job, it's availabl
 to that job only.
 
 If a variable of the same name is defined globally and for a specific job, the
-[job-specific variable overrides the global variable](../variables/README.md#priority-of-cicd-variables).
+[job-specific variable overrides the global variable](../variables/README.md#cicd-variable-precedence).
 
 All YAML-defined variables are also set to any linked
 [Docker service containers](../services/index.md).

doc/topics/autodevops/customize.md

@@ -171,7 +171,7 @@ list of options.
 ## Custom Helm chart per environment
 
 You can specify the use of a custom Helm chart per environment by scoping the CI/CD variable
-to the desired environment. See [Limiting environment scopes of variables](../../ci/variables/README.md#limit-the-environment-scopes-of-cicd-variables).
+to the desired environment. See [Limit environment scope of CI/CD variables](../../ci/variables/README.md#limit-the-environment-scope-of-a-cicd-variable).
 
 ## Customizing `.gitlab-ci.yml`
 

doc/topics/autodevops/index.md

@@ -231,7 +231,7 @@ any of the following places:
   **Continuous Integration and Delivery** section
 
 The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence
-as other environment [variables](../../ci/variables/README.md#priority-of-cicd-variables).
+as other environment [variables](../../ci/variables/README.md#cicd-variable-precedence).
 If the CI/CD variable is not set and the cluster setting is left blank, the instance-wide **Auto DevOps domain**
 setting is used if set.
 
@@ -274,7 +274,7 @@ used by Auto DevOps currently defines 3 environment names:
 Those environments are tied to jobs using [Auto Deploy](stages.md#auto-deploy), so
 except for the environment scope, they must have a different deployment domain.
 You must define a separate `KUBE_INGRESS_BASE_DOMAIN` variable for each of the above
-[based on the environment](../../ci/variables/README.md#limit-the-environment-scopes-of-cicd-variables).
+[based on the environment](../../ci/variables/README.md#limit-the-environment-scope-of-a-cicd-variable).
 
 The following table is an example of how to configure the three different clusters:
 

doc/user/group/clusters/index.md

@@ -107,7 +107,7 @@ The domain should have a wildcard DNS configured to the Ingress IP address.
 When adding more than one Kubernetes cluster to your project, you need to differentiate
 them with an environment scope. The environment scope associates clusters with
 [environments](../../../ci/environments/index.md) similar to how the
-[environment-specific variables](../../../ci/variables/README.md#limit-the-environment-scopes-of-cicd-variables)
+[environment-specific CI/CD variables](../../../ci/variables/README.md#limit-the-environment-scope-of-a-cicd-variable)
 work.
 
 While evaluating which environment matches the environment scope of a

doc/user/project/clusters/index.md

@@ -86,7 +86,7 @@ differentiates the new cluster from the rest.
 
 When adding more than one Kubernetes cluster to your project, you need to differentiate
 them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the
-[environment-specific CI/CD variables](../../../ci/variables/README.md#limit-the-environment-scopes-of-cicd-variables) work.
+[environment-specific CI/CD variables](../../../ci/variables/README.md#limit-the-environment-scope-of-a-cicd-variable) work.
 
 The default environment scope is `*`, which means all jobs, regardless of their
 environment, use that cluster. Each scope can be used only by a single cluster