Merge branch 'docs-yaml-ing-words' into 'master'

Improve CI/CD script documentation

See merge request gitlab-org/gitlab!47162

doc/.vale/gitlab/Acronyms.yml

@@ -25,6 +25,7 @@ exceptions:
   - CPU
   - CSS
   - CSV
+  - DAG
   - DAST
   - DNS
   - EKS

doc/ci/docker/using_docker_images.md

@@ -452,9 +452,9 @@ CI jobs:
    from `Dockerfile` that may be overridden in `.gitlab-ci.yml`)
 1. The runner attaches itself to a running container.
 1. The runner prepares a script (the combination of
-   [`before_script`](../yaml/README.md#before_script-and-after_script),
+   [`before_script`](../yaml/README.md#before_script),
    [`script`](../yaml/README.md#script),
-   and [`after_script`](../yaml/README.md#before_script-and-after_script)).
+   and [`after_script`](../yaml/README.md#after_script)).
 1. The runner sends the script to the container's shell STDIN and receives the
    output.
 

doc/ci/migration/jenkins.md

@@ -258,7 +258,7 @@ stages:
 ```
 
 Setting a step to be performed before and after any job can be done via the
-[`before_script` and `after_script` keywords](../yaml/README.md#before_script-and-after_script):
+[`before_script`](../yaml/README.md#before_script) and [`after_script`](../yaml/README.md#after_script) keywords:
 
 ```yaml
 default:

doc/ci/ssh_keys/README.md

@@ -93,7 +93,7 @@ to access it. This is where an SSH key pair comes in handy.
      # - git config --global user.name "User name"
    ```
 
-   The [`before_script`](../yaml/README.md#before_script-and-after_script) can be set globally
+   The [`before_script`](../yaml/README.md#before_script) can be set globally
    or per-job.
 
 1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys).

doc/ci/variables/predefined_variables.md

@@ -69,7 +69,7 @@ Kubernetes-specific environment variables are detailed in the
 | `CI_JOB_MANUAL`                               | 8.12   | all    | The flag to indicate that job was manually started                                                                                                                                                                                                                                                                                                         |
 | `CI_JOB_NAME`                                 | 9.0    | 0.5    | The name of the job as defined in `.gitlab-ci.yml`                                                                                                                                                                                                                                                                                                         |
 | `CI_JOB_STAGE`                                | 9.0    | 0.5    | The name of the stage as defined in `.gitlab-ci.yml`                                                                                                                                                                                                                                                                                                       |
-| `CI_JOB_STATUS`                               | all    | 13.5   | The state of the job as each runner stage is executed. Use with [`after_script`](../yaml/README.md#before_script-and-after_script) where `CI_JOB_STATUS` can be either: `success`, `failed` or `canceled`.                                                                                                                                                                              |
+| `CI_JOB_STATUS`                               | all    | 13.5   | The state of the job as each runner stage is executed. Use with [`after_script`](../yaml/README.md#after_script) where `CI_JOB_STATUS` can be either: `success`, `failed` or `canceled`.                                                                                                                                                                              |
 | `CI_JOB_TOKEN`                                | 9.0    | 1.2    | Token used for authenticating with [a few API endpoints](../../api/README.md#gitlab-ci-job-token) and downloading [dependent repositories](../../user/project/new_ci_build_permissions_model.md#dependent-repositories). The token is valid as long as the job is running. |
 | `CI_JOB_JWT`                                  | 12.10  | all    | RS256 JSON web token that can be used for authenticating with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). |
 | `CI_JOB_URL`                                  | 11.1   | 0.5    | Job details URL                                                                                                                                                                                                                                                                                                                                            |

doc/ci/variables/where_variables_can_be_used.md

@@ -99,7 +99,7 @@ In the case of `after_script` scripts, they can:
 - Not use variables defined in `before_script` and `script`.
 
 These restrictions are because `after_script` scripts are executed in a
-[separated shell context](../yaml/README.md#before_script-and-after_script).
+[separated shell context](../yaml/README.md#after_script).
 
 ## Persisted variables
 

doc/ci/yaml/script.md

+---
+stage: Verify
+group: Continuous Integration
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
+type: reference
+---
+
+# GitLab CI/CD script syntax
+
+You can use special syntax in [`script`](README.md#script) sections to:
+
+- [Split long commands](#split-long-commands) into multiline commands.
+- [Use color codes](#add-color-codes-to-script-output) to make job logs easier to review.
+- [Create custom collapsible sections](../pipelines/index.md#custom-collapsible-sections)
+  to simplify job log output.
+
+## Split long commands
+
+You can split long commands into multiline commands to improve readability with
+`|` (literal) and `>` (folded) [YAML multiline block scalar indicators](https://yaml-multiline.info/).
+
+CAUTION: **Warning:**
+If multiple commands are combined into one command string, only the last command's
+failure or success is reported.
+[Failures from earlier commands are ignored due to a bug](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25394).
+To work around this, run each command as a separate `script:` item, or add an `exit 1`
+command to each command string.
+
+You can use the `|` (literal) YAML multiline block scalar indicator to write
+commands over multiple lines in the `script` section of a job description.
+Each line is treated as a separate command.
+Only the first command is repeated in the job log, but additional
+commands are still executed:
+
+```yaml
+job:
+  script:
+    - |
+      echo "First command line."
+      echo "Second command line."
+      echo "Third command line."
+```
+
+The example above renders in the job log as:
+
+```shell
+$ echo First command line # collapsed multiline command
+First command line
+Second command line.
+Third command line.
+```
+
+The `>` (folded) YAML multiline block scalar indicator treats empty lines between
+sections as the start of a new command:
+
+```yaml
+job:
+  script:
+    - >
+      echo "First command line
+      is split over two lines."
+
+      echo "Second command line."
+```
+
+This behaves similarly to multiline commands without the `>` or `|` block
+scalar indicators:
+
+```yaml
+job:
+  script:
+    - echo "First command line
+      is split over two lines."
+
+      echo "Second command line."
+```
+
+Both examples above render in the job log as:
+
+```shell
+$ echo First command line is split over two lines. # collapsed multiline command
+First command line is split over two lines.
+Second command line.
+```
+
+When you omit the `>` or `|` block scalar indicators, GitLab concatenates non-empty
+lines to form the command. Make sure the lines can run when concatenated.
+
+[Shell here documents](https://en.wikipedia.org/wiki/Here_document) work with the
+`|` and `>` operators as well. The example below transliterates lower case letters
+to upper case:
+
+```yaml
+job:
+  script:
+    - |
+      tr a-z A-Z << END_TEXT
+        one two three
+        four five six
+      END_TEXT
+```
+
+Results in:
+
+```shell
+$ tr a-z A-Z << END_TEXT # collapsed multiline command
+  ONE TWO THREE
+  FOUR FIVE SIX
+```
+
+## Add color codes to script output
+
+Script output can be colored using [ANSI escape codes](https://en.wikipedia.org/wiki/ANSI_escape_code#Colors),
+or by running commands or programs that output ANSI escape codes.
+
+For example, using [Bash with color codes](https://misc.flogisoft.com/bash/tip_colors_and_formatting):
+
+```yaml
+job:
+  script:
+    - echo -e "\e[31mThis text is red,\e[0m but this text isn't\e[31m however this text is red again."
+```
+
+You can define the color codes in Shell variables, or even [custom environment variables](../variables/README.md#custom-environment-variables),
+which makes the commands easier to read and reusable.
+
+For example, using the same example as above and variables defined in a `before_script`:
+
+```yaml
+job:
+  before_script:
+    - TXT_RED="\e[31m" && TXT_CLEAR="\e[0m"
+  script:
+    - echo -e "${TXT_RED}This text is red,${TXT_CLEAR} but this part isn't${TXT_RED} however this part is again."
+    - echo "This text is not colored"
+```
+
+Or with [PowerShell color codes](https://superuser.com/a/1259916):
+
+```yaml
+job:
+  before_script:
+    - $esc="$([char]27)"; $TXT_RED="$esc[31m"; $TXT_CLEAR="$esc[0m"
+  script:
+    - Write-Host $TXT_RED"This text is red,"$TXT_CLEAR" but this text isn't"$TXT_RED" however this text is red again."
+    - Write-Host "This text is not colored"
+```

doc/development/integrations/secure.md

@@ -46,12 +46,12 @@ Because the `script` entry can't be left empty, it must be set to the command th
 It is not possible to rely on the predefined `ENTRYPOINT` and `CMD` of the Docker image
 to perform the scan automatically, without passing any command.
 
-The [`before_script`](../../ci/yaml/README.md#before_script-and-after_script)
+The [`before_script`](../../ci/yaml/README.md#before_script)
 should not be used in the job definition because users may rely on this to prepare their projects before performing the scan.
 For instance, it is common practice to use `before_script` to install system libraries
 a particular project needs before performing SAST or Dependency Scanning.
 
-Similarly, [`after_script`](../../ci/yaml/README.md#before_script-and-after_script)
+Similarly, [`after_script`](../../ci/yaml/README.md#after_script)
 should not be used in the job definition, because it may be overridden by users.
 
 ### Stage

doc/user/application_security/container_scanning/index.md

@@ -148,8 +148,8 @@ variables:
 Version `3` of the `container_scanning` Docker image uses [`centos:centos8`](https://hub.docker.com/_/centos)
 as the new base. It also removes the use of the [start.sh](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/merge_requests/77)
 script and instead executes the analyzer by default. Any customizations made to the
-`container_scanning` job's [`before_script`](../../../ci/yaml/README.md#before_script-and-after_script)
-and [`after_script`](../../../ci/yaml/README.md#before_script-and-after_script)
+`container_scanning` job's [`before_script`](../../../ci/yaml/README.md#before_script)
+and [`after_script`](../../../ci/yaml/README.md#after_script)
 blocks may not work with the new version. To roll back to the previous [`alpine:3.11.3`](https://hub.docker.com/_/alpine)-based
 Docker image, you can specify the major version through the [`CS_MAJOR_VERSION`](#available-variables)
 variable.

doc/user/compliance/license_compliance/index.md

@@ -444,7 +444,7 @@ documentation for a list of settings that you can apply.
 The `license_scanning` job runs in a [Debian 10](https://www.debian.org/releases/buster/) Docker
 image. The supplied image ships with some build tools such as [CMake](https://cmake.org/) and [GCC](https://gcc.gnu.org/).
 However, not all project types are supported by default. To install additional tools needed to
-compile dependencies, use a [`before_script`](../../../ci/yaml/README.md#before_script-and-after_script)
+compile dependencies, use a [`before_script`](../../../ci/yaml/README.md#before_script)
 to install the necessary build tools using the [`apt`](https://wiki.debian.org/PackageManagementTools)
 package manager. For a comprehensive list, consult [the Conan documentation](https://docs.conan.io/en/latest/introduction.html#all-platforms-all-build-systems-and-compilers).
 

doc/user/gitlab_com/index.md

@@ -409,7 +409,7 @@ test:
 - For the beta release, we have included a set of software packages in
   the base VM image. If your CI job requires additional software that's
   not included in this list, then you will need to add installation
-  commands to [`before_script`](../../ci/yaml/README.md#before_script-and-after_script) or [`script`](../../ci/yaml/README.md#script) to install the required
+  commands to [`before_script`](../../ci/yaml/README.md#before_script) or [`script`](../../ci/yaml/README.md#script) to install the required
   software. Note that each job runs on a new VM instance, so the
   installation of additional software packages needs to be repeated for
   each job in your pipeline.