Merge branch 'bwill/update-cilium-application-docs' into 'master'

Add docs page for Cilium application template

See merge request gitlab-org/gitlab!64646

doc/topics/autodevops/stages.md

@@ -541,7 +541,7 @@ You must use a Kubernetes network plugin that implements support for
 `NetworkPolicy`. The default network plugin for Kubernetes (`kubenet`)
 [does not implement](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#kubenet)
 support for it. The [Cilium](https://cilium.io/) network plugin can be
-installed as a [cluster application](../../user/clusters/applications.md#install-cilium-using-gitlab-cicd)
+installed as a [cluster application](../../user/project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium)
 to enable support for network policies.
 
 You can enable deployment of a network policy by setting the following
@@ -577,7 +577,7 @@ networkPolicy:
 ```
 
 For more information on installing Network Policies, see
-[Install Cilium using GitLab CI/CD](../../user/clusters/applications.md#install-cilium-using-gitlab-cicd).
+[Use the Cluster Management Template to Install Cilium](../../user/project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium).
 
 ### Cilium Network Policy
 
@@ -596,7 +596,7 @@ As the default network plugin for Kubernetes (`kubenet`)
 support for it, you must have [Cilium](https://docs.cilium.io/en/v1.8/intro/) as your Kubernetes network plugin.
 
 The [Cilium](https://cilium.io/) network plugin can be
-installed as a [cluster application](../../user/clusters/applications.md#install-cilium-using-gitlab-cicd)
+installed with a [cluster management project template](../../user/project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium)
 to enable support for network policies.
 
 #### Configuration
@@ -643,11 +643,10 @@ ciliumNetworkPolicy:
   enabled: true
   alerts:
     enabled: true
-
 ```
 
 For more information on installing Network Policies, see
-[Install Cilium using GitLab CI/CD](../../user/clusters/applications.md#install-cilium-using-gitlab-cicd).
+[Use the Cluster Management Template to Install Cilium](../../user/project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium).
 
 ### Running commands in the container
 

doc/user/application_security/threat_monitoring/index.md

@@ -27,20 +27,19 @@ your application's Kubernetes namespace. This section has the following
 prerequisites:
 
 - Your project contains at least one [environment](../../../ci/environments/index.md)
-- You've [installed Cilium](../../clusters/applications.md#install-cilium-using-gitlab-cicd)
+- You've [installed Cilium](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium)
 - You've configured the [Prometheus service](../../project/integrations/prometheus.md#enabling-prometheus-integration)
 
 If you're using custom Helm values for Cilium, you must enable Hubble
 with flow metrics for each namespace by adding the following lines to
-your [Cilium values](../../clusters/applications.md#install-cilium-using-gitlab-cicd):
+your [Cilium values](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium):
 
 ```yaml
-global:
-  hubble:
-    enabled: true
-    metrics:
-      enabled:
-        - 'flow:sourceContext=namespace;destinationContext=namespace'
+hubble:
+  enabled: true
+  metrics:
+    enabled:
+      - 'flow:sourceContext=namespace;destinationContext=namespace'
 ```
 
 The **Container Network Policy** section displays the following information
@@ -54,7 +53,11 @@ about your packet flow:
 
 If a significant percentage of packets is dropped, you should
 investigate it for potential threats by
-[examining the Cilium logs](../../clusters/applications.md#install-cilium-using-gitlab-cicd).
+examining the Cilium logs:
+
+```shell
+kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor
+```
 
 ## Container Network Policy management
 
@@ -67,7 +70,7 @@ status, and create and edit deployed policies. This section has the
 following prerequisites:
 
 - Your project contains at least one [environment](../../../ci/environments/index.md)
-- You've [installed Cilium](../../clusters/applications.md#install-cilium-using-gitlab-cicd)
+- You've [installed Cilium](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium)
 
 Network policies are fetched directly from the selected environment's
 deployment platform. Changes performed outside of this tab are

doc/user/clusters/agent/index.md

@@ -448,42 +448,21 @@ There are several components that work in concert for the Agent to generate the
 
 - A working Kubernetes cluster.
 - Cilium integration through either of these options:
-  - Installation through [GitLab Managed Apps](../applications.md#install-cilium-using-gitlab-cicd).
+  - Installation through [cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium).
   - Enablement of [hubble-relay](https://docs.cilium.io/en/v1.8/concepts/overview/#hubble) on an
     existing installation.
 - One or more network policies through any of these options:
   - Use the [Container Network Policy editor](../../application_security/threat_monitoring/index.md#container-network-policy-editor) to create and manage policies.
   - Use an [AutoDevOps](../../application_security/threat_monitoring/index.md#container-network-policy-management) configuration.
   - Add the required labels and annotations to existing network policies.
-- Use a configuration repository to inform the Agent through a `config.yaml` file, which
-  repositories can synchronize with. This repository might be the same, or a separate GitLab
-  project.
+- A configuration repository with [Cilium configured in `config.yaml`](repository.md#surface-network-security-alerts-from-cluster-to-gitlab)
 
 The setup process follows the same steps as [GitOps](#get-started-with-gitops-and-the-gitlab-agent),
 with the following differences:
 
-- When you define a configuration repository, you must do so with [Cilium settings](#define-a-configuration-repository-with-cilium-settings).
+- When you define a configuration repository, you must do so with [Cilium settings](repository.md#surface-network-security-alerts-from-cluster-to-gitlab).
 - You do not need to specify the `gitops` configuration section.
 
-### Define a configuration repository with Cilium settings
-
-You need a GitLab repository to contain your Agent configuration. The minimal repository layout
-looks like this:
-
-```plaintext
-.gitlab/agents/<agent-name>/config.yaml
-```
-
-Your `config.yaml` file must specify the `host` and `port` of your Hubble Relay service. If your
-Cilium integration was performed through [GitLab Managed Apps](../applications.md#install-cilium-using-gitlab-cicd),
-you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80`:
-
-```yaml
-cilium:
-  hubble_relay_address: "<hubble-relay-host>:<hubble-relay-port>"
-  ...
-```
-
 ## Management interfaces
 
 Users with at least the [Developer](../../permissions.md) can access the user interface

doc/user/clusters/agent/repository.md

@@ -157,7 +157,9 @@ cilium:
   hubble_relay_address: "<hubble-relay-host>:<hubble-relay-port>"
 ```
 
-If your Cilium integration was performed through GitLab Managed Apps, you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80` as the address:
+If your Cilium integration was performed through [GitLab Managed Apps](../applications.md#install-cilium-using-gitlab-cicd) or the
+[cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium),
+you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80` as the address:
 
 ```yaml
 cilium:

doc/user/clusters/applications.md

@@ -460,7 +460,7 @@ You can check Cilium's installation status on the cluster management page:
 
 WARNING:
 Installation and removal of the Cilium requires a **manual**
-[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-gke/#restart-unmanaged-pods)
+[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods)
 of all affected pods in all namespaces to ensure that they are
 [managed](https://docs.cilium.io/en/v1.8/operations/troubleshooting/#ensure-managed-pod)
 by the correct networking plugin. Whenever Hubble is enabled, its related pod might require a

doc/user/project/clusters/protect/container_network_security/quick_start_guide.md

@@ -24,16 +24,110 @@ The following steps are recommended to install and use Container Network Securit
      into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes
      cluster.
 
-1. [Install and configure Cilium](../../../../clusters/applications.md#install-cilium-using-gitlab-cicd).
+1. [Install and configure Cilium](#use-the-cluster-management-template-to-install-cilium).
 1. Be sure to restart all pods that were running before Cilium was installed by running this command
    in your cluster:
 
    `kubectl get pods --all-namespaces -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HOSTNETWORK:.spec.hostNetwork --no-headers=true | grep '<none>' | awk '{print "-n "$1" "$2}' | xargs -L 1 -r kubectl delete pod`
 
+   You can skip this step if `nodeinit.restartPods` is set to `true` on your Helm chart.
+
 It's possible to install and manage Cilium in other ways. For example, you could use the GitLab Helm
 chart to install Cilium manually in a Kubernetes cluster, and then connect it back to GitLab.
 However, such methods aren't documented or officially supported by GitLab.
 
+### Use the Cluster Management template to install Cilium
+
+[Cilium](https://cilium.io/) is a networking plug-in for Kubernetes that you can use to implement
+support for [`NetworkPolicy`](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
+resources. For more information, see [Network Policies](../../../../../topics/autodevops/stages.md#network-policy).
+
+You can use the [Cluster Management Project Template](../../../../clusters/management_project_template.md)
+to install Cilium in your Kubernetes cluster.
+
+1. In your cluster management project, go to `helmfile.yaml` and uncomment `- path: applications/cilium/helmfile.yaml`.
+1. In `applications/cilium/helmfile.yaml`, set `clusterType` to either `gke` or `eks` based on which Kubernetes provider your are using.
+
+    ```yaml
+    environments:
+      default:
+        values:
+        # Set to "gke" or "eks" based on your cluster type
+        - clusterType: ""
+    ```
+
+1. Merge or push these changes to the default branch of your cluster management project,
+and [GitLab CI/CD](../../../../../ci/README.md) will automatically install Cilium.
+
+WARNING:
+Installation and removal of the Cilium requires a **manual**
+[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods)
+of all affected pods in all namespaces to ensure that they are
+[managed](https://docs.cilium.io/en/stable/operations/troubleshooting/#ensure-managed-pod)
+by the correct networking plug-in. When Hubble is enabled, its related pod might require a
+restart depending on whether it started prior to Cilium. For more information, see
+[Failed Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment)
+in the Kubernetes docs.
+
+NOTE:
+Major upgrades might require additional setup steps. For more information, see
+the official [upgrade guide](https://docs.cilium.io/en/stable/operations/upgrade/).
+
+Support for installing the Cilium application is provided by the
+GitLab Container Security group. If you run into unknown issues,
+[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at
+least 2 people from the
+[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group).
+
+### Configure the Cilium Helm chart
+
+You can customize Cilium's Helm variables by editing the `applications/cilium/values.yaml`
+file in your cluster management project. Refer to the [Cilium Helm reference](https://docs.cilium.io/en/stable/helm-reference/)
+for the available configuration options.
+
+By default, Cilium's
+[audit mode](https://docs.cilium.io/en/stable/gettingstarted/policy-creation/#enable-policy-audit-mode)
+is enabled. In audit mode, Cilium doesn't drop disallowed packets. You
+can use `policy-verdict` log to observe policy-related decisions. You
+can disable audit mode by setting `policyAuditMode: false` in
+`applications/cilium/values.yaml`.
+
+The Cilium monitor log for traffic is logged out by the
+`cilium-monitor` sidecar container. You can check these logs with the following command:
+
+```shell
+kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor
+```
+
+You can disable the monitor log in `application/cilium/values.yaml`:
+
+```yaml
+monitor:
+  enabled: false
+```
+
+The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default
+and it's set to collect per namespace flow metrics. This metrics are accessible on the
+[Threat Monitoring](../../../../application_security/threat_monitoring/index.md)
+dashboard. You can disable Hubble by adding the following to
+`applications/cilium/values.yaml`:
+
+```yaml
+hubble:
+  enabled: false
+```
+
+You can also adjust Helm values for Hubble by using
+`applications/cilium/values.yaml`:
+
+```yaml
+hubble:
+  enabled: true
+  metrics:
+    enabled:
+    - 'flow:sourceContext=namespace;destinationContext=namespace'
+```
+
 ## Managing Network Policies
 
 Managing NetworkPolicies through GitLab is advantageous over managing the policies in Kubernetes
@@ -62,16 +156,14 @@ editor.
 To view statistics for Container Network Security, you must follow the installation steps above and
 configure GitLab integration with Prometheus. Also, if you use custom Helm values for Cilium, you
 must enable Hubble with flow metrics for each namespace by adding the following lines to
-your [Cilium values](../../../../clusters/applications.md#install-cilium-using-gitlab-cicd):
-your [Cilium values](../../../../clusters/applications.md#install-cilium-using-gitlab-cicd):
+your [Cilium values](#use-the-cluster-management-template-to-install-cilium):
 
 ```yaml
-global:
-  hubble:
-    enabled: true
-    metrics:
-      enabled:
-        - 'flow:sourceContext=namespace;destinationContext=namespace'
+hubble:
+  enabled: true
+  metrics:
+    enabled:
+      - 'flow:sourceContext=namespace;destinationContext=namespace'
 ```
 
 Additional information about the statistics page is available in the
@@ -97,15 +189,14 @@ kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor
 
 By default, Cilium is installed in Audit mode only, meaning that NetworkPolicies log policy
 violations but don't block any traffic. To set Cilium to Blocking mode, you must add the following
-lines to the `.gitlab/managed-apps/cilium/values.yaml` file in your cluster management project:
+lines to the `applications/cilium/values.yaml` file in your cluster management project:
 
 ```yaml
 config:
   policyAuditMode: false
 
-agent:
-  monitor:
-    eventTypes: ["drop"]
+monitor:
+  eventTypes: ["drop"]
 ```
 
 ### Traffic is not being allowed as expected