Merge branch '31000-api-for-instance-level-kubernetes-clusters' into 'master'

Add API support for instance clusters

Closes #31000

See merge request gitlab-org/gitlab!36001

changelogs/unreleased/31000-api-for-instance-level-kubernetes-clusters.yml

+---
+title: Add API support for instance-level Kubernetes clusters
+merge_request: 36001
+author:
+type: added

doc/api/api_resources.md

@@ -129,6 +129,7 @@ The following API resources are available outside of project and group contexts
 | [Geo Nodes](geo_nodes.md) **(PREMIUM ONLY)**       | `/geo_nodes`                                                            |
 | [Group Activity Analytics](group_activity_analytics.md) **(STARTER)**  | `/analytics/group_activity/{issues_count | merge_requests_count | new_members_count }`  |
 | [Import repository from GitHub](import.md)         | `/import/github`                                                        |
+| [Instance clusters](instance_clusters.md)          | `/admin/clusters`                                                       |
 | [Issues](issues.md)                                | `/issues` (also available for groups and projects)                      |
 | [Issues Statistics](issues_statistics.md)          | `/issues_statistics` (also available for groups and projects)           |
 | [Keys](keys.md)                                    | `/keys`                                                                 |

doc/api/instance_clusters.md

+# Instance clusters API
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36001) in GitLab 13.2.
+
+NOTE: **Note:**
+User will need admin access to use these endpoints.
+
+Use these API endpoints with your instance clusters, which enable you to use the same cluster across multiple projects. [More information](../user/instance/clusters/index.md)
+
+## List instance clusters
+
+Returns a list of instance clusters.
+
+```plaintext
+GET /admin/clusters
+```
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/admin/clusters"
+```
+
+Example response:
+
+```json
+[
+  {
+    "id": 9,
+    "name": "cluster-1",
+    "created_at": "2020-07-14T18:36:10.440Z",
+    "domain": null,
+    "provider_type": "user",
+    "platform_type": "kubernetes",
+    "environment_scope": "*",
+    "cluster_type": "instance_type",
+    "user": {
+      "id": 1,
+      "name": "Administrator",
+      "username": "root",
+      "state": "active",
+      "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+      "web_url": "https://gitlab.example.com/root"
+    },
+    "platform_kubernetes": {
+      "api_url": "https://example.com",
+      "namespace": null,
+      "authorization_type": "rbac",
+      "ca_cert":"-----BEGIN CERTIFICATE-----IxMDM1MV0ZDJkZjM...-----END CERTIFICATE-----"
+    },
+    "provider_gcp": null,
+    "management_project": null
+  },
+  {
+    "id": 10,
+    "name": "cluster-2",
+    "created_at": "2020-07-14T18:39:05.383Z",
+    "domain": null,
+    "provider_type": "user",
+    "platform_type": "kubernetes",
+    "environment_scope": "staging",
+    "cluster_type": "instance_type",
+    "user": {
+      "id": 1,
+      "name": "Administrator",
+      "username": "root",
+      "state": "active",
+      "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+      "web_url": "https://gitlab.example.com/root"
+    },
+    "platform_kubernetes": {
+      "api_url": "https://example.com",
+      "namespace": null,
+      "authorization_type": "rbac",
+      "ca_cert":"-----BEGIN CERTIFICATE-----LzEtMCadtaLGxcsGAZjM...-----END CERTIFICATE-----"
+    },
+    "provider_gcp": null,
+    "management_project": null
+  }
+  {
+    "id": 11,
+    "name": "cluster-3",
+    ...
+  }
+]
+
+```
+
+## Get a single instance cluster
+
+Returns a single instance cluster.
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `cluster_id` | integer | yes | The ID of the cluster |
+
+```plaintext
+GET /admin/clusters/:cluster_id
+```
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/admin/clusters/9"
+```
+
+Example response:
+
+```json
+{
+  "id": 9,
+  "name": "cluster-1",
+  "created_at": "2020-07-14T18:36:10.440Z",
+  "domain": null,
+  "provider_type": "user",
+  "platform_type": "kubernetes",
+  "environment_scope": "*",
+  "cluster_type": "instance_type",
+  "user": {
+    "id": 1,
+    "name": "Administrator",
+    "username": "root",
+    "state": "active",
+    "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+    "web_url": "https://gitlab.example.com/root"
+  },
+  "platform_kubernetes": {
+    "api_url": "https://example.com",
+    "namespace": null,
+    "authorization_type": "rbac",
+    "ca_cert":"-----BEGIN CERTIFICATE-----IxMDM1MV0ZDJkZjM...-----END CERTIFICATE-----"
+  },
+  "provider_gcp": null,
+  "management_project": null
+}
+```
+
+## Add existing instance cluster
+
+Adds an existing Kubernetes instance cluster.
+
+```plaintext
+POST /admin/clusters/add
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `name` | string | yes | The name of the cluster |
+| `domain` | string | no | The [base domain](../user/project/clusters/index.md#base-domain) of the cluster |
+| `environment_scope` | string | no | The associated environment to the cluster. Defaults to `*` |
+| `management_project_id` | integer | no | The ID of the [management project](../user/clusters/management_project.md) for the cluster |
+| `enabled` | boolean | no | Determines if cluster is active or not, defaults to true |
+| `managed` | boolean | no | Determines if GitLab will manage namespaces and service accounts for this cluster, defaults to true |
+| `platform_kubernetes_attributes[api_url]` | string | yes | The URL to access the Kubernetes API |
+| `platform_kubernetes_attributes[token]` | string | yes | The token to authenticate against Kubernetes |
+| `platform_kubernetes_attributes[ca_cert]` | string | no | TLS certificate. Required if API is using a self-signed TLS certificate. |
+| `platform_kubernetes_attributes[namespace]` | string | no | The unique namespace related to the project |
+| `platform_kubernetes_attributes[authorization_type]` | string | no | The cluster authorization type: `rbac`, `abac` or `unknown_authorization`. Defaults to `rbac`. |
+
+Example request:
+
+```shell
+curl --header "Private-Token:<your_access_token>" "http://gitlab.example.com/api/v4/admin/clusters/add" \
+-H "Accept:application/json" \
+-H "Content-Type:application/json" \
+-X POST --data '{"name":"cluster-3", "environment_scope":"production", "platform_kubernetes_attributes":{"api_url":"https://example.com", "token":"12345",  "ca_cert":"-----BEGIN CERTIFICATE-----qpoeiXXZafCM0ZDJkZjM...-----END CERTIFICATE-----"}}'
+
+```
+
+Example response:
+
+```json
+{
+  "id": 11,
+  "name": "cluster-3",
+  "created_at": "2020-07-14T18:42:50.805Z",
+  "domain": null,
+  "provider_type": "user",
+  "platform_type": "kubernetes",
+  "environment_scope": "production",
+  "cluster_type": "instance_type",
+  "user": {
+    "id": 1,
+    "name": "Administrator",
+    "username": "root",
+    "state": "active",
+    "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+    "web_url": "http://gitlab.example.com:3000/root"
+  },
+  "platform_kubernetes": {
+    "api_url": "https://example.com",
+    "namespace": null,
+    "authorization_type": "rbac",
+    "ca_cert":"-----BEGIN CERTIFICATE-----qpoeiXXZafCM0ZDJkZjM...-----END CERTIFICATE-----"
+  },
+  "provider_gcp": null,
+  "management_project": null
+}
+```
+
+## Edit instance cluster
+
+Updates an existing instance cluster.
+
+```shell
+PUT /admin/clusters/:cluster_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `cluster_id` | integer | yes | The ID of the cluster |
+| `name` | string | no | The name of the cluster |
+| `domain` | string | no | The [base domain](../user/project/clusters/index.md#base-domain) of the cluster |
+| `environment_scope` | string | no | The associated environment to the cluster |
+| `management_project_id` | integer | no | The ID of the [management project](../user/clusters/management_project.md) for the cluster |
+| `enabled` | boolean | no | Determines if cluster is active or not, defaults to true |
+| `platform_kubernetes_attributes[api_url]` | string | no | The URL to access the Kubernetes API |
+| `platform_kubernetes_attributes[token]` | string | no | The token to authenticate against Kubernetes |
+| `platform_kubernetes_attributes[ca_cert]` | string | no | TLS certificate. Required if API is using a self-signed TLS certificate. |
+| `platform_kubernetes_attributes[namespace]` | string | no | The unique namespace related to the project |
+
+NOTE: **Note:**
+`name`, `api_url`, `ca_cert` and `token` can only be updated if the cluster was added
+through the [Add existing Kubernetes cluster](../user/project/clusters/add_remove_clusters.md#add-existing-cluster) option or
+through the [Add existing instance cluster](#add-existing-instance-cluster) endpoint.
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "http://gitlab.example.com/api/v4/admin/clusters/9" \
+-H "Content-Type:application/json" \
+-X PUT --data '{"name":"update-cluster-name", "platform_kubernetes_attributes":{"api_url":"https://new-example.com","token":"new-token"}}'
+
+```
+
+Example response:
+
+```json
+{
+  "id": 9,
+  "name": "update-cluster-name",
+  "created_at": "2020-07-14T18:36:10.440Z",
+  "domain": null,
+  "provider_type": "user",
+  "platform_type": "kubernetes",
+  "environment_scope": "*",
+  "cluster_type": "instance_type",
+  "user": {
+    "id": 1,
+    "name": "Administrator",
+    "username": "root",
+    "state": "active",
+    "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+    "web_url": "https://gitlab.example.com/root"
+  },
+  "platform_kubernetes": {
+    "api_url": "https://new-example.com",
+    "namespace": null,
+    "authorization_type": "rbac",
+    "ca_cert":"-----BEGIN CERTIFICATE-----IxMDM1MV0ZDJkZjM...-----END CERTIFICATE-----"
+  },
+  "provider_gcp": null,
+  "management_project": null,
+  "project": null
+}
+
+```
+
+## Delete instance cluster
+
+Deletes an existing instance cluster.
+
+```plaintext
+DELETE /admin/clusters/:cluster_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `cluster_id` | integer | yes | The ID of the cluster |
+
+Example request:
+
+```shell
+curl --request DELETE --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/admin/clusters/11"
+```

lib/api/admin/instance_clusters.rb

+# frozen_string_literal: true
+
+module API
+  module Admin
+    class InstanceClusters < Grape::API::Instance
+      include PaginationParams
+
+      before do
+        authenticated_as_admin!
+      end
+
+      namespace 'admin' do
+        desc "Get list of all instance clusters" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        get '/clusters' do
+          authorize! :read_cluster, clusterable_instance
+          present paginate(clusters_for_current_user), with: Entities::Cluster
+        end
+
+        desc "Get a single instance cluster" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        params do
+          requires :cluster_id, type: Integer, desc: "The cluster ID"
+        end
+        get '/clusters/:cluster_id' do
+          authorize! :read_cluster, cluster
+
+          present cluster, with: Entities::Cluster
+        end
+
+        desc "Add an instance cluster" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        params do
+          requires :name, type: String, desc: 'Cluster name'
+          optional :enabled, type: Boolean, default: true, desc: 'Determines if cluster is active or not, defaults to true'
+          optional :environment_scope, default: '*', type: String, desc: 'The associated environment to the cluster'
+          optional :domain, type: String, desc: 'Cluster base domain'
+          optional :management_project_id, type: Integer, desc: 'The ID of the management project'
+          optional :managed, type: Boolean, default: true, desc: 'Determines if GitLab will manage namespaces and service accounts for this cluster, defaults to true'
+          requires :platform_kubernetes_attributes, type: Hash, desc: %q(Platform Kubernetes data) do
+            requires :api_url, type: String, allow_blank: false, desc: 'URL to access the Kubernetes API'
+            requires :token, type: String, desc: 'Token to authenticate against Kubernetes'
+            optional :ca_cert, type: String, desc: 'TLS certificate (needed if API is using a self-signed TLS certificate)'
+            optional :namespace, type: String, desc: 'Unique namespace related to Project'
+            optional :authorization_type, type: String, values: ::Clusters::Platforms::Kubernetes.authorization_types.keys, default: 'rbac', desc: 'Cluster authorization type, defaults to RBAC'
+          end
+        end
+        post '/clusters/add' do
+          authorize! :add_cluster, clusterable_instance
+
+          user_cluster = ::Clusters::CreateService
+            .new(current_user, create_cluster_user_params)
+            .execute
+
+          if user_cluster.persisted?
+            present user_cluster, with: Entities::Cluster
+          else
+            render_validation_error!(user_cluster)
+          end
+        end
+
+        desc "Update an instance cluster" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        params do
+          requires :cluster_id, type: Integer, desc: 'The cluster ID'
+          optional :name, type: String, desc: 'Cluster name'
+          optional :enabled, type: Boolean, desc: 'Enable or disable Gitlab\'s connection to your Kubernetes cluster'
+          optional :environment_scope, type: String, desc: 'The associated environment to the cluster'
+          optional :domain, type: String, desc: 'Cluster base domain'
+          optional :management_project_id, type: Integer, desc: 'The ID of the management project'
+          optional :platform_kubernetes_attributes, type: Hash, desc: %q(Platform Kubernetes data) do
+            optional :api_url, type: String, desc: 'URL to access the Kubernetes API'
+            optional :token, type: String, desc: 'Token to authenticate against Kubernetes'
+            optional :ca_cert, type: String, desc: 'TLS certificate (needed if API is using a self-signed TLS certificate)'
+            optional :namespace, type: String, desc: 'Unique namespace related to Project'
+          end
+        end
+        put '/clusters/:cluster_id' do
+          authorize! :update_cluster, cluster
+
+          update_service = ::Clusters::UpdateService.new(current_user, update_cluster_params)
+
+          if update_service.execute(cluster)
+            present cluster, with: Entities::ClusterProject
+          else
+            render_validation_error!(cluster)
+          end
+        end
+
+        desc "Remove a cluster" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        params do
+          requires :cluster_id, type: Integer, desc: "The cluster ID"
+        end
+        delete '/clusters/:cluster_id' do
+          authorize! :admin_cluster, cluster
+
+          destroy_conditionally!(cluster)
+        end
+      end
+
+      helpers do
+        def clusterable_instance
+          Clusters::Instance.new
+        end
+
+        def clusters_for_current_user
+          @clusters_for_current_user ||= ClustersFinder.new(clusterable_instance, current_user, :all).execute
+        end
+
+        def cluster
+          @cluster ||= clusters_for_current_user.find(params[:cluster_id])
+        end
+
+        def create_cluster_user_params
+          declared_params.merge({
+            provider_type: :user,
+            platform_type: :kubernetes,
+            clusterable: clusterable_instance
+          })
+        end
+
+        def update_cluster_params
+          declared_params(include_missing: false).without(:cluster_id)
+        end
+      end
+    end
+  end
+end

lib/api/api.rb

@@ -125,6 +125,7 @@ module API
       # Keep in alphabetical order
       mount ::API::AccessRequests
       mount ::API::Admin::Ci::Variables
+      mount ::API::Admin::InstanceClusters
       mount ::API::Admin::Sidekiq
       mount ::API::Appearance
       mount ::API::Applications