Add API support to CRUD instance clusters

Establish API endpoint for instance clusters

Specs test correctly now

Get, post, put, delete

Add get, post, put, delete API support and specs
Remove extra blank line
Remove extra new line
Add changelog entry
Add API docs for instance cluster
Add post, put, and delete
Add specs for get and post
More specs for get

Add specs for put and delete

Add specs for put and delete instance clusters, fix linting

Add documentation for instance cluster API

Add documentation for how to CRUD instance clusters via API, in the
process of documenting also tested that curl requests work

Git rebase with master

Need to rebase because behind master by a lot of commits, this was
failing doclint tests

Accidentally pushed a merge conflict

Address MR review comments

Fix docs and changelog, add specs and params

Address MR review comments

Add clarity to docs and changelog, add some specs, more api
parameters

Address remaining MR comments

Remove extra new lines

Add a spec to ensure no project clusters

Fix relative link

Actually fix the relative link this time

Put instance cluster API in admin module

Move instance cluster code and specs to admin module

Add a spec for multiple instance clusters

Added a spec for multiple instance clusters, also moved some specs into
the valid params context since they had valid params.

changelogs/unreleased/31000-api-for-instance-level-kubernetes-clusters.yml

+---
+title: Add API support for instance-level Kubernetes clusters
+merge_request: 36001
+author:
+type: added

doc/api/api_resources.md

@@ -129,6 +129,7 @@ The following API resources are available outside of project and group contexts
 | [Geo Nodes](geo_nodes.md) **(PREMIUM ONLY)**       | `/geo_nodes`                                                            |
 | [Group Activity Analytics](group_activity_analytics.md) **(STARTER)**  | `/analytics/group_activity/{issues_count | merge_requests_count | new_members_count }`  |
 | [Import repository from GitHub](import.md)         | `/import/github`                                                        |
+| [Instance clusters](instance_clusters.md)          | `/admin/clusters`                                                       |
 | [Issues](issues.md)                                | `/issues` (also available for groups and projects)                      |
 | [Issues Statistics](issues_statistics.md)          | `/issues_statistics` (also available for groups and projects)           |
 | [Keys](keys.md)                                    | `/keys`                                                                 |

doc/api/instance_clusters.md

+# Instance clusters API
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36001) in GitLab 13.2.
+
+NOTE: **Note:**
+User will need admin access to use these endpoints.
+
+Use these API endpoints with your instance clusters, which enable you to use the same cluster across multiple projects. [More information](../user/instance/clusters/index.md)
+
+## List instance clusters
+
+Returns a list of instance clusters.
+
+```plaintext
+GET /admin/clusters
+```
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/admin/clusters"
+```
+
+Example response:
+
+```json
+[
+  {
+    "id": 9,
+    "name": "cluster-1",
+    "created_at": "2020-07-14T18:36:10.440Z",
+    "domain": null,
+    "provider_type": "user",
+    "platform_type": "kubernetes",
+    "environment_scope": "*",
+    "cluster_type": "instance_type",
+    "user": {
+      "id": 1,
+      "name": "Administrator",
+      "username": "root",
+      "state": "active",
+      "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+      "web_url": "https://gitlab.example.com/root"
+    },
+    "platform_kubernetes": {
+      "api_url": "https://example.com",
+      "namespace": null,
+      "authorization_type": "rbac",
+      "ca_cert":"-----BEGIN CERTIFICATE-----IxMDM1MV0ZDJkZjM...-----END CERTIFICATE-----"
+    },
+    "provider_gcp": null,
+    "management_project": null
+  },
+  {
+    "id": 10,
+    "name": "cluster-2",
+    "created_at": "2020-07-14T18:39:05.383Z",
+    "domain": null,
+    "provider_type": "user",
+    "platform_type": "kubernetes",
+    "environment_scope": "staging",
+    "cluster_type": "instance_type",
+    "user": {
+      "id": 1,
+      "name": "Administrator",
+      "username": "root",
+      "state": "active",
+      "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+      "web_url": "https://gitlab.example.com/root"
+    },
+    "platform_kubernetes": {
+      "api_url": "https://example.com",
+      "namespace": null,
+      "authorization_type": "rbac",
+      "ca_cert":"-----BEGIN CERTIFICATE-----LzEtMCadtaLGxcsGAZjM...-----END CERTIFICATE-----"
+    },
+    "provider_gcp": null,
+    "management_project": null
+  }
+  {
+    "id": 11,
+    "name": "cluster-3",
+    ...
+  }
+]
+
+```
+
+## Get a single instance cluster
+
+Returns a single instance cluster.
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `cluster_id` | integer | yes | The ID of the cluster |
+
+```plaintext
+GET /admin/clusters/:cluster_id
+```
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/admin/clusters/9"
+```
+
+Example response:
+
+```json
+{
+  "id": 9,
+  "name": "cluster-1",
+  "created_at": "2020-07-14T18:36:10.440Z",
+  "domain": null,
+  "provider_type": "user",
+  "platform_type": "kubernetes",
+  "environment_scope": "*",
+  "cluster_type": "instance_type",
+  "user": {
+    "id": 1,
+    "name": "Administrator",
+    "username": "root",
+    "state": "active",
+    "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+    "web_url": "https://gitlab.example.com/root"
+  },
+  "platform_kubernetes": {
+    "api_url": "https://example.com",
+    "namespace": null,
+    "authorization_type": "rbac",
+    "ca_cert":"-----BEGIN CERTIFICATE-----IxMDM1MV0ZDJkZjM...-----END CERTIFICATE-----"
+  },
+  "provider_gcp": null,
+  "management_project": null
+}
+```
+
+## Add existing instance cluster
+
+Adds an existing Kubernetes instance cluster.
+
+```plaintext
+POST /admin/clusters/add
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `name` | string | yes | The name of the cluster |
+| `domain` | string | no | The [base domain](../user/project/clusters/index.md#base-domain) of the cluster |
+| `environment_scope` | string | no | The associated environment to the cluster. Defaults to `*` |
+| `management_project_id` | integer | no | The ID of the [management project](../user/clusters/management_project.md) for the cluster |
+| `enabled` | boolean | no | Determines if cluster is active or not, defaults to true |
+| `managed` | boolean | no | Determines if GitLab will manage namespaces and service accounts for this cluster, defaults to true |
+| `platform_kubernetes_attributes[api_url]` | string | yes | The URL to access the Kubernetes API |
+| `platform_kubernetes_attributes[token]` | string | yes | The token to authenticate against Kubernetes |
+| `platform_kubernetes_attributes[ca_cert]` | string | no | TLS certificate. Required if API is using a self-signed TLS certificate. |
+| `platform_kubernetes_attributes[namespace]` | string | no | The unique namespace related to the project |
+| `platform_kubernetes_attributes[authorization_type]` | string | no | The cluster authorization type: `rbac`, `abac` or `unknown_authorization`. Defaults to `rbac`. |
+
+Example request:
+
+```shell
+curl --header "Private-Token:<your_access_token>" "http://gitlab.example.com/api/v4/admin/clusters/add" \
+-H "Accept:application/json" \
+-H "Content-Type:application/json" \
+-X POST --data '{"name":"cluster-3", "environment_scope":"production", "platform_kubernetes_attributes":{"api_url":"https://example.com", "token":"12345",  "ca_cert":"-----BEGIN CERTIFICATE-----qpoeiXXZafCM0ZDJkZjM...-----END CERTIFICATE-----"}}'
+
+```
+
+Example response:
+
+```json
+{
+  "id": 11,
+  "name": "cluster-3",
+  "created_at": "2020-07-14T18:42:50.805Z",
+  "domain": null,
+  "provider_type": "user",
+  "platform_type": "kubernetes",
+  "environment_scope": "production",
+  "cluster_type": "instance_type",
+  "user": {
+    "id": 1,
+    "name": "Administrator",
+    "username": "root",
+    "state": "active",
+    "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+    "web_url": "http://gitlab.example.com:3000/root"
+  },
+  "platform_kubernetes": {
+    "api_url": "https://example.com",
+    "namespace": null,
+    "authorization_type": "rbac",
+    "ca_cert":"-----BEGIN CERTIFICATE-----qpoeiXXZafCM0ZDJkZjM...-----END CERTIFICATE-----"
+  },
+  "provider_gcp": null,
+  "management_project": null
+}
+```
+
+## Edit instance cluster
+
+Updates an existing instance cluster.
+
+```shell
+PUT /admin/clusters/:cluster_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `cluster_id` | integer | yes | The ID of the cluster |
+| `name` | string | no | The name of the cluster |
+| `domain` | string | no | The [base domain](../user/project/clusters/index.md#base-domain) of the cluster |
+| `environment_scope` | string | no | The associated environment to the cluster |
+| `management_project_id` | integer | no | The ID of the [management project](../user/clusters/management_project.md) for the cluster |
+| `enabled` | boolean | no | Determines if cluster is active or not, defaults to true |
+| `platform_kubernetes_attributes[api_url]` | string | no | The URL to access the Kubernetes API |
+| `platform_kubernetes_attributes[token]` | string | no | The token to authenticate against Kubernetes |
+| `platform_kubernetes_attributes[ca_cert]` | string | no | TLS certificate. Required if API is using a self-signed TLS certificate. |
+| `platform_kubernetes_attributes[namespace]` | string | no | The unique namespace related to the project |
+
+NOTE: **Note:**
+`name`, `api_url`, `ca_cert` and `token` can only be updated if the cluster was added
+through the [Add existing Kubernetes cluster](../user/project/clusters/add_remove_clusters.md#add-existing-cluster) option or
+through the [Add existing instance cluster](#add-existing-instance-cluster) endpoint.
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "http://gitlab.example.com/api/v4/admin/clusters/9" \
+-H "Content-Type:application/json" \
+-X PUT --data '{"name":"update-cluster-name", "platform_kubernetes_attributes":{"api_url":"https://new-example.com","token":"new-token"}}'
+
+```
+
+Example response:
+
+```json
+{
+  "id": 9,
+  "name": "update-cluster-name",
+  "created_at": "2020-07-14T18:36:10.440Z",
+  "domain": null,
+  "provider_type": "user",
+  "platform_type": "kubernetes",
+  "environment_scope": "*",
+  "cluster_type": "instance_type",
+  "user": {
+    "id": 1,
+    "name": "Administrator",
+    "username": "root",
+    "state": "active",
+    "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+    "web_url": "https://gitlab.example.com/root"
+  },
+  "platform_kubernetes": {
+    "api_url": "https://new-example.com",
+    "namespace": null,
+    "authorization_type": "rbac",
+    "ca_cert":"-----BEGIN CERTIFICATE-----IxMDM1MV0ZDJkZjM...-----END CERTIFICATE-----"
+  },
+  "provider_gcp": null,
+  "management_project": null,
+  "project": null
+}
+
+```
+
+## Delete instance cluster
+
+Deletes an existing instance cluster.
+
+```plaintext
+DELETE /admin/clusters/:cluster_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `cluster_id` | integer | yes | The ID of the cluster |
+
+Example request:
+
+```shell
+curl --request DELETE --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/admin/clusters/11"
+```

lib/api/admin/instance_clusters.rb

+# frozen_string_literal: true
+
+module API
+  module Admin
+    class InstanceClusters < Grape::API::Instance
+      include PaginationParams
+
+      before do
+        authenticated_as_admin!
+      end
+
+      namespace 'admin' do
+        desc "Get list of all instance clusters" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        get '/clusters' do
+          authorize! :read_cluster, clusterable_instance
+          present paginate(clusters_for_current_user), with: Entities::Cluster
+        end
+
+        desc "Get a single instance cluster" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        params do
+          requires :cluster_id, type: Integer, desc: "The cluster ID"
+        end
+        get '/clusters/:cluster_id' do
+          authorize! :read_cluster, cluster
+
+          present cluster, with: Entities::Cluster
+        end
+
+        desc "Add an instance cluster" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        params do
+          requires :name, type: String, desc: 'Cluster name'
+          optional :enabled, type: Boolean, default: true, desc: 'Determines if cluster is active or not, defaults to true'
+          optional :environment_scope, default: '*', type: String, desc: 'The associated environment to the cluster'
+          optional :domain, type: String, desc: 'Cluster base domain'
+          optional :management_project_id, type: Integer, desc: 'The ID of the management project'
+          optional :managed, type: Boolean, default: true, desc: 'Determines if GitLab will manage namespaces and service accounts for this cluster, defaults to true'
+          requires :platform_kubernetes_attributes, type: Hash, desc: %q(Platform Kubernetes data) do
+            requires :api_url, type: String, allow_blank: false, desc: 'URL to access the Kubernetes API'
+            requires :token, type: String, desc: 'Token to authenticate against Kubernetes'
+            optional :ca_cert, type: String, desc: 'TLS certificate (needed if API is using a self-signed TLS certificate)'
+            optional :namespace, type: String, desc: 'Unique namespace related to Project'
+            optional :authorization_type, type: String, values: ::Clusters::Platforms::Kubernetes.authorization_types.keys, default: 'rbac', desc: 'Cluster authorization type, defaults to RBAC'
+          end
+        end
+        post '/clusters/add' do
+          authorize! :add_cluster, clusterable_instance
+
+          user_cluster = ::Clusters::CreateService
+            .new(current_user, create_cluster_user_params)
+            .execute
+
+          if user_cluster.persisted?
+            present user_cluster, with: Entities::Cluster
+          else
+            render_validation_error!(user_cluster)
+          end
+        end
+
+        desc "Update an instance cluster" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        params do
+          requires :cluster_id, type: Integer, desc: 'The cluster ID'
+          optional :name, type: String, desc: 'Cluster name'
+          optional :enabled, type: Boolean, desc: 'Enable or disable Gitlab\'s connection to your Kubernetes cluster'
+          optional :environment_scope, type: String, desc: 'The associated environment to the cluster'
+          optional :domain, type: String, desc: 'Cluster base domain'
+          optional :management_project_id, type: Integer, desc: 'The ID of the management project'
+          optional :platform_kubernetes_attributes, type: Hash, desc: %q(Platform Kubernetes data) do
+            optional :api_url, type: String, desc: 'URL to access the Kubernetes API'
+            optional :token, type: String, desc: 'Token to authenticate against Kubernetes'
+            optional :ca_cert, type: String, desc: 'TLS certificate (needed if API is using a self-signed TLS certificate)'
+            optional :namespace, type: String, desc: 'Unique namespace related to Project'
+          end
+        end
+        put '/clusters/:cluster_id' do
+          authorize! :update_cluster, cluster
+
+          update_service = ::Clusters::UpdateService.new(current_user, update_cluster_params)
+
+          if update_service.execute(cluster)
+            present cluster, with: Entities::ClusterProject
+          else
+            render_validation_error!(cluster)
+          end
+        end
+
+        desc "Remove a cluster" do
+          detail "This feature was introduced in GitLab 13.2."
+        end
+        params do
+          requires :cluster_id, type: Integer, desc: "The cluster ID"
+        end
+        delete '/clusters/:cluster_id' do
+          authorize! :admin_cluster, cluster
+
+          destroy_conditionally!(cluster)
+        end
+      end
+
+      helpers do
+        def clusterable_instance
+          Clusters::Instance.new
+        end
+
+        def clusters_for_current_user
+          @clusters_for_current_user ||= ClustersFinder.new(clusterable_instance, current_user, :all).execute
+        end
+
+        def cluster
+          @cluster ||= clusters_for_current_user.find(params[:cluster_id])
+        end
+
+        def create_cluster_user_params
+          declared_params.merge({
+            provider_type: :user,
+            platform_type: :kubernetes,
+            clusterable: clusterable_instance
+          })
+        end
+
+        def update_cluster_params
+          declared_params(include_missing: false).without(:cluster_id)
+        end
+      end
+    end
+  end
+end

lib/api/api.rb

@@ -124,6 +124,7 @@ module API
       # Keep in alphabetical order
       mount ::API::AccessRequests
       mount ::API::Admin::Ci::Variables
+      mount ::API::Admin::InstanceClusters
       mount ::API::Admin::Sidekiq
       mount ::API::Appearance
       mount ::API::Applications