Merge branch 'feature/agents-rest-api' into 'master'

Implement first iteration of the agents REST API

See merge request gitlab-org/gitlab!83270

doc/api/api_resources.md

@@ -22,9 +22,10 @@ See also:
 The following API resources are available in the project context:
 
 | Resource                                                                | Available endpoints                                                                                                                                                                                   |
-|:------------------------------------------------------------------------|:--------------------|
+|:------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | [Access requests](access_requests.md)                                   | `/projects/:id/access_requests` (also available for groups)                                                                                                                                           |
 | [Access tokens](resource_access_tokens.md)                              | `/projects/:id/access_tokens` (also available for groups)                                                                                                                                             |
+| [Agents](cluster_agents.md)                                             | `/projects/:id/cluster_agents`                                                                                                                                                                        |
 | [Award emoji](award_emoji.md)                                           | `/projects/:id/issues/.../award_emoji`, `/projects/:id/merge_requests/.../award_emoji`, `/projects/:id/snippets/.../award_emoji`                                                                      |
 | [Branches](branches.md)                                                 | `/projects/:id/repository/branches/`, `/projects/:id/repository/merged_branches`                                                                                                                      |
 | [Commits](commits.md)                                                   | `/projects/:id/repository/commits`, `/projects/:id/statuses`                                                                                                                                          |

doc/api/cluster_agents.md

+---
+stage: Configure
+group: Configure
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Agents API **(FREE)**
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83270) in GitLab 14.10.
+
+Use the Agents API to work with the GitLab agent for Kubernetes.
+
+## List the agents for a project
+
+Returns the list of agents registered for the project.
+
+You must have at least the Developer role to use this endpoint.
+
+```plaintext
+GET /projects/:id/cluster_agents
+```
+
+Parameters:
+
+| Attribute | Type              | Required  | Description                                                                                                     |
+|-----------|-------------------|-----------|-----------------------------------------------------------------------------------------------------------------|
+| `id`      | integer or string | yes       | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) maintained by the authenticated user |
+
+Response:
+
+The response is a list of agents with the following fields:
+
+| Attribute                            | Type     | Description                                          |
+|--------------------------------------|----------|------------------------------------------------------|
+| `id`                                 | integer  | ID of the agent                                      |
+| `name`                               | string   | Name of the agent                                    |
+| `config_project`                     | object   | Object representing the project the agent belongs to |
+| `config_project.id`                  | integer  | ID of the project                                    |
+| `config_project.description`         | string   | Description of the project                           |
+| `config_project.name`                | string   | Name of the project                                  |
+| `config_project.name_with_namespace` | string   | Full name with namespace of the project              |
+| `config_project.path`                | string   | Path to the project                                  |
+| `config_project.path_with_namespace` | string   | Full path with namespace to the project              |
+| `config_project.created_at`          | string   | ISO8601 datetime when the project was created        |
+| `created_at`                         | string   | ISO8601 datetime when the agent was created          |
+| `created_by_user_id`                 | integer  | ID of the user who created the agent                 |
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/projects/20/cluster_agents"
+```
+
+Example response:
+
+```json
+[
+  {
+    "id": 1,
+    "name": "agent-1",
+    "config_project": {
+      "id": 20,
+      "description": "",
+      "name": "test",
+      "name_with_namespace": "Administrator / test",
+      "path": "test",
+      "path_with_namespace": "root/test",
+      "created_at": "2022-03-20T20:42:40.221Z"
+    },
+    "created_at": "2022-04-20T20:42:40.221Z",
+    "created_by_user_id": 42
+  },
+  {
+    "id": 2,
+    "name": "agent-2",
+    "config_project": {
+      "id": 20,
+      "description": "",
+      "name": "test",
+      "name_with_namespace": "Administrator / test",
+      "path": "test",
+      "path_with_namespace": "root/test",
+      "created_at": "2022-03-20T20:42:40.221Z"
+    },
+    "created_at": "2022-04-20T20:42:40.221Z",
+    "created_by_user_id": 42
+  }
+]
+```
+
+## Get details about an agent
+
+Gets a single agent details.
+
+You must have at least the Developer role to use this endpoint.
+
+```shell
+GET /projects/:id/cluster_agents/:agent_id
+```
+
+Parameters:
+
+| Attribute  | Type              | Required | Description                                                                                                     |
+|------------|-------------------|----------|-----------------------------------------------------------------------------------------------------------------|
+| `id`       | integer or string | yes      | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) maintained by the authenticated user |
+| `agent_id` | integer           | yes      | ID of the agent                                                                                                 |
+
+Response:
+
+The response is a single agent with the following fields:
+
+| Attribute                            | Type    | Description                                          |
+|--------------------------------------|---------|------------------------------------------------------|
+| `id`                                 | integer | ID of the agent                                      |
+| `name`                               | string  | Name of the agent                                    |
+| `config_project`                     | object  | Object representing the project the agent belongs to |
+| `config_project.id`                  | integer | ID of the project                                    |
+| `config_project.description`         | string  | Description of the project                           |
+| `config_project.name`                | string  | Name of the project                                  |
+| `config_project.name_with_namespace` | string  | Full name with namespace of the project              |
+| `config_project.path`                | string  | Path to the project                                  |
+| `config_project.path_with_namespace` | string  | Full path with namespace to the project              |
+| `config_project.created_at`          | string  | ISO8601 datetime when the project was created        |
+| `created_at`                         | string  | ISO8601 datetime when the agent was created          |
+| `created_by_user_id`                 | integer | ID of the user who created the agent                 |
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/projects/20/cluster_agents/1"
+```
+
+Example response:
+
+```json
+{
+  "id": 1,
+  "name": "agent-1",
+  "config_project": {
+    "id": 20,
+    "description": "",
+    "name": "test",
+    "name_with_namespace": "Administrator / test",
+    "path": "test",
+    "path_with_namespace": "root/test",
+    "created_at": "2022-03-20T20:42:40.221Z"
+  },
+  "created_at": "2022-04-20T20:42:40.221Z",
+  "created_by_user_id": 42
+}
+```
+
+## Register an agent with a project
+
+Registers an agent to the project.
+
+You must have at least the Maintainer role to use this endpoint.
+
+```shell
+POST /projects/:id/cluster_agents
+```
+
+Parameters:
+
+| Attribute | Type              | Required | Description                                                                                                     |
+|-----------|-------------------|----------|-----------------------------------------------------------------------------------------------------------------|
+| `id`      | integer or string | yes      | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) maintained by the authenticated user |
+| `name`    | string            | yes      | Name for the agent                                                                                              |
+
+Response:
+
+The response is the new agent with the following fields:
+
+| Attribute                            | Type    | Description                                          |
+|--------------------------------------|---------|------------------------------------------------------|
+| `id`                                 | integer | ID of the agent                                      |
+| `name`                               | string  | Name of the agent                                    |
+| `config_project`                     | object  | Object representing the project the agent belongs to |
+| `config_project.id`                  | integer | ID of the project                                    |
+| `config_project.description`         | string  | Description of the project                           |
+| `config_project.name`                | string  | Name of the project                                  |
+| `config_project.name_with_namespace` | string  | Full name with namespace of the project              |
+| `config_project.path`                | string  | Path to the project                                  |
+| `config_project.path_with_namespace` | string  | Full path with namespace to the project              |
+| `config_project.created_at`          | string  | ISO8601 datetime when the project was created        |
+| `created_at`                         | string  | ISO8601 datetime when the agent was created          |
+| `created_by_user_id`                 | integer | ID of the user who created the agent                 |
+
+Example request:
+
+```shell
+curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/projects/20/cluster_agents" \
+    -H "Content-Type:application/json" \
+    -X POST --data '{"name":"some-agent"}'
+```
+
+Example response:
+
+```json
+{
+  "id": 1,
+  "name": "agent-1",
+  "config_project": {
+    "id": 20,
+    "description": "",
+    "name": "test",
+    "name_with_namespace": "Administrator / test",
+    "path": "test",
+    "path_with_namespace": "root/test",
+    "created_at": "2022-03-20T20:42:40.221Z"
+  },
+  "created_at": "2022-04-20T20:42:40.221Z",
+  "created_by_user_id": 42
+}
+```
+
+## Delete a registered agent
+
+Deletes an existing agent registration.
+
+You must have at least the Maintainer role to use this endpoint.
+
+```plaintext
+DELETE /projects/:id/cluster_agents/:agent_id
+```
+
+Parameters:
+
+| Attribute  | Type              | Required | Description                                                                                                     |
+|------------|-------------------|----------|-----------------------------------------------------------------------------------------------------------------|
+| `id`       | integer or string | yes      | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) maintained by the authenticated user |
+| `agent_id` | integer           | yes      | ID of the agent                                                                                                 |
+
+Example request:
+
+```shell
+curl --request DELETE --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/projects/20/cluster_agents/1
+```

lib/api/admin/instance_clusters.rb

@@ -112,7 +112,7 @@ module API
 
       helpers do
         def clusterable_instance
-          Clusters::Instance.new
+          ::Clusters::Instance.new
         end
 
         def clusters_for_current_user

lib/api/api.rb

@@ -182,6 +182,7 @@ module API
       mount ::API::Ci::SecureFiles
       mount ::API::Ci::Triggers
       mount ::API::Ci::Variables
+      mount ::API::Clusters::Agents
       mount ::API::Commits
       mount ::API::CommitStatuses
       mount ::API::ContainerRegistryEvent

lib/api/ci/jobs.rb

@@ -197,7 +197,7 @@ module API
 
           pipeline = current_authenticated_job.pipeline
           project = current_authenticated_job.project
-          agent_authorizations = Clusters::AgentAuthorizationsFinder.new(project).execute
+          agent_authorizations = ::Clusters::AgentAuthorizationsFinder.new(project).execute
           project_groups = project.group&.self_and_ancestor_ids&.map { |id| { id: id } } || []
           user_access_level = project.team.max_member_access(current_user.id)
           roles_in_project = Gitlab::Access.sym_options_with_owner

lib/api/clusters/agents.rb

+# frozen_string_literal: true
+
+module API
+  module Clusters
+    class Agents < ::API::Base
+      include PaginationParams
+
+      before { authenticate! }
+
+      feature_category :kubernetes_management
+
+      params do
+        requires :id, type: String, desc: 'The ID of a project'
+      end
+      resource :projects, requirements: API::NAMESPACE_OR_PROJECT_REQUIREMENTS do
+        desc 'List agents' do
+          detail 'This feature was introduced in GitLab 14.10.'
+          success Entities::Clusters::Agent
+        end
+        params do
+          use :pagination
+        end
+        get ':id/cluster_agents' do
+          authorize! :read_cluster, user_project
+
+          agents = ::Clusters::AgentsFinder.new(user_project, current_user).execute
+
+          present paginate(agents), with: Entities::Clusters::Agent
+        end
+
+        desc 'Get single agent' do
+          detail 'This feature was introduced in GitLab 14.10.'
+          success Entities::Clusters::Agent
+        end
+        params do
+          requires :agent_id, type: Integer, desc: 'The ID of an agent'
+        end
+        get ':id/cluster_agents/:agent_id' do
+          authorize! :read_cluster, user_project
+
+          agent = user_project.cluster_agents.find(params[:agent_id])
+
+          present agent, with: Entities::Clusters::Agent
+        end
+
+        desc 'Add an agent to a project' do
+          detail 'This feature was introduced in GitLab 14.10.'
+          success Entities::Clusters::Agent
+        end
+        params do
+          requires :name, type: String, desc: 'The name of the agent'
+        end
+        post ':id/cluster_agents' do
+          authorize! :create_cluster, user_project
+
+          params = declared_params(include_missing: false)
+
+          result = ::Clusters::Agents::CreateService.new(user_project, current_user).execute(name: params[:name])
+
+          bad_request!(result[:message]) if result[:status] == :error
+
+          present result[:cluster_agent], with: Entities::Clusters::Agent
+        end
+
+        desc 'Delete an agent' do
+          detail 'This feature was introduced in GitLab 14.10.'
+        end
+        params do
+          requires :agent_id, type: Integer, desc: 'The ID of an agent'
+        end
+        delete ':id/cluster_agents/:agent_id' do
+          authorize! :admin_cluster, user_project
+
+          agent = user_project.cluster_agents.find(params.delete(:agent_id))
+
+          destroy_conditionally!(agent)
+        end
+      end
+    end
+  end
+end

lib/api/entities/clusters/agent.rb

@@ -5,7 +5,10 @@ module API
     module Clusters
       class Agent < Grape::Entity
         expose :id
+        expose :name
         expose :project, with: Entities::ProjectIdentity, as: :config_project
+        expose :created_at
+        expose :created_by_user_id
       end
     end
   end

lib/api/internal/kubernetes.rb

@@ -54,7 +54,7 @@ module API
         def check_agent_token
           unauthorized! unless agent_token
 
-          Clusters::AgentTokens::TrackUsageService.new(agent_token).execute
+          ::Clusters::AgentTokens::TrackUsageService.new(agent_token).execute
         end
       end
 
@@ -91,9 +91,9 @@ module API
             requires :agent_config, type: JSON, desc: 'Configuration for the Agent'
           end
           post '/' do
-            agent = Clusters::Agent.find(params[:agent_id])
+            agent = ::Clusters::Agent.find(params[:agent_id])
 
-            Clusters::Agents::RefreshAuthorizationService.new(agent, config: params[:agent_config]).execute
+            ::Clusters::Agents::RefreshAuthorizationService.new(agent, config: params[:agent_config]).execute
 
             no_content!
           end

lib/api/metrics/dashboard/annotations.rb

@@ -12,7 +12,7 @@ module API
 
         ANNOTATIONS_SOURCES = [
           { class: ::Environment, resource: :environments, create_service_param_key: :environment },
-          { class: Clusters::Cluster, resource: :clusters, create_service_param_key: :cluster }
+          { class: ::Clusters::Cluster, resource: :clusters, create_service_param_key: :cluster }
         ].freeze
 
         ANNOTATIONS_SOURCES.each do |annotations_source|

spec/fixtures/api/schemas/public_api/v4/agent.json

+{
+  "type": "object",
+  "required": [
+    "id",
+    "name",
+    "config_project",
+    "created_at",
+    "created_by_user_id"
+  ],
+  "properties": {
+    "id": { "type": "integer" },
+    "name": { "type": "string" },
+    "config_project": { "$ref": "project_identity.json" },
+    "created_at": { "type": "string", "format": "date-time" },
+    "created_by_user_id": { "type": "integer" }
+  },
+  "additionalProperties": false
+}

spec/fixtures/api/schemas/public_api/v4/agents.json

+{
+  "type": "array",
+  "items": { "$ref": "agent.json" }
+}

spec/fixtures/api/schemas/public_api/v4/project_identity.json

+{
+  "type": "object",
+  "required": [
+    "id",
+    "description",
+    "name",
+    "name_with_namespace",
+    "path",
+    "path_with_namespace",
+    "created_at"
+  ],
+  "properties": {
+    "id": { "type": "integer" },
+    "description": { "type": ["string", "null"] },
+    "name": { "type": "string" },
+    "name_with_namespace": { "type": "string" },
+    "path": { "type": "string" },
+    "path_with_namespace": { "type": "string" },
+    "created_at": { "type": "string", "format": "date-time" }
+  },
+  "additionalProperties": false
+}

spec/models/clusters/agent_spec.rb

@@ -117,6 +117,23 @@ RSpec.describe Clusters::Agent do
     end
   end
 
+  describe '#last_used_agent_tokens' do
+    let_it_be(:agent) { create(:cluster_agent) }
+
+    subject { agent.last_used_agent_tokens }
+
+    context 'agent has no tokens' do
+      it { is_expected.to be_empty }
+    end
+
+    context 'agent has active and inactive tokens' do
+      let!(:active_token) { create(:cluster_agent_token, agent: agent, last_used_at: 1.minute.ago) }
+      let!(:inactive_token) { create(:cluster_agent_token, agent: agent, last_used_at: 2.hours.ago) }
+
+      it { is_expected.to contain_exactly(active_token, inactive_token) }
+    end
+  end
+
   describe '#activity_event_deletion_cutoff' do
     let_it_be(:agent) { create(:cluster_agent) }
     let_it_be(:event1) { create(:agent_activity_event, agent: agent, recorded_at: 1.hour.ago) }

spec/requests/api/clusters/agents_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+RSpec.describe API::Clusters::Agents do
+  let_it_be(:agent) { create(:cluster_agent) }
+
+  let(:user) { agent.created_by_user }
+  let(:unauthorized_user) { create(:user) }
+  let!(:project) { agent.project }
+
+  before do
+    project.add_maintainer(user)
+  end
+
+  describe 'GET /projects/:id/cluster_agents' do
+    context 'authorized user' do
+      it 'returns project agents' do
+        get api("/projects/#{project.id}/cluster_agents", user)
+
+        aggregate_failures "testing response" do
+          expect(response).to have_gitlab_http_status(:ok)
+          expect(response).to include_pagination_headers
+          expect(response).to match_response_schema('public_api/v4/agents')
+          expect(json_response.count).to eq(1)
+          expect(json_response.first['name']).to eq(agent.name)
+        end
+      end
+    end
+
+    context 'unauthorized user' do
+      it 'unable to access agents' do
+        get api("/projects/#{project.id}/cluster_agents", unauthorized_user)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+    end
+
+    it 'avoids N+1 queries', :request_store do
+      # Establish baseline
+      get api("/projects/#{project.id}/cluster_agents", user)
+
+      control = ActiveRecord::QueryRecorder.new do
+        get api("/projects/#{project.id}/cluster_agents", user)
+      end
+
+      # Now create a second record and ensure that the API does not execute
+      # any more queries than before
+      create(:cluster_agent, project: project)
+
+      expect do
+        get api("/projects/#{project.id}/cluster_agents", user)
+      end.not_to exceed_query_limit(control)
+    end
+  end
+
+  describe 'GET /projects/:id/cluster_agents/:agent_id' do
+    context 'authorized user' do
+      it 'returns a project agent' do
+        get api("/projects/#{project.id}/cluster_agents/#{agent.id}", user)
+
+        aggregate_failures "testing response" do
+          expect(response).to have_gitlab_http_status(:ok)
+          expect(response).to match_response_schema('public_api/v4/agent')
+          expect(json_response['name']).to eq(agent.name)
+        end
+      end
+
+      it 'returns a 404 error if agent id is not available' do
+        get api("/projects/#{project.id}/cluster_agents/#{non_existing_record_id}", user)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+    end
+
+    context 'unauthorized user' do
+      it 'unable to access an existing agent' do
+        get api("/projects/#{project.id}/cluster_agents/#{agent.id}", unauthorized_user)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+    end
+  end
+
+  describe 'POST /projects/:id/cluster_agents' do
+    it 'adds agent to project' do
+      expect do
+        post(api("/projects/#{project.id}/cluster_agents", user),
+             params: { name: 'some-agent' })
+      end.to change {project.cluster_agents.count}.by(1)
+
+      aggregate_failures "testing response" do
+        expect(response).to have_gitlab_http_status(:created)
+        expect(response).to match_response_schema('public_api/v4/agent')
+        expect(json_response['name']).to eq('some-agent')
+      end
+    end
+
+    it 'returns a 400 error if name not given' do
+      post api("/projects/#{project.id}/cluster_agents", user)
+
+      expect(response).to have_gitlab_http_status(:bad_request)
+    end
+
+    it 'returns a 400 error if name is invalid' do
+      post api("/projects/#{project.id}/cluster_agents", user), params: { name: '#4^x' }
+
+      aggregate_failures "testing response" do
+        expect(response).to have_gitlab_http_status(:bad_request)
+        expect(json_response['message'])
+          .to include("Name can contain only lowercase letters, digits, and '-', but cannot start or end with '-'")
+      end
+    end
+
+    it 'returns 404 error if project does not exist' do
+      post api("/projects/#{non_existing_record_id}/cluster_agents", user), params: { name: 'some-agent' }
+
+      expect(response).to have_gitlab_http_status(:not_found)
+    end
+  end
+
+  describe 'DELETE /projects/:id/cluster_agents/:agent_id' do
+    it 'deletes agent from project' do
+      expect do
+        delete api("/projects/#{project.id}/cluster_agents/#{agent.id}", user)
+
+        expect(response).to have_gitlab_http_status(:no_content)
+      end.to change {project.cluster_agents.count}.by(-1)
+    end
+
+    it 'returns a 404 error when deleting non existent agent' do
+      delete api("/projects/#{project.id}/cluster_agents/#{non_existing_record_id}", user)
+
+      expect(response).to have_gitlab_http_status(:not_found)
+    end
+
+    it 'returns a 404 error if agent id not given' do
+      delete api("/projects/#{project.id}/cluster_agents", user)
+
+      expect(response).to have_gitlab_http_status(:not_found)
+    end
+
+    it 'returns a 404 if the user is unauthorized to delete' do
+      delete api("/projects/#{project.id}/cluster_agents/#{agent.id}", unauthorized_user)
+
+      expect(response).to have_gitlab_http_status(:not_found)
+    end
+
+    it_behaves_like '412 response' do
+      let(:request) { api("/projects/#{project.id}/cluster_agents/#{agent.id}", user) }
+    end
+  end
+end