Commit d7910628 authored by Pawel Chojnacki's avatar Pawel Chojnacki

Squashed commit of the following:

commit 4ff8db0d
Merge: 60f52508 5e8138aa
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Tue Mar 6 16:08:11 2018 +0100

    Merge remote-tracking branch 'upstream/master' into 5029-support-cluster-metrics-ce

    # Conflicts:
    #	spec/models/clusters/applications/prometheus_spec.rb

commit 60f52508
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Tue Mar 6 16:05:24 2018 +0100

    Arg.id via method pointer

commit bd684b70
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Mon Mar 5 23:32:08 2018 +0100

    one space too far

commit b5335654
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Mon Mar 5 21:02:26 2018 +0100

    Check if prometheus_adapter is properly called

commit 5ceec83a
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Mon Mar 5 19:34:59 2018 +0100

    Implemente Prometheus:AdapterService

commit 273e4142
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Thu Mar 1 20:20:41 2018 +0100

    add missing newline

commit 8ad9cd55
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Thu Mar 1 18:14:54 2018 +0100

    Implement PrometheusDeploymentLocator + fix metrics controller

commit 2c090539
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Mon Feb 26 19:57:11 2018 +0100

    use deployment_platform DI in environment and revert changes to k8s cert verification policy

commit 637c9558
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Mon Feb 26 12:40:40 2018 +0100

    Fix failing test, when deployment platform is not bound to a cluster.

commit 0feeddaa
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Sat Feb 24 01:06:08 2018 +0100

    drop the ! from synchronize_service_state! + remove unused scope

commit 22e2cad9
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Sat Feb 24 00:56:50 2018 +0100

    Use deployment platform to find cluster with prometheus application

commit 57de968a
Author: Pawel Chojnacki <pawel@chojnacki.ws>
Date:   Fri Feb 23 21:33:33 2018 +0100

    Squashed commit of the following:

    commit 22e1cb8f4b98f71d21026f69aa3e68d79946d6ae
    Merge: 766a42a9639 e966c6ae
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Fri Feb 23 21:30:46 2018 +0100

        Merge remote-tracking branch 'upstream/master' into 38783-add-cluster-metrics.yml

        # Conflicts:
        #	app/controllers/projects/prometheus/metrics_controller.rb
        #	app/controllers/projects/prometheus_controller.rb
        #	app/models/project_services/prometheus_service.rb
        #	lib/gitlab/prometheus/queries/query_additional_metrics.rb
        #	spec/controllers/projects/prometheus/metrics_controller_spec.rb
        #	spec/models/project_services/prometheus_service_spec.rb

    commit 766a42a96393f502d439c1f0beb0b6cfb2c228d4
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Fri Feb 23 17:58:45 2018 +0100

        Move prometheus adapter to app/models/concerns

    commit ca84eed49811cf3064a2e5ea611af1c947c590d7
    Merge: 66702099586 cb504cedc2a
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Fri Feb 23 15:36:03 2018 +0100

        Merge remote-tracking branch 'upstream/backport_custom_metrics_ce_components' into 38783-add-cluster-metrics.yml

        + fix failing tests

        # Conflicts:
        #	app/controllers/projects/prometheus_controller.rb
        #	app/models/project_services/prometheus_service.rb
        #	lib/gitlab/prometheus/queries/query_additional_metrics.rb
        #	spec/models/project_services/prometheus_service_spec.rb

    commit cb504cedc2a6e353ffb56833334681e3da09fc14
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Fri Feb 23 14:46:15 2018 +0100

        Fix prometheus_service found by find_or_initialize_service

    commit 928b84c72c2c4c46e1785b9a943c1822a137de16
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 22 23:15:11 2018 +0100

        additional metrics and backported tests

    commit 41291383b4b6976af94eaf9e1a7b2e2a172310e7
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 22 21:57:45 2018 +0100

        Introduce Metrics controller and retire prometheus controller

    commit 6e7492e4c7ffa9d8621f09198071bc14be875976
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 22 21:46:37 2018 +0100

        Backport PrometheusClient::Error and all->common_metrics rename

    commit 66702099586c864a7f78970a0ee0dd9f3c2beeef
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 22 20:23:03 2018 +0100

        fix failing tests

    commit ce921ea723cde61b2441ba98c9aca97606c719c8
    Author: Mike Greiling <mike@pixelcog.com>
    Date:   Wed Feb 21 23:47:30 2018 -0600

        add labels to cluster metrics

    commit 993830c6892e4fbf53d3f72da00002d642c7e9b2
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 22 01:20:49 2018 +0100

        Fix formatting probelms and few small tests

    commit 66ee65d8e3f747c90d986bc3056178422156bc8e
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 22 01:08:03 2018 +0100

        stop using in deployment tests environment.id

    commit 62c91978d15f0369988521363dae24bd7510d68d
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 22 01:06:33 2018 +0100

        fix prometheus_controller and adapter tests

    commit 977b1d34c1d03c7233582e8328f85caf634895ed
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 22 00:19:39 2018 +0100

        finish up active? -> can_query? rename

    commit e614f7daee58a9758d83ba3efe1649c8b80bc1e2
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Wed Feb 21 23:51:33 2018 +0100

        deployment prometheus adapter tests fix

    commit ebd726c114a6026fef0adf3eba6ee1972530148a
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Wed Feb 21 21:40:40 2018 +0100

        Move environment dependant tests to environment

    commit 6d31311cd3729c29233283dded70e03a4a9a3c97
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Wed Feb 21 20:40:24 2018 +0100

        update monitoring service and move adding dpeloyment_time to deployment model

    commit 60b6bf391ab36846dce122bc6b0c5196a186267c
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Wed Feb 21 20:15:46 2018 +0100

        adjust deployment spec and prometheus specs

    commit 6681662cf1c028aff2ff94aa0501732cb7119ba1
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 22:30:20 2018 +0100

        Revert changes to reactive caching

    commit e282f86c45a056889f57d3f7fd23a81c88efff6a
    Merge: 5751c73df59 6844a2df
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 22:08:17 2018 +0100

        Merge remote-tracking branch 'upstream/master' into 38783-add-cluster-metrics.yml

    commit 5751c73df59d0a03840a1b4b71b0637670f971a6
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 22:07:43 2018 +0100

        rename active? to can_query? and cleanup environment prometheus router

    commit 3f3c6e1d33dcd9315979daf26a95f2aab83a7de9
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 21:51:43 2018 +0100

        Fix tests, and only use prometheus service if its active

    commit 6345838bac584c213b665d334252ccab202cb271
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 16:24:27 2018 +0100

        Fix typo

    commit 7a585d32afe8da050b5615b1d036a550e06479f5
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 15:29:40 2018 +0100

        Cluster id is not required

    commit e6af62afb11fa380f6aff1c31a81bcc9bab3b1eb
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 15:15:46 2018 +0100

        Result transformation support

    commit f3b1bd7c67894f44efe33591ddb70093bd620c03
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 15:07:07 2018 +0100

        Fix rubocop warning and exten cluster query timeframe

    commit be77947cea64261a4d3dead33c3c57f413a9880c
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 14:42:47 2018 +0100

        Fix additional metrics test

    commit eb3922e16221abe16f59fae1c38122f227643343
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 14:34:44 2018 +0100

        rename prometheus adapter methods

    commit 045476cd08b21593818b274ae8a44d19b705523f
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 13:19:36 2018 +0100

        Make prometheus adapter a module

    commit f2daf050d8c689f72c4c61207930bc53c331f12e
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 20 12:00:44 2018 +0100

        refactoring wip

    commit 52e4ef5587794e811dc10a0f2dca522342a865da
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Sun Feb 18 20:25:55 2018 +0100

        cleanup prometheus adapter concept

    commit 3887365faab9dfcd9c00bcfc501d09ac62431a03
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Sun Feb 18 19:22:11 2018 +0100

        Refactor out deployment id, Rename PrometheusQuerier to PrometheusAdapter

    commit aa2fc2df57bd72c9a5e94f66d1f1e23990be6c3f
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Fri Feb 16 23:59:54 2018 +0100

        Refactor prometheus client

    commit e43c1ca9d9874d6cf1569f40fa1aca158d9d5d91
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 15 20:24:45 2018 +0100

        Use initial version of cluster_metrics.yml

    commit 867821ce0b2609ebf8994220aa8e3a94d66a01e0
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 15 05:24:14 2018 +0100

        Fix querying cluster metrics

    commit 1601e002a064cbb10ffe110a19433c5662858f1d
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 15 04:49:03 2018 +0100

        Queues for unicersal querier

    commit 5db198fdc925c0223be24939b76da1d544dd569c
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 15 03:35:17 2018 +0100

        refactor reactive caching and prometheus querying

    commit b0fc00e8c9d21e961ef44b0129103e2a62928b52
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Thu Feb 15 00:12:03 2018 +0100

        Add generic query additional metrics

    commit ffe76e6a9ba196bccff22c4880e0384959ad5e48
    Author: Pawel Chojnacki <pawel@chojnacki.ws>
    Date:   Tue Feb 13 15:40:15 2018 +0100

        Cluster Metric yml initial

    + Remove cluster query

    + remove cluster_metrics.yml

    + Prometheus adapter tests
parent 674be13f
...@@ -24,7 +24,7 @@ class Projects::DeploymentsController < Projects::ApplicationController ...@@ -24,7 +24,7 @@ class Projects::DeploymentsController < Projects::ApplicationController
end end
def additional_metrics def additional_metrics
return render_404 unless deployment.has_additional_metrics? return render_404 unless deployment.has_metrics?
respond_to do |format| respond_to do |format|
format.json do format.json do
......
...@@ -2,11 +2,12 @@ module Projects ...@@ -2,11 +2,12 @@ module Projects
module Prometheus module Prometheus
class MetricsController < Projects::ApplicationController class MetricsController < Projects::ApplicationController
before_action :authorize_admin_project! before_action :authorize_admin_project!
before_action :require_prometheus_metrics!
def active_common def active_common
respond_to do |format| respond_to do |format|
format.json do format.json do
matched_metrics = prometheus_service.matched_metrics || {} matched_metrics = prometheus_adapter.query(:matched_metrics) || {}
if matched_metrics.any? if matched_metrics.any?
render json: matched_metrics render json: matched_metrics
...@@ -19,8 +20,12 @@ module Projects ...@@ -19,8 +20,12 @@ module Projects
private private
def prometheus_service def prometheus_adapter
@prometheus_service ||= project.find_or_initialize_service('prometheus') @prometheus_adapter ||= ::Prometheus::AdapterService.new(project).prometheus_adapter
end
def require_prometheus_metrics!
render_404 unless prometheus_adapter.can_query?
end end
end end
end end
......
module Clusters module Clusters
module Applications module Applications
class Prometheus < ActiveRecord::Base class Prometheus < ActiveRecord::Base
include PrometheusAdapter
VERSION = "2.0.0".freeze VERSION = "2.0.0".freeze
self.table_name = 'clusters_applications_prometheus' self.table_name = 'clusters_applications_prometheus'
...@@ -39,7 +41,7 @@ module Clusters ...@@ -39,7 +41,7 @@ module Clusters
) )
end end
def proxy_client def prometheus_client
return unless kube_client return unless kube_client
proxy_url = kube_client.proxy_url('service', service_name, service_port, Gitlab::Kubernetes::Helm::NAMESPACE) proxy_url = kube_client.proxy_url('service', service_name, service_port, Gitlab::Kubernetes::Helm::NAMESPACE)
......
...@@ -53,9 +53,6 @@ module Clusters ...@@ -53,9 +53,6 @@ module Clusters
scope :enabled, -> { where(enabled: true) } scope :enabled, -> { where(enabled: true) }
scope :disabled, -> { where(enabled: false) } scope :disabled, -> { where(enabled: false) }
scope :for_environment, -> (env) { where(environment_scope: ['*', '', env.slug]) }
scope :for_all_environments, -> { where(environment_scope: ['*', '']) }
def status_name def status_name
if provider if provider
provider.status_name provider.status_name
......
module PrometheusAdapter
extend ActiveSupport::Concern
included do
include ReactiveCaching
self.reactive_cache_key = ->(adapter) { [adapter.class.model_name.singular, adapter.id] }
self.reactive_cache_lease_timeout = 30.seconds
self.reactive_cache_refresh_interval = 30.seconds
self.reactive_cache_lifetime = 1.minute
def prometheus_client
raise NotImplementedError
end
def prometheus_client_wrapper
Gitlab::PrometheusClient.new(prometheus_client)
end
def can_query?
prometheus_client.present?
end
def query(query_name, *args)
return unless can_query?
query_class = Gitlab::Prometheus::Queries.const_get("#{query_name.to_s.classify}Query")
args.map!(&:id)
with_reactive_cache(query_class.name, *args, &query_class.method(:transform_reactive_result))
end
# Cache metrics for specific environment
def calculate_reactive_cache(query_class_name, *args)
return unless prometheus_client
data = Kernel.const_get(query_class_name).new(prometheus_client_wrapper).query(*args)
{
success: true,
data: data,
last_update: Time.now.utc
}
rescue Gitlab::PrometheusClient::Error => err
{ success: false, result: err.message }
end
end
end
...@@ -98,28 +98,29 @@ class Deployment < ActiveRecord::Base ...@@ -98,28 +98,29 @@ class Deployment < ActiveRecord::Base
end end
def has_metrics? def has_metrics?
project.monitoring_service.present? prometheus_adapter&.can_query?
end end
def metrics def metrics
return {} unless has_metrics? return {} unless has_metrics?
project.monitoring_service.deployment_metrics(self) metrics = prometheus_adapter.query(:deployment, self)
end metrics&.merge(deployment_time: created_at.to_i) || {}
def has_additional_metrics?
project.prometheus_service.present?
end end
def additional_metrics def additional_metrics
return {} unless project.prometheus_service.present? return {} unless has_metrics?
metrics = project.prometheus_service.additional_deployment_metrics(self) metrics = prometheus_adapter.query(:additional_metrics_deployment, self)
metrics&.merge(deployment_time: created_at.to_i) || {} metrics&.merge(deployment_time: created_at.to_i) || {}
end end
private private
def prometheus_adapter
environment.prometheus_adapter
end
def ref_path def ref_path
File.join(environment.ref_path, 'deployments', iid.to_s) File.join(environment.ref_path, 'deployments', iid.to_s)
end end
......
...@@ -150,21 +150,19 @@ class Environment < ActiveRecord::Base ...@@ -150,21 +150,19 @@ class Environment < ActiveRecord::Base
end end
def has_metrics? def has_metrics?
project.monitoring_service.present? && available? && last_deployment.present? prometheus_adapter&.can_query? && available? && last_deployment.present?
end end
def metrics def metrics
project.monitoring_service.environment_metrics(self) if has_metrics? prometheus_adapter.query(:environment, self) if has_metrics?
end
def has_additional_metrics?
project.prometheus_service.present? && available? && last_deployment.present?
end end
def additional_metrics def additional_metrics
if has_additional_metrics? prometheus_adapter.query(:additional_metrics_environment, self) if has_metrics?
project.prometheus_service.additional_environment_metrics(self)
end end
def prometheus_adapter
@prometheus_adapter ||= Prometheus::AdapterService.new(project, deployment_platform).prometheus_adapter
end end
def slug def slug
......
...@@ -9,11 +9,11 @@ class MonitoringService < Service ...@@ -9,11 +9,11 @@ class MonitoringService < Service
%w() %w()
end end
def environment_metrics(environment) def can_query?
raise NotImplementedError raise NotImplementedError
end end
def deployment_metrics(deployment) def query(_, *_)
raise NotImplementedError raise NotImplementedError
end end
end end
class PrometheusService < MonitoringService class PrometheusService < MonitoringService
include ReactiveService include PrometheusAdapter
self.reactive_cache_lease_timeout = 30.seconds
self.reactive_cache_refresh_interval = 30.seconds
self.reactive_cache_lifetime = 1.minute
# Access to prometheus is directly through the API # Access to prometheus is directly through the API
prop_accessor :api_url prop_accessor :api_url
...@@ -13,7 +9,7 @@ class PrometheusService < MonitoringService ...@@ -13,7 +9,7 @@ class PrometheusService < MonitoringService
validates :api_url, url: true validates :api_url, url: true
end end
before_save :synchronize_service_state! before_save :synchronize_service_state
after_save :clear_reactive_cache! after_save :clear_reactive_cache!
...@@ -66,63 +62,15 @@ class PrometheusService < MonitoringService ...@@ -66,63 +62,15 @@ class PrometheusService < MonitoringService
# Check we can connect to the Prometheus API # Check we can connect to the Prometheus API
def test(*args) def test(*args)
client.ping Gitlab::PrometheusClient.new(prometheus_client).ping
{ success: true, result: 'Checked API endpoint' } { success: true, result: 'Checked API endpoint' }
rescue Gitlab::PrometheusClient::Error => err rescue Gitlab::PrometheusClient::Error => err
{ success: false, result: err } { success: false, result: err }
end end
def environment_metrics(environment) def prometheus_client
with_reactive_cache(Gitlab::Prometheus::Queries::EnvironmentQuery.name, environment.id, &rename_field(:data, :metrics)) RestClient::Resource.new(api_url) if api_url && manual_configuration? && active?
end
def deployment_metrics(deployment)
metrics = with_reactive_cache(Gitlab::Prometheus::Queries::DeploymentQuery.name, deployment.environment.id, deployment.id, &rename_field(:data, :metrics))
metrics&.merge(deployment_time: deployment.created_at.to_i) || {}
end
def additional_environment_metrics(environment)
with_reactive_cache(Gitlab::Prometheus::Queries::AdditionalMetricsEnvironmentQuery.name, environment.id, &:itself)
end
def additional_deployment_metrics(deployment)
with_reactive_cache(Gitlab::Prometheus::Queries::AdditionalMetricsDeploymentQuery.name, deployment.environment.id, deployment.id, &:itself)
end
def matched_metrics
with_reactive_cache(Gitlab::Prometheus::Queries::MatchedMetricsQuery.name, &:itself)
end
# Cache metrics for specific environment
def calculate_reactive_cache(query_class_name, *args)
return unless active? && project && !project.pending_delete?
environment_id = args.first
client = client(environment_id)
data = Kernel.const_get(query_class_name).new(client).query(*args)
{
success: true,
data: data,
last_update: Time.now.utc
}
rescue Gitlab::PrometheusClient::Error => err
{ success: false, result: err.message }
end
def client(environment_id = nil)
if manual_configuration?
Gitlab::PrometheusClient.new(RestClient::Resource.new(api_url))
else
cluster = cluster_with_prometheus(environment_id)
raise Gitlab::PrometheusClient::Error, "couldn't find cluster with Prometheus installed" unless cluster
rest_client = client_from_cluster(cluster)
raise Gitlab::PrometheusClient::Error, "couldn't create proxy Prometheus client" unless rest_client
Gitlab::PrometheusClient.new(rest_client)
end
end end
def prometheus_installed? def prometheus_installed?
...@@ -134,32 +82,7 @@ class PrometheusService < MonitoringService ...@@ -134,32 +82,7 @@ class PrometheusService < MonitoringService
private private
def cluster_with_prometheus(environment_id = nil) def synchronize_service_state
clusters = if environment_id
::Environment.find_by(id: environment_id).try do |env|
# sort results by descending order based on environment_scope being longer
# thus more closely matching environment slug
project.clusters.enabled.for_environment(env).sort_by { |c| c.environment_scope&.length }.reverse!
end
else
project.clusters.enabled.for_all_environments
end
clusters&.detect { |cluster| cluster.application_prometheus&.installed? }
end
def client_from_cluster(cluster)
cluster.application_prometheus.proxy_client
end
def rename_field(old_field, new_field)
-> (metrics) do
metrics[new_field] = metrics.delete(old_field)
metrics
end
end
def synchronize_service_state!
self.active = prometheus_installed? || manual_configuration? self.active = prometheus_installed? || manual_configuration?
true true
......
module Prometheus
class AdapterService
def initialize(project, deployment_platform = nil)
@project = project
@deployment_platform = if deployment_platform
deployment_platform
else
project.deployment_platform
end
end
attr_reader :deployment_platform, :project
def prometheus_adapter
@prometheus_adapter ||= if service_prometheus_adapter.can_query?
service_prometheus_adapter
else
cluster_prometheus_adapter
end
end
def service_prometheus_adapter
project.find_or_initialize_service('prometheus')
end
def cluster_prometheus_adapter
return unless deployment_platform.respond_to?(:cluster)
cluster = deployment_platform.cluster
return unless cluster.application_prometheus&.installed?
cluster.application_prometheus
end
end
end
module Gitlab module Gitlab
module Prometheus module Prometheus
module AdditionalMetricsParser module AdditionalMetricsParser
CONFIG_ROOT = 'config/prometheus'.freeze
MUTEX = Mutex.new
extend self extend self
def load_groups_from_yaml def load_groups_from_yaml(file_name = 'additional_metrics.yml')
additional_metrics_raw.map(&method(:group_from_entry)) yaml_metrics_raw(file_name).map(&method(:group_from_entry))
end end
private private
...@@ -22,13 +24,20 @@ module Gitlab ...@@ -22,13 +24,20 @@ module Gitlab
MetricGroup.new(entry).tap(&method(:validate!)) MetricGroup.new(entry).tap(&method(:validate!))
end end
def additional_metrics_raw def yaml_metrics_raw(file_name)
load_yaml_file&.map(&:deep_symbolize_keys).freeze load_yaml_file(file_name)&.map(&:deep_symbolize_keys).freeze
end end
def load_yaml_file # rubocop:disable Gitlab/ModuleWithInstanceVariables
@loaded_yaml_file ||= YAML.load_file(Rails.root.join('config/prometheus/additional_metrics.yml')) def load_yaml_file(file_name)
return YAML.load_file(Rails.root.join(CONFIG_ROOT, file_name)) if Rails.env.development?
MUTEX.synchronize do
@loaded_yaml_cache ||= {}
@loaded_yaml_cache[file_name] ||= YAML.load_file(Rails.root.join(CONFIG_ROOT, file_name))
end
end end
# rubocop:enable Gitlab/ModuleWithInstanceVariables
end end
end end
end end
...@@ -4,7 +4,7 @@ module Gitlab ...@@ -4,7 +4,7 @@ module Gitlab
class AdditionalMetricsDeploymentQuery < BaseQuery class AdditionalMetricsDeploymentQuery < BaseQuery
include QueryAdditionalMetrics include QueryAdditionalMetrics
def query(environment_id, deployment_id) def query(deployment_id)
Deployment.find_by(id: deployment_id).try do |deployment| Deployment.find_by(id: deployment_id).try do |deployment|
query_metrics( query_metrics(
deployment.project, deployment.project,
......
...@@ -20,6 +20,10 @@ module Gitlab ...@@ -20,6 +20,10 @@ module Gitlab
def query(*args) def query(*args)
raise NotImplementedError raise NotImplementedError
end end
def self.transform_reactive_result(result)
result
end
end end
end end
end end
......
...@@ -2,7 +2,7 @@ module Gitlab ...@@ -2,7 +2,7 @@ module Gitlab
module Prometheus module Prometheus
module Queries module Queries
class DeploymentQuery < BaseQuery class DeploymentQuery < BaseQuery
def query(environment_id, deployment_id) def query(deployment_id)
Deployment.find_by(id: deployment_id).try do |deployment| Deployment.find_by(id: deployment_id).try do |deployment|
environment_slug = deployment.environment.slug environment_slug = deployment.environment.slug
...@@ -25,6 +25,11 @@ module Gitlab ...@@ -25,6 +25,11 @@ module Gitlab
} }
end end
end end
def self.transform_reactive_result(result)
result[:metrics] = result.delete :data
result
end
end end
end end
end end
......
...@@ -19,6 +19,11 @@ module Gitlab ...@@ -19,6 +19,11 @@ module Gitlab
} }
end end
end end
def self.transform_reactive_result(result)
result[:metrics] = result.delete :data
result
end
end end
end end
end end
......
module Gitlab module Gitlab
module Prometheus module Prometheus
module Queries module Queries
class MatchedMetricsQuery < BaseQuery class MatchedMetricQuery < BaseQuery
MAX_QUERY_ITEMS = 40.freeze MAX_QUERY_ITEMS = 40.freeze
def query def query
......
...@@ -3,9 +3,16 @@ module Gitlab ...@@ -3,9 +3,16 @@ module Gitlab
module Queries module Queries
module QueryAdditionalMetrics module QueryAdditionalMetrics
def query_metrics(project, query_context) def query_metrics(project, query_context)
matched_metrics(project).map(&query_group(query_context))
.select(&method(:group_with_any_metrics))
end
protected
def query_group(query_context)
query_processor = method(:process_query).curry[query_context] query_processor = method(:process_query).curry[query_context]
groups = matched_metrics(project).map do |group| lambda do |group|
metrics = group.metrics.map do |metric| metrics = group.metrics.map do |metric|
{ {
title: metric.title, title: metric.title,
...@@ -21,8 +28,6 @@ module Gitlab ...@@ -21,8 +28,6 @@ module Gitlab
metrics: metrics.select(&method(:metric_with_any_queries)) metrics: metrics.select(&method(:metric_with_any_queries))
} }
end end
groups.select(&method(:group_with_any_metrics))
end end
private private
...@@ -72,12 +77,17 @@ module Gitlab ...@@ -72,12 +77,17 @@ module Gitlab
end end
def common_query_context(environment, timeframe_start:, timeframe_end:) def common_query_context(environment, timeframe_start:, timeframe_end:)
{ base_query_context(timeframe_start, timeframe_end).merge({
timeframe_start: timeframe_start,
timeframe_end: timeframe_end,
ci_environment_slug: environment.slug, ci_environment_slug: environment.slug,
kube_namespace: environment.project.deployment_platform(environment: environment)&.actual_namespace || '', kube_namespace: environment.project.deployment_platform(environment: environment)&.actual_namespace || '',
environment_filter: %{container_name!="POD",environment="#{environment.slug}"} environment_filter: %{container_name!="POD",environment="#{environment.slug}"}
})
end
def base_query_context(timeframe_start, timeframe_end)
{
timeframe_start: timeframe_start,
timeframe_end: timeframe_end
} }
end end
end end
......
...@@ -57,7 +57,11 @@ module Gitlab ...@@ -57,7 +57,11 @@ module Gitlab
rescue OpenSSL::SSL::SSLError rescue OpenSSL::SSL::SSLError
raise PrometheusClient::Error, "#{rest_client.url} contains invalid SSL data" raise PrometheusClient::Error, "#{rest_client.url} contains invalid SSL data"
rescue RestClient::ExceptionWithResponse => ex rescue RestClient::ExceptionWithResponse => ex
if ex.response
handle_exception_response(ex.response) handle_exception_response(ex.response)
else
raise PrometheusClient::Error, "Network connection error"
end
rescue RestClient::Exception rescue RestClient::Exception
raise PrometheusClient::Error, "Network connection error" raise PrometheusClient::Error, "Network connection error"
end end
......
...@@ -129,10 +129,10 @@ describe Projects::DeploymentsController do ...@@ -129,10 +129,10 @@ describe Projects::DeploymentsController do
end end
context 'when metrics are enabled' do context 'when metrics are enabled' do
let(:prometheus_service) { double('prometheus_service') } let(:prometheus_adapter) { double('prometheus_adapter', can_query?: true) }
before do before do
allow(deployment.project).to receive(:prometheus_service).and_return(prometheus_service) allow(deployment).to receive(:prometheus_adapter).and_return(prometheus_adapter)
end end
context 'when environment has no metrics' do context 'when environment has no metrics' do
......
...@@ -4,21 +4,22 @@ describe Projects::Prometheus::MetricsController do ...@@ -4,21 +4,22 @@ describe Projects::Prometheus::MetricsController do
let(:user) { create(:user) } let(:user) { create(:user) }
let(:project) { create(:project) } let(:project) { create(:project) }
let(:prometheus_service) { double('prometheus_service') } let(:prometheus_adapter) { double('prometheus_adapter', can_query?: true) }
before do before do
allow(controller).to receive(:project).and_return(project)
allow(project).to receive(:find_or_initialize_service).with('prometheus').and_return(prometheus_service)
project.add_master(user) project.add_master(user)
sign_in(user) sign_in(user)
end end
describe 'GET #active_common' do describe 'GET #active_common' do
before do
allow(controller).to receive(:prometheus_adapter).and_return(prometheus_adapter)
end
context 'when prometheus metrics are enabled' do context 'when prometheus metrics are enabled' do
context 'when data is not present' do context 'when data is not present' do
before do before do
allow(prometheus_service).to receive(:matched_metrics).and_return({}) allow(prometheus_adapter).to receive(:query).with(:matched_metrics).and_return({})
end end
it 'returns no content response' do it 'returns no content response' do
...@@ -32,7 +33,7 @@ describe Projects::Prometheus::MetricsController do ...@@ -32,7 +33,7 @@ describe Projects::Prometheus::MetricsController do
let(:sample_response) { { some_data: 1 } } let(:sample_response) { { some_data: 1 } }
before do before do
allow(prometheus_service).to receive(:matched_metrics).and_return(sample_response) allow(prometheus_adapter).to receive(:query).with(:matched_metrics).and_return(sample_response)
end end
it 'returns no content response' do it 'returns no content response' do
...@@ -53,6 +54,18 @@ describe Projects::Prometheus::MetricsController do ...@@ -53,6 +54,18 @@ describe Projects::Prometheus::MetricsController do
end end
end end
describe '#prometheus_adapter' do
before do
allow(controller).to receive(:project).and_return(project)
end
it 'calls prometheus adapter service' do
expect_any_instance_of(::Prometheus::AdapterService).to receive(:prometheus_adapter)
subject.__send__(:prometheus_adapter)
end
end
def project_params(opts = {}) def project_params(opts = {})
opts.reverse_merge(namespace_id: project.namespace, project_id: project) opts.reverse_merge(namespace_id: project.namespace, project_id: project)
end end
......
...@@ -7,7 +7,7 @@ describe Gitlab::Prometheus::Queries::AdditionalMetricsDeploymentQuery do ...@@ -7,7 +7,7 @@ describe Gitlab::Prometheus::Queries::AdditionalMetricsDeploymentQuery do
include_examples 'additional metrics query' do include_examples 'additional metrics query' do
let(:deployment) { create(:deployment, environment: environment) } let(:deployment) { create(:deployment, environment: environment) }
let(:query_params) { [environment.id, deployment.id] } let(:query_params) { [deployment.id] }
it 'queries using specific time' do it 'queries using specific time' do
expect(client).to receive(:query_range).with(anything, expect(client).to receive(:query_range).with(anything,
......
...@@ -31,7 +31,7 @@ describe Gitlab::Prometheus::Queries::DeploymentQuery do ...@@ -31,7 +31,7 @@ describe Gitlab::Prometheus::Queries::DeploymentQuery do
expect(client).to receive(:query).with('avg(rate(container_cpu_usage_seconds_total{container_name!="POD",environment="environment-slug"}[30m])) * 100', expect(client).to receive(:query).with('avg(rate(container_cpu_usage_seconds_total{container_name!="POD",environment="environment-slug"}[30m])) * 100',
time: stop_time) time: stop_time)
expect(subject.query(environment.id, deployment.id)).to eq(memory_values: nil, memory_before: nil, memory_after: nil, expect(subject.query(deployment.id)).to eq(memory_values: nil, memory_before: nil, memory_after: nil,
cpu_values: nil, cpu_before: nil, cpu_after: nil) cpu_values: nil, cpu_before: nil, cpu_after: nil)
end end
end end
require 'spec_helper' require 'spec_helper'
describe Gitlab::Prometheus::Queries::MatchedMetricsQuery do describe Gitlab::Prometheus::Queries::MatchedMetricQuery do
include Prometheus::MetricBuilders include Prometheus::MetricBuilders
let(:metric_group_class) { Gitlab::Prometheus::MetricGroup } let(:metric_group_class) { Gitlab::Prometheus::MetricGroup }
......
...@@ -22,11 +22,11 @@ describe Clusters::Applications::Prometheus do ...@@ -22,11 +22,11 @@ describe Clusters::Applications::Prometheus do
end end
end end
describe '#proxy_client' do describe '#prometheus_client' do
context 'cluster is nil' do context 'cluster is nil' do
it 'returns nil' do it 'returns nil' do
expect(subject.cluster).to be_nil expect(subject.cluster).to be_nil
expect(subject.proxy_client).to be_nil expect(subject.prometheus_client).to be_nil
end end
end end
...@@ -35,7 +35,7 @@ describe Clusters::Applications::Prometheus do ...@@ -35,7 +35,7 @@ describe Clusters::Applications::Prometheus do
subject { create(:clusters_applications_prometheus, cluster: cluster) } subject { create(:clusters_applications_prometheus, cluster: cluster) }
it 'returns nil' do it 'returns nil' do
expect(subject.proxy_client).to be_nil expect(subject.prometheus_client).to be_nil
end end
end end
...@@ -63,15 +63,15 @@ describe Clusters::Applications::Prometheus do ...@@ -63,15 +63,15 @@ describe Clusters::Applications::Prometheus do
end end
it 'creates proxy prometheus rest client' do it 'creates proxy prometheus rest client' do
expect(subject.proxy_client).to be_instance_of(RestClient::Resource) expect(subject.prometheus_client).to be_instance_of(RestClient::Resource)
end end
it 'creates proper url' do it 'creates proper url' do
expect(subject.proxy_client.url).to eq('http://example.com/api/v1/proxy/namespaces/gitlab-managed-apps/service/prometheus-prometheus-server:80') expect(subject.prometheus_client.url).to eq('http://example.com/api/v1/proxy/namespaces/gitlab-managed-apps/service/prometheus-prometheus-server:80')
end end
it 'copies options and headers from kube client to proxy client' do it 'copies options and headers from kube client to proxy client' do
expect(subject.proxy_client.options).to eq(kube_client.rest_client.options.merge(headers: kube_client.headers)) expect(subject.prometheus_client.options).to eq(kube_client.rest_client.options.merge(headers: kube_client.headers))
end end
end end
end end
......
require 'spec_helper'
describe PrometheusAdapter, :use_clean_rails_memory_store_caching do
include PrometheusHelpers
include ReactiveCachingHelpers
class TestClass
include PrometheusAdapter
end
let(:project) { create(:prometheus_project) }
let(:service) { project.prometheus_service }
let(:described_class) { TestClass }
let(:environment_query) { Gitlab::Prometheus::Queries::EnvironmentQuery }
describe '#query' do
describe 'environment' do
let(:environment) { build_stubbed(:environment, slug: 'env-slug') }
around do |example|
Timecop.freeze { example.run }
end
context 'with valid data' do
subject { service.query(:environment, environment) }
before do
stub_reactive_cache(service, prometheus_data, environment_query, environment.id)
end
it 'returns reactive data' do
is_expected.to eq(prometheus_metrics_data)
end
end
end
describe 'matched_metrics' do
let(:matched_metrics_query) { Gitlab::Prometheus::Queries::MatchedMetricQuery }
let(:prometheus_client_wrapper) { double(:prometheus_client_wrapper, label_values: nil) }
context 'with valid data' do
subject { service.query(:matched_metrics) }
before do
allow(service).to receive(:prometheus_client_wrapper).and_return(prometheus_client_wrapper)
synchronous_reactive_cache(service)
end
it 'returns reactive data' do
expect(subject[:success]).to be_truthy
expect(subject[:data]).to eq([])
end
end
end
describe 'deployment' do
let(:deployment) { build_stubbed(:deployment) }
let(:deployment_query) { Gitlab::Prometheus::Queries::DeploymentQuery }
around do |example|
Timecop.freeze { example.run }
end
context 'with valid data' do
subject { service.query(:deployment, deployment) }
before do
stub_reactive_cache(service, prometheus_data, deployment_query, deployment.id)
end
it 'returns reactive data' do
expect(subject).to eq(prometheus_metrics_data)
end
end
end
end
describe '#calculate_reactive_cache' do
let(:environment) { create(:environment, slug: 'env-slug') }
before do
service.manual_configuration = true
service.active = true
end
subject do
service.calculate_reactive_cache(environment_query.name, environment.id)
end
around do |example|
Timecop.freeze { example.run }
end
context 'when service is inactive' do
before do
service.active = false
end
it { is_expected.to be_nil }
end
context 'when Prometheus responds with valid data' do
before do
stub_all_prometheus_requests(environment.slug)
end
it { expect(subject.to_json).to eq(prometheus_data.to_json) }
it { expect(subject.to_json).to eq(prometheus_data.to_json) }
end
[404, 500].each do |status|
context "when Prometheus responds with #{status}" do
before do
stub_all_prometheus_requests(environment.slug, status: status, body: "QUERY FAILED!")
end
it { is_expected.to eq(success: false, result: %(#{status} - "QUERY FAILED!")) }
end
end
end
end
...@@ -64,6 +64,7 @@ describe Deployment do ...@@ -64,6 +64,7 @@ describe Deployment do
describe '#metrics' do describe '#metrics' do
let(:deployment) { create(:deployment) } let(:deployment) { create(:deployment) }
let(:prometheus_adapter) { double('prometheus_adapter', can_query?: true) }
subject { deployment.metrics } subject { deployment.metrics }
...@@ -76,17 +77,16 @@ describe Deployment do ...@@ -76,17 +77,16 @@ describe Deployment do
{ {
success: true, success: true,
metrics: {}, metrics: {},
last_update: 42, last_update: 42
deployment_time: 1494408956
} }
end end
before do before do
allow(deployment.project).to receive_message_chain(:monitoring_service, :deployment_metrics) allow(deployment).to receive(:prometheus_adapter).and_return(prometheus_adapter)
.with(any_args).and_return(simple_metrics) allow(prometheus_adapter).to receive(:query).with(:deployment, deployment).and_return(simple_metrics)
end end
it { is_expected.to eq(simple_metrics) } it { is_expected.to eq(simple_metrics.merge({ deployment_time: deployment.created_at.to_i })) }
end end
end end
...@@ -109,11 +109,11 @@ describe Deployment do ...@@ -109,11 +109,11 @@ describe Deployment do
} }
end end
let(:prometheus_service) { double('prometheus_service') } let(:prometheus_adapter) { double('prometheus_adapter', can_query?: true) }
before do before do
allow(project).to receive(:prometheus_service).and_return(prometheus_service) allow(deployment).to receive(:prometheus_adapter).and_return(prometheus_adapter)
allow(prometheus_service).to receive(:additional_deployment_metrics).and_return(simple_metrics) allow(prometheus_adapter).to receive(:query).with(:additional_metrics_deployment, deployment).and_return(simple_metrics)
end end
it { is_expected.to eq(simple_metrics.merge({ deployment_time: deployment.created_at.to_i })) } it { is_expected.to eq(simple_metrics.merge({ deployment_time: deployment.created_at.to_i })) }
......
require 'spec_helper' require 'spec_helper'
describe Environment do describe Environment do
set(:project) { create(:project) } let(:project) { create(:project) }
subject(:environment) { create(:environment, project: project) } subject(:environment) { create(:environment, project: project) }
it { is_expected.to belong_to(:project) } it { is_expected.to belong_to(:project) }
...@@ -553,8 +553,8 @@ describe Environment do ...@@ -553,8 +553,8 @@ describe Environment do
end end
it 'returns the metrics from the deployment service' do it 'returns the metrics from the deployment service' do
expect(project.monitoring_service) expect(environment.prometheus_adapter)
.to receive(:environment_metrics).with(environment) .to receive(:query).with(:environment, environment)
.and_return(:fake_metrics) .and_return(:fake_metrics)
is_expected.to eq(:fake_metrics) is_expected.to eq(:fake_metrics)
...@@ -609,12 +609,12 @@ describe Environment do ...@@ -609,12 +609,12 @@ describe Environment do
context 'when the environment has additional metrics' do context 'when the environment has additional metrics' do
before do before do
allow(environment).to receive(:has_additional_metrics?).and_return(true) allow(environment).to receive(:has_metrics?).and_return(true)
end end
it 'returns the additional metrics from the deployment service' do it 'returns the additional metrics from the deployment service' do
expect(project.prometheus_service).to receive(:additional_environment_metrics) expect(environment.prometheus_adapter).to receive(:query)
.with(environment) .with(:additional_metrics_environment, environment)
.and_return(:fake_metrics) .and_return(:fake_metrics)
is_expected.to eq(:fake_metrics) is_expected.to eq(:fake_metrics)
...@@ -623,46 +623,13 @@ describe Environment do ...@@ -623,46 +623,13 @@ describe Environment do
context 'when the environment does not have metrics' do context 'when the environment does not have metrics' do
before do before do
allow(environment).to receive(:has_additional_metrics?).and_return(false) allow(environment).to receive(:has_metrics?).and_return(false)
end end
it { is_expected.to be_nil } it { is_expected.to be_nil }
end end
end end
describe '#has_additional_metrics??' do
subject { environment.has_additional_metrics? }
context 'when the enviroment is available' do
context 'with a deployment service' do
let(:project) { create(:prometheus_project) }
context 'and a deployment' do
let!(:deployment) { create(:deployment, environment: environment) }
it { is_expected.to be_truthy }
end
context 'but no deployments' do
it { is_expected.to be_falsy }
end
end
context 'without a monitoring service' do
it { is_expected.to be_falsy }
end
end
context 'when the environment is unavailable' do
let(:project) { create(:prometheus_project) }
before do
environment.stop
end
it { is_expected.to be_falsy }
end
end
describe '#slug' do describe '#slug' do
it "is automatically generated" do it "is automatically generated" do
expect(environment.slug).not_to be_nil expect(environment.slug).not_to be_nil
...@@ -755,4 +722,12 @@ describe Environment do ...@@ -755,4 +722,12 @@ describe Environment do
end end
end end
end end
describe '#prometheus_adapter' do
it 'calls prometheus adapter service' do
expect_any_instance_of(Prometheus::AdapterService).to receive(:prometheus_adapter)
subject.prometheus_adapter
end
end
end end
...@@ -6,7 +6,6 @@ describe PrometheusService, :use_clean_rails_memory_store_caching do ...@@ -6,7 +6,6 @@ describe PrometheusService, :use_clean_rails_memory_store_caching do
let(:project) { create(:prometheus_project) } let(:project) { create(:prometheus_project) }
let(:service) { project.prometheus_service } let(:service) { project.prometheus_service }
let(:environment_query) { Gitlab::Prometheus::Queries::EnvironmentQuery }
describe "Associations" do describe "Associations" do
it { is_expected.to belong_to :project } it { is_expected.to belong_to :project }
...@@ -55,197 +54,31 @@ describe PrometheusService, :use_clean_rails_memory_store_caching do ...@@ -55,197 +54,31 @@ describe PrometheusService, :use_clean_rails_memory_store_caching do
end end
end end
describe '#environment_metrics' do describe '#prometheus_client' do
let(:environment) { build_stubbed(:environment, slug: 'env-slug') }
around do |example|
Timecop.freeze { example.run }
end
context 'with valid data' do
subject { service.environment_metrics(environment) }
before do
stub_reactive_cache(service, prometheus_data, environment_query, environment.id)
end
it 'returns reactive data' do
is_expected.to eq(prometheus_metrics_data)
end
end
end
describe '#matched_metrics' do
let(:matched_metrics_query) { Gitlab::Prometheus::Queries::MatchedMetricsQuery }
let(:client) { double(:client, label_values: nil) }
context 'with valid data' do
subject { service.matched_metrics }
before do
allow(service).to receive(:client).and_return(client)
synchronous_reactive_cache(service)
end
it 'returns reactive data' do
expect(subject[:success]).to be_truthy
expect(subject[:data]).to eq([])
end
end
end
describe '#deployment_metrics' do
let(:deployment) { build_stubbed(:deployment) }
let(:deployment_query) { Gitlab::Prometheus::Queries::DeploymentQuery }
around do |example|
Timecop.freeze { example.run }
end
context 'with valid data' do
subject { service.deployment_metrics(deployment) }
let(:fake_deployment_time) { 10 }
before do
stub_reactive_cache(service, prometheus_data, deployment_query, deployment.environment.id, deployment.id)
end
it 'returns reactive data' do
expect(deployment).to receive(:created_at).and_return(fake_deployment_time)
expect(subject).to eq(prometheus_metrics_data.merge(deployment_time: fake_deployment_time))
end
end
end
describe '#calculate_reactive_cache' do
let(:environment) { create(:environment, slug: 'env-slug') }
before do
service.manual_configuration = true
service.active = true
end
subject do
service.calculate_reactive_cache(environment_query.name, environment.id)
end
around do |example|
Timecop.freeze { example.run }
end
context 'when service is inactive' do
before do
service.active = false
end
it { is_expected.to be_nil }
end
context 'when Prometheus responds with valid data' do
before do
stub_all_prometheus_requests(environment.slug)
end
it { expect(subject.to_json).to eq(prometheus_data.to_json) }
it { expect(subject.to_json).to eq(prometheus_data.to_json) }
end
[404, 500].each do |status|
context "when Prometheus responds with #{status}" do
before do
stub_all_prometheus_requests(environment.slug, status: status, body: "QUERY FAILED!")
end
it { is_expected.to eq(success: false, result: %(#{status} - "QUERY FAILED!")) }
end
end
end
describe '#client' do
context 'manual configuration is enabled' do context 'manual configuration is enabled' do
let(:api_url) { 'http://some_url' } let(:api_url) { 'http://some_url' }
before do before do
subject.active = true
subject.manual_configuration = true subject.manual_configuration = true
subject.api_url = api_url subject.api_url = api_url
end end
it 'returns simple rest client from api_url' do it 'returns rest client from api_url' do
expect(subject.client).to be_instance_of(Gitlab::PrometheusClient) expect(subject.prometheus_client.url).to eq(api_url)
expect(subject.client.rest_client.url).to eq(api_url)
end end
end end
context 'manual configuration is disabled' do context 'manual configuration is disabled' do
let!(:cluster_for_all) { create(:cluster, environment_scope: '*', projects: [project]) } let(:api_url) { 'http://some_url' }
let!(:cluster_for_dev) { create(:cluster, environment_scope: 'dev', projects: [project]) }
let!(:prometheus_for_dev) { create(:clusters_applications_prometheus, :installed, cluster: cluster_for_dev) }
let(:proxy_client) { double('proxy_client') }
before do before do
service.manual_configuration = false subject.manual_configuration = false
end subject.api_url = api_url
context 'with cluster for all environments with prometheus installed' do
let!(:prometheus_for_all) { create(:clusters_applications_prometheus, :installed, cluster: cluster_for_all) }
context 'without environment supplied' do
it 'returns client handling all environments' do
expect(service).to receive(:client_from_cluster).with(cluster_for_all).and_return(proxy_client).twice
expect(service.client).to be_instance_of(Gitlab::PrometheusClient)
expect(service.client.rest_client).to eq(proxy_client)
end
end
context 'with dev environment supplied' do
let!(:environment) { create(:environment, project: project, name: 'dev') }
it 'returns dev cluster client' do
expect(service).to receive(:client_from_cluster).with(cluster_for_dev).and_return(proxy_client).twice
expect(service.client(environment.id)).to be_instance_of(Gitlab::PrometheusClient)
expect(service.client(environment.id).rest_client).to eq(proxy_client)
end
end
context 'with prod environment supplied' do
let!(:environment) { create(:environment, project: project, name: 'prod') }
it 'returns dev cluster client' do
expect(service).to receive(:client_from_cluster).with(cluster_for_all).and_return(proxy_client).twice
expect(service.client(environment.id)).to be_instance_of(Gitlab::PrometheusClient)
expect(service.client(environment.id).rest_client).to eq(proxy_client)
end
end
end
context 'with cluster for all environments without prometheus installed' do
context 'without environment supplied' do
it 'raises PrometheusClient::Error because cluster was not found' do
expect { service.client }.to raise_error(Gitlab::PrometheusClient::Error, /couldn't find cluster with Prometheus installed/)
end
end
context 'with dev environment supplied' do
let!(:environment) { create(:environment, project: project, name: 'dev') }
it 'returns dev cluster client' do
expect(service).to receive(:client_from_cluster).with(cluster_for_dev).and_return(proxy_client).twice
expect(service.client(environment.id)).to be_instance_of(Gitlab::PrometheusClient)
expect(service.client(environment.id).rest_client).to eq(proxy_client)
end
end end
context 'with prod environment supplied' do it 'no client provided' do
let!(:environment) { create(:environment, project: project, name: 'prod') } expect(subject.prometheus_client).to be_nil
it 'raises PrometheusClient::Error because cluster was not found' do
expect { service.client }.to raise_error(Gitlab::PrometheusClient::Error, /couldn't find cluster with Prometheus installed/)
end
end
end end
end end
end end
...@@ -284,7 +117,7 @@ describe PrometheusService, :use_clean_rails_memory_store_caching do ...@@ -284,7 +117,7 @@ describe PrometheusService, :use_clean_rails_memory_store_caching do
end end
end end
describe '#synchronize_service_state! before_save callback' do describe '#synchronize_service_state before_save callback' do
context 'no clusters with prometheus are installed' do context 'no clusters with prometheus are installed' do
context 'when service is inactive' do context 'when service is inactive' do
before do before do
......
require 'spec_helper'
describe Prometheus::AdapterService do
let(:project) { create(:project) }
subject { described_class.new(project) }
describe '#prometheus_adapter' do
let(:cluster) { create(:cluster, :provided_by_user, environment_scope: '*', projects: [project]) }
context 'prometheus service can execute queries' do
let(:prometheus_service) { double(:prometheus_service, can_query?: true) }
before do
allow(project).to receive(:find_or_initialize_service).with('prometheus').and_return prometheus_service
end
it 'return prometheus service as prometheus adapter' do
expect(subject.prometheus_adapter).to eq(prometheus_service)
end
end
context "prometheus service can't execute queries" do
let(:prometheus_service) { double(:prometheus_service, can_query?: false) }
context 'with cluster with prometheus installed' do
let!(:prometheus) { create(:clusters_applications_prometheus, :installed, cluster: cluster) }
it 'returns application handling all environments' do
expect(subject.prometheus_adapter).to eq(prometheus)
end
end
context 'with cluster without prometheus installed' do
it 'returns nil' do
expect(subject.prometheus_adapter).to be_nil
end
end
end
end
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment