Merge branch '294397-add-rake-task-to-generate-the-metric-dictionary-in-a-md-file' into 'master'

Add rake task to generate the metric dictionary in a .md file [RUN AS-IF-FOSS]

See merge request gitlab-org/gitlab!50839

doc/development/usage_ping/dictionary.md

+---
+stage: Growth
+group: Product Intelligence
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
+---
+
+<!---
+  This documentation is auto generated by a script.
+
+  Please do not edit this file directly, check generate_metrics_dictionary task on lib/tasks/gitlab/usage_data.rake.
+--->
+
+# Metrics Dictionary
+
+This file is autogenerated, please do not edit directly.
+
+To generate these files from the GitLab repository, run:
+
+```shell
+bundle exec rake gitlab:usage_data:generate_metrics_dictionary
+```
+
+The Metrics Dictionary is based on the following metrics definition YAML files:
+
+- [`config/metrics`]('https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics')
+- [`ee/config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/config/metrics)
+Each table includes a `milestone`, which corresponds to the GitLab version when the metric
+was released.
+
+## counts.deployments
+
+Total deployments count
+
+| field | value |
+| --- | --- |
+| `key_path` | **counts.deployments** |
+| `value_type` | integer |
+| `stage` | release |
+| `status` | data_available |
+| `milestone` | 8.12 |
+| `introduced_by_url` | [Introduced by](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/735) |
+| `group` | `group::ops release` |
+| `time_frame` | all |
+| `data_source` | Database |
+| `distribution` | ee, ce |
+| `tier` | free, starter, premium, ultimate, bronze, silver, gold |
+
+## counts.geo_nodes
+
+Total number of sites in a Geo deployment
+
+| field | value |
+| --- | --- |
+| `key_path` | **counts.geo_nodes** |
+| `value_type` | integer |
+| `product_category` | disaster_recovery |
+| `stage` | enablement |
+| `status` | data_available |
+| `milestone` | 11.2 |
+| `group` | `group::geo` |
+| `time_frame` | all |
+| `data_source` | Database |
+| `distribution` | ee |
+| `tier` | premium, ultimate |
+
+## counts_monthy.deployments
+
+Total deployments count for recent 28 days
+
+| field | value |
+| --- | --- |
+| `key_path` | **counts_monthy.deployments** |
+| `value_type` | integer |
+| `stage` | release |
+| `status` | data_available |
+| `milestone` | 13.2 |
+| `introduced_by_url` | [Introduced by](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35493) |
+| `group` | `group::ops release` |
+| `time_frame` | 28d |
+| `data_source` | Database |
+| `distribution` | ee, ce |
+| `tier` | free, starter, premium, ultimate, bronze, silver, gold |
+
+## database.adapter
+
+This metric only returns a value of PostgreSQL in supported versions of GitLab. It could be removed from the usage ping. Historically MySQL was also supported.
+
+| field | value |
+| --- | --- |
+| `key_path` | **database.adapter** |
+| `value_type` | string |
+| `product_category` | collection |
+| `stage` | growth |
+| `status` | data_available |
+| `group` | `group::enablement distribution` |
+| `time_frame` | none |
+| `data_source` | Database |
+| `distribution` | ee, ce |
+| `tier` | free, starter, premium, ultimate, bronze, silver, gold |
+
+## recorded_at
+
+When the Usage Ping computation was started
+
+| field | value |
+| --- | --- |
+| `key_path` | **recorded_at** |
+| `value_type` | string |
+| `product_category` | collection |
+| `stage` | growth |
+| `status` | data_available |
+| `milestone` | 8.1 |
+| `introduced_by_url` | [Introduced by](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/557) |
+| `group` | `group::product analytics` |
+| `time_frame` | none |
+| `data_source` | Ruby |
+| `distribution` | ee, ce |
+| `tier` | free, starter, premium, ultimate, bronze, silver, gold |
+
+## redis_hll_counters.issues_edit.g_project_management_issue_title_changed_weekly
+
+Distinct users count that changed issue title in a group for last recent week
+
+| field | value |
+| --- | --- |
+| `key_path` | **redis_hll_counters.issues_edit.g_project_management_issue_title_changed_weekly** |
+| `value_type` | integer |
+| `product_category` | issue_tracking |
+| `stage` | plan |
+| `status` | data_available |
+| `milestone` | 13.6 |
+| `introduced_by_url` | [Introduced by](https://gitlab.com/gitlab-org/gitlab/-/issues/229918) |
+| `group` | `group::project management` |
+| `time_frame` | 7d |
+| `data_source` | Redis_hll |
+| `distribution` | ee, ce |
+| `tier` | free, starter, premium, ultimate, bronze, silver, gold |
+
+## uuid
+
+GitLab instance unique identifier
+
+| field | value |
+| --- | --- |
+| `key_path` | **uuid** |
+| `value_type` | string |
+| `product_category` | collection |
+| `stage` | growth |
+| `status` | data_available |
+| `milestone` | 9.1 |
+| `introduced_by_url` | [Introduced by](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1521) |
+| `group` | `group::product analytics` |
+| `time_frame` | none |
+| `data_source` | Database |
+| `distribution` | ee, ce |
+| `tier` | free, starter, premium, ultimate, bronze, silver, gold |

ee/lib/ee/gitlab/usage/metric_definition.rb

@@ -5,6 +5,7 @@ module EE
     module Usage
       module MetricDefinition
         extend ActiveSupport::Concern
+
         class_methods do
           extend ::Gitlab::Utils::Override
 

lib/gitlab/usage/docs/helper.rb

+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Docs
+      # Helper with functions to be used by HAML templates
+      module Helper
+        HEADER = %w(field value).freeze
+        SKIP_KEYS = %i(description).freeze
+
+        def auto_generated_comment
+          <<-MARKDOWN.strip_heredoc
+            ---
+            stage: Growth
+            group: Product Intelligence
+            info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
+            ---
+
+            <!---
+              This documentation is auto generated by a script.
+
+              Please do not edit this file directly, check generate_metrics_dictionary task on lib/tasks/gitlab/usage_data.rake.
+            --->
+          MARKDOWN
+        end
+
+        def render_name(name)
+          "## #{name}\n"
+        end
+
+        def render_description(object)
+          object.description
+        end
+
+        def render_attribute_row(key, value)
+          value = Gitlab::Usage::Docs::ValueFormatter.format(key, value)
+          table_row(["`#{key}`", value])
+        end
+
+        def render_attributes_table(object)
+          <<~MARKDOWN
+
+            #{table_row(HEADER)}
+            #{table_row(HEADER.map { '---' })}
+            #{table_value_rows(object.attributes)}
+          MARKDOWN
+        end
+
+        def table_value_rows(attributes)
+          attributes.reject { |k, _| k.in?(SKIP_KEYS) }.map do |key, value|
+            render_attribute_row(key, value)
+          end.join("\n")
+        end
+
+        def table_row(array)
+          "| #{array.join(' | ')} |"
+        end
+      end
+    end
+  end
+end

lib/gitlab/usage/docs/renderer.rb

+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Docs
+      class Renderer
+        include Gitlab::Usage::Docs::Helper
+        DICTIONARY_PATH = Rails.root.join('doc', 'development', 'usage_ping')
+        TEMPLATE_PATH = Rails.root.join('lib', 'gitlab', 'usage', 'docs', 'templates', 'default.md.haml')
+
+        def initialize(metrics_definitions)
+          @layout = Haml::Engine.new(File.read(TEMPLATE_PATH))
+          @metrics_definitions = metrics_definitions.sort
+        end
+
+        def contents
+          # Render and remove an extra trailing new line
+          @contents ||= @layout.render(self, metrics_definitions: @metrics_definitions).sub!(/\n(?=\Z)/, '')
+        end
+
+        def write
+          filename = DICTIONARY_PATH.join('dictionary.md').to_s
+
+          FileUtils.mkdir_p(DICTIONARY_PATH)
+          File.write(filename, contents)
+
+          filename
+        end
+      end
+    end
+  end
+end

lib/gitlab/usage/docs/templates/default.md.haml

+= auto_generated_comment
+
+:plain
+  # Metrics Dictionary
+
+  This file is autogenerated, please do not edit directly.
+
+  To generate these files from the GitLab repository, run:
+
+  ```shell
+  bundle exec rake gitlab:usage_data:generate_metrics_dictionary
+  ```
+
+  The Metrics Dictionary is based on the following metrics definition YAML files:
+
+  - [`config/metrics`]('https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics')
+  - [`ee/config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/config/metrics)
+
+Each table includes a `milestone`, which corresponds to the GitLab version when the metric
+was released.
+\
+- metrics_definitions.each do |name, object|
+
+  = render_name(name)
+
+  = render_description(object)
+
+  = render_attributes_table(object)

lib/gitlab/usage/docs/value_formatter.rb

+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Docs
+      class ValueFormatter
+        def self.format(key, value)
+          case key
+          when :key_path
+            "**#{value}**"
+          when :data_source
+            value.capitalize
+          when :group
+            "`#{value}`"
+          when :introduced_by_url
+            "[Introduced by](#{value})"
+          when :distribution, :tier
+            Array(value).join(', ')
+          else
+            value
+          end
+        end
+      end
+    end
+  end
+end

lib/tasks/gitlab/usage_data.rake

@@ -21,5 +21,11 @@ namespace :gitlab do
 
       puts Gitlab::Json.pretty_generate(result.attributes)
     end
+
+    desc 'GitLab | UsageData | Generate metrics dictionary'
+    task generate_metrics_dictionary: :environment do
+      items = Gitlab::Usage::MetricDefinition.definitions
+      Gitlab::Usage::Docs::Renderer.new(items).write
+    end
   end
 end

spec/lib/gitlab/usage/docs/renderer_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+RSpec.describe Gitlab::Usage::Docs::Renderer do
+  describe 'contents' do
+    let(:dictionary_path) { Gitlab::Usage::Docs::Renderer::DICTIONARY_PATH }
+    let(:items) { Gitlab::Usage::MetricDefinition.definitions }
+
+    it 'generates dictionary for given items' do
+      generated_dictionary = described_class.new(items).contents
+      generated_dictionary_keys = RDoc::Markdown
+        .parse(generated_dictionary)
+        .table_of_contents
+        .select { |metric_doc| metric_doc.level == 2 && !metric_doc.text.start_with?('info:') }
+        .map(&:text)
+
+      expect(generated_dictionary_keys).to match_array(items.keys)
+    end
+  end
+end

spec/lib/gitlab/usage/docs/value_formatter_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+RSpec.describe Gitlab::Usage::Docs::ValueFormatter do
+  describe '.format' do
+    using RSpec::Parameterized::TableSyntax
+    where(:key, :value, :expected_value) do
+      :group             | 'growth::product intelligence' | '`growth::product intelligence`'
+      :data_source       | 'redis'                        | 'Redis'
+      :data_source       | 'ruby'                         | 'Ruby'
+      :introduced_by_url | 'http://test.com'              | '[Introduced by](http://test.com)'
+      :tier              | %w(gold premium)               | 'gold, premium'
+      :distribution      | %w(ce ee)                      | 'ce, ee'
+      :key_path          | 'key.path'                     | '**key.path**'
+      :milestone         | '13.4'                         | '13.4'
+      :status            | 'data_available'               | 'data_available'
+    end
+
+    with_them do
+      subject { described_class.format(key, value) }
+
+      it { is_expected.to eq(expected_value) }
+    end
+  end
+end