Merge branch 'mwaw/284637-define-and-describe-metrics-retiring-process' into 'master'

Define and describe metrics retiring process

See merge request gitlab-org/gitlab!59000

doc/development/usage_ping/index.md

@@ -178,10 +178,132 @@ The correct approach is to add a new metric for GitLab 12.6 release with updated
 
 and update existing business analysis artefacts to use `example_metric_without_archived` instead of `example_metric`
 
-### 3. Metrics deprecation and removal
+### 3. Deprecate a metric
 
-The process for deprecating and removing metrics is under development. For
-more information, see the following [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/284637).
+If a metric is obsolete and you no longer use it, you can mark it as deprecated.
+
+For an example of the metric deprecation process take a look at this [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/59899)
+
+To deprecate a metric:
+
+1. Check the following YAML files and verify the metric is not used in an aggregate:
+   - [`config/metrics/aggregates/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/aggregates/)
+   - [`ee/config/metrics/aggregates/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/metrics/aggregates/)
+
+1. Create an issue in the [GitLab Data Team
+   project](https://gitlab.com/gitlab-data/analytics/-/issues). Ask for
+   confirmation that the metric is not used by other teams, or in any of the SiSense
+   dashboards.
+
+1. Verify the metric is not used to calculate the conversational index. The
+   conversational index is a measure that reports back to self-managed instances
+   to inform administrators of the progress of DevOps adoption for the instance.
+
+   You can check
+   [`CalculateConvIndexService`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/app/services/calculate_conv_index_service.rb)
+   to view the metrics that are used. The metrics are represented
+   as the keys that are passed as a field argument into the `get_value` method.
+
+1. Document the deprecation in the metric's YAML definition. Set
+   the `status:` attribute to `deprecated`, for example:
+
+   ```yaml
+   ---
+   key_path: analytics_unique_visits.analytics_unique_visits_for_any_target_monthly
+   description: Visits to any of the pages listed above per month
+   product_section: dev
+   product_stage: manage
+   product_group: group::analytics
+   product_category:
+   value_type: number
+   status: deprecated
+   time_frame: 28d
+   data_source:
+   distribution:
+   - ce
+   tier:
+   - free
+   ```
+
+1. Replace the metric's instrumentation with a fixed value. This avoids wasting
+   resources to calculate the deprecated metric. In
+   [`lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb)
+   or
+   [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb),
+   replace the code that calculates the metric's value with a fixed value that
+   indicates it's deprecated:
+
+   ```ruby
+   module Gitlab
+     class UsageData
+       DEPRECATED_VALUE = -1000
+
+       def analytics_unique_visits_data
+         results['analytics_unique_visits_for_any_target'] = redis_usage_data { unique_visit_service.unique_visits_for(targets: :analytics) }
+         results['analytics_unique_visits_for_any_target_monthly'] = DEPRECATED_VALUE
+
+         { analytics_unique_visits: results }
+       end
+     # ...
+     end
+   end
+   ```
+ 
+1. Update the Metrics Dictionary following [guidelines instructions](dictionary.md).
+
+### 4. Remove a metric
+
+Only deprecated metrics can be removed from Usage Ping. 
+
+For an example of the metric removal process take a look at this [example issue](https://gitlab.com/gitlab-org/gitlab/-/issues/297029)
+
+To remove a deprecated metric:
+
+1. Verify that removing the metric from the Usage Ping payload does not cause
+   errors in [Version App](https://gitlab.com/gitlab-services/version-gitlab-com)
+   when the updated payload is collected and processed. Version App collects
+   and persists all Usage Ping reports. To do that you can modify 
+   [fixtures](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/spec/support/usage_data_helpers.rb#L540)
+   used to test 
+   [`UsageDataController#create`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/3760ef28/spec/controllers/usage_data_controller_spec.rb#L75)
+   endpoint, and assure that test suite does not fail when metric that you wish to remove is not included into test payload.  
+
+1. Create an issue in the
+   [GitLab Data Team project](https://gitlab.com/gitlab-data/analytics/-/issues).
+   Ask for confirmation that the metric is not referred to in any SiSense dashboards and
+   can be safely removed from Usage Ping. Use this
+   [example issue](https://gitlab.com/gitlab-data/analytics/-/issues/7539) for guidance.
+   This step can be skipped if verification done during [deprecation process](#3-deprecate-a-metric) 
+   reported that metric is not required by any data transformation in Snowflake data warehouse nor it is
+   used by any of SiSense dashboards.
+  
+1. After you verify the metric can be safely removed,
+   update the attributes of the metric's YAML definition:
+
+   - Set the `status:` to `removed`.
+   - Set `milestone_removed:` to the number of the
+     milestone in which the metric was removed.
+
+   Do not remove the metric's YAML definition altogether. Some self-managed
+   instances might not immediately update to the latest version of GitLab, and
+   therefore continue to report the removed metric. The Product Intelligence team
+   requires a record of all removed metrics in order to identify and filter them.
+    
+   For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#b01f429a54843feb22265100c0e4fec1b7da1240_10_10).
+    
+1. After you verify the metric can be safely removed,
+   remove the metric's instrumentation from
+   [`lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb)
+   or
+   [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb).
+   
+   For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#6335dc533bd21df26db9de90a02dd66278c2390d_167_167).
+
+1. Remove any other records related to the metric:
+   - The feature flag YAML file at [`config/feature_flags/*/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/feature_flags).
+   - The entry in the known events YAML file at [`lib/gitlab/usage_data_counters/known_events/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/usage_data_counters/known_events).
+
+1. Update the Metrics Dictionary following [guidelines instructions](dictionary.md).
 
 ## Implementing Usage Ping