Commit 43aa71eb authored by Russell Dickenson's avatar Russell Dickenson

Merge branch '235457-add-docs-for-redis-hll-counters' into 'master'

Add docs for Redis HLL counters

Closes #235457

See merge request gitlab-org/gitlab!39298
parents afa5bff1 12aa69de
......@@ -213,11 +213,47 @@ Arguments:
- `counter`: a counter from `Gitlab::UsageDataCounters`, that has `fallback_totals` method implemented
- or a `block`: which is evaluated
#### Ordinary Redis Counters
Examples of implementation:
- Using Redis methods [`INCR`](https://redis.io/commands/incr), [`GET`](https://redis.io/commands/get), and [`Gitlab::UsageDataCounters::WikiPageCounter`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/wiki_page_counter.rb)
- Using Redis methods [`HINCRBY`](https://redis.io/commands/hincrby), [`HGETALL`](https://redis.io/commands/hgetall), and [`Gitlab::UsageCounters::PodLogs`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_counters/pod_logs.rb)
#### Redis HLL Counters
With `Gitlab::Redis::HLL` we have available data structures used to count unique values.
Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd) and [PFCOUNT](https://redis.io/commands/pfcount).
Recommendations:
- Key should expire in 29 days.
- If possible, data granularity should be a week. For example a key could be composed from the metric's name and week of the year, `2020-33-{metric_name}`.
- Use a [feature flag](../../operations/feature_flags.md) in order to have a control over the impact when adding new metrics.
- If possible, data granularity should be week, for example a key could be composed from metric name and week of the year, 2020-33-{metric_name}
- Use a [feature flag](../../operations/feature_flags.md) in order to have a control over the impact when adding new metrics
Examples of implementation:
- [`Gitlab::UsageDataCounters::TrackUniqueActions`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/track_unique_actions.rb)
- [`Gitlab::Analytics::UniqueVisits`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/analytics/unique_visits.rb)
Example of usage:
```ruby
# Redis Counters
redis_usage_data(Gitlab::UsageDataCounters::WikiPageCounter)
redis_usage_data { ::Gitlab::UsageCounters::PodLogs.usage_totals[:total] }
# Redis HLL counter
counter = Gitlab::UsageDataCounters::TrackUniqueActions
redis_usage_data do
counter.count_unique_events(
event_action: Gitlab::UsageDataCounters::TrackUniqueActions::PUSH_ACTION,
date_from: time_period[:created_at].first,
date_to: time_period[:created_at].last
)
```
Note that Redis counters are in the [process of being deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/216330) and you should instead try to use Snowplow events instead. We're in the process of building [self-managed event tracking](https://gitlab.com/gitlab-org/telemetry/-/issues/373) and once this is available, we will convert all Redis counters into Snowplow events.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment