Allow configuring Redis trace chunks instance

Allow configuring a Redis instance for storing trace chunks. For almost
every GitLab instance except GitLab.com, all Redis instances are the
same instance. The trace chunks REdis is only used in the
`ci_enable_live_trace` feature flag is enabled (the flag is off by
default).

If that flag is enabled and the instance was already using the
persistent Redis, then the only way to cleanly migrate is to take
downtime. We are not providing additional migration support at this time
as this is a niche use-case on top of a niche use-case, and so the
additional complexity in the application isn't worth it right now.

Changelog: added

app/models/ci/build_trace_chunk.rb

@@ -14,19 +14,7 @@ module Ci
 
     belongs_to :build, class_name: "Ci::Build", foreign_key: :build_id
 
-    default_value_for :data_store do |chunk|
-      # We're using the safe operator here to get to the project for which we're
-      # creating a TraceChunk because the build attribute would not be populated
-      # when the chunk was initialized by FactoryBot:
-      # https://github.com/thoughtbot/factory_bot/wiki/How-factory_bot-interacts-with-ActiveRecord
-      # While the `default_value_for` gem depends on an `after_initialize`
-      # callback.
-      if Feature.enabled?(:dedicated_redis_trace_chunks, chunk.build&.project, type: :ops)
-        :redis_trace_chunks
-      else
-        :redis
-      end
-    end
+    default_value_for :data_store, :redis_trace_chunks
 
     after_create { metrics.increment_trace_operation(operation: :chunked) }
 

config/feature_flags/ops/dedicated_redis_trace_chunks.yml

----
-name: dedicated_redis_trace_chunks
-introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62938
-rollout_issue_url: https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1096
-milestone: '14.0'
-type: ops
-group: team::Scalability
-default_enabled: false

doc/administration/job_logs.md

@@ -162,7 +162,7 @@ file storage. Redis is used as first-class storage, and it stores up-to 128KB
 of data. After the full chunk is sent, it is flushed to a persistent store, either object storage (temporary directory) or database.
 After a while, the data in Redis and a persistent store is archived to [object storage](#uploading-logs-to-object-storage).
 
-The data are stored in the following Redis namespace: `Gitlab::Redis::SharedState`.
+The data are stored in the following Redis namespace: `Gitlab::Redis::TraceChunks`.
 
 Here is the detailed data flow:
 

doc/administration/redis/replication_and_failover.md

@@ -646,6 +646,7 @@ persistence classes.
 | `queues`       | Store Sidekiq background jobs.                   |
 | `shared_state` | Store session-related and other persistent data. |
 | `actioncable`  | Pub/Sub queue backend for ActionCable.           |
+| `trace_chunks` | Store [CI trace chunks](../job_logs.md#enable-or-disable-incremental-logging) data. |
 
 To make this work with Sentinel:
 
@@ -657,6 +658,7 @@ To make this work with Sentinel:
    gitlab_rails['redis_queues_instance'] = REDIS_QUEUES_URL
    gitlab_rails['redis_shared_state_instance'] = REDIS_SHARED_STATE_URL
    gitlab_rails['redis_actioncable_instance'] = REDIS_ACTIONCABLE_URL
+   gitlab_rails['redis_trace_chunks_instance'] = REDIS_TRACE_CHUNKS_URL
 
    # Configure the Sentinels
    gitlab_rails['redis_cache_sentinels'] = [
@@ -675,6 +677,10 @@ To make this work with Sentinel:
      { host: ACTIONCABLE_SENTINEL_HOST, port: 26379 },
      { host: ACTIONCABLE_SENTINEL_HOST2, port: 26379 }
    ]
+   gitlab_rails['redis_trace_chunks_sentinels'] = [
+     { host: TRACE_CHUNKS_SENTINEL_HOST, port: 26379 },
+     { host: TRACE_CHUNKS_SENTINEL_HOST2, port: 26379 }
+   ]
    ```
 
    Note that:

spec/models/ci/build_trace_chunk_spec.rb

@@ -152,14 +152,6 @@ RSpec.describe Ci::BuildTraceChunk, :clean_gitlab_redis_shared_state, :clean_git
 
     context 'default value' do
       it { expect(subject).to eq('redis_trace_chunks') }
-
-      context 'when dedicated_redis_trace_chunks is disabled' do
-        before do
-          stub_feature_flags(dedicated_redis_trace_chunks: false)
-        end
-
-        it { expect(subject).to eq('redis') }
-      end
     end
   end