Merge branch 'migrations-for-multiple-databases' into 'master'

Add documentation for migrations for multiple databases

See merge request gitlab-org/gitlab!84764

doc/development/database/index.md

@@ -23,6 +23,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 ## Migrations
 
+- [Migrations for multiple databases](migrations_for_multiple_databases.md)
 - [Avoiding downtime in migrations](avoiding_downtime_in_migrations.md)
 - [SQL guidelines](../sql.md) for working with SQL queries
 - [Migrations style guide](../migration_style_guide.md) for creating safe SQL migrations

doc/development/database/migrations_for_multiple_databases.md

+---
+stage: Enablement
+group: Database
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Migrations for Multiple databases
+
+> Support for describing migration purposes was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73756) in GitLab 14.8.
+
+This document describes how to properly write database migrations
+for [the decomposed GitLab application using multiple databases](https://gitlab.com/groups/gitlab-org/-/epics/6168).
+
+Learn more about general multiple databases support in a [separate document](multiple_databases.md).
+
+WARNING:
+If you experience any issues using `Gitlab::Database::Migration[2.0]`,
+you can temporarily revert back to the previous behavior by changing the version to `Gitlab::Database::Migration[1.0]`.
+Please report any issues with `Gitlab::Database::Migration[2.0]` in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/358430).
+
+The design for multiple databases (except for the Geo database) assumes
+that all decomposed databases have **the same structure** (for example, schema), but **the data is different** in each database. This means that some tables do not contain data on each database.
+
+## Operations
+
+Depending on the used constructs, we can classify migrations to be either:
+
+1. Modifying structure ([DDL - Data Definition Language](https://www.postgresql.org/docs/current/ddl.html)) (for example, `ALTER TABLE`).
+1. Modifying data ([DML - Data Manipulation Language](https://www.postgresql.org/docs/current/dml.html)) (for example, `UPDATE`).
+1. Performing [other queries](https://www.postgresql.org/docs/current/queries.html) (for example, `SELECT`) that are treated as **DML** for the purposes of our migrations.
+
+**The usage of `Gitlab::Database::Migration[2.0]` requires migrations to always be of a single purpose**.
+Migrations cannot mix **DDL** and **DML** changes as the application requires the structure
+(as described by `db/structure.sql`) to be exactly the same across all decomposed databases.
+
+### Data Definition Language (DDL) 
+
+The DDL migrations are all migrations that:
+
+1. Create or drop a table (for example, `create_table`).
+1. Add or remove an index (for example, `add_index`, `add_index_concurrently`).
+1. Add or remove a foreign key (for example `add_foreign_key`, `add_foreign_key_concurrently`).
+1. Add or remove a column with or without a default value (for example, `add_column`).
+1. Create or drop trigger functions (for example, `create_trigger_function`).
+1. Attach or detach triggers from tables (for example, `track_record_deletions`, `untrack_record_deletions`).
+1. Prepare or not async indexes (for example, `prepare_async_index`, `unprepare_async_index_by_name`).
+
+As such DDL migrations **CANNOT**:
+
+1. Read or modify data in any form, via SQL statements or ActiveRecord models.
+1. Update column values (for example, `update_column_in_batches`).
+1. Schedule background migrations (for example, `queue_background_migration_jobs_by_range_at_intervals`).
+1. Read the state of feature flags since they are stored in `main:` (a `features` and `feature_gates`).
+1. Read application settings (as settings are stored in `main:`).
+
+As the majority of migrations in the GitLab codebase are of the DDL-type,
+this is also the default mode of operation and requires no further changes
+to the migrations files.
+
+#### Example: perform DDL on all databases
+
+Example migration adding a concurrent index that is treated as change of the structure (DDL)
+that is executed on all configured databases.
+
+```ruby
+class AddUserIdAndStateIndexToMergeRequestReviewers < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  INDEX_NAME = 'index_on_merge_request_reviewers_user_id_and_state'
+
+  def up
+    add_concurrent_index :merge_request_reviewers, [:user_id, :state], where: 'state = 2', name: INDEX_NAME
+  end
+
+  def down
+    remove_concurrent_index_by_name :merge_request_reviewers, INDEX_NAME
+  end
+end
+```
+
+### Data Manipulation Language (DML)
+
+The DML migrations are all migrations that:
+
+1. Read data via SQL statements (for example, `SELECT * FROM projects WHERE id=1`).
+1. Read data via ActiveRecord models (for example, `User < MigrationRecord`).
+1. Create, update or delete data via ActiveRecord models (for example, `User.create!(...)`).
+1. Create, update or delete data via SQL statements (for example, `DELETE FROM projects WHERE id=1`).
+1. Update columns in batches (for example, `update_column_in_batches(:projects, :archived, true)`).
+1. Schedule background migrations (for example, `queue_background_migration_jobs_by_range_at_intervals`).
+1. Access application settings (for example, `ApplicationSetting.last` if run for `main:` database).
+1. Read and modify feature flags if run for the `main:` database.
+
+The DML migrations **CANNOT**:
+
+1. Make any changes to DDL since this breaks the rule of keeping `structure.sql` coherent across
+   all decomposed databases.
+1. **Read data from another database**.
+
+To indicate the `DML` migration type, a migration must use the `restrict_gitlab_migration gitlab_schema:`
+syntax in a migration class. This marks the given migration as DML and restricts access to it.
+
+#### Example: perform DML only in context of the database containing the given `gitlab_schema`
+
+Example migration updating `archived` column of `projects` that is executed
+only for the database containing `gitlab_main` schema.
+
+```ruby
+class UpdateProjectsArchivedState < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  restrict_gitlab_migration gitlab_schema: :gitlab_main
+
+  def up
+    update_column_in_batches(:projects, :archived, true) do |table, query|
+      query.where(table[:archived].eq(false)) # rubocop:disable CodeReuse/ActiveRecord
+    end
+  end
+
+  def down
+    # no-op
+  end
+end
+```
+
+#### Example: usage of `ActiveRecord` classes
+
+A migration using `ActiveRecord` class to perform data manipulation
+must use the `MigrationRecord` class. This class is guaranteed to provide
+a correct connection in a context of a given migration.
+
+Underneath the `MigrationRecord == ActiveRecord::Base`, as once the `db:migrate`
+runs, it switches the active connection of `ActiveRecord::Base.establish_connection :ci`.
+To avoid confusion to using the `ActiveRecord::Base`, `MigrationRecord` is required.
+
+This implies that DML migrations are forbidden to read data from other
+databases. For example, running migration in context of `ci:` and reading feature flags
+from `main:`, as no established connection to another database is present.
+
+```ruby
+class UpdateProjectsArchivedState < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  restrict_gitlab_migration gitlab_schema: :gitlab_main
+
+  class Project < MigrationRecord
+  end
+
+  def up
+    Project.where(archived: false).each_batch of |batch|
+      batch.update_all(archived: true)
+    end
+  end
+
+  def down
+  end
+end
+```
+
+### The special purpose of `gitlab_shared`
+
+As described in [gitlab_schema](multiple_databases.md#the-special-purpose-of-gitlab_shared),
+the `gitlab_shared` tables are allowed to contain data across all databases. This implies
+that such migrations should run across all databases to modify structure (DDL) or modify data (DML).
+
+As such migrations accessing `gitlab_shared` do not need to use `restrict_gitlab_migration gitlab_schema:`,
+migrations without restriction run across all databases and are allowed to modify data on each of them.
+If the `restrict_gitlab_migration gitlab_schema:` is specified, the `DML` migration
+runs only in a context of a database containing the given `gitlab_schema`.
+
+#### Example: run DML `gitlab_shared` migration on all databases
+
+Example migration updating `loose_foreign_keys_deleted_records` table
+that is marked in `lib/gitlab/database/gitlab_schemas.yml` as `gitlab_shared`.
+
+This migration is executed across all configured databases.
+
+```ruby
+class DeleteAllLooseForeignKeyRecords < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  def up
+    execute("DELETE FROM loose_foreign_keys_deleted_records")
+  end
+
+  def down
+    # no-op
+  end
+end
+```
+
+#### Example: run DML `gitlab_shared` only on the database containing the given `gitlab_schema`
+
+Example migration updating `loose_foreign_keys_deleted_records` table
+that is marked in `lib/gitlab/database/gitlab_schemas.yml` as `gitlab_shared`.
+
+This migration since it configures restriction on `gitlab_ci` is executed only
+in context of database containing `gitlab_ci` schema.
+
+```ruby
+class DeleteCiBuildsLooseForeignKeyRecords < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  restrict_gitlab_migration gitlab_schema: :gitlab_ci
+
+  def up
+    execute("DELETE FROM loose_foreign_keys_deleted_records WHERE fully_qualified_table_name='ci_builds'")
+  end
+
+  def down
+    # no-op
+  end
+end
+```
+
+### The behavior of skipping migrations
+
+The only migrations that are skipped are the ones performing **DML** changes.
+The **DDL** migrations are **always and unconditionally** executed.
+
+The implemented [solution](https://gitlab.com/gitlab-org/gitlab/-/issues/355014#solution-2-use-database_tasks)
+uses the `database_tasks:` as a way to indicate which additional database configurations
+(in `config/database.yml`) share the same primary database. The database configurations
+marked with `database_tasks: false` are exempt from executing `db:migrate` for those
+database configurations.
+
+If database configurations do not share databases (all do have `database_tasks: true`),
+each migration runs for every database configuration:
+
+1. The DDL migration applies all structure changes on all databases.
+1. The DML migration runs only in the context of a database containing the given `gitlab_schema:`.
+1. If the DML migration is not eligible to run, it is skipped. It's still
+   marked as executed in `schema_migrations`. While running `db:migrate`, the skipped
+   migration outputs `Current migration is skipped since it modifies 'gitlab_ci' which is outside of 'gitlab_main, gitlab_shared`.
+
+To prevent loss of migrations if the `database_tasks: false` is configured, a dedicated
+Rake task is used [`gitlab:db:validate_config`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83118).
+The `gitlab:db:validate_config` validates the correctness of `database_tasks:` by checking database identifiers
+of each underlying database configuration. The ones that share the database are required to have
+the `database_tasks: false` set. `gitlab:db:validate_config` always runs before `db:migrate`.
+
+## Validation
+
+Validation in a nutshell uses [pg_query](https://github.com/pganalyze/pg_query) to analyze
+each query and classify tables with information from [`gitlab_schema.yml`](multiple_databases.md#gitlab-schema).
+The migration is skipped if the specified `gitlab_schema` is outside of a list of schemas
+managed by a given database connection (`Gitlab::Database::gitlab_schemas_for_connection`).
+
+The `Gitlab::Database::Migration[2.0]` includes `Gitlab::Database::MigrationHelpers::RestrictGitlabSchema`
+which extends the `#migrate` method. For the duration of a migration a dedicated query analyzer
+is installed `Gitlab::Database::QueryAnalyzers::RestrictAllowedSchemas` that accepts
+a list of allowed schemas as defined by `restrict_gitlab_migration:`. If the executed query
+is outside of allowed schemas, it raises an exception.
+
+## Exceptions
+
+Depending on misuse or lack of `restrict_gitlab_migration` various exceptions can be raised
+as part of the migration run and prevent the migration from being completed.
+
+### Exception 1: migration running in DDL mode does DML select
+
+```ruby
+class UpdateProjectsArchivedState < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  # Missing:
+  # restrict_gitlab_migration gitlab_schema: :gitlab_main
+
+  def up
+    update_column_in_batches(:projects, :archived, true) do |table, query|
+      query.where(table[:archived].eq(false)) # rubocop:disable CodeReuse/ActiveRecord
+    end
+  end
+
+  def down
+    # no-op
+  end
+end
+```
+
+```plaintext
+Select/DML queries (SELECT/UPDATE/DELETE) are disallowed in the DDL (structure) mode
+Modifying of 'projects' (gitlab_main) with 'SELECT * FROM projects...
+```
+
+The current migration do not use `restrict_gitlab_migration`. The lack indicates a migration
+running in **DDL** mode, but the executed payload appears to be reading data from `projects`.
+
+**The solution** is to add `restrict_gitlab_migration gitlab_schema: :gitlab_main`.
+
+### Exception 2: migration running in DML mode changes the structure
+
+```ruby
+class AddUserIdAndStateIndexToMergeRequestReviewers < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  # restrict_gitlab_migration if defined indicates DML, it should be removed
+  restrict_gitlab_migration gitlab_schema: :gitlab_main
+
+  INDEX_NAME = 'index_on_merge_request_reviewers_user_id_and_state'
+
+  def up
+    add_concurrent_index :merge_request_reviewers, [:user_id, :state], where: 'state = 2', name: INDEX_NAME
+  end
+
+  def down
+    remove_concurrent_index_by_name :merge_request_reviewers, INDEX_NAME
+  end
+end
+```
+
+```plaintext
+DDL queries (structure) are disallowed in the Select/DML (SELECT/UPDATE/DELETE) mode.
+Modifying of 'merge_request_reviewers' with 'CREATE INDEX...
+```
+
+The current migration do use `restrict_gitlab_migration`. The presence indicates **DML** mode,
+but the executed payload appears to be doing structure changes (DDL).
+
+**The solution** is to remove `restrict_gitlab_migration gitlab_schema: :gitlab_main`.
+
+### Exception 3: migration running in DML mode accesses data from a table in another schema
+
+```ruby
+class UpdateProjectsArchivedState < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  # Since it modifies `projects` it should use `gitlab_main`
+  restrict_gitlab_migration gitlab_schema: :gitlab_ci
+
+  def up
+    update_column_in_batches(:projects, :archived, true) do |table, query|
+      query.where(table[:archived].eq(false)) # rubocop:disable CodeReuse/ActiveRecord
+    end
+  end
+
+  def down
+    # no-op
+  end
+end
+```
+
+```plaintext
+Select/DML queries (SELECT/UPDATE/DELETE) do access 'projects' (gitlab_main) " \
+which is outside of list of allowed schemas: 'gitlab_ci'
+```
+
+The current migration do restrict the migration to `gitlab_ci`, but appears to modify
+data in `gitlab_main`.
+
+**The solution** is to change `restrict_gitlab_migration gitlab_schema: :gitlab_ci`.
+
+### Exception 4: mixing DDL and DML mode
+
+```ruby
+class UpdateProjectsArchivedState < Gitlab::Database::Migration[2.0]
+  disable_ddl_transaction!
+
+  # This migration is invalid regardless of specification
+  # as it cannot modify structure and data at the same time
+  restrict_gitlab_migration gitlab_schema: :gitlab_ci
+
+  def up
+    add_concurrent_index :merge_request_reviewers, [:user_id, :state], where: 'state = 2', name: 'index_on_merge_request_reviewers'
+    update_column_in_batches(:projects, :archived, true) do |table, query|
+      query.where(table[:archived].eq(false)) # rubocop:disable CodeReuse/ActiveRecord
+    end
+  end
+
+  def down
+    # no-op
+  end
+end
+```
+
+The migrations mixing **DDL** and **DML** depending on ordering of operations raises
+one of the prior exceptions.
+
+## Upcoming changes on multiple database migrations
+
+The `restrict_gitlab_migration` using `gitlab_schema:` is considered as a first iteration
+of this feature for running migrations selectively depending on a context. It is possible
+to add additional restrictions to DML-only migrations (as the structure coherency is likely
+to stay as-is until further notice) to restrict when they run.
+
+A Potential extension is to limit running DML migration only to specific environments:
+
+```ruby
+restrict_gitlab_migration gitlab_schema: :gitlab_main, gitlab_env: :gitlab_com
+```

doc/development/database/multiple_databases.md

@@ -9,6 +9,63 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 To scale GitLab, the we are
 [decomposing the GitLab application database into multiple databases](https://gitlab.com/groups/gitlab-org/-/epics/6168).
 
+## GitLab Schema
+
+For properly discovering allowed patterns between different databases
+the GitLab application implements the `lib/gitlab/database/gitlab_schemas.yml` YAML file.
+
+This file provides a virtual classification of tables into a `gitlab_schema`
+which conceptually is similar to [PostgreSQL Schema](https://www.postgresql.org/docs/current/ddl-schemas.html).
+We decided as part of [using database schemas to better isolated CI decomposed features](https://gitlab.com/gitlab-org/gitlab/-/issues/333415)
+that we cannot use PostgreSQL schema due to complex migration procedures. Instead we implemented
+the concept of application-level classification.
+Each table of GitLab needs to have a `gitlab_schema` assigned:
+
+- `gitlab_main`: describes all tables that are being stored in the `main:` database (for example, like `projects`, `users`).
+- `gitlab_ci`: describes all CI tables that are being stored in the `ci:` database (for example, `ci_pipelines`, `ci_builds`).
+- `gitlab_shared`: describe all application tables that contain data across all decomposed databases (for example, `loose_foreign_keys_deleted_records`).
+- `...`: more schemas to be introduced with additional decomposed databases
+
+The usage of schema enforces the base class to be used:
+
+- `ApplicationRecord` for `gitlab_main`
+- `Ci::ApplicationRecord` for `gitlab_ci`
+- `Gitlab::Database::SharedModel` for `gitlab_shared`
+
+### The impact of `gitlab_schema`
+
+The usage of `gitlab_schema` has a significant impact on the application.
+The `gitlab_schema` primary purpose is to introduce a barrier between different data access patterns.
+
+This is used as a primary source of classification for:
+
+- [Discovering cross-joins across tables from different schemas](#removing-joins-between-ci_-and-non-ci_-tables)
+- [Discovering cross-database transactions across tables from different schemas](#removing-cross-database-transactions)
+
+### The special purpose of `gitlab_shared`
+
+`gitlab_shared` is a special case describing tables or views that by design contain data across
+all decomposed databases. This does describe application-defined tables (like `loose_foreign_keys_deleted_records`),
+Rails-defined tables (like `schema_migrations` or `ar_internal_metadata` as well as internal PostgreSQL tables
+(for example, `pg_attribute`).
+
+**Be careful** to use `gitlab_shared` as it requires special handling while accessing data.
+Since `gitlab_shared` shares not only structure but also data, the application needs to be written in a way
+that traverses all data from all databases in sequential manner.
+
+```ruby
+Gitlab::Database::EachDatabase.each_model_connection([MySharedModel]) do |connection, connection_name|
+  MySharedModel.select_all_data...
+end
+```
+
+As such, migrations modifying data of `gitlab_shared` tables are expected to run across
+all decomposed databases.
+
+## Migrations
+
+Read [Migrations for Multiple Databases](migrations_for_multiple_databases.md).
+
 ## CI/CD Database
 
 > Support for configuring the GitLab Rails application to use a distinct

doc/development/migration_style_guide.md

@@ -240,7 +240,7 @@ of migration helpers.
 In this example, we use version 1.0 of the migration class:
 
 ```ruby
-class TestMigration < Gitlab::Database::Migration[1.0]
+class TestMigration < Gitlab::Database::Migration[2.0]
   def change
   end
 end
@@ -253,7 +253,7 @@ version of migration helpers automatically.
 Migration helpers and versioning were [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68986)
 in GitLab 14.3.
 For merge requests targeting previous stable branches, use the old format and still inherit from
-`ActiveRecord::Migration[6.1]` instead of `Gitlab::Database::Migration[1.0]`.
+`ActiveRecord::Migration[6.1]` instead of `Gitlab::Database::Migration[2.0]`.
 
 ## Retry mechanism when acquiring database locks
 
@@ -535,7 +535,7 @@ by calling the method `disable_ddl_transaction!` in the body of your migration
 class like so:
 
 ```ruby
-class MyMigration < Gitlab::Database::Migration[1.0]
+class MyMigration < Gitlab::Database::Migration[2.0]
   disable_ddl_transaction!
 
   INDEX_NAME = 'index_name'
@@ -586,7 +586,7 @@ by calling the method `disable_ddl_transaction!` in the body of your migration
 class like so:
 
 ```ruby
-class MyMigration < Gitlab::Database::Migration[1.0]
+class MyMigration < Gitlab::Database::Migration[2.0]
   disable_ddl_transaction!
 
   INDEX_NAME = 'index_name'
@@ -629,7 +629,7 @@ The easiest way to test for existence of an index by name is to use the
 be used with a name option. For example:
 
 ```ruby
-class MyMigration < Gitlab::Database::Migration[1.0]
+class MyMigration < Gitlab::Database::Migration[2.0]
   INDEX_NAME = 'index_name'
 
   def up
@@ -664,7 +664,7 @@ Here's an example where we add a new column with a foreign key
 constraint. Note it includes `index: true` to create an index for it.
 
 ```ruby
-class Migration < Gitlab::Database::Migration[1.0]
+class Migration < Gitlab::Database::Migration[2.0]
 
   def change
     add_reference :model, :other_model, index: true, foreign_key: { on_delete: :cascade }
@@ -710,7 +710,7 @@ expensive and disruptive operation for larger tables, but in reality it's not.
 Take the following migration as an example:
 
 ```ruby
-class DefaultRequestAccessGroups < Gitlab::Database::Migration[1.0]
+class DefaultRequestAccessGroups < Gitlab::Database::Migration[2.0]
   def change
     change_column_default(:namespaces, :request_access_enabled, from: false, to: true)
   end
@@ -943,7 +943,7 @@ The Rails 5 natively supports `JSONB` (binary JSON) column type.
 Example migration adding this column:
 
 ```ruby
-class AddOptionsToBuildMetadata < Gitlab::Database::Migration[1.0]
+class AddOptionsToBuildMetadata < Gitlab::Database::Migration[2.0]
   def change
     add_column :ci_builds_metadata, :config_options, :jsonb
   end
@@ -975,7 +975,7 @@ Do not store `attr_encrypted` attributes as `:text` in the database; use
 efficient:
 
 ```ruby
-class AddSecretToSomething < Gitlab::Database::Migration[1.0]
+class AddSecretToSomething < Gitlab::Database::Migration[2.0]
   def change
     add_column :something, :encrypted_secret, :binary
     add_column :something, :encrypted_secret_iv, :binary
@@ -1033,8 +1033,8 @@ If you need more complex logic, you can define and use models local to a
 migration. For example:
 
 ```ruby
-class MyMigration < Gitlab::Database::Migration[1.0]
-  class Project < ActiveRecord::Base
+class MyMigration < Gitlab::Database::Migration[2.0]
+  class Project < MigrationRecord
     self.table_name = 'projects'
   end
 
@@ -1132,8 +1132,8 @@ in a previous migration.
 It is important not to leave out the `User.reset_column_information` command, in order to ensure that the old schema is dropped from the cache and ActiveRecord loads the updated schema information.
 
 ```ruby
-class AddAndSeedMyColumn < Gitlab::Database::Migration[1.0]
-  class User < ActiveRecord::Base
+class AddAndSeedMyColumn < Gitlab::Database::Migration[2.0]
+  class User < MigrationRecord
     self.table_name = 'users'
   end
 

doc/development/single_table_inheritance.md

@@ -31,8 +31,8 @@ could result in loading unexpected code or associations which may cause unintend
 side effects or failures during upgrades.
 
 ```ruby
-class SomeMigration < Gitlab::Database::Migration[1.0]
-  class Services < ActiveRecord::Base
+class SomeMigration < Gitlab::Database::Migration[2.0]
+  class Services < MigrationRecord
     self.table_name = 'services'
     self.inheritance_column = :_type_disabled
   end

doc/development/sql.md

@@ -254,13 +254,13 @@ of records plucked. `MAX_PLUCK` defaults to `1_000` in `ApplicationRecord`.
 
 ## Inherit from ApplicationRecord
 
-Most models in the GitLab codebase should inherit from `ApplicationRecord`,
-rather than from `ActiveRecord::Base`. This allows helper methods to be easily
-added.
+Most models in the GitLab codebase should inherit from `ApplicationRecord`
+or `Ci::ApplicationRecord` rather than from `ActiveRecord::Base`. This allows
+helper methods to be easily added.
 
 An exception to this rule exists for models created in database migrations. As
 these should be isolated from application code, they should continue to subclass
-from `ActiveRecord::Base`.
+from `MigrationRecord` which is available only in migration context.
 
 ## Use UNIONs
 

generator_templates/active_record/migration/migration.rb

@@ -16,6 +16,10 @@ class <%= migration_class_name %> < Gitlab::Database::Migration[<%= Gitlab::Data
   # To disable transactions uncomment the following line and remove these
   # comments:
   # disable_ddl_transaction!
+  #
+  # Configure the `gitlab_schema` to perform data manipulation (DML).
+  # Visit: https://docs.gitlab.com/ee/development/database/migrations_for_multiple_databases.html
+  # restrict_gitlab_migration gitlab_schema: :gitlab_main
 
 <%- if migration_action == 'add' -%>
   def change

generator_templates/post_deployment_migration/post_deployment_migration/migration.rb

@@ -16,6 +16,10 @@ class <%= migration_class_name %> < Gitlab::Database::Migration[<%= Gitlab::Data
   # To disable transactions uncomment the following line and remove these
   # comments:
   # disable_ddl_transaction!
+  #
+  # Configure the `gitlab_schema` to perform data manipulation (DML).
+  # Visit: https://docs.gitlab.com/ee/development/database/migrations_for_multiple_databases.html
+  # restrict_gitlab_migration gitlab_schema: :gitlab_main
 
   def up
   end

lib/gitlab/database/migration.rb

@@ -41,6 +41,12 @@ module Gitlab
 
       class V2_0 < V1_0 # rubocop:disable Naming/ClassAndModuleCamelCase
         include Gitlab::Database::MigrationHelpers::RestrictGitlabSchema
+
+        # When running migrations, the `db:migrate` switches connection of
+        # ActiveRecord::Base depending where the migration runs.
+        # This helper class is provided to avoid confusion using `ActiveRecord::Base`
+        class MigrationRecord < ActiveRecord::Base
+        end
       end
 
       def self.[](version)
@@ -53,7 +59,7 @@ module Gitlab
 
       # The current version to be used in new migrations
       def self.current_version
-        1.0
+        2.0
       end
     end
   end

lib/gitlab/database/query_analyzers/restrict_allowed_schemas.rb

@@ -69,8 +69,10 @@ module Gitlab
             schemas = self.dml_schemas(tables)
 
             if (schemas - self.allowed_gitlab_schemas).any?
-              raise DMLAccessDeniedError, "Select/DML queries (SELECT/UPDATE/DELETE) do access '#{tables}' (#{schemas.to_a}) " \
-                "which is outside of list of allowed schemas: '#{self.allowed_gitlab_schemas}'."
+              raise DMLAccessDeniedError, \
+                "Select/DML queries (SELECT/UPDATE/DELETE) do access '#{tables}' (#{schemas.to_a}) " \
+                "which is outside of list of allowed schemas: '#{self.allowed_gitlab_schemas}'. " \
+                "#{documentation_url}"
             end
           end
 
@@ -93,11 +95,19 @@ module Gitlab
           end
 
           def raise_dml_not_allowed_error(message)
-            raise DMLNotAllowedError, "Select/DML queries (SELECT/UPDATE/DELETE) are disallowed in the DDL (structure) mode. #{message}"
+            raise DMLNotAllowedError, \
+              "Select/DML queries (SELECT/UPDATE/DELETE) are disallowed in the DDL (structure) mode. " \
+              "#{message}. #{documentation_url}" \
           end
 
           def raise_ddl_not_allowed_error(message)
-            raise DDLNotAllowedError, "DDL queries (structure) are disallowed in the Select/DML (SELECT/UPDATE/DELETE) mode. #{message}"
+            raise DDLNotAllowedError, \
+              "DDL queries (structure) are disallowed in the Select/DML (SELECT/UPDATE/DELETE) mode. " \
+              "#{message}. #{documentation_url}"
+          end
+
+          def documentation_url
+            "For more information visit: https://docs.gitlab.com/ee/development/database/migrations_for_multiple_databases.html"
           end
         end
       end

spec/lib/gitlab/database/migration_helpers/restrict_gitlab_schema_spec.rb

@@ -240,7 +240,7 @@ RSpec.describe Gitlab::Database::MigrationHelpers::RestrictGitlabSchema, query_a
             end
 
             def software_license_class
-              Class.new(ActiveRecord::Base) do
+              Class.new(Gitlab::Database::Migration[2.0]::MigrationRecord) do
                 self.table_name = 'software_licenses'
               end
             end
@@ -272,7 +272,7 @@ RSpec.describe Gitlab::Database::MigrationHelpers::RestrictGitlabSchema, query_a
             end
 
             def ci_instance_variables_class
-              Class.new(ActiveRecord::Base) do
+              Class.new(Gitlab::Database::Migration[2.0]::MigrationRecord) do
                 self.table_name = 'ci_instance_variables'
               end
             end
@@ -303,7 +303,7 @@ RSpec.describe Gitlab::Database::MigrationHelpers::RestrictGitlabSchema, query_a
             end
 
             def detached_partitions_class
-              Class.new(ActiveRecord::Base) do
+              Class.new(Gitlab::Database::Migration[2.0]::MigrationRecord) do
                 self.table_name = 'detached_partitions'
               end
             end

spec/lib/gitlab/database/migration_spec.rb

@@ -32,7 +32,7 @@ RSpec.describe Gitlab::Database::Migration do
       # This breaks upon Rails upgrade. In that case, we'll add a new version in Gitlab::Database::Migration::MIGRATION_CLASSES,
       # bump .current_version and leave existing migrations and already defined versions of Gitlab::Database::Migration
       # untouched.
-      expect(described_class[described_class.current_version].superclass).to eq(ActiveRecord::Migration::Current)
+      expect(described_class[described_class.current_version]).to be < ActiveRecord::Migration::Current
     end
   end