migration_style_guide.md 4.87 KB
Newer Older
1 2 3
# Migration Style Guide

When writing migrations for GitLab, you have to take into account that
4
these will be ran by hundreds of thousands of organizations of all sizes, some with
5 6 7 8
many years of data in their database.

In addition, having to take a server offline for a an upgrade small or big is
a big burden for most organizations. For this reason it is important that your
9
migrations are written carefully, can be applied online and adhere to the style guide below.
10

11 12 13
Migrations should not require GitLab installations to be taken offline unless
_absolutely_ necessary. If a migration requires downtime this should be
clearly mentioned during the review process as well as being documented in the
14 15
monthly release post. For more information see the "Downtime Tagging" section
below.
16

17 18 19 20
When writing your migrations, also consider that databases might have stale data
or inconsistencies and guard for that. Try to make as little assumptions as possible
about the state of the database.

21
Please don't depend on GitLab specific code since it can change in future versions.
Prayag Verma's avatar
Prayag Verma committed
22
If needed copy-paste GitLab code into the migration to make it forward compatible.
23

24
## Downtime Tagging
25

26 27 28
Every migration must specify if it requires downtime or not, and if it should
require downtime it must also specify a reason for this. To do so, add the
following two constants to the migration class' body:
29

30 31 32 33
* `DOWNTIME`: a boolean that when set to `true` indicates the migration requires
  downtime.
* `DOWNTIME_REASON`: a String containing the reason for the migration requiring
  downtime. This constant **must** be set when `DOWNTIME` is set to `true`.
34

35
For example:
36

37
```ruby
38
class MyMigration < ActiveRecord::Migration
39 40
  DOWNTIME = true
  DOWNTIME_REASON = 'This migration requires downtime because ...'
41

42 43 44 45 46
  def change
    ...
  end
end
```
47

48 49
It is an error (that is, CI will fail) if the `DOWNTIME` constant is missing
from a migration class.
50

51
## Reversibility
52 53 54 55 56

Your migration should be reversible. This is very important, as it should
be possible to downgrade in case of a vulnerability or bugs.

In your migration, add a comment describing how the reversibility of the
57 58 59 60 61 62 63 64 65 66 67 68 69 70
migration was tested.

## Removing indices

If you need to remove index, please add a condition like in following example:

```
remove_index :namespaces, column: :name if index_exists?(:namespaces, :name)
```

## Adding indices

If you need to add an unique index please keep in mind there is possibility of existing duplicates. If it is possible write a separate migration for handling this situation. It can be just removing or removing with overwriting all references to these duplicates depend on situation.

71 72 73 74 75 76 77 78 79
When adding an index make sure to use the method `add_concurrent_index` instead
of the regular `add_index` method. The `add_concurrent_index` method
automatically creates concurrent indexes when using PostgreSQL, removing the
need for downtime. To use this method you must disable transactions by calling
the method `disable_ddl_transaction!` in the body of your migration class like
so:

```
class MyMigration < ActiveRecord::Migration
80
  include Gitlab::Database::MigrationHelpers
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100
  disable_ddl_transaction!

  def change

  end
end
```

## Adding Columns With Default Values

When adding columns with default values you should use the method
`add_column_with_default`. This method ensures the table is updated without
requiring downtime. This method is not reversible so you must manually define
the `up` and `down` methods in your migration class.

For example, to add the column `foo` to the `projects` table with a default
value of `10` you'd write the following:

```
class MyMigration < ActiveRecord::Migration
101 102
  include Gitlab::Database::MigrationHelpers
  disable_ddl_transaction!
103

104
  def up
105
    add_column_with_default(:projects, :foo, :integer, default: 10)
106 107 108 109 110 111 112 113
  end

  def down
    remove_column(:projects, :foo)
  end
end
```

114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129
## Testing

Make sure that your migration works with MySQL and PostgreSQL with data. An empty database does not guarantee that your migration is correct.

Make sure your migration can be reversed.

## Data migration

Please prefer Arel and plain SQL over usual ActiveRecord syntax. In case of using plain SQL you need to quote all input manually with `quote_string` helper.

Example with Arel:

```
users = Arel::Table.new(:users)
users.group(users[:user_id]).having(users[:id].count.gt(5))

Chris Spicer's avatar
Chris Spicer committed
130
#update other tables with these results
131 132 133 134 135 136 137 138 139 140 141 142 143 144
```

Example with plain SQL and `quote_string` helper:

```
select_all("SELECT name, COUNT(id) as cnt FROM tags GROUP BY name HAVING COUNT(id) > 1").each do |tag|
  tag_name = quote_string(tag["name"])
  duplicate_ids = select_all("SELECT id FROM tags WHERE name = '#{tag_name}'").map{|tag| tag["id"]}
  origin_tag_id = duplicate_ids.first
  duplicate_ids.delete origin_tag_id

  execute("UPDATE taggings SET tag_id = #{origin_tag_id} WHERE tag_id IN(#{duplicate_ids.join(",")})")
  execute("DELETE FROM tags WHERE id IN(#{duplicate_ids.join(",")})")
end
145
```