• Vladimir Shushlin's avatar
    Automatically migrate pages to zip storage · 63bc19ed
    Vladimir Shushlin authored
    Currently, the only way to migrate pages to zip storage is to manually
    run the rake task:
    
    ```
    gitlab-rake gitlab:pages:migrate_legacy_storage
    ```
    
    It requires human intervention on migration and can take a long time
    to  complete. (it took about a week on gitlab.com).
    
    We can't remove this rake task because it allows users to control the
    process and fix some errors:
    * it prints all errors by default, such as invalid symlinks and
    projects marked as deployed by absent on pages storage
    * it also allows for fix these errors by ignoring invalid symlinks and
    marking projects as not deployed
    * and it allows users to see the progress/check if everything has been
    migrated
    
    However, making every instance admin run this task when upgrading to
    `14.0` may create too much work for them.
    So we want to run the same migration code as the background
    migration.
    This way we will migrate the data for most of the users without their
    intervention.
    
    It still will be recommended to run the rake task and make sure
    everything has been migrated properly.
    But in most cases everything should be migrated automatically and even
    if users forgot to run the rake task everything should be fine.
    
    * Refactor existing batch migration service to support
    calling without threads and just proccessing the batch or projects
    * Subsequently refactor rake task due to changed interface
    * Shedule a background migration and execute zip migration service
    there
    63bc19ed
322001-poc-for-migrating-pages-to-zip-storage-in-the-background.yml 109 Bytes