Merge branch 'create_hooks_rake' into 'master'

Add Rake task to create/repair GitLab Shell hooks symlinks

## What does this MR do?

Adds a Rake task wrapper around `gitlab-shell` `bin/create_hooks` script. This makes it easier for an administrator to repair hooks symlinks without worrying about where repository data is located, etc. Just run this task and it takes care of everything based on current configuration. 

## Are there points in the code the reviewer needs to double check?

No.

## Why was this MR needed?

`bin/create_hooks` was not well-documented. It requires parameters specifying where repository data is located. It also needs to be run by the `git` user. Wrapping it in a Rake task allows us to take current configuration in to account and makes it easier on the administrator. 


See merge request !5634

CHANGELOG.md

@@ -38,6 +38,7 @@ Please view this file on the master branch, on stable branches it's out of date.
 - Optimize Event queries by removing default order
 - Remove duplicate links from sidebar
 - API: Fix project deploy keys 400 and 500 errors when adding an existing key. !6784 (Joshua Welsh)
+- Add Rake task to create/repair GitLab Shell hooks symlinks !5634
 - Add job for removal of unreferenced LFS objects from both the database and the filesystem (Frank Groeneveld)
 - Replace jquery.cookie plugin with js.cookie !7085
 - Use MergeRequestsClosingIssues cache data on Issue#closed_by_merge_requests method

doc/administration/raketasks/maintenance.md

+# Maintenance Rake Tasks
+
+## Gather information about GitLab and the system it runs on
+
+This command gathers information about your GitLab installation and the System it runs on. These may be useful when asking for help or reporting issues.
+
+**Omnibus Installation**
+
+```
+sudo gitlab-rake gitlab:env:info
+```
+
+**Source Installation**
+
+```
+bundle exec rake gitlab:env:info RAILS_ENV=production
+```
+
+Example output:
+
+```
+System information
+System:           Debian 7.8
+Current User:     git
+Using RVM:        no
+Ruby Version:     2.1.5p273
+Gem Version:      2.4.3
+Bundler Version:  1.7.6
+Rake Version:     10.3.2
+Sidekiq Version:  2.17.8
+
+GitLab information
+Version:          7.7.1
+Revision:         41ab9e1
+Directory:        /home/git/gitlab
+DB Adapter:       postgresql
+URL:              https://gitlab.example.com
+HTTP Clone URL:   https://gitlab.example.com/some-project.git
+SSH Clone URL:    git@gitlab.example.com:some-project.git
+Using LDAP:       no
+Using Omniauth:   no
+
+GitLab Shell
+Version:          2.4.1
+Repositories:     /home/git/repositories/
+Hooks:            /home/git/gitlab-shell/hooks/
+Git:              /usr/bin/git
+```
+
+## Check GitLab configuration
+
+Runs the following rake tasks:
+
+- `gitlab:gitlab_shell:check`
+- `gitlab:sidekiq:check`
+- `gitlab:app:check`
+
+It will check that each component was setup according to the installation guide and suggest fixes for issues found.
+
+You may also have a look at our [Trouble Shooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide).
+
+**Omnibus Installation**
+
+```
+sudo gitlab-rake gitlab:check
+```
+
+**Source Installation**
+
+```
+bundle exec rake gitlab:check RAILS_ENV=production
+```
+
+NOTE: Use SANITIZE=true for gitlab:check if you want to omit project names from the output.
+
+Example output:
+
+```
+Checking Environment ...
+
+Git configured for git user? ... yes
+Has python2? ... yes
+python2 is supported version? ... yes
+
+Checking Environment ... Finished
+
+Checking GitLab Shell ...
+
+GitLab Shell version? ... OK (1.2.0)
+Repo base directory exists? ... yes
+Repo base directory is a symlink? ... no
+Repo base owned by git:git? ... yes
+Repo base access is drwxrws---? ... yes
+post-receive hook up-to-date? ... yes
+post-receive hooks in repos are links: ... yes
+
+Checking GitLab Shell ... Finished
+
+Checking Sidekiq ...
+
+Running? ... yes
+
+Checking Sidekiq ... Finished
+
+Checking GitLab ...
+
+Database config exists? ... yes
+Database is SQLite ... no
+All migrations up? ... yes
+GitLab config exists? ... yes
+GitLab config outdated? ... no
+Log directory writable? ... yes
+Tmp directory writable? ... yes
+Init script exists? ... yes
+Init script up-to-date? ... yes
+Redis version >= 2.0.0? ... yes
+
+Checking GitLab ... Finished
+```
+
+## Rebuild authorized_keys file
+
+In some case it is necessary to rebuild the `authorized_keys` file.
+
+**Omnibus Installation**
+
+```
+sudo gitlab-rake gitlab:shell:setup
+```
+
+**Source Installation**
+
+```
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production
+```
+
+```
+This will rebuild an authorized_keys file.
+You will lose any data stored in authorized_keys file.
+Do you want to continue (yes/no)? yes
+```
+
+## Clear redis cache
+
+If for some reason the dashboard shows wrong information you might want to
+clear Redis' cache.
+
+**Omnibus Installation**
+
+```
+sudo gitlab-rake cache:clear
+```
+
+**Source Installation**
+
+```
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production
+```
+
+## Precompile the assets
+
+Sometimes during version upgrades you might end up with some wrong CSS or
+missing some icons. In that case, try to precompile the assets again.
+
+Note that this only applies to source installations and does NOT apply to
+Omnibus packages.
+
+**Source Installation**
+
+```
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production
+```
+
+For omnibus versions, the unoptimized assets (JavaScript, CSS) are frozen at
+the release of upstream GitLab. The omnibus version includes optimized versions
+of those assets. Unless you are modifying the JavaScript / CSS code on your
+production machine after installing the package, there should be no reason to redo
+rake assets:precompile on the production machine. If you suspect that assets
+have been corrupted, you should reinstall the omnibus package.
+
+## Tracking Deployments
+
+GitLab provides a Rake task that lets you track deployments in GitLab
+Performance Monitoring. This Rake task simply stores the current GitLab version
+in the GitLab Performance Monitoring database.
+
+**Omnibus Installation**
+
+```
+sudo gitlab-rake gitlab:track_deployment
+```
+
+**Source Installation**
+
+```
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:track_deployment RAILS_ENV=production
+```
+
+## Create or repair repository hooks symlink
+
+If the GitLab shell hooks directory location changes or another circumstance
+leads to the hooks symlink becoming missing or invalid, run this Rake task
+to create or repair the symlinks.
+
+**Omnibus Installation**
+
+```
+sudo gitlab-rake gitlab:shell:create_hooks
+```
+
+**Source Installation**
+
+```
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:shell:create_hooks RAILS_ENV=production
+```

doc/development/testing.md

@@ -132,6 +132,42 @@ Adding new Spinach scenarios is acceptable _only if_ the new scenario requires
 no more than one new `step` definition. If more than that is required, the
 test should be re-implemented using RSpec instead.
 
+## Testing Rake Tasks
+
+To make testing Rake tasks a little easier, there is a helper that can be included
+in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use
+`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures
+a few other things to make testing Rake tasks easier.
+
+At a minimum, requiring the Rake helper will redirect `stdout`, include the
+runtime task helpers, and include the `RakeHelpers` Spec support module.
+
+The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make
+executing tasks simple. See `spec/support/rake_helpers.rb` for all available
+methods.
+
+Example:
+
+```ruby
+require 'rake_helper'
+
+describe 'gitlab:shell rake tasks' do
+  before do
+    Rake.application.rake_require 'tasks/gitlab/shell'
+
+    stub_warn_user_is_not_gitlab
+  end
+
+ describe 'install task' do
+    it 'invokes create_hooks task' do
+      expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke)
+
+      run_rake_task('gitlab:shell:install')
+    end
+  end
+end
+```
+
 ---
 
 [Return to Development documentation](README.md)

doc/raketasks/maintenance.md

-# Maintenance
+# Maintenance Rake Tasks
 
-## Gather information about GitLab and the system it runs on
-
-This command gathers information about your GitLab installation and the System it runs on. These may be useful when asking for help or reporting issues.
-
-```
-# omnibus-gitlab
-sudo gitlab-rake gitlab:env:info
-
-# installation from source
-bundle exec rake gitlab:env:info RAILS_ENV=production
-```
-
-Example output:
-
-```
-System information
-System:           Debian 7.8
-Current User:     git
-Using RVM:        no
-Ruby Version:     2.1.5p273
-Gem Version:      2.4.3
-Bundler Version:  1.7.6
-Rake Version:     10.3.2
-Sidekiq Version:  2.17.8
-
-GitLab information
-Version:          7.7.1
-Revision:         41ab9e1
-Directory:        /home/git/gitlab
-DB Adapter:       postgresql
-URL:              https://gitlab.example.com
-HTTP Clone URL:   https://gitlab.example.com/some-project.git
-SSH Clone URL:    git@gitlab.example.com:some-project.git
-Using LDAP:       no
-Using Omniauth:   no
-
-GitLab Shell
-Version:          2.4.1
-Repositories:     /home/git/repositories/
-Hooks:            /home/git/gitlab-shell/hooks/
-Git:              /usr/bin/git
-```
-
-## Check GitLab configuration
-
-Runs the following rake tasks:
-
-- `gitlab:gitlab_shell:check`
-- `gitlab:sidekiq:check`
-- `gitlab:app:check`
-
-It will check that each component was setup according to the installation guide and suggest fixes for issues found.
-
-You may also have a look at our [Trouble Shooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide).
-
-```
-# omnibus-gitlab
-sudo gitlab-rake gitlab:check
-
-# installation from source
-bundle exec rake gitlab:check RAILS_ENV=production
-```
-
-NOTE: Use SANITIZE=true for gitlab:check if you want to omit project names from the output.
-
-Example output:
-
-```
-Checking Environment ...
-
-Git configured for git user? ... yes
-Has python2? ... yes
-python2 is supported version? ... yes
-
-Checking Environment ... Finished
-
-Checking GitLab Shell ...
-
-GitLab Shell version? ... OK (1.2.0)
-Repo base directory exists? ... yes
-Repo base directory is a symlink? ... no
-Repo base owned by git:git? ... yes
-Repo base access is drwxrws---? ... yes
-post-receive hook up-to-date? ... yes
-post-receive hooks in repos are links: ... yes
-
-Checking GitLab Shell ... Finished
-
-Checking Sidekiq ...
-
-Running? ... yes
-
-Checking Sidekiq ... Finished
-
-Checking GitLab ...
-
-Database config exists? ... yes
-Database is SQLite ... no
-All migrations up? ... yes
-GitLab config exists? ... yes
-GitLab config outdated? ... no
-Log directory writable? ... yes
-Tmp directory writable? ... yes
-Init script exists? ... yes
-Init script up-to-date? ... yes
-Redis version >= 2.0.0? ... yes
-
-Checking GitLab ... Finished
-```
-
-## Rebuild authorized_keys file
-
-In some case it is necessary to rebuild the `authorized_keys` file.
-
-For Omnibus-packages:
-```
-sudo gitlab-rake gitlab:shell:setup
-```
-
-For installations from source:
-```
-cd /home/git/gitlab
-sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production
-```
-
-```
-This will rebuild an authorized_keys file.
-You will lose any data stored in authorized_keys file.
-Do you want to continue (yes/no)? yes
-```
-
-## Clear redis cache
-
-If for some reason the dashboard shows wrong information you might want to
-clear Redis' cache.
-
-For Omnibus-packages:
-```
-sudo gitlab-rake cache:clear
-```
-
-For installations from source:
-```
-cd /home/git/gitlab
-sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production
-```
-
-## Precompile the assets
-
-Sometimes during version upgrades you might end up with some wrong CSS or
-missing some icons. In that case, try to precompile the assets again.
-
-Note that this only applies to source installations and does NOT apply to
-omnibus packages.
-
-For installations from source:
-```
-cd /home/git/gitlab
-sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production
-```
-
-For omnibus versions, the unoptimized assets (JavaScript, CSS) are frozen at
-the release of upstream GitLab. The omnibus version includes optimized versions
-of those assets. Unless you are modifying the JavaScript / CSS code on your
-production machine after installing the package, there should be no reason to redo
-rake assets:precompile on the production machine. If you suspect that assets
-have been corrupted, you should reinstall the omnibus package.
-
-## Tracking Deployments
-
-GitLab provides a Rake task that lets you track deployments in GitLab
-Performance Monitoring. This Rake task simply stores the current GitLab version
-in the GitLab Performance Monitoring database.
-
-For Omnibus-packages:
-
-```
-sudo gitlab-rake gitlab:track_deployment
-```
-
-For installations from source:
-
-```
-cd /home/git/gitlab
-sudo -u git -H bundle exec rake gitlab:track_deployment RAILS_ENV=production
-```
+This document was moved to [administration/raketasks/maintenance](../administration/raketasks/maintenance.md).

features/steps/dashboard/help.rb

@@ -8,7 +8,7 @@ class Spinach::Features::DashboardHelp < Spinach::FeatureSteps
   end
 
   step 'I visit the "Rake Tasks" help page' do
-    visit help_page_path("raketasks/maintenance")
+    visit help_page_path("administration/raketasks/maintenance")
   end
 
   step 'I should see "Rake Tasks" page markdown rendered' do

lib/tasks/gitlab/shell.rake

@@ -63,11 +63,11 @@ namespace :gitlab do
 
         # Launch installation process
         system(*%W(bin/install) + repository_storage_paths_args)
-
-        # (Re)create hooks
-        system(*%W(bin/create-hooks) + repository_storage_paths_args)
       end
 
+      # (Re)create hooks
+      Rake::Task['gitlab:shell:create_hooks'].invoke
+
       # Required for debian packaging with PKGR: Setup .ssh/environment with
       # the current PATH, so that the correct ruby version gets loaded
       # Requires to set "PermitUserEnvironment yes" in sshd config (should not
@@ -102,6 +102,15 @@ namespace :gitlab do
         end
       end
     end
+
+    desc 'Create or repair repository hooks symlink'
+    task create_hooks: :environment do
+      warn_user_is_not_gitlab
+
+      puts 'Creating/Repairing hooks symlinks for all repositories'
+      system(*%W(#{Gitlab.config.gitlab_shell.path}/bin/create-hooks) + repository_storage_paths_args)
+      puts 'done'.color(:green)
+    end
   end
 
   def setup

spec/rake_helper.rb

+require 'spec_helper'
+require 'rake'
+
+RSpec.configure do |config|
+  config.include RakeHelpers
+
+  # Redirect stdout so specs don't have so much noise
+  config.before(:all) do
+    $stdout = StringIO.new
+
+    Rake.application.rake_require 'tasks/gitlab/task_helpers'
+    Rake::Task.define_task :environment
+  end
+
+  # Reset stdout
+  config.after(:all) do
+    $stdout = STDOUT
+  end
+end

spec/support/rake_helpers.rb

+module RakeHelpers
+  def run_rake_task(task_name)
+    Rake::Task[task_name].reenable
+    Rake.application.invoke_task task_name
+  end
+
+  def stub_warn_user_is_not_gitlab
+    allow_any_instance_of(Object).to receive(:warn_user_is_not_gitlab)
+  end
+end

spec/tasks/gitlab/shell_rake_spec.rb

+require 'rake_helper'
+
+describe 'gitlab:shell rake tasks' do
+  before do
+    Rake.application.rake_require 'tasks/gitlab/shell'
+
+    stub_warn_user_is_not_gitlab
+  end
+
+  describe 'install task' do
+    it 'invokes create_hooks task' do
+      expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke)
+
+      run_rake_task('gitlab:shell:install')
+    end
+  end
+
+  describe 'create_hooks task' do
+    it 'calls gitlab-shell bin/create_hooks' do
+      expect_any_instance_of(Object).to receive(:system)
+        .with("#{Gitlab.config.gitlab_shell.path}/bin/create-hooks", *repository_storage_paths_args)
+
+      run_rake_task('gitlab:shell:create_hooks')
+    end
+  end
+end