Merge master from CE

Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

CHANGELOG

 Please view this file on the master branch, on stable branches it's out of date.
 
 v 7.14.0 (unreleased)
+  - Provide more feedback what went wrong if HipChat service failed test (Stan Hu)
+  - Disable turbolinks when linking to Bitbucket import status (Stan Hu)
+  - Fix broken code import and display error messages if something went wrong with creating project (Stan Hu)
+  - Fix corrupted binary files when using API files endpoint (Stan Hu)
+  - Bump Haml to 4.0.7 to speed up textarea rendering (Stan Hu)
   - Show incompatible projects in Bitbucket import status (Stan Hu)
   - Fix coloring of diffs on MR Discussion-tab (Gert Goet)
   - Fix "Network" and "Graphs" pages for branches with encoded slashes (Stan Hu)
@@ -24,11 +29,10 @@ v 7.14.0 (unreleased)
   - Fix file upload dialog for comment editing (Daniel Gerhardt)
   - Set OmniAuth full_host parameter to ensure redirect URIs are correct (Stan Hu)
   - Return comments in created order in merge request API (Stan Hu)
+  - Disable internal issue tracker controller if external tracker is used (Stan Hu)
   - Expire Rails cache entries after two weeks to prevent endless Redis growth
   - Add support for destroying project milestones (Stan Hu)
-  - Add fetch command to the MR page.
   - Allow custom backup archive permissions
-  - Add fetch command to the MR page
   - Add project star and fork count, group avatar URL and user/group web URL attributes to API
   - Fix bug causing Bitbucket importer to crash when OAuth application had been removed.
   - Add fetch command to the MR page.
@@ -46,15 +50,34 @@ v 7.14.0 (unreleased)
   - Remove redis-store TTL monkey patch
   - Add support for CI skipped status
   - Fetch code from forks to refs/merge-requests/:id/head when merge request created
-  - Remove satellites
   - Remove comments and email addresses when publicly exposing ssh keys (Zeger-Jan van de Weg)
+  - Add "Check out branch" button to the MR page.
   - Improve MR merge widget text and UI consistency.
   - Improve text in MR "How To Merge" modal.
   - Cache all events
+  - Order commits by date when comparing branches
+  - Fix bug causing error when the target branch of a symbolic ref was deleted
+  - Include branch/tag name in archive file and directory name
+  - Add dropzone upload progress
+  - Add a label for merged branches on branches page (Florent Baldino)
+  - Detect .mkd and .mkdn files as markdown (Ben Boeckel)
+  - Fix: User search feature in admin area does not respect filters
+  - Set max-width for README, issue and merge request description for easier read on big screens
+  - Update Flowdock integration to support new Flowdock API (Boyan Tabakov)
+  - Remove author from files view (Sven Strickroth)
+
+v 7.13.5
+  - Satellites reverted
+
+v 7.13.4
+  - Allow users to send abuse reports
+  - Fix redirection after sign in when using auto_sign_in_with_provider
 
 v 7.13.3
   - Fix bug causing Bitbucket importer to crash when OAuth application had been removed.
   - Allow users to send abuse reports
+  - Remove satellites 
+  - Link username to profile on Group Members page (Tom Webster)
 
 v 7.13.2
   - Fix randomly failed spec
@@ -196,7 +219,6 @@ v 7.12.0
   - Add SAML support as an omniauth provider
   - Allow to configure a URL to show after sign out
   - Add an option to automatically sign-in with an Omniauth provider
-  - Better performance for web editor (switched from satellites to rugged)
   - GitLab CI service sends .gitlab-ci.yml in each push call
   - When remove project - move repository and schedule it removal
   - Improve group removing logic

Gemfile

@@ -24,7 +24,7 @@ gem 'omniauth-shibboleth'
 gem 'omniauth-kerberos', group: :kerberos
 gem 'omniauth-gitlab'
 gem 'omniauth-bitbucket'
-gem 'omniauth-saml'
+gem 'omniauth-saml', '~> 1.4.0'
 gem 'doorkeeper', '2.1.3'
 gem "rack-oauth2", "~> 1.0.5"
 
@@ -38,7 +38,7 @@ gem "browser", '~> 0.8.0'
 
 # Extracting information from a git repository
 # Provide access to Gitlab::Git library
-gem "gitlab_git", '~> 7.2.6'
+gem "gitlab_git", '~> 7.2.12'
 
 # Ruby/Rack Git Smart-HTTP Server Handler
 # GitLab fork with a lot of changes (improved thread-safety, better memory usage etc)
@@ -151,7 +151,7 @@ gem 'tinder', '~> 1.9.2'
 gem 'hipchat', '~> 1.5.0'
 
 # Flowdock integration
-gem "gitlab-flowdock-git-hook", "~> 0.4.2"
+gem "gitlab-flowdock-git-hook", "~> 1.0.1"
 
 # Gemnasium integration
 gem "gemnasium-gitlab-service", "~> 0.2"

Gemfile.lock

@@ -183,6 +183,9 @@ GEM
     ffi (1.9.8)
     fission (0.5.0)
       CFPropertyList (~> 2.2)
+    flowdock (0.7.0)
+      httparty (~> 0.7)
+      multi_json
     fog (1.25.0)
       fog-brightbox (~> 0.4)
       fog-core (~> 1.25)
@@ -255,7 +258,8 @@ GEM
       racc
     github-markup (1.3.1)
       posix-spawn (~> 0.3.8)
-    gitlab-flowdock-git-hook (0.4.2.2)
+    gitlab-flowdock-git-hook (1.0.1)
+      flowdock (~> 0.7)
       gitlab-grit (>= 2.4.1)
       multi_json
     gitlab-grack (2.0.2)
@@ -272,7 +276,7 @@ GEM
       mime-types (~> 1.19)
     gitlab_emoji (0.1.0)
       gemojione (~> 2.0)
-    gitlab_git (7.2.6)
+    gitlab_git (7.2.12)
       activesupport (~> 4.0)
       charlock_holmes (~> 0.6)
       gitlab-linguist (~> 3.0)
@@ -308,7 +312,7 @@ GEM
     grape-entity (0.4.2)
       activesupport
       multi_json (>= 1.3.2)
-    haml (4.0.5)
+    haml (4.0.7)
       tilt
     haml-rails (0.5.3)
       actionpack (>= 4.0.1)
@@ -423,9 +427,9 @@ GEM
     omniauth-oauth2 (1.1.1)
       oauth2 (~> 0.8.0)
       omniauth (~> 1.0)
-    omniauth-saml (1.3.1)
+    omniauth-saml (1.4.1)
       omniauth (~> 1.1)
-      ruby-saml (~> 0.8.1)
+      ruby-saml (~> 1.0.0)
     omniauth-shibboleth (1.1.1)
       omniauth (>= 1.0.0)
     omniauth-twitter (1.0.1)
@@ -569,8 +573,8 @@ GEM
       rainbow (>= 1.99.1, < 3.0)
       ruby-progressbar (~> 1.4)
     ruby-progressbar (1.7.1)
-    ruby-saml (0.8.2)
-      nokogiri (>= 1.5.0)
+    ruby-saml (1.0.0)
+      nokogiri (>= 1.5.10)
       uuid (~> 2.3)
     ruby2ruby (2.1.3)
       ruby_parser (~> 3.1)
@@ -710,7 +714,7 @@ GEM
       raindrops (~> 0.7)
     unicorn-worker-killer (0.4.2)
       unicorn (~> 4)
-    uuid (2.3.7)
+    uuid (2.3.8)
       macaddr (~> 1.0)
     version_sorter (2.0.0)
     virtus (1.0.1)
@@ -780,12 +784,12 @@ DEPENDENCIES
   fuubar (~> 2.0.0)
   gemnasium-gitlab-service (~> 0.2)
   github-markup
-  gitlab-flowdock-git-hook (~> 0.4.2)
+  gitlab-flowdock-git-hook (~> 1.0.1)
   gitlab-grack (~> 2.0.2)
   gitlab-license (~> 0.0.2)
   gitlab-linguist (~> 3.0.1)
   gitlab_emoji (~> 0.1)
-  gitlab_git (~> 7.2.6)
+  gitlab_git (~> 7.2.12)
   gitlab_meta (= 7.0)
   gitlab_omniauth-ldap (= 1.2.1)
   gollum-lib (~> 4.0.2)
@@ -816,7 +820,7 @@ DEPENDENCIES
   omniauth-gitlab
   omniauth-google-oauth2
   omniauth-kerberos
-  omniauth-saml
+  omniauth-saml (~> 1.4.0)
   omniauth-shibboleth
   omniauth-twitter
   org-ruby (= 0.9.12)
@@ -878,4 +882,4 @@ DEPENDENCIES
   wikicloth (= 0.8.1)
 
 BUNDLED WITH
-   1.10.4
+   1.10.6

VERSION

-7.13.0.pre-ee
+7.14.0.pre-ee

app/assets/javascripts/merge_request_widget.js.coffee

@@ -19,7 +19,7 @@ class @MergeRequestWidget
           when 'merged'
             location.reload()
           else
-            setTimeout(merge_request_widget.mergeInProgress, 2000)
+            setTimeout(merge_request_widget.mergeInProgress, 3000)
       dataType: 'json'
 
   getMergeStatus: ->

app/assets/javascripts/project.js.coffee

@@ -24,8 +24,3 @@ class @Project
       $.cookie('hide_no_password_message', 'false', { path: path })
       $(@).parents('.no-password-message').remove()
       e.preventDefault()
-
-    $('.js-toggle-clone-holder').on 'click', (e) ->
-      cloneHolder.toggle()
-
-    cloneHolder.hide() unless $('.empty-project').length

app/assets/stylesheets/base/variables.scss

@@ -13,6 +13,7 @@ $code_line_height: 1.5;
 $border-color: #E5E5E5;
 $background-color: #f5f5f5;
 $header-height: 50px;
+$readable-width: 1100px;
 
 
 /*

app/assets/stylesheets/generic/files.scss

@@ -90,12 +90,7 @@
           border-right: none;
         }
         background: #fff;
-        padding: 5px;
-      }
-      .author,
-      .blame_commit {
-        background: $background-color;
-        vertical-align: top;
+        padding: 8px;
       }
       .lines {
         pre {

app/assets/stylesheets/pages/issuable.scss

@@ -45,3 +45,9 @@
 
   .btn { font-size: 13px; }
 }
+
+.issuable-details {
+  .description {
+    max-width: $readable-width;
+  }
+}

app/assets/stylesheets/pages/issues.scss

@@ -4,14 +4,25 @@
     position: relative;
 
     .issue-title {
-      margin-bottom: 5px;
       font-size: $list-font-size;
+      margin-bottom: 5px;
       font-weight: bold;
+      float: left;
+      width: 97.7%;
+    }
+    
+    @media screen and (max-width: 1100px) {
+      .issues-list .issue .issue-title { 
+        width: 97%;
+      }
     }
 
     .issue-info {
       color: #999;
       font-size: 13px;
+      display: inline-block;
+      width: 100%;
+      padding: 0 0 0 20px;
     }
 
     .issue-check {

app/assets/stylesheets/pages/merge_requests.scss

@@ -186,3 +186,7 @@
 #modal_merge_info .modal-dialog {
   width: 600px;
 }
+
+.mr-source-target {
+  line-height: 31px;
+}

app/assets/stylesheets/pages/tree.scss

@@ -89,6 +89,10 @@
     td.blame-commit {
       background: #f9f9f9;
       min-width: 350px;
+
+      .commit-author-link {
+        color: #888;
+      }
     }
     td.blame-numbers {
       pre {
@@ -112,6 +116,9 @@
 }
 
 .readme-holder {
+  margin: 0 auto;
+  max-width: $readable-width;
+
   .readme-file-title {
     font-size: 14px;
     font-weight: bold;

app/controllers/projects/blame_controller.rb

@@ -7,7 +7,7 @@ class Projects::BlameController < Projects::ApplicationController
   before_action :authorize_download_code!
 
   def show
+    @blob = @repository.blob_at(@commit.id, @path)
     @blame = Gitlab::Git::Blame.new(@repository, @commit.id, @path)
-    @blob = @blame.blob
   end
 end

app/controllers/projects/blob_controller.rb

@@ -13,20 +13,27 @@ class Projects::BlobController < Projects::ApplicationController
   before_action :commit, except: [:new, :create]
   before_action :blob, except: [:new, :create]
   before_action :from_merge_request, only: [:edit, :update]
-  before_action :require_branch_head, only: [:edit, :update]
-  before_action :editor_variables, except: [:show, :preview, :diff]
   before_action :after_edit_path, only: [:edit, :update]
+  before_action :require_branch_head, only: [:edit, :update]
 
   def new
     commit unless @repository.empty?
   end
 
   def create
-    result = Files::CreateService.new(@project, current_user, @commit_params).execute
+    file_path = File.join(@path, File.basename(params[:file_name]))
+    result = Files::CreateService.new(
+      @project,
+      current_user,
+      params.merge(new_branch: sanitized_new_branch_name),
+      @ref,
+      file_path
+    ).execute
 
     if result[:status] == :success
       flash[:notice] = "Your changes have been successfully committed"
-      redirect_to namespace_project_blob_path(@project.namespace, @project, File.join(@target_branch, @file_path))
+      ref = sanitized_new_branch_name.presence || @ref
+      redirect_to namespace_project_blob_path(@project.namespace, @project, File.join(ref, file_path))
     else
       flash[:alert] = result[:message]
       render :new
@@ -41,10 +48,22 @@ class Projects::BlobController < Projects::ApplicationController
   end
 
   def update
-    result = Files::UpdateService.new(@project, current_user, @commit_params).execute
+    result = Files::UpdateService.
+      new(
+        @project,
+        current_user,
+        params.merge(new_branch: sanitized_new_branch_name),
+        @ref,
+        @path
+      ).execute
 
     if result[:status] == :success
       flash[:notice] = "Your changes have been successfully committed"
+
+      if from_merge_request
+        from_merge_request.reload_code
+      end
+
       redirect_to after_edit_path
     else
       flash[:alert] = result[:message]
@@ -61,11 +80,12 @@ class Projects::BlobController < Projects::ApplicationController
   end
 
   def destroy
-    result = Files::DeleteService.new(@project, current_user, @commit_params).execute
+    result = Files::DeleteService.new(@project, current_user, params, @ref, @path).execute
 
     if result[:status] == :success
       flash[:notice] = "Your changes have been successfully committed"
-      redirect_to namespace_project_tree_path(@project.namespace, @project, @target_branch)
+      redirect_to namespace_project_tree_path(@project.namespace, @project,
+                                              @ref)
     else
       flash[:alert] = result[:message]
       render :show
@@ -115,6 +135,7 @@ class Projects::BlobController < Projects::ApplicationController
     @id = params[:id]
     @ref, @path = extract_ref(@id)
 
+
   rescue InvalidPathError
     not_found!
   end
@@ -124,8 +145,8 @@ class Projects::BlobController < Projects::ApplicationController
       if from_merge_request
         diffs_namespace_project_merge_request_path(from_merge_request.target_project.namespace, from_merge_request.target_project, from_merge_request) +
           "#file-path-#{hexdigest(@path)}"
-      elsif @target_branch.present?
-        namespace_project_blob_path(@project.namespace, @project, File.join(@target_branch, @path))
+      elsif sanitized_new_branch_name.present?
+        namespace_project_blob_path(@project.namespace, @project, File.join(sanitized_new_branch_name, @path))
       else
         namespace_project_blob_path(@project.namespace, @project, @id)
       end
@@ -139,25 +160,4 @@ class Projects::BlobController < Projects::ApplicationController
   def sanitized_new_branch_name
     @new_branch ||= sanitize(strip_tags(params[:new_branch]))
   end
-
-  def editor_variables
-    @current_branch = @ref
-    @target_branch = (sanitized_new_branch_name || @ref)
-
-    @file_path =
-      if action_name.to_s == 'create'
-        File.join(@path, File.basename(params[:file_name]))
-      else
-        @path
-      end
-
-    @commit_params = {
-      file_path: @file_path,
-      current_branch: @current_branch,
-      target_branch: @target_branch,
-      commit_message: params[:commit_message],
-      file_content: params[:content],
-      file_content_encoding: params[:encoding]
-    }
-  end
 end

app/controllers/projects/compare_controller.rb

@@ -13,8 +13,13 @@ class Projects::CompareController < Projects::ApplicationController
     base_ref = Addressable::URI.unescape(params[:from])
     @ref = head_ref = Addressable::URI.unescape(params[:to])
 
-    compare_result = CompareService.new.
-      execute(@project, head_ref, @project, base_ref)
+    compare_result = CompareService.new.execute(
+      current_user,
+      @project,
+      head_ref,
+      @project,
+      base_ref
+    )
 
     @commits = compare_result.commits
     @diffs = compare_result.diffs

app/controllers/projects/issues_controller.rb

@@ -131,7 +131,7 @@ class Projects::IssuesController < Projects::ApplicationController
   end
 
   def module_enabled
-    return render_404 unless @project.issues_enabled
+    return render_404 unless @project.issues_enabled && @project.default_issues_tracker?
   end
 
   # Since iids are implemented only in 6.1

app/controllers/projects/merge_requests_controller.rb

+require 'gitlab/satellite/satellite'
+
 class Projects::MergeRequestsController < Projects::ApplicationController
   before_action :module_enabled
   before_action :merge_request, only: [
-    :edit, :update, :show, :diffs, :commits, :merge, :merge_check,
+    :edit, :update, :show, :diffs, :commits, :automerge, :automerge_check,
     :ci_status, :toggle_subscription, :approve
   ]
   before_action :closes_issues, only: [:edit, :update, :show, :diffs, :commits]
@@ -139,7 +141,7 @@ class Projects::MergeRequestsController < Projects::ApplicationController
     end
   end
 
-  def merge_check
+  def automerge_check
     if @merge_request.unchecked?
       @merge_request.check_if_can_be_merged
     end
@@ -149,12 +151,12 @@ class Projects::MergeRequestsController < Projects::ApplicationController
     render partial: "projects/merge_requests/widget/show.html.haml", layout: false
   end
 
-  def merge
+  def automerge
     return access_denied! unless @merge_request.can_be_merged_by?(current_user)
     return render_404 unless @merge_request.approved?
 
-    if @merge_request.mergeable?
-      MergeWorker.perform_async(@merge_request.id, current_user.id, params)
+    if @merge_request.automergeable?
+      AutoMergeWorker.perform_async(@merge_request.id, current_user.id, params)
       @status = true
     else
       @status = false

app/controllers/projects/services_controller.rb

@@ -41,10 +41,13 @@ class Projects::ServicesController < Projects::ApplicationController
 
   def test
     data = Gitlab::PushDataBuilder.build_sample(project, current_user)
-    if @service.execute(data)
+    outcome = @service.test(data)
+    if outcome[:success]
       message = { notice: 'We sent a request to the provided URL' }
     else
-      message = { alert: 'We tried to send a request to the provided URL but an error occured' }
+      error_message = "We tried to send a request to the provided URL but an error occurred"
+      error_message << ": #{outcome[:result]}" if outcome[:result].present?
+      message = { alert: error_message }
     end
 
     redirect_to :back, message

app/controllers/sessions_controller.rb

@@ -2,27 +2,10 @@ class SessionsController < Devise::SessionsController
   include AuthenticatesWithTwoFactor
 
   prepend_before_action :authenticate_with_two_factor, only: [:create]
+  prepend_before_action :store_redirect_path, only: [:new]
   before_action :auto_sign_in_with_provider, only: [:new]
 
   def new
-    redirect_path =
-      if request.referer.present? && (params['redirect_to_referer'] == 'yes')
-        referer_uri = URI(request.referer)
-        if referer_uri.host == Gitlab.config.gitlab.host
-          referer_uri.path
-        else
-          request.fullpath
-        end
-      else
-        request.fullpath
-      end
-
-    # Prevent a 'you are already signed in' message directly after signing:
-    # we should never redirect to '/users/sign_in' after signing in successfully.
-    unless redirect_path == new_user_session_path
-      store_location_for(:redirect, redirect_path)
-    end
-
     if Gitlab.config.ldap.enabled
       @ldap_servers = Gitlab::LDAP::Config.servers
     end
@@ -55,6 +38,26 @@ class SessionsController < Devise::SessionsController
       User.find(session[:otp_user_id])
     end
   end
+  
+  def store_redirect_path
+    redirect_path =
+      if request.referer.present? && (params['redirect_to_referer'] == 'yes')
+        referer_uri = URI(request.referer)
+        if referer_uri.host == Gitlab.config.gitlab.host
+          referer_uri.path
+        else
+          request.fullpath
+        end
+      else
+        request.fullpath
+      end
+
+    # Prevent a 'you are already signed in' message directly after signing:
+    # we should never redirect to '/users/sign_in' after signing in successfully.
+    unless redirect_path == new_user_session_path
+      store_location_for(:redirect, redirect_path)
+    end
+  end
 
   def authenticate_with_two_factor
     user = self.resource = find_user

app/helpers/projects_helper.rb

@@ -231,37 +231,20 @@ module ProjectsHelper
     end
   end
 
+  def readme_path(project)
+    filename_path(project, :readme)
+  end
+
   def changelog_path(project)
-    if project && changelog = project.repository.changelog
-      namespace_project_blob_path(
-        project.namespace,
-        project,
-        tree_join(project.default_branch,
-                  changelog.name)
-      )
-    end
+    filename_path(project, :changelog)
   end
 
   def license_path(project)
-    if project && license = project.repository.license
-      namespace_project_blob_path(
-        project.namespace,
-        project,
-        tree_join(project.default_branch,
-                  license.name)
-      )
-    end
+    filename_path(project, :license)
   end
 
   def version_path(project)
-    if project && version = project.repository.version
-      namespace_project_blob_path(
-        project.namespace,
-        project,
-        tree_join(project.default_branch,
-                  version.name)
-      )
-    end
+    filename_path(project, :version)
   end
 
   def hidden_pass_url(original_url)
@@ -339,4 +322,17 @@ module ProjectsHelper
       count
     end
   end
+
+  private
+
+  def filename_path(project, filename)
+    if project && blob = project.repository.send(filename)
+      namespace_project_blob_path(
+          project.namespace,
+          project,
+          tree_join(project.default_branch,
+                    blob.name)
+      )
+    end
+  end
 end

app/models/merge_request.rb

@@ -43,6 +43,8 @@ class MergeRequest < ActiveRecord::Base
 
   delegate :commits, :diffs, :last_commit, :last_commit_short_sha, to: :merge_request_diff, prefix: nil
 
+  attr_accessor :should_remove_source_branch
+
   # When this attribute is true some MR validation is ignored
   # It allows us to close or modify broken merge requests
   attr_accessor :allow_broken
@@ -57,7 +59,7 @@ class MergeRequest < ActiveRecord::Base
       transition [:reopened, :opened] => :closed
     end
 
-    event :mark_as_merged do
+    event :merge do
       transition [:reopened, :opened, :locked] => :merged
     end
 
@@ -205,10 +207,7 @@ class MergeRequest < ActiveRecord::Base
   end
 
   def check_if_can_be_merged
-    can_be_merged =
-      project.repository.can_be_merged?(source_sha, target_branch)
-
-    if can_be_merged
+    if Gitlab::Satellite::MergeAction.new(self.author, self).can_be_merged?
       mark_as_mergeable
     else
       mark_as_unmergeable
@@ -223,6 +222,18 @@ class MergeRequest < ActiveRecord::Base
     self.target_project.events.where(target_id: self.id, target_type: "MergeRequest", action: Event::CLOSED).last
   end
 
+  def automerge!(current_user, commit_message = nil)
+    return unless automergeable?
+
+    MergeRequests::AutoMergeService.
+      new(target_project, current_user).
+      execute(self, commit_message)
+  end
+
+  def remove_source_branch?
+    self.should_remove_source_branch && !self.source_project.root_ref?(self.source_branch) && !self.for_fork?
+  end
+
   def open?
     opened? || reopened?
   end
@@ -231,11 +242,11 @@ class MergeRequest < ActiveRecord::Base
     title =~ /\A\[?WIP\]?:? /i
   end
 
-  def mergeable?
+  def automergeable?
     open? && !work_in_progress? && can_be_merged?
   end
 
-  def gitlab_merge_status
+  def automerge_status
     if work_in_progress?
       "work_in_progress"
     else
@@ -262,14 +273,14 @@ class MergeRequest < ActiveRecord::Base
   #
   # see "git diff"
   def to_diff(current_user)
-    target_project.repository.diff_text(target_branch, source_sha)
+    Gitlab::Satellite::MergeAction.new(current_user, self).diff_in_satellite
   end
 
   # Returns the commit as a series of email patches.
   #
   # see "git format-patch"
   def to_patch(current_user)
-    target_project.repository.format_patch(target_branch, source_sha)
+    Gitlab::Satellite::MergeAction.new(current_user, self).format_patch
   end
 
   def hook_attrs
@@ -472,30 +483,4 @@ class MergeRequest < ActiveRecord::Base
       approvers.find_or_initialize_by(user_id: user_id, target_id: id)
     end
   end
-
-  def target_sha
-    @target_sha ||= target_project.
-      repository.commit(target_branch).sha
-  end
-
-  def source_sha
-    commits.first.sha
-  end
-
-  def fetch_ref
-    target_project.repository.fetch_ref(
-      source_project.repository.path_to_repo,
-      "refs/heads/#{source_branch}",
-      "refs/merge-requests/#{iid}/head"
-    )
-  end
-
-  def in_locked_state
-    begin
-      lock_mr
-      yield
-    ensure
-      unlock_mr if locked?
-    end
-  end
 end

app/models/merge_request_diff.rb

@@ -16,8 +16,9 @@ require Rails.root.join("app/models/commit")
 class MergeRequestDiff < ActiveRecord::Base
   include Sortable
 
-  # Prevent store of diff if commits amount more then 500
-  COMMITS_SAFE_SIZE = 500
+  # Prevent store of diff
+  # if commits amount more then 200
+  COMMITS_SAFE_SIZE = 200
 
   attr_reader :commits, :diffs
 
@@ -123,12 +124,12 @@ class MergeRequestDiff < ActiveRecord::Base
     if new_diffs.any?
       if new_diffs.size > Commit::DIFF_HARD_LIMIT_FILES
         self.state = :overflow_diff_files_limit
-        new_diffs = new_diffs.first[Commit::DIFF_HARD_LIMIT_LINES]
+        new_diffs = []
       end
 
       if new_diffs.sum { |diff| diff.diff.lines.count } > Commit::DIFF_HARD_LIMIT_LINES
         self.state = :overflow_diff_lines_limit
-        new_diffs = new_diffs.first[Commit::DIFF_HARD_LIMIT_LINES]
+        new_diffs = []
       end
     end
 
@@ -159,21 +160,12 @@ class MergeRequestDiff < ActiveRecord::Base
   private
 
   def compare_result
-    @compare_result ||=
-      begin
-        # Update ref for merge request
-        merge_request.fetch_ref
-
-        # Get latest sha of branch from source project
-        source_sha = merge_request.source_project.commit(source_branch).sha
-
-        Gitlab::CompareResult.new(
-          Gitlab::Git::Compare.new(
-            merge_request.target_project.repository.raw_repository,
-            merge_request.target_branch,
-            source_sha,
-          )
-        )
-      end
+    @compare_result ||= CompareService.new.execute(
+      merge_request.author,
+      merge_request.source_project,
+      merge_request.source_branch,
+      merge_request.target_project,
+      merge_request.target_branch,
+    )
   end
 end

app/models/namespace.rb

@@ -118,11 +118,12 @@ class Namespace < ActiveRecord::Base
     gitlab_shell.add_namespace(path_was)
 
     if gitlab_shell.mv_namespace(path_was, path)
-      # If repositories moved successfully we need to
-      # send update instructions to users.
+      # If repositories moved successfully we need to remove old satellites
+      # and send update instructions to users.
       # However we cannot allow rollback since we moved namespace dir
       # So we basically we mute exceptions in next actions
       begin
+        gitlab_shell.rm_satellites(path_was)
         send_update_instructions
       rescue
         # Returning false does not rollback after_* transaction but gives

app/models/note.rb

@@ -360,6 +360,10 @@ class Note < ActiveRecord::Base
     create_new_cross_references!(project, author)
   end
 
+  def system?
+    read_attribute(:system)
+  end
+
   def editable?
     !read_attribute(:system)
   end

app/models/project.rb

@@ -540,6 +540,14 @@ class Project < ActiveRecord::Base
     !repository.exists? || repository.empty?
   end
 
+  def ensure_satellite_exists
+    self.satellite.create unless self.satellite.exists?
+  end
+
+  def satellite
+    @satellite ||= Gitlab::Satellite::Satellite.new(self)
+  end
+
   def repo
     repository.raw
   end
@@ -609,11 +617,14 @@ class Project < ActiveRecord::Base
     new_path_with_namespace = File.join(namespace_dir, path)
 
     if gitlab_shell.mv_repository(old_path_with_namespace, new_path_with_namespace)
-      # If repository moved successfully we need to send update instructions to users.
+      # If repository moved successfully we need to remove old satellite
+      # and send update instructions to users.
       # However we cannot allow rollback since we moved repository
       # So we basically we mute exceptions in next actions
       begin
         gitlab_shell.mv_repository("#{old_path_with_namespace}.wiki", "#{new_path_with_namespace}.wiki")
+        gitlab_shell.rm_satellites(old_path_with_namespace)
+        ensure_satellite_exists
         send_move_instructions
         reset_events_cache
       rescue
@@ -719,6 +730,7 @@ class Project < ActiveRecord::Base
   def create_repository
     if forked?
       if gitlab_shell.fork_repository(forked_from_project.path_with_namespace, self.namespace.path)
+        ensure_satellite_exists
         true
       else
         errors.add(:base, 'Failed to fork repository via gitlab-shell')

app/models/project_services/flowdock_service.rb

@@ -38,7 +38,7 @@ class FlowdockService < Service
 
   def fields
     [
-      { type: 'text', name: 'token',     placeholder: '' }
+      { type: 'text', name: 'token', placeholder: 'Flowdock Git source token' }
     ]
   end
 

app/models/project_services/gitlab_ci_service.rb

@@ -74,8 +74,6 @@ class GitlabCiService < CiService
     else
       :error
     end
-  rescue Errno::ECONNREFUSED
-    :error
   end
 
   def fork_registration(new_project, private_token)
@@ -105,8 +103,6 @@ class GitlabCiService < CiService
     if response.code == 200 and response["coverage"]
       response["coverage"]
     end
-  rescue Errno::ECONNREFUSED
-    nil
   end
 
   def build_page(sha, ref)

app/models/project_services/hipchat_service.rb

@@ -60,6 +60,16 @@ class HipchatService < Service
     gate[room].send('GitLab', message, message_options)
   end
 
+  def test(data)
+    begin
+      result = execute(data)
+    rescue StandardError => error
+      return { success: false, result: error }
+    end
+
+    { success: true, result: result }
+  end
+
   private
 
   def gate

app/models/repository.rb

@@ -364,83 +364,17 @@ class Repository
     @root_ref ||= raw_repository.root_ref
   end
 
-  def commit_file(user, path, content, message, ref)
-    path[0] = '' if path[0] == '/'
+  def merged_to_root_ref?(branch_name)
+    branch_commit = commit(branch_name)
+    root_ref_commit = commit(root_ref)
 
-    committer = user_to_comitter(user)
-    options = {}
-    options[:committer] = committer
-    options[:author] = committer
-    options[:commit] = {
-      message: message,
-      branch: ref
-    }
-
-    options[:file] = {
-      content: content,
-      path: path
-    }
-
-    Gitlab::Git::Blob.commit(raw_repository, options)
-  end
-
-  def remove_file(user, path, message, ref)
-    path[0] = '' if path[0] == '/'
-
-    committer = user_to_comitter(user)
-    options = {}
-    options[:committer] = committer
-    options[:author] = committer
-    options[:commit] = {
-      message: message,
-      branch: ref
-    }
-
-    options[:file] = {
-      path: path
-    }
-
-    Gitlab::Git::Blob.remove(raw_repository, options)
-  end
-
-  def user_to_comitter(user)
-    {
-      email: user.email,
-      name: user.name,
-      time: Time.now
-    }
-  end
-
-  def can_be_merged?(source_sha, target_branch)
-    our_commit = rugged.branches[target_branch].target
-    their_commit = rugged.lookup(source_sha)
-
-    if our_commit && their_commit
-      !rugged.merge_commits(our_commit, their_commit).conflicts?
+    if branch_commit
+      rugged.merge_base(root_ref_commit.id, branch_commit.id) == branch_commit.id
     else
-      false
+      nil
     end
   end
 
-  def merge(source_sha, target_branch, options = {})
-    our_commit = rugged.branches[target_branch].target
-    their_commit = rugged.lookup(source_sha)
-
-    raise "Invalid merge target" if our_commit.nil?
-    raise "Invalid merge source" if their_commit.nil?
-
-    merge_index = rugged.merge_commits(our_commit, their_commit)
-    return false if merge_index.conflicts?
-
-    actual_options = options.merge(
-      parents: [our_commit, their_commit],
-      tree: merge_index.write_tree(rugged),
-      update_ref: "refs/heads/#{target_branch}"
-    )
-
-    Rugged::Commit.create(rugged, actual_options)
-  end
-
   def search_files(query, ref)
     offset = 2
     args = %W(git grep -i -n --before-context #{offset} --after-context #{offset} #{query} #{ref || root_ref})
@@ -474,11 +408,6 @@ class Repository
     )
   end
 
-  def fetch_ref(source_path, source_ref, target_ref)
-    args = %W(git fetch #{source_path} #{source_ref}:#{target_ref})
-    Gitlab::Popen.popen(args, path_to_repo)
-  end
-
   private
 
   def cache

app/models/service.rb

@@ -87,10 +87,16 @@ class Service < ActiveRecord::Base
     %w(push tag_push issue merge_request)
   end
 
-  def execute
+  def execute(data)
     # implement inside child
   end
 
+  def test(data)
+    # default implementation
+    result = execute(data)
+    { success: result.present?, result: result }
+  end
+
   def can_test?
     !project.empty_repo?
   end

app/services/base_service.rb

@@ -31,10 +31,6 @@ class BaseService
     SystemHooksService.new
   end
 
-  def repository
-    project.repository
-  end
-
   # Add an error to the specified model for restricted visibility levels
   def deny_visibility_level(model, denied_visibility_level = nil)
     denied_visibility_level ||= model.visibility_level

app/services/compare_service.rb

-require 'securerandom'
-
 # Compare 2 branches for one repo or between repositories
 # and return Gitlab::CompareResult object that responds to commits and diffs
 class CompareService
-  def execute(source_project, source_branch, target_project, target_branch)
-    source_sha = source_project.commit(source_branch).sha
-
-    # If compare with other project we need to fetch ref first
-    unless target_project == source_project
-      random_string = SecureRandom.hex
-
-      target_project.repository.fetch_ref(
-        source_project.repository.path_to_repo,
-        "refs/heads/#{source_branch}",
-        "refs/tmp/#{random_string}/head"
+  def execute(current_user, source_project, source_branch, target_project, target_branch)
+    # Try to compare branches to get commits list and diffs
+    #
+    # Note: Use satellite only when need to compare between two repos
+    # because satellites are slower than operations on bare repo
+    if target_project == source_project
+      Gitlab::CompareResult.new(
+        Gitlab::Git::Compare.new(
+          target_project.repository.raw_repository,
+          target_branch,
+          source_branch,
+        )
       )
-    end
-
-    Gitlab::CompareResult.new(
-      Gitlab::Git::Compare.new(
-        target_project.repository.raw_repository,
+    else
+      Gitlab::Satellite::CompareAction.new(
+        current_user,
+        target_project,
         target_branch,
-        source_sha,
-      )
-    )
+        source_project,
+        source_branch
+      ).result
+    end
   end
 end

app/services/files/base_service.rb

 module Files
   class BaseService < ::BaseService
-    class ValidationError < StandardError; end
+    attr_reader :ref, :path
 
-    def execute
-      @current_branch = params[:current_branch]
-      @target_branch  = params[:target_branch]
-      @commit_message = params[:commit_message]
-      @file_path      = params[:file_path]
-      @file_content   = if params[:file_content_encoding] == 'base64'
-                          Base64.decode64(params[:file_content])
-                        else
-                          params[:file_content]
-                        end
-
-      # Validate parameters
-      validate
-
-      # Create new branch if it different from current_branch
-      if @target_branch != @current_branch
-        create_target_branch
-      end
-
-      if sha = commit
-        after_commit(sha, @target_branch)
-        success
-      else
-        error("Something went wrong. Your changes were not committed")
-      end
-    rescue ValidationError => ex
-      error(ex.message)
+    def initialize(project, user, params, ref, path = nil)
+      @project, @current_user, @params = project, user, params.dup
+      @ref = ref
+      @path = path
     end
 
     private
@@ -40,53 +17,5 @@ module Files
     def git_hook
       project.git_hook
     end
-
-    def after_commit(sha, branch)
-      PostCommitService.new(project, current_user).execute(sha, branch)
-    end
-
-    def current_branch
-      @current_branch ||= params[:current_branch]
-    end
-
-    def target_branch
-      @target_branch ||= params[:target_branch]
-    end
-
-    def raise_error(message)
-      raise ValidationError.new(message)
-    end
-
-    def validate
-      allowed = ::Gitlab::GitAccess.new(current_user, project).can_push_to_branch?(@target_branch)
-
-      unless allowed
-        raise_error("You are not allowed to push into this branch")
-      end
-
-      if git_hook && !git_hook.commit_message_allowed?(@commit_message)
-        raise_error("Commit message must match next format: #{git_hook.commit_message_regex}")
-      end
-
-      unless project.empty_repo?
-        unless repository.branch_names.include?(@current_branch)
-          raise_error("You can only create files if you are on top of a branch")
-        end
-
-        if @current_branch != @target_branch
-          if repository.branch_names.include?(@target_branch)
-            raise_error("Branch with such name already exists. You need to switch to this branch in order to make changes")
-          end
-        end
-      end
-    end
-
-    def create_target_branch
-      result = CreateBranchService.new(project, current_user).execute(@target_branch, @current_branch)
-
-      unless result[:status] == :success
-        raise_error("Something went wrong when we tried to create #{@target_branch} for you")
-      end
-    end
   end
 end

app/services/files/create_service.rb

 require_relative "base_service"
 
 module Files
-  class CreateService < Files::BaseService
-    def commit
-      repository.commit_file(current_user, @file_path, @file_content, @commit_message, @target_branch)
-    end
+  class CreateService < BaseService
+    def execute
+      allowed = Gitlab::GitAccess.new(current_user, project).can_push_to_branch?(ref)
 
-    def validate
-      super
+      unless allowed
+        return error("You are not allowed to create file in this branch")
+      end
 
-      file_name = File.basename(@file_path)
+      file_name = File.basename(path)
+      file_path = path
 
       unless file_name =~ Gitlab::Regex.file_name_regex
-        raise_error(
+        return error(
           'Your changes could not be committed, because the file name ' +
           Gitlab::Regex.file_name_regex_message
         )
       end
 
-      unless project.empty_repo?
-        blob = repository.blob_at_branch(@current_branch, @file_path)
+      if project.empty_repo?
+        # everything is ok because repo does not have a commits yet
+      else
+        unless repository.branch_names.include?(ref)
+          return error("You can only create files if you are on top of a branch")
+        end
+
+        blob = repository.blob_at_branch(ref, file_path)
 
         if blob
-          raise_error("Your changes could not be committed, because file with such name exists")
+          return error("Your changes could not be committed, because file with such name exists")
         end
       end
+
+
+      new_file_action = Gitlab::Satellite::NewFileAction.new(current_user, project, ref, file_path)
+      created_successfully = new_file_action.commit!(
+        params[:content],
+        params[:commit_message],
+        params[:encoding],
+        params[:new_branch]
+      )
+
+      if created_successfully
+        success
+      else
+        error("Your changes could not be committed, because the file has been changed")
+      end
     end
   end
 end

app/services/files/delete_service.rb

 require_relative "base_service"
 
 module Files
-  class DeleteService < Files::BaseService
-    def commit
-      repository.remove_file(current_user, @file_path, @commit_message, @target_branch)
+  class DeleteService < BaseService
+    def execute
+      allowed = ::Gitlab::GitAccess.new(current_user, project).can_push_to_branch?(ref)
+
+      unless allowed
+        return error("You are not allowed to push into this branch")
+      end
+
+      unless repository.branch_names.include?(ref)
+        return error("You can only create files if you are on top of a branch")
+      end
+
+      blob = repository.blob_at_branch(ref, path)
+
+      unless blob
+        return error("You can only edit text files")
+      end
+
+      delete_file_action = Gitlab::Satellite::DeleteFileAction.new(current_user, project, ref, path)
+
+      deleted_successfully = delete_file_action.commit!(
+        nil,
+        params[:commit_message]
+      )
+
+      if deleted_successfully
+        success
+      else
+        error("Your changes could not be committed, because the file has been changed")
+      end
     end
   end
 end

app/services/files/update_service.rb

 require_relative "base_service"
 
 module Files
-  class UpdateService < Files::BaseService
-    def commit
-      repository.commit_file(current_user, @file_path, @file_content, @commit_message, @target_branch)
+  class UpdateService < BaseService
+    def execute
+      allowed = ::Gitlab::GitAccess.new(current_user, project).can_push_to_branch?(ref)
+
+      unless allowed
+        return error("You are not allowed to push into this branch")
+      end
+
+      unless repository.branch_names.include?(ref)
+        return error("You can only create files if you are on top of a branch")
+      end
+
+      blob = repository.blob_at_branch(ref, path)
+
+      unless blob
+        return error("You can only edit text files")
+      end
+
+      edit_file_action = Gitlab::Satellite::EditFileAction.new(current_user, project, ref, path)
+      edit_file_action.commit!(
+        params[:content],
+        params[:commit_message],
+        params[:encoding],
+        params[:new_branch]
+      )
+
+      success
+    rescue Gitlab::Satellite::CheckoutFailed => ex
+      error("Your changes could not be committed because ref '#{ref}' could not be checked out", 400)
+    rescue Gitlab::Satellite::CommitFailed => ex
+      error("Your changes could not be committed. Maybe there was nothing to commit?", 409)
+    rescue Gitlab::Satellite::PushFailed => ex
+      error("Your changes could not be committed. Maybe the file was changed by another process?", 409)
     end
   end
 end

app/services/git_push_service.rb

@@ -10,14 +10,16 @@ class GitPushService
   #
   # Next, this method:
   #  1. Creates the push event
-  #  2. Updates merge requests
-  #  3. Recognizes cross-references from commit messages
-  #  4. Executes the project's web hooks
-  #  5. Executes the project's services
+  #  2. Ensures that the project satellite exists
+  #  3. Updates merge requests
+  #  4. Recognizes cross-references from commit messages
+  #  5. Executes the project's web hooks
+  #  6. Executes the project's services
   #
   def execute(project, user, oldrev, newrev, ref)
     @project, @user = project, user
 
+    project.ensure_satellite_exists
     project.repository.expire_cache
 
     if push_remove_branch?(ref, newrev)
@@ -131,8 +133,7 @@ class GitPushService
   end
 
   def is_default_branch?(ref)
-    Gitlab::Git.branch_ref?(ref) &&
-      (Gitlab::Git.ref_name(ref) == project.default_branch || project.default_branch.nil?)
+    Gitlab::Git.branch_ref?(ref) && Gitlab::Git.ref_name(ref) == project.default_branch
   end
 
   def commit_user(commit)

app/services/merge_requests/auto_merge_service.rb

+module MergeRequests
+  # AutoMergeService class
+  #
+  # Do git merge in satellite and in case of success
+  # mark merge request as merged and execute all hooks and notifications
+  # Called when you do merge via GitLab UI
+  class AutoMergeService < BaseMergeService
+    def execute(merge_request, commit_message)
+      merge_request.lock_mr
+
+      if Gitlab::Satellite::MergeAction.new(current_user, merge_request).merge!(commit_message)
+        merge_request.merge
+
+        create_merge_event(merge_request, current_user)
+        create_note(merge_request)
+        notification_service.merge_mr(merge_request, current_user)
+        execute_hooks(merge_request, 'merge')
+
+        true
+      else
+        merge_request.unlock_mr
+        false
+      end
+    rescue
+      merge_request.unlock_mr if merge_request.locked?
+      merge_request.mark_as_unmergeable
+      false
+    end
+  end
+end

app/services/merge_requests/base_merge_service.rb

+module MergeRequests
+  class BaseMergeService < MergeRequests::BaseService
+
+    private
+
+    def create_merge_event(merge_request, current_user)
+      EventCreateService.new.merge_mr(merge_request, current_user)
+    end
+  end
+end

app/services/merge_requests/build_service.rb

@@ -12,16 +12,12 @@ module MergeRequests
       merge_request.target_project ||= (project.forked_from_project || project)
       merge_request.target_branch ||= merge_request.target_project.default_branch
 
-      if merge_request.target_branch.blank? || merge_request.source_branch.blank?
-        message =
-          if params[:source_branch] || params[:target_branch]
-            "You must select source and target branch"
-          end
-
-        return build_failed(merge_request, message)
+      unless merge_request.target_branch && merge_request.source_branch
+        return build_failed(merge_request, nil)
       end
 
       compare_result = CompareService.new.execute(
+        current_user,
         merge_request.source_project,
         merge_request.source_branch,
         merge_request.target_project,
@@ -44,6 +40,7 @@ module MergeRequests
           merge_request.compare_diffs = diffs
 
         elsif diffs == false
+          # satellite timeout return false
           merge_request.can_be_created = false
           merge_request.compare_failed = true
         end
@@ -67,6 +64,9 @@ module MergeRequests
       end
 
       merge_request
+
+    rescue Gitlab::Satellite::BranchesWithoutParent
+      return build_failed(merge_request, "Selected branches have no common commit so they cannot be merged.")
     end
 
     def build_failed(merge_request, message)

app/services/merge_requests/merge_service.rb

 module MergeRequests
   # MergeService class
   #
-  # Do git merge and in case of success
-  # mark merge request as merged and execute all hooks and notifications
-  # Executed when you do merge via GitLab UI
-  #
-  class MergeService < MergeRequests::BaseService
-    attr_reader :merge_request, :commit_message
-
+  # Mark existing merge request as merged
+  # and execute all hooks and notifications
+  # Called when you do merge via command line and push code
+  # to target branch
+  class MergeService < BaseMergeService
     def execute(merge_request, commit_message)
-      @commit_message = commit_message
-      @merge_request = merge_request
+      merge_request.merge
 
-      unless @merge_request.mergeable?
-        return error('Merge request is not mergeable')
-      end
-
-      merge_request.in_locked_state do
-        if merge_changes
-          after_merge
-          success
-        else
-          error('Can not merge changes')
-        end
-      end
-    end
-
-    private
-
-    def merge_changes
-      if sha = commit
-        after_commit(sha, merge_request.target_branch)
-      end
-    end
-
-    def commit
-      committer = repository.user_to_comitter(current_user)
-
-      options = {
-        message: commit_message,
-        author: committer,
-        committer: committer
-      }
-
-      repository.merge(merge_request.source_sha, merge_request.target_branch, options)
-    end
-
-    def after_commit(sha, branch)
-      PostCommitService.new(project, current_user).execute(sha, branch)
-    end
+      create_merge_event(merge_request, current_user)
+      create_note(merge_request)
+      notification_service.merge_mr(merge_request, current_user)
+      execute_hooks(merge_request, 'merge')
 
-    def after_merge
-      MergeRequests::PostMergeService.new(project, current_user).execute(merge_request)
+      true
+    rescue
+      false
     end
   end
 end

app/services/merge_requests/post_merge_service.rb

-module MergeRequests
-  # PostMergeService class
-  #
-  # Mark existing merge request as merged
-  # and execute all hooks and notifications
-  #
-  class PostMergeService < MergeRequests::BaseService
-    def execute(merge_request)
-      merge_request.mark_as_merged
-      create_merge_event(merge_request, current_user)
-      create_note(merge_request)
-      notification_service.merge_mr(merge_request, current_user)
-      execute_hooks(merge_request, 'merge')
-    end
-
-    private
-
-    def create_merge_event(merge_request, current_user)
-      EventCreateService.new.merge_mr(merge_request, current_user)
-    end
-  end
-end

app/services/merge_requests/refresh_service.rb

@@ -34,9 +34,9 @@ module MergeRequests
 
 
       merge_requests.uniq.select(&:source_project).each do |merge_request|
-        MergeRequests::PostMergeService.
+        MergeRequests::MergeService.
           new(merge_request.target_project, @current_user).
-          execute(merge_request)
+          execute(merge_request, nil)
       end
     end
 

app/services/post_commit_service.rb

-class PostCommitService < BaseService
-  include Gitlab::Popen
-
-  attr_reader :changes, :repo_path
-
-  def execute(sha, branch)
-    commit = repository.commit(sha)
-    full_ref = Gitlab::Git::BRANCH_REF_PREFIX + branch
-    old_sha = commit.parent_id || Gitlab::Git::BLANK_SHA
-    @changes = "#{old_sha} #{sha} #{full_ref}"
-    @repo_path = repository.path_to_repo
-
-    post_receive
-  end
-
-  private
-
-  def post_receive
-    hook = hook_file('post-receive', repo_path)
-    return true if hook.nil?
-    call_receive_hook(hook)
-  end
-
-  def call_receive_hook(hook)
-    # function  will return true if succesful
-    exit_status = false
-
-    vars = {
-      'GL_ID' => Gitlab::ShellEnv.gl_id(current_user),
-      'PWD' => repo_path
-    }
-
-    options = {
-      chdir: repo_path
-    }
-
-    # we combine both stdout and stderr as we don't know what stream
-    # will be used by the custom hook
-    Open3.popen2e(vars, hook, options) do |stdin, stdout_stderr, wait_thr|
-      exit_status = true
-      stdin.sync = true
-
-      # in git, pre- and post- receive hooks may just exit without
-      # reading stdin. We catch the exception to avoid a broken pipe
-      # warning
-      begin
-        # inject all the changes as stdin to the hook
-        changes.lines do |line|
-          stdin.puts line
-        end
-      rescue Errno::EPIPE
-      end
-
-      # need to close stdin before reading stdout
-      stdin.close
-
-      # only output stdut_stderr if scripts doesn't return 0
-      unless wait_thr.value == 0
-        exit_status = false
-      end
-    end
-
-    exit_status
-  end
-
-  def hook_file(hook_type, repo_path)
-    hook_path = File.join(repo_path.strip, 'hooks')
-    hook_file = "#{hook_path}/#{hook_type}"
-    hook_file if File.exist?(hook_file)
-  end
-end

app/services/projects/destroy_service.rb

@@ -27,6 +27,7 @@ module Projects
         end
       end
 
+      project.satellite.destroy
       log_info("Project \"#{project.name}\" was removed")
       system_hook_service.execute_hooks_for(project, :destroy)
       true

app/services/projects/transfer_service.rb

@@ -33,6 +33,9 @@ module Projects
           raise TransferError.new("Project with same path in target namespace already exists")
         end
 
+        # Remove old satellite
+        project.satellite.destroy
+
         # Apply new namespace id
         project.namespace = new_namespace
         project.save!
@@ -48,6 +51,9 @@ module Projects
         # Move wiki repo also if present
         gitlab_shell.mv_repository("#{old_path}.wiki", "#{new_path}.wiki")
 
+        # Create a new satellite (reload project from DB)
+        Project.find(project.id).ensure_satellite_exists
+
         # clear project cached events
         project.reset_events_cache
 

app/views/admin/users/index.html.haml

@@ -33,6 +33,7 @@
       = form_tag admin_users_path, method: :get, class: 'form-inline' do
         .form-group
           = search_field_tag :name, params[:name], placeholder: 'Name, email or username', class: 'form-control'
+          = hidden_field_tag "filter", params[:filter]
         = button_tag class: 'btn btn-primary' do
           %i.fa.fa-search
       %hr

app/views/groups/group_members/_group_member.html.haml

@@ -6,7 +6,8 @@
   %span{class: ("list-item-name" if show_controls)}
     - if member.user
       = image_tag avatar_icon(user.email, 16), class: "avatar s16", alt: ''
-      %strong= user.name
+      %strong
+        = link_to user.name, user_path(user)
       %span.cgray= user.username
       - if user == current_user
         %span.label.label-success It's you

app/views/import/base/create.js.haml

@@ -14,12 +14,16 @@
   :plain
     job = $("tr#repo_#{@repo_id}")
     job.find(".import-actions").html("<p class='alert alert-danger'>Access denied! Please verify you can add deploy keys to this repository.</p>")
-- else
+- elsif @project.persisted?
   :plain
     job = $("tr#repo_#{@repo_id}")
     job.attr("id", "project_#{@project.id}")
     target_field = job.find(".import-target")
     target_field.empty()
-    target_field.append('<strong>#{link_to @project.path_with_namespace, [@project.namespace.becomes(Namespace), @project]}</strong>')
+    target_field.append('<strong>#{link_to @project.path_with_namespace, namespace_project_path(@project.namespace, @project)}</strong>')
     $("table.import-jobs tbody").prepend(job)
     job.addClass("active").find(".import-actions").html("<i class='fa fa-spinner fa-spin'></i> started")
+- else
+  :plain
+    job = $("tr#repo_#{@repo_id}")
+    job.find(".import-actions").html("<p class='alert alert-danger'>Error saving project: #{escape_javascript(@project.errors.messages.to_s)}</p>")

app/views/import/bitbucket/status.html.haml

@@ -61,7 +61,7 @@
     rather than Git. Please convert
     = link_to "them to Git,", "https://www.atlassian.com/git/tutorials/migrating-overview"
     and go through the
-    = link_to "import flow", status_import_bitbucket_path
+    = link_to "import flow", status_import_bitbucket_path, "data-no-turbolink" => "true"
     again.
 
 

app/views/projects/_home_panel.html.haml

@@ -20,11 +20,6 @@
           Forked from
           = forked_from_project.namespace.try(:name)
 
-      - if can? current_user, :download_code, @project
-        = link_to "#", class: 'btn js-toggle-clone-holder' do
-          = icon('cloud-download fw')
-          Clone
-
       - if can? current_user, :download_code, @project
         = link_to archive_namespace_project_repository_path(@project.namespace, @project, ref: @ref, format: 'zip'), class: 'btn', rel: 'nofollow' do
           = icon('download fw')

app/views/projects/blame/show.html.haml

@@ -12,25 +12,31 @@
         = render "projects/blob/actions"
     .file-content.blame.highlight
       %table
-        - @blame.each do |commit, lines, since|
-          - commit = Commit.new(commit, @project)
+        - current_line = 1
+        - @blame.each do |raw_commit, line|
           %tr
             %td.blame-commit
-              %span.commit
-                = link_to commit.short_id, namespace_project_commit_path(@project.namespace, @project, commit), class: "commit_short_id"
-                 
-                = commit_author_link(commit, avatar: true, size: 16)
-                 
-                = link_to_gfm truncate(commit.title, length: 20), namespace_project_commit_path(@project.namespace, @project, commit.id), class: "row_title"
+              .commit
+                - unless @prev_commit && @prev_commit.sha == raw_commit.sha
+                  - commit = Commit.new(raw_commit, @project)
+                  .commit-row-title
+                    %strong
+                      = link_to_gfm truncate(commit.title, length: 35), namespace_project_commit_path(@project.namespace, @project, commit.id), class: "cdark"
+                    .pull-right
+                      = link_to commit.short_id, namespace_project_commit_path(@project.namespace, @project, commit), class: "monospace"
+                     
+                  .light
+                    = commit_author_link(commit, avatar: false)
+                    authored
+                    #{time_ago_with_tooltip(commit.committed_date)}
+                - @prev_commit = raw_commit
             %td.lines.blame-numbers
               %pre
-                - (since...(since + lines.count)).each do |i|
-                  = i
-                  \
+                = current_line
+                - current_line += 1
             %td.lines
               %pre{class: 'code highlight white'}
                 %code
                   :erb
-                    <% lines.each do |line| %>
-                      <%= highlight(@blob.name, line, nowrap: true, continue: true).html_safe %>
-                    <% end %>
+                    <%= highlight(@blob.name, line, nowrap: true, continue: true).html_safe %>
+

app/views/projects/blob/_editor.html.haml

@@ -12,8 +12,8 @@
         \/
         = text_field_tag 'file_name', params[:file_name], placeholder: "File name",
           required: true, class: 'form-control new-file-name'
-      .pull-right
-        = select_tag :encoding, options_for_select([ "base64", "text" ], "text"), class: 'form-control'
+        .pull-right
+          = select_tag :encoding, options_for_select([ "base64", "text" ], "text"), class: 'form-control'
 
   .file-content.code
     %pre.js-edit-mode-pane#editor

app/views/projects/blob/new.html.haml

@@ -6,12 +6,11 @@
     = render 'shared/commit_message_container', params: params,
              placeholder: 'Add new file'
 
-    - unless @project.empty_repo?
-      .form-group.branch
-        = label_tag 'branch', class: 'control-label' do
-          Branch
-        .col-sm-10
-          = text_field_tag 'new_branch', @ref, class: "form-control"
+    .form-group.branch
+      = label_tag 'branch', class: 'control-label' do
+        Branch
+      .col-sm-10
+        = text_field_tag 'new_branch', @ref, class: "form-control"
 
     = hidden_field_tag 'content', '', id: 'file-content'
     = render 'projects/commit_button', ref: @ref,

app/views/projects/branches/_branch.html.haml

@@ -5,6 +5,11 @@
       %strong.str-truncated= branch.name
       - if branch.name == @repository.root_ref
         %span.label.label-info default
+      - elsif @repository.merged_to_root_ref? branch.name
+        %span.label.label-primary.has_tooltip(title="Merged into #{@repository.root_ref}")
+          %i.fa.fa-check
+          merged
+
       - if @project.protected_branch? branch.name
         %span.label.label-success
           %i.fa.fa-lock

app/views/projects/empty.html.haml

@@ -43,6 +43,8 @@
         cd existing_folder
         git init
         git remote add origin #{ content_tag(:span, default_url_to_repo, class: 'clone')}
+        git add .
+        git commit
         git push -u origin master
 
 - if can? current_user, :remove_project, @project

app/views/projects/merge_requests/_new_compare.html.haml

@@ -35,7 +35,7 @@
     - if @merge_request.compare_failed
       .alert.alert-danger
         %h4 Compare failed
-        %p We can't compare selected branches. It may be because of huge diff. Please try again or select different branches.
+        %p We can't compare selected branches. It may be because of huge diff or satellite timeout. Please try again or select different branches.
     - else
       .light-well
         .center

app/views/projects/merge_requests/_new_submit.html.haml

@@ -24,7 +24,7 @@
         = icon('history')
         Commits
         %span.badge= @commits.size
-    %li.diffs-tab.active
+    %li.diffs-tab
       = link_to url_for(params), data: {target: '#diffs', action: 'diffs', toggle: 'tab'} do
         = icon('list-alt')
         Changes
@@ -33,7 +33,7 @@
   .tab-content
     #commits.commits.tab-pane
       = render "projects/commits/commits", project: @project
-    #diffs.diffs.tab-pane.active
+    #diffs.diffs.tab-pane
       - if @diffs.present?
         = render "projects/diffs/diffs", diffs: @diffs, project: @project
       - elsif @commits.size > MergeRequestDiff::COMMITS_SAFE_SIZE

app/views/projects/merge_requests/_show.html.haml

@@ -5,30 +5,36 @@
     %hr
     = render "projects/merge_requests/show/mr_box"
     %hr
-    .append-bottom-20
+    .append-bottom-20.mr-source-target
       - if @merge_request.open?
-        .btn-group.btn-group-sm.pull-right
-          %a.btn.btn-sm.dropdown-toggle{ data: {toggle: :dropdown} }
-            = icon('download')
-            Download as
-            %span.caret
-          %ul.dropdown-menu
-            %li= link_to "Email Patches", merge_request_path(@merge_request, format: :patch)
-            %li= link_to "Plain Diff",    merge_request_path(@merge_request, format: :diff)
+        .pull-right
+          - if @merge_request.source_branch_exists?
+            = link_to "#modal_merge_info", class: "btn btn-sm", "data-toggle" => "modal" do
+              = icon('cloud-download fw')
+              Check out branch
+
+          %span.dropdown
+            %a.btn.btn-sm.dropdown-toggle{ data: {toggle: :dropdown} }
+              = icon('download')
+              Download as
+              %span.caret
+            %ul.dropdown-menu
+              %li= link_to "Email Patches", merge_request_path(@merge_request, format: :patch)
+              %li= link_to "Plain Diff",    merge_request_path(@merge_request, format: :diff)
       .light
-        %div
-          %span From
-          %span.label-branch #{source_branch_with_namespace(@merge_request)}
-          %span into
-          %span.label-branch #{@merge_request.target_branch}
-        - if @merge_request.open? && !@merge_request.branch_missing?
-          %div
-            If you want to try or merge this request manually, you can use the
-            = link_to "command line", "#modal_merge_info", class: "how_to_merge_link vlink", title: "How To Merge", "data-toggle" => "modal"
+        %span Request to merge
+        %span.label-branch #{source_branch_with_namespace(@merge_request)}
+        %span into
+        %span.label-branch #{@merge_request.target_branch}
 
     = render "projects/merge_requests/show/how_to_merge"
     = render "projects/merge_requests/widget/show.html.haml"
 
+    - if @merge_request.open? && @merge_request.can_be_merged?
+      .light
+        You can also accept this merge request manually using the
+        = link_to "command line", "#modal_merge_info", class: "how_to_merge_link vlink", title: "How To Merge", "data-toggle" => "modal"
+
   - if @commits.present?
     %ul.nav.nav-tabs.merge-request-tabs
       %li.notes-tab

app/views/projects/merge_requests/show/_commits.html.haml

-= render "projects/commits/commits", project: @merge_request.project
+= render "projects/commits/commits", project: @merge_request.source_project

app/views/projects/merge_requests/show/_diffs.html.haml

 - if @merge_request_diff.collected?
-  = render "projects/diffs/diffs", diffs: @merge_request.diffs, project: @merge_request.project
+  = render "projects/diffs/diffs", diffs: @merge_request.diffs, project: @merge_request.source_project
 - elsif @merge_request_diff.empty?
   .nothing-here-block Nothing to merge from #{@merge_request.source_branch} into #{@merge_request.target_branch}
 - else

app/views/projects/merge_requests/widget/_open.html.haml

@@ -3,6 +3,8 @@
   .mr-widget-body
     - if @project.archived?
       = render 'projects/merge_requests/widget/open/archived'
+    - elsif !@project.satellite.exists?
+      = render 'projects/merge_requests/widget/open/no_satellite'
     - elsif @merge_request.commits.blank?
       = render 'projects/merge_requests/widget/open/nothing'
     - elsif @merge_request.branch_missing?

app/views/projects/merge_requests/widget/_show.html.haml

@@ -11,10 +11,10 @@
   var merge_request_widget;
 
   merge_request_widget = new MergeRequestWidget({
-    url_to_automerge_check: "#{merge_check_namespace_project_merge_request_path(@project.namespace, @project, @merge_request)}",
+    url_to_automerge_check: "#{automerge_check_namespace_project_merge_request_path(@project.namespace, @project, @merge_request)}",
     check_enable: #{@merge_request.unchecked? ? "true" : "false"},
     url_to_ci_check: "#{ci_status_namespace_project_merge_request_path(@project.namespace, @project, @merge_request)}",
     ci_enable: #{@project.ci_service ? "true" : "false"},
-    current_status: "#{@merge_request.gitlab_merge_status}",
+    current_status: "#{@merge_request.automerge_status}",
   });
 

app/views/projects/merge_requests/widget/open/_accept.html.haml

-= form_for [:merge, @project.namespace.becomes(Namespace), @project, @merge_request], remote: true, method: :post, html: { class: 'accept-mr-form js-requires-input' } do |f|
+= form_for [:automerge, @project.namespace.becomes(Namespace), @project, @merge_request], remote: true, method: :post, html: { class: 'accept-mr-form js-requires-input' } do |f|
   = hidden_field_tag :authenticity_token, form_authenticity_token
   .accept-merge-holder.clearfix.js-toggle-container
     .accept-action
@@ -29,4 +29,3 @@
       btn = $('.accept_merge_request')
       btn.disable()
       btn.html("<i class='fa fa-spinner fa-spin'></i> Merge in progress")
-

app/views/projects/merge_requests/widget/open/_no_satellite.html.haml

+%p
+  %span
+    %strong This repository does not have a satellite. Please ask an administrator to fix this issue!

app/views/projects/new.html.haml

@@ -40,7 +40,7 @@
 
 
             - if bitbucket_import_enabled?
-              = link_to status_import_bitbucket_path, class: 'btn' do
+              = link_to status_import_bitbucket_path, class: 'btn', "data-no-turbolink" => "true" do
                 %i.fa.fa-bitbucket
                 Bitbucket
             - else

app/views/projects/show.html.haml

@@ -23,18 +23,21 @@
       = link_to namespace_project_tags_path(@project.namespace, @project) do
         = pluralize(number_with_delimiter(@repository.tag_names.count), 'tag')
 
+    - if !prefer_readme? && @repository.readme
+      %li
+        = link_to 'Readme', readme_path(@project)
+
     - if @repository.changelog
       %li
-        = link_to changelog_path(@project) do
-          Changelog
+        = link_to 'Changelog', changelog_path(@project)
+
     - if @repository.license
       %li
-        = link_to license_path(@project) do
-          License
+        = link_to 'License', license_path(@project)
+
     - if @repository.contribution_guide
       %li
-        = link_to contribution_guide_path(@project) do
-          Contribution guide
+        = link_to 'Contribution guide', contribution_guide_path(@project)
 
     - if current_user && can_push_branch?(@project, @project.default_branch)
       - unless @repository.changelog

app/views/projects/tree/_tree_commit_column.html.haml

 %span.str-truncated
-  %span.tree_author= commit_author_link(commit, avatar: true, size: 16)
   = link_to_gfm commit.title, namespace_project_commit_path(@project.namespace, @project, commit.id), class: "tree-commit-link"

app/workers/merge_worker.rb → app/workers/auto_merge_worker.rb

-class MergeWorker
+class AutoMergeWorker
   include Sidekiq::Worker
 
   sidekiq_options queue: :default
@@ -7,13 +7,7 @@ class MergeWorker
     params = params.with_indifferent_access
     current_user = User.find(current_user_id)
     merge_request = MergeRequest.find(merge_request_id)
-
-    result = MergeRequests::MergeService.new(merge_request.target_project, current_user).
-      execute(merge_request, params[:commit_message])
-
-    if result[:status] == :success && params[:should_remove_source_branch].present?
-      DeleteBranchService.new(merge_request.source_project, current_user).
-        execute(merge_request.source_branch)
-    end
+    merge_request.should_remove_source_branch = params[:should_remove_source_branch]
+    merge_request.automerge!(current_user, params[:commit_message])
   end
 end

app/workers/repository_import_worker.rb

@@ -27,6 +27,7 @@ class RepositoryImportWorker
 
     project.import_finish
     project.save
+    project.satellite.create unless project.satellite.exists?
     ProjectCacheWorker.perform_async(project.id)
     Gitlab::BitbucketImport::KeyDeleter.new(project).execute if project.import_type == 'bitbucket'
   end

config/routes.rb

@@ -499,8 +499,8 @@ Gitlab::Application.routes.draw do
           member do
             get :diffs
             get :commits
-            post :merge
-            get :merge_check
+            post :automerge
+            get :automerge_check
             get :ci_status
             post :toggle_subscription
             post :approve

doc/api/notes.md

@@ -31,7 +31,10 @@ Parameters:
       "state": "active",
       "created_at": "2013-09-30T13:46:01Z"
     },
-    "created_at": "2013-10-02T09:22:45Z"
+    "created_at": "2013-10-02T09:22:45Z",
+    "system": true,
+    "upvote": false,
+    "downvote": false
   },
   {
     "id": 305,
@@ -45,7 +48,10 @@ Parameters:
       "state": "active",
       "created_at": "2013-09-30T13:46:01Z"
     },
-    "created_at": "2013-10-02T09:56:03Z"
+    "created_at": "2013-10-02T09:56:03Z",
+    "system": false,
+    "upvote": false,
+    "downvote": false
   }
 ]
 ```

doc/customization/libravatar.md

 # Use Libravatar service with GitLab
 
-GitLab by default supports [Gravatar](gravatar.com) avatar service.
+GitLab by default supports [Gravatar](https://gravatar.com) avatar service.
 Libravatar is a service which delivers your avatar (profile picture) to other websites and their API is
 [heavily based on gravatar](http://wiki.libravatar.org/api/).
 

doc/development/architecture.md

@@ -56,9 +56,9 @@ To serve repositories over SSH there's an add-on application called gitlab-shell
 
 A typical install of GitLab will be on GNU/Linux. It uses Nginx or Apache as a web front end to proxypass the Unicorn web server. By default, communication between Unicorn and the front end is via a Unix domain socket but forwarding requests via TCP is also supported. The web front end accesses `/home/git/gitlab/public` bypassing the Unicorn server to serve static pages, uploads (e.g. avatar images or attachments), and precompiled assets. GitLab serves web pages and a [GitLab API](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/api) using the Unicorn web server. It uses Sidekiq as a job queue which, in turn, uses redis as a non-persistent database backend for job information, meta data, and incoming jobs.
 
-The GitLab web app uses MySQL or PostgreSQL for persistent database information (e.g. users, permissions, issues, other meta data). GitLab stores the bare git repositories it serves in `/home/git/repositories` by default. It also keeps default branch and hook information with the bare repository.
+The GitLab web app uses MySQL or PostgreSQL for persistent database information (e.g. users, permissions, issues, other meta data). GitLab stores the bare git repositories it serves in `/home/git/repositories` by default. It also keeps default branch and hook information with the bare repository. `/home/git/gitlab-satellites` keeps checked out repositories when performing actions such as a merge request, editing files in the web interface, etc.
 
-When serving repositories over HTTP/HTTPS GitLab utilizes the GitLab API to resolve authorization and access as well as serving git objects.
+The satellite repository is used by the web interface for editing repositories and the wiki which is also a git repository. When serving repositories over HTTP/HTTPS GitLab utilizes the GitLab API to resolve authorization and access as well as serving git objects.
 
 The add-on component gitlab-shell serves repositories over SSH. It manages the SSH keys within `/home/git/.ssh/authorized_keys` which should not be manually edited. gitlab-shell accesses the bare repositories directly to serve git objects and communicates with redis to submit jobs to Sidekiq for GitLab to process. gitlab-shell queries the GitLab API to determine authorization and access.
 
@@ -129,7 +129,7 @@ Note: `/home/git/` is shorthand for `/home/git`.
 
 gitlabhq (includes Unicorn and Sidekiq logs)
 
-- `/home/git/gitlab/log/` contains `application.log`, `production.log`, `sidekiq.log`, `unicorn.stdout.log`, `githost.log` and `unicorn.stderr.log` normally.
+- `/home/git/gitlab/log/` contains `application.log`, `production.log`, `sidekiq.log`, `unicorn.stdout.log`, `githost.log`, `satellites.log`, and `unicorn.stderr.log` normally.
 
 gitlab-shell
 

doc/gitlab-basics/README.md

@@ -20,4 +20,6 @@ Step-by-step guides on the basics of working with Git and GitLab.
 
 * [Add a file](add-file.md)
 
+* [Add an image](add-image.md)
+
 * [Create a Merge Request](add-merge-request.md)

doc/gitlab-basics/add-image.md

+# How to add an image
+
+The following are the steps to add images to your repository in
+GitLab:
+
+Find the image that you’d like to add.
+
+In your computer files, find the GitLab project to which you'd like to add the image
+(you'll find it as a regular file). Click on every file until you find exactly where you'd
+like to add the image. There, paste the image.
+
+Go to your [shell](command-line-commands.md), and add the following commands:
+
+Add this command for every directory that you'd like to open:
+```
+cd NAME-OF-FILE-YOU'D-LIKE-TO-OPEN
+```
+
+Create a new branch:
+```
+git checkout -b NAME-OF-BRANCH
+```
+
+Check if your image was correctly added to the directory:
+```
+ls
+```
+
+You should see the name of the image in the list shown.
+
+Move up the hierarchy through directories:
+```
+cd ../
+```
+
+Check the status and you should see your image’s name in red:
+```
+git status
+```
+
+Add your changes:
+```
+git add NAME-OF-YOUR-IMAGE
+```
+
+Check the status and you should see your image’s name in green:
+```
+git status
+```
+
+Add the commit:
+```
+git commit -m “DESCRIBE COMMIT IN A FEW WORDS”
+```
+
+Now you can push (send) your changes (in the branch NAME-OF-BRANCH) to GitLab (the git remote named 'origin'):
+```
+git push origin NAME-OF-BRANCH
+```
+
+Your image will be added to your branch in your repository in GitLab. Create a [Merge Request](add-merge-request.md)
+to integrate your changes to your project.

doc/install/installation.md

@@ -216,6 +216,10 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da
     sudo chmod -R u+rwX,go-w log/
     sudo chmod -R u+rwX tmp/
 
+    # Create directory for satellites
+    sudo -u git -H mkdir /home/git/gitlab-satellites
+    sudo chmod u+rwx,g=rx,o-rwx /home/git/gitlab-satellites
+
     # Make sure GitLab can write to the tmp/pids/ and tmp/sockets/ directories
     sudo chmod -R u+rwX tmp/pids/
     sudo chmod -R u+rwX tmp/sockets/

doc/install/requirements.md

@@ -40,7 +40,7 @@ We love [JRuby](http://jruby.org/) and [Rubinius](http://rubini.us/) but GitLab
 
 ### Storage
 
-The necessary hard drive space largely depends on the size of the repos you want to store in GitLab but as a *rule of thumb* you should have at least as much free space as all your repos combined take up. 
+The necessary hard drive space largely depends on the size of the repos you want to store in GitLab but as a *rule of thumb* you should have at least twice as much free space as all your repos combined take up. You need twice the storage because [GitLab satellites](structure.md) contain an extra copy of each repo.
 
 If you want to be flexible about growing your hard drive space in the future consider mounting it using LVM so you can add more hard drives when you need them.
 
@@ -93,7 +93,7 @@ To change the Unicorn workers when you have the Omnibus package please see [the
 
 ## Database
 
-If you want to run the database separately, the **recommended** database size is **1 MB per user**.
+If you want to run the database separately expect a size of about 1 MB per user.
 
 ## Redis and Sidekiq
 

doc/install/structure.md

@@ -6,14 +6,16 @@ This is the directory structure you will end up with following the instructions
     |   |-- git
     |       |-- .ssh
     |       |-- gitlab
+    |       |-- gitlab-satellites
     |       |-- gitlab-shell
     |       |-- repositories
 
 * `/home/git/.ssh` - contains openssh settings.  Specifically the `authorized_keys` file managed by gitlab-shell.
 * `/home/git/gitlab` - GitLab core software.
+* `/home/git/gitlab-satellites` - checked out repositories for merge requests and file editing from web UI.  This can be treated as a temporary files directory.
 * `/home/git/gitlab-shell` - Core add-on component of GitLab.  Maintains SSH cloning and other functionality.
 * `/home/git/repositories` - bare repositories for all projects organized by namespace.  This is where the git repositories which are pushed/pulled are maintained for all projects.  **This area is critical data for projects.  [Keep a backup](../raketasks/backup_restore.md)**
 
-*Note: the default locations for repositories can be configured in `config/gitlab.yml` of GitLab and `config.yml` of gitlab-shell.*
+*Note: the default locations for gitlab-satellites and repositories can be configured in `config/gitlab.yml` of GitLab and `config.yml` of gitlab-shell.*
 
 To see a more in-depth overview see the [GitLab architecture doc](../development/architecture.md).

doc/logs/logs.md

@@ -51,6 +51,16 @@ December 03, 2014 13:20 -> ERROR -> Command failed [1]: /usr/bin/git --git-dir=/
 error: failed to push some refs to '/Users/vsizov/gitlab-development-kit/repositories/gitlabhq/gitlab_git.git'
 ```
 
+#### satellites.log
+This file lives in `/var/log/gitlab/gitlab-rails/satellites.log` for omnibus package or in `/home/git/gitlab/log/satellites.log` for installations from the source.
+
+In some cases GitLab should perform write actions to git repository, for example when it is needed to merge the merge request or edit a file with online editor. If something went wrong you can look into this file to find out what exactly happened.
+```
+October 07, 2014 11:36: Failed to create satellite for Chesley Weimann III / project1817
+October 07, 2014 11:36: PID: 1872: git clone /Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/repositories/conrad6841/gitlabhq.git /Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/gitlab-satellites/conrad6841/gitlabhq
+October 07, 2014 11:36: PID: 1872: -> fatal: repository '/Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/repositories/conrad6841/gitlabhq.git' does not exist
+```
+
 #### sidekiq.log
 This file lives in `/var/log/gitlab/gitlab-rails/sidekiq.log` for omnibus package or in `/home/git/gitlab/log/sidekiq.log` for installations from the source.
 
@@ -89,4 +99,4 @@ W, [2015-02-13T07:16:01.313000 #9094]  WARN -- : Unicorn::WorkerKiller send SIGQ
 I, [2015-02-13T07:16:01.530733 #9047]  INFO -- : reaped #<Process::Status: pid 9094 exit 0> worker=1
 I, [2015-02-13T07:16:01.534501 #13379]  INFO -- : worker=1 spawned pid=13379
 I, [2015-02-13T07:16:01.534848 #13379]  INFO -- : worker=1 ready
-```
+```
\ No newline at end of file

doc/raketasks/maintenance.md

@@ -105,11 +105,24 @@ Log directory writable? ... yes
 Tmp directory writable? ... yes
 Init script exists? ... yes
 Init script up-to-date? ... yes
+Projects have satellites? ... yes
 Redis version >= 2.0.0? ... yes
 
 Checking GitLab ... Finished
 ```
 
+## (Re-)Create satellite repositories
+
+This will create satellite repositories for all your projects.
+
+If necessary, remove the `repo_satellites` directory and rerun the commands below.
+
+```
+sudo -u git -H mkdir -p /home/git/gitlab-satellites
+sudo -u git -H bundle exec rake gitlab:satellites:create RAILS_ENV=production
+sudo chmod u+rwx,g=rx,o-rwx /home/git/gitlab-satellites
+```
+
 ## Rebuild authorized_keys file
 
 In some case it is necessary to rebuild the `authorized_keys` file.

doc/workflow/README.md

@@ -20,5 +20,4 @@
 - [Share projects with other groups](share_projects_with_other_groups.md)
 - [Two-factor Authentication (2FA)](two_factor_authentication.md)
 - [Web Editor](web_editor.md)
-- [Merge Requests](merge_requests.md)
 - ["Work In Progress" Merge Requests](wip_merge_requests.md)

doc/workflow/merge_requests.md

-# Merge Requests
-
-Merge requests allow you to exchange changes you made to source code
-
-## Checkout merge requests locally
-
-Locate the section for your GitLab remote in the `.git/config` file. It looks like this:
-
-```
-[remote "origin"]
-  url = https://gitlab.com/gitlab-org/gitlab-ce.git
-  fetch = +refs/heads/*:refs/remotes/origin/*
-```
-
-Now add the line `fetch = +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*` to this section. 
-
-It should looks like this: 
-
-```
-[remote "origin"]
-  url = https://gitlab.com/gitlab-org/gitlab-ce.git
-  fetch = +refs/heads/*:refs/remotes/origin/*
-  fetch = +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*
-```
-
-Now you can fetch all the merge requests requests:
-
-```
-$ git fetch origin
-From https://gitlab.com/gitlab-org/gitlab-ce.git
- * [new ref]         refs/merge-requests/1/head -> origin/merge-requests/1
- * [new ref]         refs/merge-requests/2/head -> origin/merge-requests/2
-...
-```
-
-To check out a particular merge request:
-
-```
-$ git checkout origin/merge-requests/1
-```

doc/workflow/notifications.md

@@ -55,7 +55,7 @@ Below is the table of events users can be notified of:
 | User added to project        | User                                                              | Sent when user is added to project |
 | Project access level changed | User                                                              | Sent when user project access level is changed |
 | User added to group          | User                                                              | Sent when user is added to group |
-| Group access level changed   | User                                                              | Sent when user group access level is changed |
+| Group access level changed   | User                                                              | Sent when user group access level is changed | 
 | Project moved                | Project members [1]                                               | [1] not disabled |
 
 ### Issue / Merge Request events
@@ -84,3 +84,8 @@ In all of the below cases, the notification will be sent to:
 | Reopen merge request   | |
 | Merge merge request    | |
 | New comment            | The above, plus anyone mentioned by `@username` in the comment, with notification level "Mention" or higher |
+
+You won't receive notifications for Issues, Merge Requests or Milestones
+created by yourself. You will only receive automatic notifications when
+somebody else comments or adds changes to the ones that you've created or
+mentions you.

features/steps/dashboard/help.rb

@@ -16,6 +16,6 @@ class Spinach::Features::DashboardHelp < Spinach::FeatureSteps
   end
 
   step 'Header "Rebuild project satellites" should have correct ids and links' do
-    header_should_have_correct_id_and_link(2, 'Check GitLab configuration', 'check-gitlab-configuration', '.documentation')
+    header_should_have_correct_id_and_link(2, '(Re-)Create satellite repositories', 're-create-satellite-repositories', '.documentation')
   end
 end

features/steps/dashboard/merge_requests.rb

@@ -66,7 +66,7 @@ class Spinach::Features::DashboardMergeRequests < Spinach::FeatureSteps
 
   def authored_merge_request
     @authored_merge_request ||= create :merge_request,
-                                  source_branch: 'markdown',
+                                  source_branch: 'simple_merge_request',
                                   author: current_user,
                                   target_project: project,
                                   source_project: project
@@ -74,14 +74,14 @@ class Spinach::Features::DashboardMergeRequests < Spinach::FeatureSteps
 
   def other_merge_request
     @other_merge_request ||= create :merge_request,
-                              source_branch: 'fix',
+                              source_branch: '2_3_notes_fix',
                               target_project: project,
                               source_project: project
   end
 
   def authored_merge_request_from_fork
     @authored_merge_request_from_fork ||= create :merge_request,
-                                            source_branch: 'feature_conflict',
+                                            source_branch: 'basic_page',
                                             author: current_user,
                                             target_project: public_project,
                                             source_project: forked_project
@@ -89,7 +89,7 @@ class Spinach::Features::DashboardMergeRequests < Spinach::FeatureSteps
 
   def assigned_merge_request_from_fork
     @assigned_merge_request_from_fork ||= create :merge_request,
-                                            source_branch: 'markdown',
+                                            source_branch: 'basic_page_fix',
                                             assignee: current_user,
                                             target_project: public_project,
                                             source_project: forked_project

features/steps/project/forked_merge_requests.rb

@@ -9,6 +9,7 @@ class Spinach::Features::ProjectForkedMergeRequests < Spinach::FeatureSteps
     @project = Project.find_by(name: "Shop")
     @project ||= create(:project, name: "Shop")
     @project.team << [@user, :reporter]
+    @project.ensure_satellite_exists
   end
 
   step 'I have a project forked off of "Shop" called "Forked Shop"' do

features/steps/project/merge_requests.rb

@@ -198,10 +198,15 @@ class Spinach::Features::ProjectMergeRequests < Spinach::FeatureSteps
   end
 
   step 'merge request "Bug NS-05" is mergeable' do
+    merge_request.project.satellite.create
     merge_request.mark_as_mergeable
   end
 
   step 'I accept this merge request' do
+    Gitlab::Satellite::MergeAction.any_instance.stub(
+      merge!: true,
+    )
+
     page.within '.mr-state-widget' do
       click_button "Accept Merge Request"
     end
@@ -332,7 +337,7 @@ class Spinach::Features::ProjectMergeRequests < Spinach::FeatureSteps
   end
 
   step 'I should see new target branch changes' do
-    expect(page).to have_content 'From fix into feature'
+    expect(page).to have_content 'Request to merge fix into feature'
     expect(page).to have_content 'Target branch changed from master to feature'
   end
 

lib/api/entities.rb

@@ -215,6 +215,9 @@ module API
       expose :attachment_identifier, as: :attachment
       expose :author, using: Entities::UserBasic
       expose :created_at
+      expose :system?, as: :system
+      expose :upvote?, as: :upvote
+      expose :downvote?, as: :downvote
     end
 
     class MRNote < Grape::Entity

lib/api/files.rb

@@ -3,26 +3,6 @@ module API
   class Files < Grape::API
     before { authenticate! }
 
-    helpers do
-      def commit_params(attrs)
-        {
-          file_path: attrs[:file_path],
-          current_branch: attrs[:branch_name],
-          target_branch: attrs[:branch_name],
-          commit_message: attrs[:commit_message],
-          file_content: attrs[:content],
-          file_content_encoding: attrs[:encoding]
-        }
-      end
-
-      def commit_response(attrs)
-        {
-          file_path: attrs[:file_path],
-          branch_name: attrs[:branch_name],
-        }
-      end
-    end
-
     resource :projects do
       # Get file from repository
       # File content is Base64 encoded
@@ -67,7 +47,7 @@ module API
             file_path: blob.path,
             size: blob.size,
             encoding: "base64",
-            content: Base64.encode64(blob.data),
+            content: Base64.strict_encode64(blob.data),
             ref: ref,
             blob_id: blob.id,
             commit_id: commit.id,
@@ -93,11 +73,17 @@ module API
 
         required_attributes! [:file_path, :branch_name, :content, :commit_message]
         attrs = attributes_for_keys [:file_path, :branch_name, :content, :commit_message, :encoding]
-        result = ::Files::CreateService.new(user_project, current_user, commit_params(attrs)).execute
+        branch_name = attrs.delete(:branch_name)
+        file_path = attrs.delete(:file_path)
+        result = ::Files::CreateService.new(user_project, current_user, attrs, branch_name, file_path).execute
 
         if result[:status] == :success
           status(201)
-          commit_response(attrs)
+
+          {
+            file_path: file_path,
+            branch_name: branch_name
+          }
         else
           render_api_error!(result[:message], 400)
         end
@@ -119,11 +105,17 @@ module API
 
         required_attributes! [:file_path, :branch_name, :content, :commit_message]
         attrs = attributes_for_keys [:file_path, :branch_name, :content, :commit_message, :encoding]
-        result = ::Files::UpdateService.new(user_project, current_user, commit_params(attrs)).execute
+        branch_name = attrs.delete(:branch_name)
+        file_path = attrs.delete(:file_path)
+        result = ::Files::UpdateService.new(user_project, current_user, attrs, branch_name, file_path).execute
 
         if result[:status] == :success
           status(200)
-          commit_response(attrs)
+
+          {
+            file_path: file_path,
+            branch_name: branch_name
+          }
         else
           http_status = result[:http_status] || 400
           render_api_error!(result[:message], http_status)
@@ -146,11 +138,17 @@ module API
 
         required_attributes! [:file_path, :branch_name, :commit_message]
         attrs = attributes_for_keys [:file_path, :branch_name, :commit_message]
-        result = ::Files::DeleteService.new(user_project, current_user, commit_params(attrs)).execute
+        branch_name = attrs.delete(:branch_name)
+        file_path = attrs.delete(:file_path)
+        result = ::Files::DeleteService.new(user_project, current_user, attrs, branch_name, file_path).execute
 
         if result[:status] == :success
           status(200)
-          commit_response(attrs)
+
+          {
+            file_path: file_path,
+            branch_name: branch_name
+          }
         else
           render_api_error!(result[:message], 400)
         end

lib/api/merge_requests.rb

@@ -198,11 +198,7 @@ module API
 
           if merge_request.open? && !merge_request.work_in_progress?
             if merge_request.can_be_merged?
-              commit_message = params[:merge_commit_message] || merge_request.merge_commit_message
-
-              ::MergeRequests::MergeService.new(merge_request.target_project, current_user).
-                execute(merge_request, commit_message)
-
+              merge_request.automerge!(current_user, params[:merge_commit_message] || merge_request.merge_commit_message)
               present merge_request, with: Entities::MergeRequest
             else
               render_api_error!('Branch cannot be merged', 405)

lib/backup/database.rb

@@ -18,6 +18,8 @@ module Backup
       success = case config["adapter"]
       when /^mysql/ then
         $progress.print "Dumping MySQL database #{config['database']} ... "
+        # Workaround warnings from MySQL 5.6 about passwords on cmd line
+        ENV['MYSQL_PWD'] = config["password"].to_s if config["password"]
         system('mysqldump', *mysql_args, config['database'], out: db_file_name)
       when "postgresql" then
         $progress.print "Dumping PostgreSQL database #{config['database']} ... "
@@ -43,6 +45,8 @@ module Backup
       success = case config["adapter"]
       when /^mysql/ then
         $progress.print "Restoring MySQL database #{config['database']} ... "
+        # Workaround warnings from MySQL 5.6 about passwords on cmd line
+        ENV['MYSQL_PWD'] = config["password"].to_s if config["password"]
         system('mysql', *mysql_args, config['database'], in: db_file_name)
       when "postgresql" then
         $progress.print "Restoring PostgreSQL database #{config['database']} ... "
@@ -69,8 +73,7 @@ module Backup
         'port'      => '--port',
         'socket'    => '--socket',
         'username'  => '--user',
-        'encoding'  => '--default-character-set',
-        'password'  => '--password'
+        'encoding'  => '--default-character-set'
       }
       args.map { |opt, arg| "#{arg}=#{config[opt]}" if config[opt] }.compact
     end

lib/gitlab.rb

 require 'gitlab/git'
 
 module Gitlab
+  autoload :Satellite, 'gitlab/satellite/satellite'
 end

lib/gitlab/backend/shell.rb

@@ -217,6 +217,20 @@ module Gitlab
       FileUtils.mv(full_path(old_name), full_path(new_name))
     end
 
+    # Remove GitLab Satellites for provided path (namespace or repo dir)
+    #
+    # Ex.
+    #   rm_satellites("gitlab")
+    #
+    #   rm_satellites("gitlab/gitlab-ci.git")
+    #
+    def rm_satellites(path)
+      raise ArgumentError.new("Path can't be blank") if path.blank?
+
+      satellites_path = File.join(Gitlab.config.satellites.path, path)
+      FileUtils.rm_r(satellites_path, force: true)
+    end
+
     def url_to_repo(path)
       Gitlab.config.gitlab_shell.ssh_path_prefix + "#{path}.git"
     end

lib/gitlab/markup_helper.rb

@@ -21,7 +21,7 @@ module Gitlab
     #
     # Returns boolean
     def gitlab_markdown?(filename)
-      filename.downcase.end_with?(*%w(.mdown .md .markdown))
+      filename.downcase.end_with?(*%w(.mdown .mkd .mkdn .md .markdown))
     end
 
     # Public: Determines if the given filename has AsciiDoc extension.

lib/gitlab/satellite/action.rb

+module Gitlab
+  module Satellite
+    class Action
+      DEFAULT_OPTIONS = { git_timeout: Gitlab.config.satellites.timeout.seconds }
+
+      attr_accessor :options, :project, :user
+
+      def initialize(user, project, options = {})
+        @options = DEFAULT_OPTIONS.merge(options)
+        @project = project
+        @user = user
+      end
+
+      protected
+
+      # * Sets a 30s timeout for Git
+      # * Locks the satellite repo
+      # * Yields the prepared satellite repo
+      def in_locked_and_timed_satellite
+        Gitlab::ShellEnv.set_env(user)
+
+        Grit::Git.with_timeout(options[:git_timeout]) do
+          project.satellite.lock do
+            return yield project.satellite.repo
+          end
+        end
+      rescue Errno::ENOMEM => ex
+        return handle_exception(ex)
+      rescue Grit::Git::GitTimeout => ex
+        return handle_exception(ex)
+      ensure
+        Gitlab::ShellEnv.reset_env
+      end
+
+      # * Recreates the satellite
+      # * Sets up Git variables for the user
+      #
+      # Note: use this within #in_locked_and_timed_satellite
+      def prepare_satellite!(repo)
+        project.satellite.clear_and_update!
+
+        if user
+          repo.config['user.name'] = user.name
+          repo.config['user.email'] = user.email
+        end
+      end
+
+      def default_options(options = {})
+        { raise: true, timeout: true }.merge(options)
+      end
+
+      def handle_exception(exception)
+        Gitlab::GitLogger.error(exception.message)
+        false
+      end
+    end
+  end
+end

lib/gitlab/satellite/compare_action.rb

+module Gitlab
+  module Satellite
+    class BranchesWithoutParent < StandardError; end
+
+    class CompareAction < Action
+      def initialize(user, target_project, target_branch, source_project, source_branch)
+        super user, target_project
+
+        @target_project, @target_branch = target_project, target_branch
+        @source_project, @source_branch = source_project, source_branch
+      end
+
+      # Compare 2 repositories and return Gitlab::CompareResult object
+      def result
+        in_locked_and_timed_satellite do |target_repo|
+          prepare_satellite!(target_repo)
+          update_satellite_source_and_target!(target_repo)
+
+          Gitlab::CompareResult.new(compare(target_repo))
+        end
+      rescue Grit::Git::CommandFailed => ex
+        raise BranchesWithoutParent
+      end
+
+      private
+
+      # Assumes a satellite exists that is a fresh clone of the projects repo, prepares satellite for diffs
+      def update_satellite_source_and_target!(target_repo)
+        target_repo.remote_add('source', @source_project.repository.path_to_repo)
+        target_repo.remote_fetch('source')
+      rescue Grit::Git::CommandFailed => ex
+        handle_exception(ex)
+      end
+
+      def compare(repo)
+        @compare ||= Gitlab::Git::Compare.new(
+          Gitlab::Git::Repository.new(repo.path),
+          "origin/#{@target_branch}",
+          "source/#{@source_branch}"
+        )
+      end
+    end
+  end
+end

lib/gitlab/satellite/files/delete_file_action.rb

+require_relative 'file_action'
+
+module Gitlab
+  module Satellite
+    class DeleteFileAction < FileAction
+      # Deletes file and creates a new commit for it
+      #
+      # Returns false if committing the change fails
+      # Returns false if pushing from the satellite to bare repo failed or was rejected
+      # Returns true otherwise
+      def commit!(content, commit_message)
+        in_locked_and_timed_satellite do |repo|
+          prepare_satellite!(repo)
+
+          # create target branch in satellite at the corresponding commit from bare repo
+          repo.git.checkout({ raise: true, timeout: true, b: true }, ref, "origin/#{ref}")
+
+          # update the file in the satellite's working dir
+          file_path_in_satellite = File.join(repo.working_dir, file_path)
+
+          # Prevent relative links
+          unless safe_path?(file_path_in_satellite)
+            Gitlab::GitLogger.error("FileAction: Relative path not allowed")
+            return false
+          end
+
+          File.delete(file_path_in_satellite)
+
+          # add removed file
+          repo.remove(file_path_in_satellite)
+
+          # commit the changes
+          # will raise CommandFailed when commit fails
+          repo.git.commit(raise: true, timeout: true, a: true, m: commit_message)
+
+
+          # push commit back to bare repo
+          # will raise CommandFailed when push fails
+          repo.git.push({ raise: true, timeout: true }, :origin, ref)
+
+          # everything worked
+          true
+        end
+      rescue Grit::Git::CommandFailed => ex
+        Gitlab::GitLogger.error(ex.message)
+        false
+      end
+    end
+  end
+end

lib/gitlab/satellite/files/edit_file_action.rb

+require_relative 'file_action'
+
+module Gitlab
+  module Satellite
+    # GitLab server-side file update and commit
+    class EditFileAction < FileAction
+      # Updates the files content and creates a new commit for it
+      #
+      # Returns false if the ref has been updated while editing the file
+      # Returns false if committing the change fails
+      # Returns false if pushing from the satellite to bare repo failed or was rejected
+      # Returns true otherwise
+      def commit!(content, commit_message, encoding, new_branch = nil)
+        in_locked_and_timed_satellite do |repo|
+          prepare_satellite!(repo)
+
+          # create target branch in satellite at the corresponding commit from bare repo
+          begin
+            repo.git.checkout({ raise: true, timeout: true, b: true }, ref, "origin/#{ref}")
+          rescue Grit::Git::CommandFailed => ex
+            log_and_raise(CheckoutFailed, ex.message)
+          end
+
+          # update the file in the satellite's working dir
+          file_path_in_satellite = File.join(repo.working_dir, file_path)
+
+          # Prevent relative links
+          unless safe_path?(file_path_in_satellite)
+            Gitlab::GitLogger.error("FileAction: Relative path not allowed")
+            return false
+          end
+
+          # Write file
+          write_file(file_path_in_satellite, content, encoding)
+
+          # commit the changes
+          # will raise CommandFailed when commit fails
+          begin
+            repo.git.commit(raise: true, timeout: true, a: true, m: commit_message)
+          rescue Grit::Git::CommandFailed => ex
+            log_and_raise(CommitFailed, ex.message)
+          end
+
+
+          target_branch = new_branch.present? ? "#{ref}:#{new_branch}" : ref
+
+          # push commit back to bare repo
+          # will raise CommandFailed when push fails
+          begin
+            repo.git.push({ raise: true, timeout: true }, :origin, target_branch)
+          rescue Grit::Git::CommandFailed => ex
+            log_and_raise(PushFailed, ex.message)
+          end
+
+          # everything worked
+          true
+        end
+      end
+
+      private
+
+      def log_and_raise(errorClass, message)
+        Gitlab::GitLogger.error(message)
+        raise(errorClass, message)
+      end
+    end
+  end
+end

lib/gitlab/satellite/files/file_action.rb

+module Gitlab
+  module Satellite
+    class FileAction < Action
+      attr_accessor :file_path, :ref
+
+      def initialize(user, project, ref, file_path)
+        super user, project
+        @file_path = file_path
+        @ref = ref
+      end
+
+      def safe_path?(path)
+        File.absolute_path(path) == path
+      end
+
+      def write_file(abs_file_path, content, file_encoding = 'text')
+        if file_encoding == 'base64'
+          File.open(abs_file_path, 'wb') { |f| f.write(Base64.decode64(content)) }
+        else
+          File.open(abs_file_path, 'w') { |f| f.write(content) }
+        end
+      end
+    end
+  end
+end

lib/gitlab/satellite/files/new_file_action.rb

+require_relative 'file_action'
+
+module Gitlab
+  module Satellite
+    class NewFileAction < FileAction
+      # Updates the files content and creates a new commit for it
+      #
+      # Returns false if the ref has been updated while editing the file
+      # Returns false if committing the change fails
+      # Returns false if pushing from the satellite to bare repo failed or was rejected
+      # Returns true otherwise
+      def commit!(content, commit_message, encoding, new_branch = nil)
+        in_locked_and_timed_satellite do |repo|
+          prepare_satellite!(repo)
+
+          # create target branch in satellite at the corresponding commit from bare repo
+          current_ref =
+            if @project.empty_repo?
+              # skip this step if we want to add first file to empty repo
+              Satellite::PARKING_BRANCH
+            else
+              repo.git.checkout({ raise: true, timeout: true, b: true }, ref, "origin/#{ref}")
+              ref
+            end
+
+          file_path_in_satellite = File.join(repo.working_dir, file_path)
+          dir_name_in_satellite = File.dirname(file_path_in_satellite)
+
+          # Prevent relative links
+          unless safe_path?(file_path_in_satellite)
+            Gitlab::GitLogger.error("FileAction: Relative path not allowed")
+            return false
+          end
+
+          # Create dir if not exists
+          FileUtils.mkdir_p(dir_name_in_satellite)
+
+          # Write file
+          write_file(file_path_in_satellite, content, encoding)
+
+          # add new file
+          repo.add(file_path_in_satellite)
+
+          # commit the changes
+          # will raise CommandFailed when commit fails
+          repo.git.commit(raise: true, timeout: true, a: true, m: commit_message)
+
+          target_branch = if new_branch.present? && !@project.empty_repo?
+                            "#{ref}:#{new_branch}"
+                          else
+                            "#{current_ref}:#{ref}"
+                          end
+
+          # push commit back to bare repo
+          # will raise CommandFailed when push fails
+          repo.git.push({ raise: true, timeout: true }, :origin, target_branch)
+
+          # everything worked
+          true
+        end
+      rescue Grit::Git::CommandFailed => ex
+        Gitlab::GitLogger.error(ex.message)
+        false
+      end
+    end
+  end
+end

lib/gitlab/satellite/logger.rb

+module Gitlab
+  module Satellite
+    class Logger < Gitlab::Logger
+      def self.file_name
+        'satellites.log'
+      end
+
+      def format_message(severity, timestamp, progname, msg)
+        "#{timestamp.to_s(:long)}: #{msg}\n"
+      end
+    end
+  end
+end

lib/gitlab/satellite/merge_action.rb

+module Gitlab
+  module Satellite
+    # GitLab server-side merge
+    class MergeAction < Action
+      attr_accessor :merge_request
+
+      def initialize(user, merge_request)
+        super user, merge_request.target_project
+        @merge_request = merge_request
+      end
+
+      # Checks if a merge request can be executed without user interaction
+      def can_be_merged?
+        in_locked_and_timed_satellite do |merge_repo|
+          prepare_satellite!(merge_repo)
+          merge_in_satellite!(merge_repo)
+        end
+      end
+
+      # Merges the source branch into the target branch in the satellite and
+      # pushes it back to the repository.
+      # It also removes the source branch if requested in the merge request (and this is permitted by the merge request).
+      #
+      # Returns false if the merge produced conflicts
+      # Returns false if pushing from the satellite to the repository failed or was rejected
+      # Returns true otherwise
+      def merge!(merge_commit_message = nil)
+        in_locked_and_timed_satellite do |merge_repo|
+          prepare_satellite!(merge_repo)
+          if merge_in_satellite!(merge_repo, merge_commit_message)
+            # push merge back to bare repo
+            # will raise CommandFailed when push fails
+            merge_repo.git.push(default_options, :origin, merge_request.target_branch)
+
+            # remove source branch
+            if merge_request.remove_source_branch?
+              # will raise CommandFailed when push fails
+              merge_repo.git.push(default_options, :origin, ":#{merge_request.source_branch}")
+            end
+            # merge, push and branch removal successful
+            true
+          end
+        end
+      rescue Grit::Git::CommandFailed => ex
+        handle_exception(ex)
+      end
+
+      def diff_in_satellite
+        in_locked_and_timed_satellite do |merge_repo|
+          prepare_satellite!(merge_repo)
+          update_satellite_source_and_target!(merge_repo)
+
+          # Only show what is new in the source branch compared to the target branch, not the other way around.
+          # The line below with merge_base is equivalent to diff with three dots (git diff branch1...branch2)
+          # From the git documentation: "git diff A...B" is equivalent to "git diff $(git-merge-base A B) B"
+          common_commit = merge_repo.git.native(:merge_base, default_options, ["origin/#{merge_request.target_branch}", "source/#{merge_request.source_branch}"]).strip
+          merge_repo.git.native(:diff, default_options, common_commit, "source/#{merge_request.source_branch}")
+        end
+      rescue Grit::Git::CommandFailed => ex
+        handle_exception(ex)
+      end
+
+      def diffs_between_satellite
+        in_locked_and_timed_satellite do |merge_repo|
+          prepare_satellite!(merge_repo)
+          update_satellite_source_and_target!(merge_repo)
+          if merge_request.for_fork?
+            repository = Gitlab::Git::Repository.new(merge_repo.path)
+            diffs = Gitlab::Git::Diff.between(
+              repository,
+              "source/#{merge_request.source_branch}",
+              "origin/#{merge_request.target_branch}"
+            )
+          else
+            raise "Attempt to determine diffs between for a non forked merge request in satellite MergeRequest.id:[#{merge_request.id}]"
+          end
+
+          return diffs
+        end
+      rescue Grit::Git::CommandFailed => ex
+        handle_exception(ex)
+      end
+
+      # Get commit as an email patch
+      def format_patch
+        in_locked_and_timed_satellite do |merge_repo|
+          prepare_satellite!(merge_repo)
+          update_satellite_source_and_target!(merge_repo)
+          patch = merge_repo.git.format_patch(default_options({ stdout: true }), "origin/#{merge_request.target_branch}..source/#{merge_request.source_branch}")
+        end
+      rescue Grit::Git::CommandFailed => ex
+        handle_exception(ex)
+      end
+
+      # Retrieve an array of commits between the source and the target
+      def commits_between
+        in_locked_and_timed_satellite do |merge_repo|
+          prepare_satellite!(merge_repo)
+          update_satellite_source_and_target!(merge_repo)
+          if merge_request.for_fork?
+            repository = Gitlab::Git::Repository.new(merge_repo.path)
+            commits = Gitlab::Git::Commit.between(
+              repository,
+              "origin/#{merge_request.target_branch}",
+              "source/#{merge_request.source_branch}"
+            )
+          else
+            raise "Attempt to determine commits between for a non forked merge request in satellite MergeRequest.id:[#{merge_request.id}]"
+          end
+
+          return commits
+        end
+      rescue Grit::Git::CommandFailed => ex
+        handle_exception(ex)
+      end
+
+      private
+      # Merges the source_branch into the target_branch in the satellite.
+      #
+      # Note: it will clear out the satellite before doing anything
+      #
+      # Returns false if the merge produced conflicts
+      # Returns true otherwise
+      def merge_in_satellite!(repo, message = nil)
+        update_satellite_source_and_target!(repo)
+
+        message ||= "Merge branch '#{merge_request.source_branch}' into '#{merge_request.target_branch}'"
+
+        # merge the source branch into the satellite
+        # will raise CommandFailed when merge fails
+        repo.git.merge(default_options({ no_ff: true }), "-m#{message}", "source/#{merge_request.source_branch}")
+      rescue Grit::Git::CommandFailed => ex
+        handle_exception(ex)
+      end
+
+      # Assumes a satellite exists that is a fresh clone of the projects repo, prepares satellite for merges, diffs etc
+      def update_satellite_source_and_target!(repo)
+        repo.remote_add('source', merge_request.source_project.repository.path_to_repo)
+        repo.remote_fetch('source')
+        repo.git.checkout(default_options({ b: true }), merge_request.target_branch, "origin/#{merge_request.target_branch}")
+      rescue Grit::Git::CommandFailed => ex
+        handle_exception(ex)
+      end
+    end
+  end
+end

lib/gitlab/satellite/satellite.rb

+module Gitlab
+  module Satellite
+    autoload :DeleteFileAction, 'gitlab/satellite/files/delete_file_action'
+    autoload :EditFileAction,   'gitlab/satellite/files/edit_file_action'
+    autoload :FileAction,       'gitlab/satellite/files/file_action'
+    autoload :NewFileAction,    'gitlab/satellite/files/new_file_action'
+
+    class CheckoutFailed < StandardError; end
+    class CommitFailed < StandardError; end
+    class PushFailed < StandardError; end
+
+    class Satellite
+      include Gitlab::Popen
+
+      PARKING_BRANCH = "__parking_branch"
+
+      attr_accessor :project
+
+      def initialize(project)
+        @project = project
+      end
+
+      def log(message)
+        Gitlab::Satellite::Logger.error(message)
+      end
+
+      def clear_and_update!
+        project.ensure_satellite_exists
+
+        @repo = nil
+        clear_working_dir!
+        delete_heads!
+        remove_remotes!
+        update_from_source!
+      end
+
+      def create
+        output, status = popen(%W(git clone -- #{project.repository.path_to_repo} #{path}),
+                               Gitlab.config.satellites.path)
+
+        log("PID: #{project.id}: git clone #{project.repository.path_to_repo} #{path}")
+        log("PID: #{project.id}: -> #{output}")
+
+        if status.zero?
+          true
+        else
+          log("Failed to create satellite for #{project.name_with_namespace}")
+          false
+        end
+      end
+
+      def exists?
+        File.exists? path
+      end
+
+      # * Locks the satellite
+      # * Changes the current directory to the satellite's working dir
+      # * Yields
+      def lock
+        project.ensure_satellite_exists
+
+        File.open(lock_file, "w+") do |f|
+          begin
+            f.flock File::LOCK_EX
+            yield
+          ensure
+            f.flock File::LOCK_UN
+          end
+        end
+      end
+
+      def lock_file
+        create_locks_dir unless File.exists?(lock_files_dir)
+        File.join(lock_files_dir, "satellite_#{project.id}.lock")
+      end
+
+      def path
+        File.join(Gitlab.config.satellites.path, project.path_with_namespace)
+      end
+
+      def repo
+        project.ensure_satellite_exists
+
+        @repo ||= Grit::Repo.new(path)
+      end
+
+      def destroy
+        FileUtils.rm_rf(path)
+      end
+
+      private
+
+      # Clear the working directory
+      def clear_working_dir!
+        repo.git.reset(hard: true)
+        repo.git.clean(f: true, d: true, x: true)
+      end
+
+      # Deletes all branches except the parking branch
+      #
+      # This ensures we have no name clashes or issues updating branches when
+      # working with the satellite.
+      def delete_heads!
+        heads = repo.heads.map(&:name)
+
+        # update or create the parking branch
+        repo.git.checkout(default_options({ B: true }), PARKING_BRANCH)
+
+        # remove the parking branch from the list of heads ...
+        heads.delete(PARKING_BRANCH)
+        # ... and delete all others
+        heads.each { |head| repo.git.branch(default_options({ D: true }), head) }
+      end
+
+      # Deletes all remotes except origin
+      #
+      # This ensures we have no remote name clashes or issues updating branches when
+      # working with the satellite.
+      def remove_remotes!
+        remotes = repo.git.remote.split(' ')
+        remotes.delete('origin')
+        remotes.each { |name| repo.git.remote(default_options,'rm', name)}
+      end
+
+      # Updates the satellite from bare repo
+      #
+      # Note: this will only update remote branches (i.e. origin/*)
+      def update_from_source!
+        repo.git.remote(default_options, 'set-url', :origin, project.repository.path_to_repo)
+        repo.git.fetch(default_options, :origin)
+      end
+
+      def default_options(options = {})
+        { raise: true, timeout: true }.merge(options)
+      end
+
+      # Create directory for storing
+      # satellites lock files
+      def create_locks_dir
+        FileUtils.mkdir_p(lock_files_dir)
+      end
+
+      def lock_files_dir
+        @lock_files_dir ||= File.join(Gitlab.config.satellites.path, "tmp")
+      end
+    end
+  end
+end

lib/rouge/formatters/html_gitlab.rb

@@ -93,16 +93,27 @@ module Rouge
       end
 
       def process_tokens(tokens)
-        num_lines = 0
-        last_val = ''
-        rendered = ''
+        rendered = []
+        current_line = ''
 
         tokens.each do |tok, val|
-          last_val = val
-          num_lines += val.scan(/\n/).size
-          rendered << span(tok, val)
+          # In the case of multi-line values (e.g. comments), we need to apply
+          # styling to each line since span elements are inline.
+          val.lines.each do |line|
+            stripped = line.chomp
+            current_line << span(tok, stripped)
+
+            if line.end_with?("\n")
+              rendered << current_line
+              current_line = ''
+            end
+          end
         end
 
+        # Add leftover text
+        rendered << current_line if current_line.present?
+
+        num_lines = rendered.size
         numbers = (@linenostart..num_lines + @linenostart - 1).to_a
 
         { numbers: numbers, code: rendered }
@@ -117,9 +128,8 @@ module Rouge
         numbers.join("\n")
       end
 
-      def wrap_lines(rendered)
+      def wrap_lines(lines)
         if @lineanchors
-          lines = rendered.split("\n")
           lines = lines.each_with_index.map do |line, index|
             number = index + @linenostart
 
@@ -136,24 +146,17 @@ module Rouge
           lines.join("\n")
         else
           if @linenos == 'inline'
-            lines = rendered.split("\n")
             lines = lines.each_with_index.map do |line, index|
               number = index + @linenostart
               "<span class=\"linenos\">#{number}</span>#{line}"
             end
             lines.join("\n")
           else
-            rendered
+            lines.join("\n")
           end
         end
       end
 
-      def wrap_values(val, element)
-        lines = val.split("\n")
-        lines = lines.map{ |x| "<span #{element}>#{x}</span>" }
-        lines.join("\n")
-      end
-
       def span(tok, val)
         # http://stackoverflow.com/a/1600584/2587286
         val = CGI.escapeHTML(val)
@@ -161,13 +164,11 @@ module Rouge
         if tok.shortname.empty?
           val
         else
-          # In the case of multi-line values (e.g. comments), we need to apply
-          # styling to each line since span elements are inline.
           if @inline_theme
             rules = @inline_theme.style_for(tok).rendered_rules
-            wrap_values(val, "style=\"#{rules.to_a.join(';')}\"")
+            "<span style=\"#{rules.to_a.join(';')}\"#{val}</span>"
           else
-            wrap_values(val, "class=\"#{tok.shortname}\"")
+            "<span class=\"#{tok.shortname}\">#{val}</span>"
           end
         end
       end

lib/tasks/gitlab/check.rake

@@ -25,6 +25,7 @@ namespace :gitlab do
       check_init_script_exists
       check_init_script_up_to_date
       check_projects_have_namespace
+      check_satellites_exist
       check_redis_version
       check_ruby_version
       check_git_version
@@ -237,6 +238,37 @@ namespace :gitlab do
       end
     end
 
+    def check_satellites_exist
+      print "Projects have satellites? ... "
+
+      unless Project.count > 0
+        puts "can't check, you have no projects".magenta
+        return
+      end
+      puts ""
+
+      Project.find_each(batch_size: 100) do |project|
+        print sanitized_message(project)
+
+        if project.satellite.exists?
+          puts "yes".green
+        elsif project.empty_repo?
+          puts "can't create, repository is empty".magenta
+        else
+          puts "no".red
+          try_fixing_it(
+            sudo_gitlab("bundle exec rake gitlab:satellites:create RAILS_ENV=production"),
+            "If necessary, remove the tmp/repo_satellites directory ...",
+            "... and rerun the above command"
+          )
+          for_more_information(
+            "doc/raketasks/maintenance.md "
+          )
+          fix_and_rerun
+        end
+      end
+    end
+
     def check_log_writable
       print "Log directory writable? ... "
 
@@ -307,6 +339,7 @@ namespace :gitlab do
       check_repo_base_is_not_symlink
       check_repo_base_user_and_group
       check_repo_base_permissions
+      check_satellites_permissions
       check_repos_hooks_directory_is_link
       check_gitlab_shell_self_test
 
@@ -384,6 +417,29 @@ namespace :gitlab do
       end
     end
 
+    def check_satellites_permissions
+      print "Satellites access is drwxr-x---? ... "
+
+      satellites_path = Gitlab.config.satellites.path
+      unless File.exists?(satellites_path)
+        puts "can't check because of previous errors".magenta
+        return
+      end
+
+      if File.stat(satellites_path).mode.to_s(8).ends_with?("0750")
+        puts "yes".green
+      else
+        puts "no".red
+        try_fixing_it(
+          "sudo chmod u+rwx,g=rx,o-rwx #{satellites_path}",
+        )
+        for_more_information(
+          see_installation_guide_section "GitLab"
+        )
+        fix_and_rerun
+      end
+    end
+
     def check_repo_base_user_and_group
       gitlab_shell_ssh_user = Gitlab.config.gitlab_shell.ssh_user
       gitlab_shell_owner_group = Gitlab.config.gitlab_shell.owner_group
@@ -693,7 +749,7 @@ namespace :gitlab do
   end
 
   def check_ruby_version
-    required_version = Gitlab::VersionInfo.new(2, 0, 0)
+    required_version = Gitlab::VersionInfo.new(2, 1, 0)
     current_version = Gitlab::VersionInfo.parse(run(%W(ruby --version)))
 
     print "Ruby version >= #{required_version} ? ... "

lib/tasks/gitlab/enable_automerge.rake

+namespace :gitlab do
+  namespace :satellites do
+    desc "GitLab | Create satellite repos"
+    task create: :environment do
+      create_satellites
+    end
+  end
+
+  def create_satellites
+    warn_user_is_not_gitlab
+
+    print "Creating satellites for ..."
+    unless Project.count > 0
+      puts "skipping, because you have no projects".magenta
+      return
+    end
+    puts ""
+
+    Project.find_each(batch_size: 100) do |project|
+      print "#{project.name_with_namespace.yellow} ... "
+
+      unless project.repo_exists?
+        puts "skipping, because the repo is empty".magenta
+        next
+      end
+
+      if project.satellite.exists?
+        puts "exists already".green
+      else
+        print "\n... "
+        if project.satellite.create
+          puts "created".green
+        else
+          puts "error".red
+        end
+      end
+    end
+  end
+end

spec/controllers/projects/issues_controller_spec.rb

+require('spec_helper')
+
+describe Projects::IssuesController do
+  let(:project) { create(:project) }
+  let(:user)    { create(:user) }
+  let(:issue) { create(:issue, project: project) }
+
+  before do
+    sign_in(user)
+    project.team << [user, :developer]
+    controller.instance_variable_set(:@project, project)
+  end
+
+  describe "GET #index" do
+    it "returns index" do
+      get :index, namespace_id: project.namespace.id, project_id: project.id
+
+      expect(response.status).to eq(200)
+    end
+
+    it "returns 404 when issues are disabled" do
+      project.issues_enabled = false
+      project.save
+
+      get :index, namespace_id: project.namespace.id, project_id: project.id
+      expect(response.status).to eq(404)
+    end
+
+    it "returns 404 when external issue tracker is enabled" do
+      allow(project).to receive(:default_issues_tracker?).and_return(false)
+
+      get :index, namespace_id: project.namespace.id, project_id: project.id
+      expect(response.status).to eq(404)
+    end
+
+  end
+end

spec/controllers/projects/services_controller_spec.rb

+require 'spec_helper'
+
+describe Projects::ServicesController do
+  let(:project) { create(:project) }
+  let(:user)    { create(:user) }
+  let(:service) { create(:service, project: project) }
+
+  before do
+    sign_in(user)
+    project.team << [user, :master]
+    controller.instance_variable_set(:@project, project)
+    controller.instance_variable_set(:@service, service)
+    request.env["HTTP_REFERER"] = "/"
+  end
+
+  describe "#test" do
+    context 'success' do
+      it "should redirect and show success message" do
+        expect(service).to receive(:test).and_return({ success: true, result: 'done' })
+        get :test, namespace_id: project.namespace.id, project_id: project.id, id: service.id, format: :html
+        expect(response.status).to redirect_to('/')
+        expect(flash[:notice]).to eq('We sent a request to the provided URL')
+      end
+    end
+
+    context 'failure' do
+      it "should redirect and show failure message" do
+        expect(service).to receive(:test).and_return({ success: false, result: 'Bad test' })
+        get :test, namespace_id: project.namespace.id, project_id: project.id, id: service.id, format: :html
+        expect(response.status).to redirect_to('/')
+        expect(flash[:alert]).to eq('We tried to send a request to the provided URL but an error occurred: Bad test')
+      end
+    end
+  end
+end

spec/helpers/blob_helper_spec.rb

@@ -47,5 +47,21 @@ describe BlobHelper do
       expect(lines[1].text).to eq('        This is line 2.')
       expect(lines[2].text).to eq('        """')
     end
+
+    context 'diff highlighting' do
+      let(:blob_name) { 'test.diff' }
+      let(:blob_content) { "+aaa\n+bbb\n- ccc\n ddd\n"}
+      let(:expected) do
+        %q(<span id="LC1" class="line"><span class="gi">+aaa</span></span>
+<span id="LC2" class="line"><span class="gi">+bbb</span></span>
+<span id="LC3" class="line"><span class="gd">- ccc</span></span>
+<span id="LC4" class="line"> ddd</span>)
+      end
+
+      it 'should highlight each line properly' do
+        result = highlight(blob_name, blob_content, nowrap: true, continue: false)
+        expect(result).to eq(expected)
+      end
+    end
   end
 end

spec/helpers/events_helper_spec.rb

@@ -58,7 +58,7 @@ describe EventsHelper do
     expected = '<pre class="code highlight white ruby">' \
       "<code><span class=\"k\">def</span> <span class=\"nf\">test</span>\n" \
       "  <span class=\"s1\">\'hello world\'</span>\n" \
-      "<span class=\"k\">end</span>\n" \
+      "<span class=\"k\">end</span>" \
       '</code></pre>'
     expect(event_note(input)).to eq(expected)
   end

spec/lib/gitlab/markup_helper_spec.rb

@@ -15,7 +15,7 @@ describe Gitlab::MarkupHelper do
   end
 
   describe '#gitlab_markdown?' do
-    %w(mdown md markdown).each do |type|
+    %w(mdown mkd mkdn md markdown).each do |type|
       it "returns true for #{type} files" do
         expect(Gitlab::MarkupHelper.gitlab_markdown?("README.#{type}")).to be_truthy
       end

spec/lib/gitlab/reference_extractor_spec.rb

@@ -25,7 +25,7 @@ describe Gitlab::ReferenceExtractor do
     project.team << [@u_bar, :guest]
 
     subject.analyze(%Q{
-      Inline code: `@foo`
+      Inline code: `@foo` 
 
       Code block:
 
@@ -33,7 +33,7 @@ describe Gitlab::ReferenceExtractor do
       @bar
       ```
 
-      Quote:
+      Quote: 
 
       > @offteam
     })
@@ -49,8 +49,8 @@ describe Gitlab::ReferenceExtractor do
   end
 
   it 'accesses valid merge requests' do
-    @m0 = create(:merge_request, source_project: project, target_project: project, source_branch: 'markdown')
-    @m1 = create(:merge_request, source_project: project, target_project: project, source_branch: 'feature_conflict')
+    @m0 = create(:merge_request, source_project: project, target_project: project, source_branch: 'aaa')
+    @m1 = create(:merge_request, source_project: project, target_project: project, source_branch: 'bbb')
 
     subject.analyze("!999, !#{@m1.iid}, and !#{@m0.iid}.")
     expect(subject.merge_requests).to eq([@m1, @m0])

spec/lib/gitlab/satellite/action_spec.rb

+require 'spec_helper'
+
+describe 'Gitlab::Satellite::Action' do
+  let(:project) { create(:project) }
+  let(:user) { create(:user) }
+
+  describe '#prepare_satellite!' do
+    it 'should be able to fetch timeout from conf' do
+      expect(Gitlab::Satellite::Action::DEFAULT_OPTIONS[:git_timeout]).to eq(30.seconds)
+    end
+
+    it 'create a repository with a parking branch and one remote: origin' do
+      repo = project.satellite.repo
+
+      #now lets dirty it up
+
+      starting_remote_count = repo.git.list_remotes.size
+      expect(starting_remote_count).to be >= 1
+      #kind of hookey way to add a second remote
+      origin_uri = repo.git.remote({ v: true }).split(" ")[1]
+
+      repo.git.remote({ raise: true }, 'add', 'another-remote', origin_uri)
+      repo.git.branch({ raise: true }, 'a-new-branch')
+
+      expect(repo.heads.size).to be > (starting_remote_count)
+      expect(repo.git.remote().split(" ").size).to be > (starting_remote_count)
+
+      repo.git.config({}, "user.name", "#{user.name} -- foo")
+      repo.git.config({}, "user.email", "#{user.email} -- foo")
+      expect(repo.config['user.name']).to eq("#{user.name} -- foo")
+      expect(repo.config['user.email']).to eq("#{user.email} -- foo")
+
+
+      #These must happen in the context of the satellite directory...
+      satellite_action = Gitlab::Satellite::Action.new(user, project)
+      project.satellite.lock do
+        #Now clean it up, use send to get around prepare_satellite! being protected
+        satellite_action.send(:prepare_satellite!, repo)
+      end
+
+      #verify it's clean
+      heads = repo.heads.map(&:name)
+      expect(heads.size).to eq(1)
+      expect(heads.include?(Gitlab::Satellite::Satellite::PARKING_BRANCH)).to eq(true)
+      remotes = repo.git.remote().split(' ')
+      expect(remotes.size).to eq(1)
+      expect(remotes.include?('origin')).to eq(true)
+      expect(repo.config['user.name']).to eq(user.name)
+      expect(repo.config['user.email']).to eq(user.email)
+    end
+  end
+
+  describe '#in_locked_and_timed_satellite' do
+
+    it 'should make use of a lockfile' do
+      repo = project.satellite.repo
+      called = false
+
+      #set assumptions
+      FileUtils.rm_f(project.satellite.lock_file)
+
+      expect(File.exists?(project.satellite.lock_file)).to be_falsey
+
+      satellite_action = Gitlab::Satellite::Action.new(user, project)
+      satellite_action.send(:in_locked_and_timed_satellite) do |sat_repo|
+        expect(repo).to eq(sat_repo)
+        expect(File.exists? project.satellite.lock_file).to be_truthy
+        called = true
+      end
+
+      expect(called).to be_truthy
+
+    end
+
+    it 'should be able to use the satellite after locking' do
+      repo = project.satellite.repo
+      called = false
+
+      # Set base assumptions
+      if File.exists? project.satellite.lock_file
+        expect(FileLockStatusChecker.new(project.satellite.lock_file).flocked?).to be_falsey
+      end
+
+      satellite_action = Gitlab::Satellite::Action.new(user, project)
+      satellite_action.send(:in_locked_and_timed_satellite) do |sat_repo|
+        called = true
+        expect(repo).to eq(sat_repo)
+        expect(File.exists? project.satellite.lock_file).to be_truthy
+        expect(FileLockStatusChecker.new(project.satellite.lock_file).flocked?).to be_truthy
+      end
+
+      expect(called).to be_truthy
+      expect(FileLockStatusChecker.new(project.satellite.lock_file).flocked?).to be_falsey
+
+    end
+
+    class FileLockStatusChecker < File
+      def flocked?(&block)
+        status = flock LOCK_EX|LOCK_NB
+        case status
+        when false
+          return true
+        when 0
+          begin
+            block ? block.call : false
+          ensure
+            flock LOCK_UN
+          end
+        else
+          raise SystemCallError, status
+        end
+      end
+    end
+
+  end
+end

spec/lib/gitlab/satellite/merge_action_spec.rb

+require 'spec_helper'
+
+describe 'Gitlab::Satellite::MergeAction' do
+  include RepoHelpers
+
+  let(:project) { create(:project, namespace: create(:group)) }
+  let(:fork_project) { create(:project, namespace: create(:group), forked_from_project: project) }
+  let(:merge_request) { create(:merge_request, source_project: project, target_project: project) }
+  let(:merge_request_fork) { create(:merge_request, source_project: fork_project, target_project: project) }
+
+  let(:merge_request_with_conflict) { create(:merge_request, :conflict, source_project: project, target_project: project) }
+  let(:merge_request_fork_with_conflict) { create(:merge_request, :conflict, source_project: project, target_project: project) }
+
+  describe '#commits_between' do
+    def verify_commits(commits, first_commit_sha, last_commit_sha)
+      commits.each { |commit| expect(commit.class).to eq(Gitlab::Git::Commit) }
+      expect(commits.first.id).to eq(first_commit_sha)
+      expect(commits.last.id).to eq(last_commit_sha)
+    end
+
+    context 'on fork' do
+      it 'should get proper commits between' do
+        commits = Gitlab::Satellite::MergeAction.new(merge_request_fork.author, merge_request_fork).commits_between
+        verify_commits(commits, sample_compare.commits.first, sample_compare.commits.last)
+      end
+    end
+
+    context 'between branches' do
+      it 'should raise exception -- not expected to be used by non forks' do
+        expect { Gitlab::Satellite::MergeAction.new(merge_request.author, merge_request).commits_between }.to raise_error(RuntimeError)
+      end
+    end
+  end
+
+  describe '#format_patch' do
+    def verify_content(patch)
+      sample_compare.commits.each do |commit|
+        expect(patch.include?(commit)).to be_truthy
+      end
+    end
+
+    context 'on fork' do
+      it 'should build a format patch' do
+        patch = Gitlab::Satellite::MergeAction.new(merge_request_fork.author, merge_request_fork).format_patch
+        verify_content(patch)
+      end
+    end
+
+    context 'between branches' do
+      it 'should build a format patch' do
+        patch = Gitlab::Satellite::MergeAction.new(merge_request_fork.author, merge_request).format_patch
+        verify_content(patch)
+      end
+    end
+  end
+
+  describe '#diffs_between_satellite tested against diff_in_satellite' do
+    def is_a_matching_diff(diff, diffs)
+      diff_count = diff.scan('diff --git').size
+      expect(diff_count).to be >= 1
+      expect(diffs.size).to eq(diff_count)
+      diffs.each do |a_diff|
+        expect(a_diff.class).to eq(Gitlab::Git::Diff)
+        expect(diff.include? a_diff.diff).to be_truthy
+      end
+    end
+
+    context 'on fork' do
+      it 'should get proper diffs' do
+        diffs = Gitlab::Satellite::MergeAction.new(merge_request_fork.author, merge_request_fork).diffs_between_satellite
+        diff = Gitlab::Satellite::MergeAction.new(merge_request.author, merge_request_fork).diff_in_satellite
+        is_a_matching_diff(diff, diffs)
+      end
+    end
+
+    context 'between branches' do
+      it 'should get proper diffs' do
+        expect{ Gitlab::Satellite::MergeAction.new(merge_request.author, merge_request).diffs_between_satellite }.to raise_error(RuntimeError)
+      end
+    end
+  end
+
+  describe '#can_be_merged?' do
+    context 'on fork' do
+      it do
+        expect(Gitlab::Satellite::MergeAction.new(merge_request_fork.author, merge_request_fork).can_be_merged?).to be_truthy
+      end
+
+      it do
+        expect(Gitlab::Satellite::MergeAction.new(merge_request_fork_with_conflict.author, merge_request_fork_with_conflict).can_be_merged?).to be_falsey
+      end
+    end
+
+    context 'between branches' do
+      it do
+        expect(Gitlab::Satellite::MergeAction.new(merge_request.author, merge_request).can_be_merged?).to be_truthy
+      end
+
+      it do
+        expect(Gitlab::Satellite::MergeAction.new(merge_request_with_conflict.author, merge_request_with_conflict).can_be_merged?).to be_falsey
+      end
+    end
+  end
+end

spec/models/merge_request_spec.rb

@@ -200,7 +200,7 @@ describe MergeRequest do
   end
 
   it_behaves_like 'an editable mentionable' do
-    subject { create(:merge_request) }
+    subject { create(:merge_request, source_project: project) }
 
     let(:backref_text) { "merge request #{subject.to_reference}" }
     let(:set_mentionable_text) { ->(txt){ subject.description = txt } }

spec/models/project_services/flowdock_service_spec.rb

@@ -39,15 +39,19 @@ describe FlowdockService do
         token: 'verySecret'
       )
       @sample_data = Gitlab::PushDataBuilder.build_sample(project, user)
-      @api_url = 'https://api.flowdock.com/v1/git/verySecret'
+      @api_url = 'https://api.flowdock.com/v1/messages'
       WebMock.stub_request(:post, @api_url)
     end
 
     it "should call FlowDock API" do
       @flowdock_service.execute(@sample_data)
-      expect(WebMock).to have_requested(:post, @api_url).with(
-        body: /#{@sample_data[:before]}.*#{@sample_data[:after]}.*#{project.path}/
-      ).once
+      @sample_data[:commits].each do |commit|
+        # One request to Flowdock per new commit
+        next if commit[:id] == @sample_data[:before]
+        expect(WebMock).to have_requested(:post, @api_url).with(
+          body: /#{commit[:id]}.*#{project.path}/
+        ).once
+      end
     end
   end
 end

spec/models/project_services/hipchat_service_spec.rb

@@ -47,6 +47,14 @@ describe HipchatService do
       WebMock.stub_request(:post, api_url)
     end
 
+    it 'should test and return errors' do
+      allow(hipchat).to receive(:execute).and_raise(StandardError, 'no such room')
+      result = hipchat.test(push_sample_data)
+
+      expect(result[:success]).to be_falsey
+      expect(result[:result].to_s).to eq('no such room')
+    end
+
     it 'should use v1 if version is provided' do
       allow(hipchat).to receive(:api_version).and_return('v1')
       expect(HipChat::Client).to receive(:new).

spec/models/project_services/slack_service_spec.rb

@@ -67,7 +67,7 @@ describe SlackService do
       opts = {
         title: 'Awesome merge_request',
         description: 'please fix',
-        source_branch: 'feature',
+        source_branch: 'stable',
         target_branch: 'master'
       }
       merge_service = MergeRequests::CreateService.new(project,

spec/models/project_spec.rb

@@ -91,6 +91,7 @@ describe Project do
   describe 'Respond to' do
     it { is_expected.to respond_to(:url_to_repo) }
     it { is_expected.to respond_to(:repo_exists?) }
+    it { is_expected.to respond_to(:satellite) }
     it { is_expected.to respond_to(:update_merge_requests) }
     it { is_expected.to respond_to(:execute_hooks) }
     it { is_expected.to respond_to(:name_with_namespace) }

spec/models/repository_spec.rb

@@ -34,18 +34,24 @@ describe Repository do
     end
   end
 
-  describe :can_be_merged? do
-    context 'mergeable branches' do
-      subject { repository.can_be_merged?('0b4bc9a49b562e85de7cc9e834518ea6828729b9', 'master') }
+  describe :merged_to_root_ref? do
+    context 'merged branch' do
+      subject { repository.merged_to_root_ref?('improve/awesome') }
 
       it { is_expected.to be_truthy }
     end
 
-    context 'non-mergeable branches' do
-      subject { repository.can_be_merged?('bb5206fee213d983da88c47f9cf4cc6caf9c66dc', 'feature') }
+    context 'non merged branch' do
+      subject { repository.merged_to_root_ref?('fix') }
 
       it { is_expected.to be_falsey }
     end
+
+    context 'non existent branch' do
+      subject { repository.merged_to_root_ref?('non_existent_branch') }
+
+      it { is_expected.to be_nil }
+    end
   end
 
   describe "search_files" do

spec/models/service_spec.rb

@@ -46,6 +46,16 @@ describe Service do
       describe :can_test do
         it { expect(@testable).to eq(true) }
       end
+
+      describe :test do
+        let(:data) { 'test' }
+
+        it 'test runs execute' do
+          expect(@service).to receive(:execute).with(data)
+
+          @service.test(data)
+        end
+      end
     end
 
     describe "With commits" do

spec/requests/api/files_spec.rb

@@ -49,6 +49,8 @@ describe API::API, api: true  do
     end
 
     it "should create a new file in project repo" do
+      expect_any_instance_of(Gitlab::Satellite::NewFileAction).to receive(:commit!).and_return(true)
+
       post api("/projects/#{project.id}/repository/files", user), valid_params
       expect(response.status).to eq(201)
       expect(json_response['file_path']).to eq('newfile.rb')
@@ -59,9 +61,8 @@ describe API::API, api: true  do
       expect(response.status).to eq(400)
     end
 
-    it "should return a 400 if editor fails to create file" do
-      allow_any_instance_of(Repository).to receive(:commit_file).
-        and_return(false)
+    it "should return a 400 if satellite fails to create file" do
+      expect_any_instance_of(Gitlab::Satellite::NewFileAction).to receive(:commit!).and_return(false)
 
       post api("/projects/#{project.id}/repository/files", user), valid_params
       expect(response.status).to eq(400)
@@ -79,6 +80,8 @@ describe API::API, api: true  do
     end
 
     it "should update existing file in project repo" do
+      expect_any_instance_of(Gitlab::Satellite::EditFileAction).to receive(:commit!).and_return(true)
+
       put api("/projects/#{project.id}/repository/files", user), valid_params
       expect(response.status).to eq(200)
       expect(json_response['file_path']).to eq(file_path)
@@ -88,6 +91,32 @@ describe API::API, api: true  do
       put api("/projects/#{project.id}/repository/files", user)
       expect(response.status).to eq(400)
     end
+
+    it 'should return a 400 if the checkout fails' do
+      expect_any_instance_of(Gitlab::Satellite::EditFileAction).to receive(:commit!).and_raise(Gitlab::Satellite::CheckoutFailed)
+
+      put api("/projects/#{project.id}/repository/files", user), valid_params
+      expect(response.status).to eq(400)
+
+      ref = valid_params[:branch_name]
+      expect(response.body).to match("ref '#{ref}' could not be checked out")
+    end
+
+    it 'should return a 409 if the file was not modified' do
+      expect_any_instance_of(Gitlab::Satellite::EditFileAction).to receive(:commit!).and_raise(Gitlab::Satellite::CommitFailed)
+
+      put api("/projects/#{project.id}/repository/files", user), valid_params
+      expect(response.status).to eq(409)
+      expect(response.body).to match("Maybe there was nothing to commit?")
+    end
+
+    it 'should return a 409 if the push fails' do
+      expect_any_instance_of(Gitlab::Satellite::EditFileAction).to receive(:commit!).and_raise(Gitlab::Satellite::PushFailed)
+
+      put api("/projects/#{project.id}/repository/files", user), valid_params
+      expect(response.status).to eq(409)
+      expect(response.body).to match("Maybe the file was changed by another process?")
+    end
   end
 
   describe "DELETE /projects/:id/repository/files" do
@@ -100,6 +129,7 @@ describe API::API, api: true  do
     end
 
     it "should delete existing file in project repo" do
+      expect_any_instance_of(Gitlab::Satellite::DeleteFileAction).to receive(:commit!).and_return(true)
       delete api("/projects/#{project.id}/repository/files", user), valid_params
       expect(response.status).to eq(200)
       expect(json_response['file_path']).to eq(file_path)
@@ -110,11 +140,42 @@ describe API::API, api: true  do
       expect(response.status).to eq(400)
     end
 
-    it "should return a 400 if fails to create file" do
-      allow_any_instance_of(Repository).to receive(:remove_file).and_return(false)
+    it "should return a 400 if satellite fails to create file" do
+      expect_any_instance_of(Gitlab::Satellite::DeleteFileAction).to receive(:commit!).and_return(false)
 
       delete api("/projects/#{project.id}/repository/files", user), valid_params
       expect(response.status).to eq(400)
     end
   end
+
+  describe "POST /projects/:id/repository/files with binary file" do
+    let(:file_path) { 'test.bin' }
+    let(:put_params) do
+      {
+        file_path: file_path,
+        branch_name: 'master',
+        content: 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABAQMAAAAl21bKAAAAA1BMVEUAAACnej3aAAAAAXRSTlMAQObYZgAAAApJREFUCNdjYAAAAAIAAeIhvDMAAAAASUVORK5CYII=',
+        commit_message: 'Binary file with a \n should not be touched',
+        encoding: 'base64'
+      }
+    end
+    let(:get_params) do
+      {
+        file_path: file_path,
+        ref: 'master',
+      }
+    end
+
+    before do
+      post api("/projects/#{project.id}/repository/files", user), put_params
+    end
+
+    it "remains unchanged" do
+      get api("/projects/#{project.id}/repository/files", user), get_params
+      expect(response.status).to eq(200)
+      expect(json_response['file_path']).to eq(file_path)
+      expect(json_response['file_name']).to eq(file_path)
+      expect(json_response['content']).to eq(put_params[:content])
+    end
+  end
 end

spec/requests/api/merge_requests_spec.rb

@@ -148,7 +148,7 @@ describe API::API, api: true  do
       it "should return merge_request" do
         post api("/projects/#{project.id}/merge_requests", user),
              title: 'Test merge_request',
-             source_branch: 'feature_conflict',
+             source_branch: 'stable',
              target_branch: 'master',
              author: user,
              labels: 'label, label2'
@@ -171,20 +171,20 @@ describe API::API, api: true  do
 
       it "should return 400 when target_branch is missing" do
         post api("/projects/#{project.id}/merge_requests", user),
-        title: "Test merge_request", source_branch: "markdown", author: user
+        title: "Test merge_request", source_branch: "stable", author: user
         expect(response.status).to eq(400)
       end
 
       it "should return 400 when title is missing" do
         post api("/projects/#{project.id}/merge_requests", user),
-        target_branch: 'master', source_branch: 'markdown'
+        target_branch: 'master', source_branch: 'stable'
         expect(response.status).to eq(400)
       end
 
       it 'should return 400 on invalid label names' do
         post api("/projects/#{project.id}/merge_requests", user),
              title: 'Test merge_request',
-             source_branch: 'markdown',
+             source_branch: 'stable',
              target_branch: 'master',
              author: user,
              labels: 'label, ?'
@@ -198,7 +198,7 @@ describe API::API, api: true  do
         before do
           post api("/projects/#{project.id}/merge_requests", user),
                title: 'Test merge_request',
-               source_branch: 'feature_conflict',
+               source_branch: 'stable',
                target_branch: 'master',
                author: user
           @mr = MergeRequest.all.last
@@ -208,7 +208,7 @@ describe API::API, api: true  do
           expect do
             post api("/projects/#{project.id}/merge_requests", user),
                  title: 'New test merge_request',
-                 source_branch: 'feature_conflict',
+                 source_branch: 'stable',
                  target_branch: 'master',
                  author: user
           end.to change { MergeRequest.count }.by(0)
@@ -228,8 +228,7 @@ describe API::API, api: true  do
 
       it "should return merge_request" do
         post api("/projects/#{fork_project.id}/merge_requests", user2),
-          title: 'Test merge_request', source_branch: "feature_conflict", target_branch: "master",
-          author: user2, target_project_id: project.id, description: 'Test description for Test merge_request'
+        title: 'Test merge_request', source_branch: "stable", target_branch: "master", author: user2, target_project_id: project.id, description: 'Test description for Test merge_request'
         expect(response.status).to eq(201)
         expect(json_response['title']).to eq('Test merge_request')
         expect(json_response['description']).to eq('Test description for Test merge_request')
@@ -259,7 +258,7 @@ describe API::API, api: true  do
 
       it "should return 400 when title is missing" do
         post api("/projects/#{fork_project.id}/merge_requests", user2),
-        target_branch: 'master', source_branch: 'markdown', author: user2, target_project_id: project.id
+        target_branch: 'master', source_branch: 'stable', author: user2, target_project_id: project.id
         expect(response.status).to eq(400)
       end
 
@@ -268,7 +267,7 @@ describe API::API, api: true  do
           post api("/projects/#{project.id}/merge_requests", user),
                title: 'Test merge_request',
                target_branch: 'master',
-               source_branch: 'markdown',
+               source_branch: 'stable',
                author: user,
                target_project_id: fork_project.id
           expect(response.status).to eq(422)
@@ -278,7 +277,7 @@ describe API::API, api: true  do
           post api("/projects/#{fork_project.id}/merge_requests", user2),
                title: 'Test merge_request',
                target_branch: 'master',
-               source_branch: 'markdown',
+               source_branch: 'stable',
                author: user2,
                target_project_id: unrelated_project.id
           expect(response.status).to eq(422)
@@ -287,7 +286,7 @@ describe API::API, api: true  do
 
       it "should return 201 when target_branch is specified and for the same project" do
         post api("/projects/#{fork_project.id}/merge_requests", user2),
-        title: 'Test merge_request', target_branch: 'master', source_branch: 'markdown', author: user2, target_project_id: fork_project.id
+        title: 'Test merge_request', target_branch: 'master', source_branch: 'stable', author: user2, target_project_id: fork_project.id
         expect(response.status).to eq(201)
       end
     end
@@ -303,6 +302,9 @@ describe API::API, api: true  do
 
   describe "PUT /projects/:id/merge_request/:merge_request_id/merge" do
     it "should return merge_request in case of success" do
+      allow_any_instance_of(MergeRequest).
+        to receive_messages(can_be_merged?: true, automerge!: true)
+
       put api("/projects/#{project.id}/merge_request/#{merge_request.id}/merge", user)
 
       expect(response.status).to eq(200)

spec/routing/project_routing_spec.rb

@@ -210,8 +210,8 @@ end
 
 #               diffs_namespace_project_merge_request GET      /:namespace_id/:project_id/merge_requests/:id/diffs(.:format)               projects/merge_requests#diffs
 #             commits_namespace_project_merge_request GET      /:namespace_id/:project_id/merge_requests/:id/commits(.:format)             projects/merge_requests#commits
-#           merge_namespace_project_merge_request POST     /:namespace_id/:project_id/merge_requests/:id/merge(.:format)           projects/merge_requests#merge
-#     merge_check_namespace_project_merge_request GET      /:namespace_id/:project_id/merge_requests/:id/merge_check(.:format)     projects/merge_requests#merge_check
+#           automerge_namespace_project_merge_request POST     /:namespace_id/:project_id/merge_requests/:id/automerge(.:format)           projects/merge_requests#automerge
+#     automerge_check_namespace_project_merge_request GET      /:namespace_id/:project_id/merge_requests/:id/automerge_check(.:format)     projects/merge_requests#automerge_check
 #           ci_status_namespace_project_merge_request GET      /:namespace_id/:project_id/merge_requests/:id/ci_status(.:format)           projects/merge_requests#ci_status
 # toggle_subscription_namespace_project_merge_request POST     /:namespace_id/:project_id/merge_requests/:id/toggle_subscription(.:format) projects/merge_requests#toggle_subscription
 #        branch_from_namespace_project_merge_requests GET      /:namespace_id/:project_id/merge_requests/branch_from(.:format)             projects/merge_requests#branch_from
@@ -233,15 +233,15 @@ describe Projects::MergeRequestsController, 'routing' do
     expect(get('/gitlab/gitlabhq/merge_requests/1/commits')).to route_to('projects/merge_requests#commits', namespace_id: 'gitlab', project_id: 'gitlabhq', id: '1')
   end
 
-  it 'to #merge' do
-    expect(post('/gitlab/gitlabhq/merge_requests/1/merge')).to route_to(
-      'projects/merge_requests#merge',
+  it 'to #automerge' do
+    expect(post('/gitlab/gitlabhq/merge_requests/1/automerge')).to route_to(
+      'projects/merge_requests#automerge',
       namespace_id: 'gitlab', project_id: 'gitlabhq', id: '1'
     )
   end
 
-  it 'to #merge_check' do
-    expect(get('/gitlab/gitlabhq/merge_requests/1/merge_check')).to route_to('projects/merge_requests#merge_check', namespace_id: 'gitlab', project_id: 'gitlabhq', id: '1')
+  it 'to #automerge_check' do
+    expect(get('/gitlab/gitlabhq/merge_requests/1/automerge_check')).to route_to('projects/merge_requests#automerge_check', namespace_id: 'gitlab', project_id: 'gitlabhq', id: '1')
   end
 
   it 'to #branch_from' do

spec/services/merge_requests/create_service_spec.rb

@@ -10,7 +10,7 @@ describe MergeRequests::CreateService do
         {
           title: 'Awesome merge_request',
           description: 'please fix',
-          source_branch: 'feature',
+          source_branch: 'stable',
           target_branch: 'master'
         }
       end

spec/services/merge_requests/merge_service_spec.rb

@@ -24,6 +24,11 @@ describe MergeRequests::MergeService do
       it { expect(merge_request).to be_valid }
       it { expect(merge_request).to be_merged }
 
+      it 'should execute hooks with merge action' do
+        expect(service).to have_received(:execute_hooks).
+                               with(merge_request, 'merge')
+      end
+
       it 'should send email to user2 about merge of new merge_request' do
         email = ActionMailer::Base.deliveries.last
         expect(email.to.first).to eq(user2.email)

spec/support/mentionable_shared_examples.rb

@@ -9,7 +9,7 @@ def common_mentionable_setup
   let(:author)  { subject.author }
 
   let(:mentioned_issue)  { create(:issue, project: project) }
-  let!(:mentioned_mr)     { create(:merge_request, :simple, source_project: project) }
+  let(:mentioned_mr)     { create(:merge_request, :simple, source_project: project) }
   let(:mentioned_commit) { project.commit }
 
   let(:ext_proj)   { create(:project, :public) }