Merge branch 'merge-request-push-compare-ui' of...

Merge branch 'merge-request-push-compare-ui' of gitlab.com:gitlab-org/gitlab-ce into merge-request-push-compare-ui

CHANGELOG

@@ -5,7 +5,9 @@ v 8.12.0 (unreleased)
   - Only check :can_resolve permission if the note is resolvable
   - Bump fog-aws to v0.11.0 to support ap-south-1 region
   - Add ability to fork to a specific namespace using API. (ritave)
+  - Allow to set request_access_enabled for groups and projects
   - Cleanup misalignments in Issue list view !6206
+  - Only create a protected branch upon a push to a new branch if a rule for that branch doesn't exist
   - Prune events older than 12 months. (ritave)
   - Prepend blank line to `Closes` message on merge request linked to issue (lukehowell)
   - Fix issues/merge-request templates dropdown for forked projects
@@ -15,6 +17,7 @@ v 8.12.0 (unreleased)
   - Make push events have equal vertical spacing.
   - Add two-factor recovery endpoint to internal API !5510
   - Pass the "Remember me" value to the U2F authentication form
+  - Only update projects.last_activity_at once per hour when creating a new event
   - Remove vendor prefixes for linear-gradient CSS (ClemMakesApps)
   - Move pushes_since_gc from the database to Redis
   - Add font color contrast to external label in admin area (ClemMakesApps)
@@ -22,9 +25,12 @@ v 8.12.0 (unreleased)
   - Instructions for enabling Git packfile bitmaps !6104
   - Use Search::GlobalService.new in the `GET /projects/search/:query` endpoint
   - Fix pagination on user snippets page
+  - Run CI builds with the permissions of users !5735
   - Fix sorting of issues in API
   - Sort project variables by key. !6275 (Diego Souza)
   - Ensure specs on sorting of issues in API are deterministic on MySQL
+  - Added ability to use predefined CI variables for environment name
+  - Added ability to specify URL in environment configuration in gitlab-ci.yml
   - Escape search term before passing it to Regexp.new !6241 (winniehell)
   - Fix pinned sidebar behavior in smaller viewports !6169
   - Fix file permissions change when updating a file on the Gitlab UI !5979
@@ -139,6 +145,7 @@ v 8.12.0 (unreleased)
   - Use default clone protocol on "check out, review, and merge locally" help page URL
   - API for Ci Lint !5953 (Katarzyna Kobierska Urszula Budziszewska)
   - Allow bulk update merge requests from merge requests index page
+  - Ensure validation messages are shown within the milestone form
   - Add notification_settings API calls !5632 (mahcsig)
   - Remove duplication between project builds and admin builds view !5680 (Katarzyna Kobierska Ula Budziszewska)
   - Fix URLs with anchors in wiki !6300 (houqp)
@@ -148,7 +155,9 @@ v 8.12.0 (unreleased)
   - Add specs to removing project (Katarzyna Kobierska Ula Budziszewska)
   - Clean environment variables when running git hooks
   - Add UX improvements for merge request version diffs
+  - Fix Import/Export issues importing protected branches and some specific models
   - Fix non-master branch readme display in tree view
+  - Add UX improvements for merge request version diffs
 
 v 8.11.6
   - Fix unnecessary horizontal scroll area in pipeline visualizations. !6005

README.md

@@ -3,10 +3,11 @@
 [![build status](https://gitlab.com/gitlab-org/gitlab-ce/badges/master/build.svg)](https://gitlab.com/gitlab-org/gitlab-ce/commits/master)
 [![coverage report](https://gitlab.com/gitlab-org/gitlab-ce/badges/master/coverage.svg?job=coverage)](https://gitlab.com/gitlab-org/gitlab-ce/commits/master)
 [![Code Climate](https://codeclimate.com/github/gitlabhq/gitlabhq.svg)](https://codeclimate.com/github/gitlabhq/gitlabhq)
+[![Core Infrastructure Initiative Best Practices](https://bestpractices.coreinfrastructure.org/projects/42/badge)](https://bestpractices.coreinfrastructure.org/projects/42)
 
 ## Canonical source
 
-The source of GitLab Community Edition is [hosted on GitLab.com](https://gitlab.com/gitlab-org/gitlab-ce/) and there are mirrors to make [contributing](CONTRIBUTING.md) as easy as possible.
+The cannonical source of GitLab Community Edition is [hosted on GitLab.com](https://gitlab.com/gitlab-org/gitlab-ce/).
 
 ## Open source software to collaborate on code
 

app/controllers/jwt_controller.rb

@@ -11,7 +11,10 @@ class JwtController < ApplicationController
     service = SERVICES[params[:service]]
     return head :not_found unless service
 
-    result = service.new(@project, @user, auth_params).execute
+    @authentication_result ||= Gitlab::Auth::Result.new
+
+    result = service.new(@authentication_result.project, @authentication_result.actor, auth_params).
+      execute(authentication_abilities: @authentication_result.authentication_abilities)
 
     render json: result, status: result[:http_status]
   end
@@ -20,30 +23,23 @@ class JwtController < ApplicationController
 
   def authenticate_project_or_user
     authenticate_with_http_basic do |login, password|
-      # if it's possible we first try to authenticate project with login and password
-      @project = authenticate_project(login, password)
-      return if @project
-
-      @user = authenticate_user(login, password)
-      return if @user
+      @authentication_result = Gitlab::Auth.find_for_git_client(login, password, project: nil, ip: request.ip)
 
-      render_403
+      render_403 unless @authentication_result.success? &&
+        (@authentication_result.actor.nil? || @authentication_result.actor.is_a?(User))
     end
+  rescue Gitlab::Auth::MissingPersonalTokenError
+    render_missing_personal_token
   end
 
-  def auth_params
-    params.permit(:service, :scope, :account, :client_id)
+  def render_missing_personal_token
+    render plain: "HTTP Basic: Access denied\n" \
+                  "You have 2FA enabled, please use a personal access token for Git over HTTP.\n" \
+                  "You can generate one at #{profile_personal_access_tokens_url}",
+           status: 401
   end
 
-  def authenticate_project(login, password)
-    if login == 'gitlab-ci-token'
-      Project.with_builds_enabled.find_by(runners_token: password)
-    end
-  end
-
-  def authenticate_user(login, password)
-    user = Gitlab::Auth.find_with_user_password(login, password)
-    Gitlab::Auth.rate_limit!(request.ip, success: user.present?, login: login)
-    user
+  def auth_params
+    params.permit(:service, :scope, :account, :client_id)
   end
 end

app/controllers/projects/builds_controller.rb

@@ -35,7 +35,11 @@ class Projects::BuildsController < Projects::ApplicationController
     respond_to do |format|
       format.html
       format.json do
-        render json: @build.to_json(methods: :trace_html)
+        render json: {
+          id: @build.id,
+          status: @build.status,
+          trace_html: @build.trace_html
+        }
       end
     end
   end

app/controllers/projects/git_http_client_controller.rb

@@ -4,7 +4,11 @@ class Projects::GitHttpClientController < Projects::ApplicationController
   include ActionController::HttpAuthentication::Basic
   include KerberosSpnegoHelper
 
-  attr_reader :user
+  attr_reader :authentication_result
+
+  delegate :actor, :authentication_abilities, to: :authentication_result, allow_nil: true
+
+  alias_method :user, :actor
 
   # Git clients will not know what authenticity token to send along
   skip_before_action :verify_authenticity_token
@@ -15,32 +19,25 @@ class Projects::GitHttpClientController < Projects::ApplicationController
   private
 
   def authenticate_user
+    @authentication_result = Gitlab::Auth::Result.new
+
     if project && project.public? && download_request?
       return # Allow access
     end
 
     if allow_basic_auth? && basic_auth_provided?
       login, password = user_name_and_password(request)
-      auth_result = Gitlab::Auth.find_for_git_client(login, password, project: project, ip: request.ip)
-
-      if auth_result.type == :ci && download_request?
-        @ci = true
-      elsif auth_result.type == :oauth && !download_request?
-        # Not allowed
-      elsif auth_result.type == :missing_personal_token
-        render_missing_personal_token
-        return # Render above denied access, nothing left to do
-      else
-        @user = auth_result.user
-      end
 
-      if ci? || user
+      if handle_basic_authentication(login, password)
         return # Allow access
       end
     elsif allow_kerberos_spnego_auth? && spnego_provided?
-      @user = find_kerberos_user
+      user = find_kerberos_user
 
       if user
+        @authentication_result = Gitlab::Auth::Result.new(
+          user, nil, :kerberos, Gitlab::Auth.full_authentication_abilities)
+
         send_final_spnego_response
         return # Allow access
       end
@@ -48,6 +45,8 @@ class Projects::GitHttpClientController < Projects::ApplicationController
 
     send_challenges
     render plain: "HTTP Basic: Access denied\n", status: 401
+  rescue Gitlab::Auth::MissingPersonalTokenError
+    render_missing_personal_token
   end
 
   def basic_auth_provided?
@@ -114,8 +113,39 @@ class Projects::GitHttpClientController < Projects::ApplicationController
     render plain: 'Not Found', status: :not_found
   end
 
+  def handle_basic_authentication(login, password)
+    @authentication_result = Gitlab::Auth.find_for_git_client(
+      login, password, project: project, ip: request.ip)
+
+    return false unless @authentication_result.success?
+
+    if download_request?
+      authentication_has_download_access?
+    else
+      authentication_has_upload_access?
+    end
+  end
+
   def ci?
-    @ci.present?
+    authentication_result.ci? &&
+      authentication_project &&
+      authentication_project == project
+  end
+
+  def authentication_has_download_access?
+    has_authentication_ability?(:download_code) || has_authentication_ability?(:build_download_code)
+  end
+
+  def authentication_has_upload_access?
+    has_authentication_ability?(:push_code)
+  end
+
+  def has_authentication_ability?(capability)
+    (authentication_abilities || []).include?(capability)
+  end
+
+  def authentication_project
+    authentication_result.project
   end
 
   def verify_workhorse_api!

app/controllers/projects/git_http_controller.rb

@@ -86,7 +86,7 @@ class Projects::GitHttpController < Projects::GitHttpClientController
   end
 
   def access
-    @access ||= Gitlab::GitAccess.new(user, project, 'http')
+    @access ||= Gitlab::GitAccess.new(user, project, 'http', authentication_abilities: authentication_abilities)
   end
 
   def access_check

app/controllers/projects/merge_requests_controller.rb

@@ -428,6 +428,10 @@ class Projects::MergeRequestsController < Projects::ApplicationController
   end
 
   def validates_merge_request
+    # If source project was removed and merge request for some reason
+    # wasn't close (Ex. mr from fork to origin)
+    return invalid_mr if !@merge_request.source_project && @merge_request.open?
+
     # Show git not found page
     # if there is no saved commits between source & target branch
     if @merge_request.commits.blank?

app/helpers/lfs_helper.rb

@@ -25,13 +25,21 @@ module LfsHelper
   def lfs_download_access?
     return false unless project.lfs_enabled?
 
-    project.public? || ci? || (user && user.can?(:download_code, project))
+    project.public? || ci? || user_can_download_code? || build_can_download_code?
+  end
+
+  def user_can_download_code?
+    has_authentication_ability?(:download_code) && can?(user, :download_code, project)
+  end
+
+  def build_can_download_code?
+    has_authentication_ability?(:build_download_code) && can?(user, :build_download_code, project)
   end
 
   def lfs_upload_access?
     return false unless project.lfs_enabled?
 
-    user && user.can?(:push_code, project)
+    has_authentication_ability?(:push_code) && can?(user, :push_code, project)
   end
 
   def render_lfs_forbidden

app/models/ci/build.rb

 module Ci
   class Build < CommitStatus
+    include TokenAuthenticatable
+
     belongs_to :runner, class_name: 'Ci::Runner'
     belongs_to :trigger_request, class_name: 'Ci::TriggerRequest'
     belongs_to :erased_by, class_name: 'User'
@@ -23,7 +25,10 @@ module Ci
 
     acts_as_taggable
 
+    add_authentication_token_field :token
+
     before_save :update_artifacts_size, if: :artifacts_file_changed?
+    before_save :ensure_token
     before_destroy { project }
 
     after_create :execute_hooks
@@ -38,6 +43,7 @@ module Ci
         new_build.status = 'pending'
         new_build.runner_id = nil
         new_build.trigger_request_id = nil
+        new_build.token = nil
         new_build.save
       end
 
@@ -79,11 +85,14 @@ module Ci
 
       after_transition any => [:success] do |build|
         if build.environment.present?
-          service = CreateDeploymentService.new(build.project, build.user,
-                                                environment: build.environment,
-                                                sha: build.sha,
-                                                ref: build.ref,
-                                                tag: build.tag)
+          service = CreateDeploymentService.new(
+            build.project, build.user,
+            environment: build.environment,
+            sha: build.sha,
+            ref: build.ref,
+            tag: build.tag,
+            options: build.options[:environment],
+            variables: build.variables)
           service.execute(build)
         end
       end
@@ -173,7 +182,7 @@ module Ci
     end
 
     def repo_url
-      auth = "gitlab-ci-token:#{token}@"
+      auth = "gitlab-ci-token:#{ensure_token!}@"
       project.http_url_to_repo.sub(/^https?:\/\//) do |prefix|
         prefix + auth
       end
@@ -235,12 +244,7 @@ module Ci
     end
 
     def trace
-      trace = raw_trace
-      if project && trace.present? && project.runners_token.present?
-        trace.gsub(project.runners_token, 'xxxxxx')
-      else
-        trace
-      end
+      hide_secrets(raw_trace)
     end
 
     def trace_length
@@ -253,6 +257,7 @@ module Ci
 
     def trace=(trace)
       recreate_trace_dir
+      trace = hide_secrets(trace)
       File.write(path_to_trace, trace)
     end
 
@@ -266,6 +271,8 @@ module Ci
     def append_trace(trace_part, offset)
       recreate_trace_dir
 
+      trace_part = hide_secrets(trace_part)
+
       File.truncate(path_to_trace, offset) if File.exist?(path_to_trace)
       File.open(path_to_trace, 'ab') do |f|
         f.write(trace_part)
@@ -341,12 +348,8 @@ module Ci
       )
     end
 
-    def token
-      project.runners_token
-    end
-
     def valid_token?(token)
-      project.valid_runners_token?(token)
+      self.token && ActiveSupport::SecurityUtils.variable_size_secure_compare(token, self.token)
     end
 
     def has_tags?
@@ -488,5 +491,11 @@ module Ci
 
       pipeline.config_processor.build_attributes(name)
     end
+
+    def hide_secrets(trace)
+      trace = Ci::MaskSecret.mask(trace, project.runners_token) if project
+      trace = Ci::MaskSecret.mask(trace, token)
+      trace
+    end
   end
 end

app/models/ci/pipeline.rb

@@ -2,6 +2,7 @@ module Ci
   class Pipeline < ActiveRecord::Base
     extend Ci::Model
     include HasStatus
+    include Importable
 
     self.table_name = 'ci_commits'
 
@@ -12,12 +13,12 @@ module Ci
     has_many :builds, class_name: 'Ci::Build', foreign_key: :commit_id
     has_many :trigger_requests, dependent: :destroy, class_name: 'Ci::TriggerRequest', foreign_key: :commit_id
 
-    validates_presence_of :sha
-    validates_presence_of :ref
-    validates_presence_of :status
-    validate :valid_commit_sha
+    validates_presence_of :sha, unless: :importing?
+    validates_presence_of :ref, unless: :importing?
+    validates_presence_of :status, unless: :importing?
+    validate :valid_commit_sha, unless: :importing?
 
-    after_save :keep_around_commits
+    after_save :keep_around_commits, unless: :importing?
 
     delegate :stages, to: :statuses
 

app/models/concerns/has_status.rb

@@ -20,7 +20,7 @@ module HasStatus
       skipped = scope.skipped.select('count(*)').to_sql
 
       deduce_status = "(CASE
-        WHEN (#{builds})=(#{created}) THEN NULL
+        WHEN (#{builds})=(#{created}) THEN 'created'
         WHEN (#{builds})=(#{skipped}) THEN 'skipped'
         WHEN (#{builds})=(#{success})+(#{ignored})+(#{skipped}) THEN 'success'
         WHEN (#{builds})=(#{created})+(#{pending})+(#{skipped}) THEN 'pending'

app/models/environment.rb

@@ -4,6 +4,7 @@ class Environment < ActiveRecord::Base
   has_many :deployments
 
   before_validation :nullify_external_url
+  before_save :set_environment_type
 
   validates :name,
             presence: true,
@@ -26,6 +27,17 @@ class Environment < ActiveRecord::Base
     self.external_url = nil if self.external_url.blank?
   end
 
+  def set_environment_type
+    names = name.split('/')
+
+    self.environment_type =
+      if names.many?
+        names.first
+      else
+        nil
+      end
+  end
+
   def includes_commit?(commit)
     return false unless last_deployment
 

app/models/event.rb

@@ -13,6 +13,8 @@ class Event < ActiveRecord::Base
   LEFT      = 9 # User left project
   DESTROYED = 10
 
+  RESET_PROJECT_ACTIVITY_INTERVAL = 1.hour
+
   delegate :name, :email, to: :author, prefix: true, allow_nil: true
   delegate :title, to: :issue, prefix: true, allow_nil: true
   delegate :title, to: :merge_request, prefix: true, allow_nil: true
@@ -324,8 +326,17 @@ class Event < ActiveRecord::Base
   end
 
   def reset_project_activity
-    if project && Gitlab::ExclusiveLease.new("project:update_last_activity_at:#{project.id}", timeout: 60).try_obtain
-      project.update_column(:last_activity_at, self.created_at)
-    end
+    return unless project
+
+    # Don't even bother obtaining a lock if the last update happened less than
+    # 60 minutes ago.
+    return if project.last_activity_at > RESET_PROJECT_ACTIVITY_INTERVAL.ago
+
+    return unless Gitlab::ExclusiveLease.
+      new("project:update_last_activity_at:#{project.id}",
+          timeout: RESET_PROJECT_ACTIVITY_INTERVAL.to_i).
+      try_obtain
+
+    project.update_column(:last_activity_at, created_at)
   end
 end

app/models/project.rb

@@ -1137,12 +1137,6 @@ class Project < ActiveRecord::Base
     self.runners_token && ActiveSupport::SecurityUtils.variable_size_secure_compare(token, self.runners_token)
   end
 
-  # TODO (ayufan): For now we use runners_token (backward compatibility)
-  # In 8.4 every build will have its own individual token valid for time of build
-  def valid_build_token?(token)
-    self.builds_enabled? && self.runners_token && ActiveSupport::SecurityUtils.variable_size_secure_compare(token, self.runners_token)
-  end
-
   def build_coverage_enabled?
     build_coverage_regex.present?
   end

app/policies/project_policy.rb

@@ -64,6 +64,12 @@ class ProjectPolicy < BasePolicy
     can! :read_deployment
   end
 
+  # Permissions given when an user is team member of a project
+  def team_member_reporter_access!
+    can! :build_download_code
+    can! :build_read_container_image
+  end
+
   def developer_access!
     can! :admin_merge_request
     can! :update_merge_request
@@ -109,6 +115,8 @@ class ProjectPolicy < BasePolicy
     can! :read_commit_status
     can! :read_pipeline
     can! :read_container_image
+    can! :build_download_code
+    can! :build_read_container_image
   end
 
   def owner_access!
@@ -130,10 +138,11 @@ class ProjectPolicy < BasePolicy
   def team_access!(user)
     access = project.team.max_member_access(user.id)
 
-    guest_access!     if access >= Gitlab::Access::GUEST
-    reporter_access!  if access >= Gitlab::Access::REPORTER
-    developer_access! if access >= Gitlab::Access::DEVELOPER
-    master_access!    if access >= Gitlab::Access::MASTER
+    guest_access!                if access >= Gitlab::Access::GUEST
+    reporter_access!             if access >= Gitlab::Access::REPORTER
+    team_member_reporter_access! if access >= Gitlab::Access::REPORTER
+    developer_access!            if access >= Gitlab::Access::DEVELOPER
+    master_access!               if access >= Gitlab::Access::MASTER
   end
 
   def archived_access!

app/services/auth/container_registry_authentication_service.rb

@@ -4,7 +4,9 @@ module Auth
 
     AUDIENCE = 'container_registry'
 
-    def execute
+    def execute(authentication_abilities:)
+      @authentication_abilities = authentication_abilities || []
+
       return error('not found', 404) unless registry.enabled
 
       unless current_user || project
@@ -74,9 +76,9 @@ module Auth
 
       case requested_action
       when 'pull'
-        requested_project == project || can?(current_user, :read_container_image, requested_project)
+        requested_project.public? || build_can_pull?(requested_project) || user_can_pull?(requested_project)
       when 'push'
-        requested_project == project || can?(current_user, :create_container_image, requested_project)
+        build_can_push?(requested_project) || user_can_push?(requested_project)
       else
         false
       end
@@ -85,5 +87,29 @@ module Auth
     def registry
       Gitlab.config.registry
     end
+
+    def build_can_pull?(requested_project)
+      # Build can:
+      # 1. pull from its own project (for ex. a build)
+      # 2. read images from dependent projects if creator of build is a team member
+      @authentication_abilities.include?(:build_read_container_image) &&
+        (requested_project == project || can?(current_user, :build_read_container_image, requested_project))
+    end
+
+    def user_can_pull?(requested_project)
+      @authentication_abilities.include?(:read_container_image) &&
+        can?(current_user, :read_container_image, requested_project)
+    end
+
+    def build_can_push?(requested_project)
+      # Build can push only to the project from which it originates
+      @authentication_abilities.include?(:build_create_container_image) &&
+        requested_project == project
+    end
+
+    def user_can_push?(requested_project)
+      @authentication_abilities.include?(:create_container_image) &&
+        can?(current_user, :create_container_image, requested_project)
+    end
   end
 end

app/services/create_deployment_service.rb

@@ -2,9 +2,7 @@ require_relative 'base_service'
 
 class CreateDeploymentService < BaseService
   def execute(deployable = nil)
-    environment = project.environments.find_or_create_by(
-      name: params[:environment]
-    )
+    environment = find_or_create_environment
 
     project.deployments.create(
       environment: environment,
@@ -15,4 +13,38 @@ class CreateDeploymentService < BaseService
       deployable: deployable
     )
   end
+
+  private
+
+  def find_or_create_environment
+    project.environments.find_or_create_by(name: expanded_name) do |environment|
+      environment.external_url = expanded_url
+    end
+  end
+
+  def expanded_name
+    ExpandVariables.expand(name, variables)
+  end
+
+  def expanded_url
+    return unless url
+
+    @expanded_url ||= ExpandVariables.expand(url, variables)
+  end
+
+  def name
+    params[:environment]
+  end
+
+  def url
+    options[:url]
+  end
+
+  def options
+    params[:options] || {}
+  end
+
+  def variables
+    params[:variables] || []
+  end
 end

app/services/git_push_service.rb

@@ -87,7 +87,7 @@ class GitPushService < BaseService
     project.change_head(branch_name)
 
     # Set protection on the default branch if configured
-    if current_application_settings.default_branch_protection != PROTECTION_NONE
+    if current_application_settings.default_branch_protection != PROTECTION_NONE && !@project.protected_branch?(@project.default_branch)
 
       params = {
         name: @project.default_branch,

app/services/milestones/create_service.rb

@@ -3,7 +3,7 @@ module Milestones
     def execute
       milestone = project.milestones.new(params)
 
-      if milestone.save!
+      if milestone.save
         event_service.open_milestone(milestone, current_user)
       end
 

db/migrate/20160808085531_add_token_to_build.rb

+class AddTokenToBuild < ActiveRecord::Migration
+  include Gitlab::Database::MigrationHelpers
+
+  # Set this constant to true if this migration requires downtime.
+  DOWNTIME = false
+
+  def change
+    add_column :ci_builds, :token, :string
+  end
+end

db/migrate/20160808085602_add_index_for_build_token.rb

+class AddIndexForBuildToken < ActiveRecord::Migration
+  include Gitlab::Database::MigrationHelpers
+
+  # Set this constant to true if this migration requires downtime.
+  DOWNTIME = false
+
+  disable_ddl_transaction!
+
+  def change
+    add_concurrent_index :ci_builds, :token, unique: true
+  end
+end

db/migrate/20160907131111_add_environment_type_to_environments.rb

+class AddEnvironmentTypeToEnvironments < ActiveRecord::Migration
+  include Gitlab::Database::MigrationHelpers
+
+  DOWNTIME = false
+
+  def change
+    add_column :environments, :environment_type, :string
+  end
+end

db/schema.rb

@@ -181,6 +181,7 @@ ActiveRecord::Schema.define(version: 20160913212128) do
     t.string   "when"
     t.text     "yaml_variables"
     t.datetime "queued_at"
+    t.string   "token"
   end
 
   add_index "ci_builds", ["commit_id", "stage_idx", "created_at"], name: "index_ci_builds_on_commit_id_and_stage_idx_and_created_at", using: :btree
@@ -192,6 +193,7 @@ ActiveRecord::Schema.define(version: 20160913212128) do
   add_index "ci_builds", ["project_id"], name: "index_ci_builds_on_project_id", using: :btree
   add_index "ci_builds", ["runner_id"], name: "index_ci_builds_on_runner_id", using: :btree
   add_index "ci_builds", ["status"], name: "index_ci_builds_on_status", using: :btree
+  add_index "ci_builds", ["token"], name: "index_ci_builds_on_token", unique: true, using: :btree
 
   create_table "ci_commits", force: :cascade do |t|
     t.integer  "project_id"
@@ -390,10 +392,11 @@ ActiveRecord::Schema.define(version: 20160913212128) do
 
   create_table "environments", force: :cascade do |t|
     t.integer  "project_id"
-    t.string   "name",         null: false
+    t.string   "name",             null: false
     t.datetime "created_at"
     t.datetime "updated_at"
     t.string   "external_url"
+    t.string   "environment_type"
   end
 
   add_index "environments", ["project_id", "name"], name: "index_environments_on_project_id_and_name", using: :btree

doc/api/groups.md

@@ -84,7 +84,8 @@ Parameters:
     "forks_count": 0,
     "open_issues_count": 3,
     "public_builds": true,
-    "shared_with_groups": []
+    "shared_with_groups": [],
+    "request_access_enabled": false
   }
 ]
 ```
@@ -118,6 +119,7 @@ Example response:
   "visibility_level": 20,
   "avatar_url": null,
   "web_url": "https://gitlab.example.com/groups/twitter",
+  "request_access_enabled": false,
   "projects": [
     {
       "id": 7,
@@ -163,7 +165,8 @@ Example response:
       "forks_count": 0,
       "open_issues_count": 3,
       "public_builds": true,
-      "shared_with_groups": []
+      "shared_with_groups": [],
+      "request_access_enabled": false
     },
     {
       "id": 6,
@@ -209,7 +212,8 @@ Example response:
       "forks_count": 0,
       "open_issues_count": 8,
       "public_builds": true,
-      "shared_with_groups": []
+      "shared_with_groups": [],
+      "request_access_enabled": false
     }
   ],
   "shared_projects": [
@@ -289,6 +293,7 @@ Parameters:
 - `description` (optional) - The group's description
 - `visibility_level` (optional) - The group's visibility. 0 for private, 10 for internal, 20 for public.
 - `lfs_enabled` (optional)      - Enable/disable Large File Storage (LFS) for the projects in this group
+- `request_access_enabled` (optional) - Allow users to request member access.
 
 ## Transfer project to group
 
@@ -319,6 +324,7 @@ PUT /groups/:id
 | `description` | string | no | The description of the group |
 | `visibility_level` | integer | no | The visibility level of the group. 0 for private, 10 for internal, 20 for public. |
 | `lfs_enabled` (optional) | boolean | no | Enable/disable Large File Storage (LFS) for the projects in this group |
+| `request_access_enabled` | boolean | no | Allow users to request member access. |
 
 ```bash
 curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/groups/5?name=Experimental"
@@ -336,6 +342,7 @@ Example response:
   "visibility_level": 10,
   "avatar_url": null,
   "web_url": "http://gitlab.example.com/groups/h5bp",
+  "request_access_enabled": false,
   "projects": [
     {
       "id": 9,
@@ -380,7 +387,8 @@ Example response:
       "forks_count": 0,
       "open_issues_count": 3,
       "public_builds": true,
-      "shared_with_groups": []
+      "shared_with_groups": [],
+      "request_access_enabled": false
     }
   ]
 }

doc/api/projects.md

@@ -85,7 +85,8 @@ Parameters:
     "runners_token": "b8547b1dc37721d05889db52fa2f02",
     "public_builds": true,
     "shared_with_groups": [],
-    "only_allow_merge_if_build_succeeds": false
+    "only_allow_merge_if_build_succeeds": false,
+    "request_access_enabled": false
   },
   {
     "id": 6,
@@ -146,7 +147,8 @@ Parameters:
     "runners_token": "b8547b1dc37721d05889db52fa2f02",
     "public_builds": true,
     "shared_with_groups": [],
-    "only_allow_merge_if_build_succeeds": false
+    "only_allow_merge_if_build_succeeds": false,
+    "request_access_enabled": false
   }
 ]
 ```
@@ -283,7 +285,8 @@ Parameters:
       "group_access_level": 10
     }
   ],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 
@@ -453,6 +456,7 @@ Parameters:
 - `public_builds` (optional)
 - `only_allow_merge_if_build_succeeds` (optional)
 - `lfs_enabled` (optional)
+- `request_access_enabled` (optional) - Allow users to request member access.
 
 ### Create project for user
 
@@ -480,6 +484,7 @@ Parameters:
 - `public_builds` (optional)
 - `only_allow_merge_if_build_succeeds` (optional)
 - `lfs_enabled` (optional)
+- `request_access_enabled` (optional) - Allow users to request member access.
 
 ### Edit project
 
@@ -508,6 +513,7 @@ Parameters:
 - `public_builds` (optional)
 - `only_allow_merge_if_build_succeeds` (optional)
 - `lfs_enabled` (optional)
+- `request_access_enabled` (optional) - Allow users to request member access.
 
 On success, method returns 200 with the updated project. If parameters are
 invalid, 400 is returned.
@@ -588,7 +594,8 @@ Example response:
   "star_count": 1,
   "public_builds": true,
   "shared_with_groups": [],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 
@@ -655,7 +662,8 @@ Example response:
   "star_count": 0,
   "public_builds": true,
   "shared_with_groups": [],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 
@@ -742,7 +750,8 @@ Example response:
   "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b",
   "public_builds": true,
   "shared_with_groups": [],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 
@@ -829,7 +838,8 @@ Example response:
   "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b",
   "public_builds": true,
   "shared_with_groups": [],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 

doc/api/settings.md

@@ -67,7 +67,7 @@ PUT /application/settings
 | `default_snippet_visibility` | integer | no | What visibility level new snippets receive. Can take `0` _(Private)_, `1` _(Internal)_ and `2` _(Public)_ as a parameter. Default is `0`.|
 | `domain_whitelist` | array of strings | no | Force people to use only corporate emails for sign-up. Default is null, meaning there is no restriction. |
 | `domain_blacklist_enabled` | boolean | no | Enable/disable the `domain_blacklist` |
-| `domain_blacklist` | array of strings | yes (if `domain_whitelist_enabled` is `true` | People trying to sign-up with emails from this domain will not be allowed to do so. |
+| `domain_blacklist` | array of strings | yes (if `domain_blacklist_enabled` is `true`) | People trying to sign-up with emails from this domain will not be allowed to do so. |
 | `user_oauth_applications` | boolean | no | Allow users to register any application to use GitLab as an OAuth provider |
 | `after_sign_out_path` | string | no | Where to redirect users after logout |
 | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes |

doc/ci/examples/README.md

@@ -16,4 +16,4 @@ Apart from those, here is an collection of tutorials and guides on setting up yo
 - [Repo's with examples for various languages](https://gitlab.com/groups/gitlab-examples)
 - [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml)
 
-[gitlab-ci-templates][https://gitlab.com/gitlab-org/gitlab-ci-yml]
+[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml

doc/ci/yaml/README.md

@@ -90,8 +90,7 @@ builds, including deploy builds. This can be an array or a multi-line string.
 
 ### after_script
 
->**Note:**
-Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
+> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
 
 `after_script` is used to define the command that will be run after for all
 builds. This has to be an array or a multi-line string.
@@ -135,8 +134,7 @@ Alias for [stages](#stages).
 
 ### variables
 
->**Note:**
-Introduced in GitLab Runner v0.5.0.
+> Introduced in GitLab Runner v0.5.0.
 
 GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the
 build environment. The variables are stored in the Git repository and are meant
@@ -158,8 +156,7 @@ Variables can be also defined on [job level](#job-variables).
 
 ### cache
 
->**Note:**
-Introduced in GitLab Runner v0.7.0.
+> Introduced in GitLab Runner v0.7.0.
 
 `cache` is used to specify a list of files and directories which should be
 cached between builds.
@@ -220,8 +217,7 @@ will be always present. For implementation details, please check GitLab Runner.
 
 #### cache:key
 
->**Note:**
-Introduced in GitLab Runner v1.0.0.
+> Introduced in GitLab Runner v1.0.0.
 
 The `key` directive allows you to define the affinity of caching
 between jobs, allowing to have a single cache for all jobs,
@@ -531,8 +527,7 @@ The above script will:
 
 #### Manual actions
 
->**Note:**
-Introduced in GitLab 8.10.
+> Introduced in GitLab 8.10.
 
 Manual actions are a special type of job that are not executed automatically;
 they need to be explicitly started by a user. Manual actions can be started
@@ -543,17 +538,16 @@ An example usage of manual actions is deployment to production.
 
 ### environment
 
->**Note:**
-Introduced in GitLab 8.9.
+> Introduced in GitLab 8.9.
 
-`environment` is used to define that a job deploys to a specific environment.
+`environment` is used to define that a job deploys to a specific [environment].
 This allows easy tracking of all deployments to your environments straight from
 GitLab.
 
 If `environment` is specified and no environment under that name exists, a new
 one will be created automatically.
 
-The `environment` name must contain only letters, digits, '-' and '_'. Common
+The `environment` name must contain only letters, digits, '-', '_', '/', '$', '{', '}' and spaces. Common
 names are `qa`, `staging`, and `production`, but you can use whatever name works
 with your workflow.
 
@@ -571,6 +565,35 @@ deploy to production:
 The `deploy to production` job will be marked as doing deployment to
 `production` environment.
 
+#### dynamic environments
+
+> [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6.
+
+`environment` can also represent a configuration hash with `name` and `url`.
+These parameters can use any of the defined CI [variables](#variables)
+(including predefined, secure variables and `.gitlab-ci.yml` variables).
+
+The common use case is to create dynamic environments for branches and use them
+as review apps.
+
+---
+
+**Example configurations**
+
+```
+deploy as review app:
+  stage: deploy
+  script: ...
+  environment:
+    name: review-apps/$CI_BUILD_REF_NAME
+    url: https://$CI_BUILD_REF_NAME.review.example.com/
+```
+
+The `deploy as review app` job will be marked as deployment to dynamically
+create the `review-apps/branch-name` environment.
+
+This environment should be accessible under `https://branch-name.review.example.com/`.
+
 ### artifacts
 
 >**Notes:**
@@ -638,8 +661,7 @@ be available for download in the GitLab UI.
 
 #### artifacts:name
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.0.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.0.
 
 The `name` directive allows you to define the name of the created artifacts
 archive. That way, you can have a unique name for every archive which could be
@@ -702,8 +724,7 @@ job:
 
 #### artifacts:when
 
->**Note:**
-Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
+> Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
 
 `artifacts:when` is used to upload artifacts on build failure or despite the
 failure.
@@ -728,8 +749,7 @@ job:
 
 #### artifacts:expire_in
 
->**Note:**
-Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
+> Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
 
 `artifacts:expire_in` is used to delete uploaded artifacts after the specified
 time. By default, artifacts are stored on GitLab forever. `expire_in` allows you
@@ -764,8 +784,7 @@ job:
 
 ### dependencies
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 This feature should be used in conjunction with [`artifacts`](#artifacts) and
 allows you to define the artifacts to pass between different builds.
@@ -839,9 +858,8 @@ job:
 
 ## Git Strategy
 
->**Note:**
-Introduced in GitLab 8.9 as an experimental feature. May change in future
-releases or be removed completely.
+> Introduced in GitLab 8.9 as an experimental feature. May change in future
+  releases or be removed completely.
 
 You can set the `GIT_STRATEGY` used for getting recent application code. `clone`
 is slower, but makes sure you have a clean directory before every build. `fetch`
@@ -863,8 +881,7 @@ variables:
 
 ## Shallow cloning
 
->**Note:**
-Introduced in GitLab 8.9 as an experimental feature. May change in future
+> Introduced in GitLab 8.9 as an experimental feature. May change in future
 releases or be removed completely.
 
 You can specify the depth of fetching and cloning using `GIT_DEPTH`. This allows
@@ -894,8 +911,7 @@ variables:
 
 ## Hidden keys
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 Keys that start with a dot (`.`) will be not processed by GitLab CI. You can
 use this feature to ignore jobs, or use the
@@ -923,8 +939,7 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya
 
 ### Anchors
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 YAML also has a handy feature called 'anchors', which let you easily duplicate
 content across your document. Anchors can be used to duplicate/inherit
@@ -1067,3 +1082,5 @@ Visit the [examples README][examples] to see a list of examples using GitLab
 CI with various languages.
 
 [examples]: ../examples/README.md
+[ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323
+[environment]: ../environments.md

doc/container_registry/README.md

@@ -78,9 +78,9 @@ delete them.
 > **Note:**
 This feature requires GitLab 8.8 and GitLab Runner 1.2.
 
-Make sure that your GitLab Runner is configured to allow building docker images.
-You have to check the [Using Docker Build documentation](../ci/docker/using_docker_build.md).
-Then see the CI documentation on [Using the GitLab Container Registry](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry).
+Make sure that your GitLab Runner is configured to allow building Docker images by
+following the [Using Docker Build](../ci/docker/using_docker_build.md)
+and [Using the GitLab Container Registry documentation](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry).
 
 ## Limitations
 

doc/raketasks/backup_restore.md

@@ -79,6 +79,9 @@ gitlab_rails['backup_upload_connection'] = {
   'region' => 'eu-west-1',
   'aws_access_key_id' => 'AKIAKIAKI',
   'aws_secret_access_key' => 'secret123'
+  # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty
+  # ie. 'aws_access_key_id' => '',
+  # 'use_iam_profile' => 'true'
 }
 gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket'
 ```
@@ -95,6 +98,9 @@ For installations from source:
         region: eu-west-1
         aws_access_key_id: AKIAKIAKI
         aws_secret_access_key: 'secret123'
+        # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty
+        # ie. aws_access_key_id: ''
+        # use_iam_profile: 'true'
       # The remote 'directory' to store your backups. For S3, this would be the bucket name.
       remote_directory: 'my.s3.bucket'
       # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional

doc/user/project/builds/artifacts.md

@@ -101,4 +101,36 @@ inside GitLab that make that possible.
 
     ![Build artifacts browser](img/build_artifacts_browser.png)
 
+## Downloading the latest build artifacts
+
+It is possible to download the latest artifacts of a build via a well known URL
+so you can use it for scripting purposes.
+
+The structure of the URL is the following:
+
+```
+https://example.com/<namespace>/<project>/builds/artifacts/<ref>/download?job=<job_name>
+```
+
+For example, to download the latest artifacts of the job named `rspec 6 20` of
+the `master` branch of the `gitlab-ce` project that belongs to the `gitlab-org`
+namespace, the URL would be:
+
+```
+https://gitlab.com/gitlab-org/gitlab-ce/builds/artifacts/master/download?job=rspec+6+20
+```
+
+The latest builds are also exposed in the UI in various places. Specifically,
+look for the download button in:
+
+- the main project's page
+- the branches page
+- the tags page
+
+If the latest build has failed to upload the artifacts, you can see that
+information in the UI.
+
+![Latest artifacts button](img/build_latest_artifacts_browser.png)
+
+
 [gitlab workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse "GitLab Workhorse repository"

doc/user/project/settings/import_export.md

@@ -3,8 +3,8 @@
 >**Notes:**
 >
 >  - [Introduced][ce-3050] in GitLab 8.9.
->  - Importing will not be possible if the import instance version is lower
->    than that of the exporter.
+>  - Importing will not be possible if the import instance version differs from
+>    that of the exporter.
 >  - For existing installations, the project import option has to be enabled in
 >    application settings (`/admin/application_settings`) under 'Import sources'.
 >    You will have to be an administrator to enable and use the import functionality.
@@ -17,6 +17,20 @@
 Existing projects running on any GitLab instance or GitLab.com can be exported
 with all their related data and be moved into a new GitLab instance.
 
+## Version history
+
+| GitLab version | Import/Export version |
+| -------- | -------- |
+| 8.12.0 to current  | 0.1.4    |
+| 8.10.3   | 0.1.3    |
+| 8.10.0   | 0.1.2    |
+| 8.9.5    | 0.1.1    |
+| 8.9.0    | 0.1.0    |
+ 
+ > The table reflects what GitLab version we updated the Import/Export version at.
+ > For instance, 8.10.3 and 8.11 will have the same Import/Export version (0.1.3)
+ > and the exports between them will be compatible.
+
 ## Exported contents
 
 The following items will be exported:

doc/workflow/importing/import_projects_from_github.md

 # Import your project from GitHub to GitLab
 
+Import your projects from GitHub to GitLab with minimal effort.
+
+## Overview
+
 >**Note:**
-In order to enable the GitHub import setting, you may also want to
-enable the [GitHub integration][gh-import] in your GitLab instance. This
-configuration is optional, you will be able import your GitHub repositories
-with a Personal Access Token.
+If you are an administrator you can enable the [GitHub integration][gh-import]
+in your GitLab instance sitewide. This configuration is optional, users will be
+able import their GitHub repositories with a [personal access token][gh-token].
 
-At its current state, GitHub importer can import:
+- At its current state, GitHub importer can import:
+  - the repository description (GitLab 7.7+)
+  - the Git repository data (GitLab 7.7+)
+  - the issues (GitLab 7.7+)
+  - the pull requests (GitLab 8.4+)
+  - the wiki pages (GitLab 8.4+)
+  - the milestones (GitLab 8.7+)
+  - the labels (GitLab 8.7+)
+  - the release note descriptions (GitLab 8.12+)
+- References to pull requests and issues are preserved (GitLab 8.7+)
+- Repository public access is retained. If a repository is private in GitHub
+  it will be created as private in GitLab as well.
 
-- the repository description (introduced in GitLab 7.7)
-- the git repository data (introduced in GitLab 7.7)
-- the issues (introduced in GitLab 7.7)
-- the pull requests (introduced in GitLab 8.4)
-- the wiki pages (introduced in GitLab 8.4)
-- the milestones (introduced in GitLab 8.7)
-- the labels (introduced in GitLab 8.7)
-- the release note descriptions (introduced in GitLab 8.12)
+## How it works
 
-With GitLab 8.7+, references to pull requests and issues are preserved.
+When issues/pull requests are being imported, the GitHub importer tries to find
+the GitHub author/assignee in GitLab's database using the GitHub ID. For this
+to work, the GitHub author/assignee should have signed in beforehand in GitLab
+and [**associated their GitHub account**][social sign-in]. If the user is not
+found in GitLab's database, the project creator (most of the times the current
+user that started the import process) is set as the author, but a reference on
+the issue about the original GitHub author is kept.
 
-The importer page is visible when you [create a new project][new-project].
-Click on the **GitHub** link and, if you are logged in via the GitHub
-integration, you will be redirected to GitHub for permission to access your
-projects. After accepting, you'll be automatically redirected to the importer.
+The importer will create any new namespaces (groups) if they don't exist or in
+the case the namespace is taken, the repository will be imported under the user's
+namespace that started the import process.
 
-If you are not using the GitHub integration, you can still perform a one-off
-authorization with GitHub to access your projects.
+## Importing your GitHub repositories
 
-Alternatively, you can also enter a GitHub Personal Access Token. Once you enter
-your token, you'll be taken to the importer.
+The importer page is visible when you create a new project.
 
 ![New project page on GitLab](img/import_projects_from_github_new_project_page.png)
 
----
+Click on the **GitHub** link and the import authorization process will start.
+There are two ways to authorize access to your GitHub repositories:
 
-While at the GitHub importer page, you can see the import statuses of your
-GitHub projects. Those that are being imported will show a _started_ status,
-those already imported will be green, whereas those that are not yet imported
-have an **Import** button on the right side of the table. If you want, you can
-import all your GitHub projects in one go by hitting **Import all projects**
-in the upper left corner.
+1. [Using the GitHub integration][gh-integration] (if it's enabled by your
+   GitLab administrator). This is the preferred way as it's possible to
+   preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works)
+   section.
+1. [Using a personal access token][gh-token] provided by GitHub.
 
-![GitHub importer page](img/import_projects_from_github_importer.png)
+![Select authentication method](img/import_projects_from_github_select_auth_method.png)
+
+### Authorize access to your repositories using the GitHub integration
 
----
+If the [GitHub integration][gh-import] is enabled by your GitLab administrator,
+you can use it instead of the personal access token.
+
+1. First you may want to connect your GitHub account to GitLab in order for
+   the username mapping to be correct. Follow the [social sign-in] documentation
+   on how to do so.
+1. Once you connect GitHub, click the **List your GitHub repositories** button
+   and you will be redirected to GitHub for permission to access your projects.
+1. After accepting, you'll be automatically redirected to the importer.
+
+You can now go on and [select which repositories to import](#select-which-repositories-to-import).
+
+### Authorize access to your repositories using a personal access token
+
+>**Note:**
+For a proper author/assignee mapping for issues and pull requests, the
+[GitHub integration][gh-integration] should be used instead of the
+[personal access token][gh-token]. If the GitHub integration is enabled by your
+GitLab administrator, it should be the preferred method to import your repositories.
+Read more in the [How it works](#how-it-works) section.
 
-The importer will create any new namespaces if they don't exist or in the
-case the namespace is taken, the project will be imported on the user's
-namespace.
+If you are not using the GitHub integration, you can still perform a one-off
+authorization with GitHub to grant GitLab access your repositories:
+
+1. Go to <https://github.com/settings/tokens/new>.
+1. Enter a token description.
+1. Check the `repo` scope.
+1. Click **Generate token**.
+1. Copy the token hash.
+1. Go back to GitLab and provide the token to the GitHub importer.
+1. Hit the **List your GitHub repositories** button and wait while GitLab reads
+   your repositories' information. Once done, you'll be taken to the importer
+   page to select the repositories to import.
+
+### Select which repositories to import
+
+After you've authorized access to your GitHub repositories, you will be
+redirected to the GitHub importer page.
+
+From there, you can see the import statuses of your GitHub repositories.
+
+- Those that are being imported will show a _started_ status,
+- those already successfully imported will be green with a _done_ status,
+- whereas those that are not yet imported will have an **Import** button on the
+  right side of the table.
+
+If you want, you can import all your GitHub projects in one go by hitting
+**Import all projects** in the upper left corner.
+
+![GitHub importer page](img/import_projects_from_github_importer.png)
 
 [gh-import]: ../../integration/github.md "GitHub integration"
-[ee-gh]: http://docs.gitlab.com/ee/integration/github.html "GitHub integration for GitLab EE"
 [new-project]: ../../gitlab-basics/create-project.md "How to create a new project in GitLab"
+[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration
+[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token
+[social sign-in]: ../../profile/account/social_sign_in.md

lib/api/entities.rb

@@ -100,6 +100,7 @@ module API
         SharedGroup.represent(project.project_group_links.all, options)
       end
       expose :only_allow_merge_if_build_succeeds
+      expose :request_access_enabled
     end
 
     class Member < UserBasic
@@ -125,6 +126,7 @@ module API
       expose :lfs_enabled?, as: :lfs_enabled
       expose :avatar_url
       expose :web_url
+      expose :request_access_enabled
     end
 
     class GroupDetail < Group

lib/api/groups.rb

@@ -23,18 +23,19 @@ module API
       # Create group. Available only for users who can create groups.
       #
       # Parameters:
-      #   name (required)             - The name of the group
-      #   path (required)             - The path of the group
-      #   description (optional)      - The description of the group
-      #   visibility_level (optional) - The visibility level of the group
-      #   lfs_enabled (optional)      - Enable/disable LFS for the projects in this group
+      #   name (required)                   - The name of the group
+      #   path (required)                   - The path of the group
+      #   description (optional)            - The description of the group
+      #   visibility_level (optional)       - The visibility level of the group
+      #   lfs_enabled (optional)            - Enable/disable LFS for the projects in this group
+      #   request_access_enabled (optional) - Allow users to request member access
       # Example Request:
       #   POST /groups
       post do
         authorize! :create_group
         required_attributes! [:name, :path]
 
-        attrs = attributes_for_keys [:name, :path, :description, :visibility_level, :lfs_enabled]
+        attrs = attributes_for_keys [:name, :path, :description, :visibility_level, :lfs_enabled, :request_access_enabled]
         @group = Group.new(attrs)
 
         if @group.save
@@ -48,18 +49,19 @@ module API
       # Update group. Available only for users who can administrate groups.
       #
       # Parameters:
-      #   id (required)               - The ID of a group
-      #   path (optional)             - The path of the group
-      #   description (optional)      - The description of the group
-      #   visibility_level (optional) - The visibility level of the group
-      #   lfs_enabled (optional)      - Enable/disable LFS for the projects in this group
+      #   id (required)                     - The ID of a group
+      #   path (optional)                   - The path of the group
+      #   description (optional)            - The description of the group
+      #   visibility_level (optional)       - The visibility level of the group
+      #   lfs_enabled (optional)            - Enable/disable LFS for the projects in this group
+      #   request_access_enabled (optional) - Allow users to request member access
       # Example Request:
       #   PUT /groups/:id
       put ':id' do
         group = find_group(params[:id])
         authorize! :admin_group, group
 
-        attrs = attributes_for_keys [:name, :path, :description, :visibility_level, :lfs_enabled]
+        attrs = attributes_for_keys [:name, :path, :description, :visibility_level, :lfs_enabled, :request_access_enabled]
 
         if ::Groups::UpdateService.new(group, current_user, attrs).execute
           present group, with: Entities::GroupDetail

lib/api/internal.rb

@@ -35,6 +35,14 @@ module API
             Project.find_with_namespace(project_path)
           end
         end
+
+        def ssh_authentication_abilities
+          [
+            :read_project,
+            :download_code,
+            :push_code
+          ]
+        end
       end
 
       post "/allowed" do
@@ -51,9 +59,9 @@ module API
 
         access =
           if wiki?
-            Gitlab::GitAccessWiki.new(actor, project, protocol)
+            Gitlab::GitAccessWiki.new(actor, project, protocol, authentication_abilities: ssh_authentication_abilities)
           else
-            Gitlab::GitAccess.new(actor, project, protocol)
+            Gitlab::GitAccess.new(actor, project, protocol, authentication_abilities: ssh_authentication_abilities)
           end
 
         access_status = access.check(params[:action], params[:changes])

lib/api/projects.rb

@@ -91,8 +91,8 @@ module API
       # Create new project
       #
       # Parameters:
-      #   name (required) - name for new project
-      #   description (optional) - short project description
+      #   name (required)                   - name for new project
+      #   description (optional)            - short project description
       #   issues_enabled (optional)
       #   merge_requests_enabled (optional)
       #   builds_enabled (optional)
@@ -100,33 +100,35 @@ module API
       #   snippets_enabled (optional)
       #   container_registry_enabled (optional)
       #   shared_runners_enabled (optional)
-      #   namespace_id (optional) - defaults to user namespace
-      #   public (optional) - if true same as setting visibility_level = 20
-      #   visibility_level (optional) - 0 by default
+      #   namespace_id (optional)           - defaults to user namespace
+      #   public (optional)                 - if true same as setting visibility_level = 20
+      #   visibility_level (optional)       - 0 by default
       #   import_url (optional)
       #   public_builds (optional)
       #   lfs_enabled (optional)
+      #   request_access_enabled (optional) - Allow users to request member access
       # Example Request
       #   POST /projects
       post do
         required_attributes! [:name]
-        attrs = attributes_for_keys [:name,
-                                     :path,
+        attrs = attributes_for_keys [:builds_enabled,
+                                     :container_registry_enabled,
                                      :description,
+                                     :import_url,
                                      :issues_enabled,
+                                     :lfs_enabled,
                                      :merge_requests_enabled,
-                                     :builds_enabled,
-                                     :wiki_enabled,
-                                     :snippets_enabled,
-                                     :container_registry_enabled,
-                                     :shared_runners_enabled,
+                                     :name,
                                      :namespace_id,
+                                     :only_allow_merge_if_build_succeeds,
+                                     :path,
                                      :public,
-                                     :visibility_level,
-                                     :import_url,
                                      :public_builds,
-                                     :only_allow_merge_if_build_succeeds,
-                                     :lfs_enabled]
+                                     :request_access_enabled,
+                                     :shared_runners_enabled,
+                                     :snippets_enabled,
+                                     :visibility_level,
+                                     :wiki_enabled]
         attrs = map_public_to_visibility_level(attrs)
         @project = ::Projects::CreateService.new(current_user, attrs).execute
         if @project.saved?
@@ -143,10 +145,10 @@ module API
       # Create new project for a specified user.  Only available to admin users.
       #
       # Parameters:
-      #   user_id (required) - The ID of a user
-      #   name (required) - name for new project
-      #   description (optional) - short project description
-      #   default_branch (optional) - 'master' by default
+      #   user_id (required)                - The ID of a user
+      #   name (required)                   - name for new project
+      #   description (optional)            - short project description
+      #   default_branch (optional)         - 'master' by default
       #   issues_enabled (optional)
       #   merge_requests_enabled (optional)
       #   builds_enabled (optional)
@@ -154,31 +156,33 @@ module API
       #   snippets_enabled (optional)
       #   container_registry_enabled (optional)
       #   shared_runners_enabled (optional)
-      #   public (optional) - if true same as setting visibility_level = 20
+      #   public (optional)                 - if true same as setting visibility_level = 20
       #   visibility_level (optional)
       #   import_url (optional)
       #   public_builds (optional)
       #   lfs_enabled (optional)
+      #   request_access_enabled (optional) - Allow users to request member access
       # Example Request
       #   POST /projects/user/:user_id
       post "user/:user_id" do
         authenticated_as_admin!
         user = User.find(params[:user_id])
-        attrs = attributes_for_keys [:name,
-                                     :description,
+        attrs = attributes_for_keys [:builds_enabled,
                                      :default_branch,
+                                     :description,
+                                     :import_url,
                                      :issues_enabled,
+                                     :lfs_enabled,
                                      :merge_requests_enabled,
-                                     :builds_enabled,
-                                     :wiki_enabled,
-                                     :snippets_enabled,
-                                     :shared_runners_enabled,
+                                     :name,
+                                     :only_allow_merge_if_build_succeeds,
                                      :public,
-                                     :visibility_level,
-                                     :import_url,
                                      :public_builds,
-                                     :only_allow_merge_if_build_succeeds,
-                                     :lfs_enabled]
+                                     :request_access_enabled,
+                                     :shared_runners_enabled,
+                                     :snippets_enabled,
+                                     :visibility_level,
+                                     :wiki_enabled]
         attrs = map_public_to_visibility_level(attrs)
         @project = ::Projects::CreateService.new(user, attrs).execute
         if @project.saved?
@@ -242,22 +246,23 @@ module API
       # Example Request
       #   PUT /projects/:id
       put ':id' do
-        attrs = attributes_for_keys [:name,
-                                     :path,
-                                     :description,
+        attrs = attributes_for_keys [:builds_enabled,
+                                     :container_registry_enabled,
                                      :default_branch,
+                                     :description,
                                      :issues_enabled,
+                                     :lfs_enabled,
                                      :merge_requests_enabled,
-                                     :builds_enabled,
-                                     :wiki_enabled,
-                                     :snippets_enabled,
-                                     :container_registry_enabled,
-                                     :shared_runners_enabled,
+                                     :name,
+                                     :only_allow_merge_if_build_succeeds,
+                                     :path,
                                      :public,
-                                     :visibility_level,
                                      :public_builds,
-                                     :only_allow_merge_if_build_succeeds,
-                                     :lfs_enabled]
+                                     :request_access_enabled,
+                                     :shared_runners_enabled,
+                                     :snippets_enabled,
+                                     :visibility_level,
+                                     :wiki_enabled]
         attrs = map_public_to_visibility_level(attrs)
         authorize_admin_project
         authorize! :rename_project, user_project if attrs[:name].present?

lib/ci/api/entities.rb

@@ -15,6 +15,15 @@ module Ci
         expose :filename, :size
       end
 
+      class BuildOptions < Grape::Entity
+        expose :image
+        expose :services
+        expose :artifacts
+        expose :cache
+        expose :dependencies
+        expose :after_script
+      end
+
       class Build < Grape::Entity
         expose :id, :ref, :tag, :sha, :status
         expose :name, :token, :stage

lib/ci/api/helpers.rb

@@ -14,12 +14,20 @@ module Ci
       end
 
       def authenticate_build_token!(build)
-        token = (params[BUILD_TOKEN_PARAM] || env[BUILD_TOKEN_HEADER]).to_s
-        forbidden! unless token && build.valid_token?(token)
+        forbidden! unless build_token_valid?(build)
       end
 
       def runner_registration_token_valid?
-        params[:token] == current_application_settings.runners_registration_token
+        ActiveSupport::SecurityUtils.variable_size_secure_compare(
+          params[:token],
+          current_application_settings.runners_registration_token)
+      end
+
+      def build_token_valid?(build)
+        token = (params[BUILD_TOKEN_PARAM] || env[BUILD_TOKEN_HEADER]).to_s
+
+        # We require to also check `runners_token` to maintain compatibility with old version of runners
+        token && (build.valid_token?(token) || build.project.valid_runners_token?(token))
       end
 
       def update_runner_last_contact(save: true)

lib/ci/gitlab_ci_yaml_processor.rb

@@ -60,7 +60,7 @@ module Ci
         name: job[:name].to_s,
         allow_failure: job[:allow_failure] || false,
         when: job[:when] || 'on_success',
-        environment: job[:environment],
+        environment: job[:environment_name],
         yaml_variables: yaml_variables(name),
         options: {
           image: job[:image],
@@ -69,6 +69,7 @@ module Ci
           cache: job[:cache],
           dependencies: job[:dependencies],
           after_script: job[:after_script],
+          environment: job[:environment],
         }.compact
       }
     end

lib/ci/mask_secret.rb

+module Ci::MaskSecret
+  class << self
+    def mask(value, token)
+      return value unless value.present? && token.present?
+
+      value.gsub(token, 'x' * token.length)
+    end
+  end
+end

lib/expand_variables.rb

+module ExpandVariables
+  class << self
+    def expand(value, variables)
+      # Convert hash array to variables
+      if variables.is_a?(Array)
+        variables = variables.reduce({}) do |hash, variable|
+          hash[variable[:key]] = variable[:value]
+          hash
+        end
+      end
+
+      value.gsub(/\$([a-zA-Z_][a-zA-Z0-9_]*)|\${\g<1>}|%\g<1>%/) do
+        variables[$1 || $2]
+      end
+    end
+  end
+end

lib/gitlab/auth.rb

 module Gitlab
   module Auth
-    Result = Struct.new(:user, :type)
+    class MissingPersonalTokenError < StandardError; end
 
     class << self
       def find_for_git_client(login, password, project:, ip:)
         raise "Must provide an IP for rate limiting" if ip.nil?
 
-        result = Result.new
+        result =
+          service_request_check(login, password, project) ||
+          build_access_token_check(login, password) ||
+          user_with_password_for_git(login, password) ||
+          oauth_access_token_check(login, password) ||
+          personal_access_token_check(login, password) ||
+          Gitlab::Auth::Result.new
 
-        if valid_ci_request?(login, password, project)
-          result.type = :ci
-        else
-          result = populate_result(login, password)
-        end
+        rate_limit!(ip, success: result.success?, login: login)
 
-        success = result.user.present? || [:ci, :missing_personal_token].include?(result.type)
-        rate_limit!(ip, success: success, login: login)
         result
       end
 
@@ -57,44 +57,31 @@ module Gitlab
 
       private
 
-      def valid_ci_request?(login, password, project)
+      def service_request_check(login, password, project)
         matched_login = /(?<service>^[a-zA-Z]*-ci)-token$/.match(login)
 
-        return false unless project && matched_login.present?
+        return unless project && matched_login.present?
 
         underscored_service = matched_login['service'].underscore
 
-        if underscored_service == 'gitlab_ci'
-          project && project.valid_build_token?(password)
-        elsif Service.available_services_names.include?(underscored_service)
+        if Service.available_services_names.include?(underscored_service)
           # We treat underscored_service as a trusted input because it is included
           # in the Service.available_services_names whitelist.
           service = project.public_send("#{underscored_service}_service")
 
-          service && service.activated? && service.valid_token?(password)
-        end
-      end
-
-      def populate_result(login, password)
-        result =
-          user_with_password_for_git(login, password) ||
-          oauth_access_token_check(login, password) ||
-          personal_access_token_check(login, password)
-
-        if result
-          result.type = nil unless result.user
-
-          if result.user && result.user.two_factor_enabled? && result.type == :gitlab_or_ldap
-            result.type = :missing_personal_token
+          if service && service.activated? && service.valid_token?(password)
+            Gitlab::Auth::Result.new(nil, project, :ci, build_authentication_abilities)
           end
         end
-
-        result || Result.new
       end
 
       def user_with_password_for_git(login, password)
         user = find_with_user_password(login, password)
-        Result.new(user, :gitlab_or_ldap) if user
+        return unless user
+
+        raise Gitlab::Auth::MissingPersonalTokenError if user.two_factor_enabled?
+
+        Gitlab::Auth::Result.new(user, nil, :gitlab_or_ldap, full_authentication_abilities)
       end
 
       def oauth_access_token_check(login, password)
@@ -102,7 +89,7 @@ module Gitlab
           token = Doorkeeper::AccessToken.by_token(password)
           if token && token.accessible?
             user = User.find_by(id: token.resource_owner_id)
-            Result.new(user, :oauth)
+            Gitlab::Auth::Result.new(user, nil, :oauth, read_authentication_abilities)
           end
         end
       end
@@ -111,9 +98,52 @@ module Gitlab
         if login && password
           user = User.find_by_personal_access_token(password)
           validation = User.by_login(login)
-          Result.new(user, :personal_token) if user == validation
+          Gitlab::Auth::Result.new(user, nil, :personal_token, full_authentication_abilities) if user.present? && user == validation
+        end
+      end
+
+      def build_access_token_check(login, password)
+        return unless login == 'gitlab-ci-token'
+        return unless password
+
+        build = ::Ci::Build.running.find_by_token(password)
+        return unless build
+        return unless build.project.builds_enabled?
+
+        if build.user
+          # If user is assigned to build, use restricted credentials of user
+          Gitlab::Auth::Result.new(build.user, build.project, :build, build_authentication_abilities)
+        else
+          # Otherwise use generic CI credentials (backward compatibility)
+          Gitlab::Auth::Result.new(nil, build.project, :ci, build_authentication_abilities)
         end
       end
+
+      public
+
+      def build_authentication_abilities
+        [
+          :read_project,
+          :build_download_code,
+          :build_read_container_image,
+          :build_create_container_image
+        ]
+      end
+
+      def read_authentication_abilities
+        [
+          :read_project,
+          :download_code,
+          :read_container_image
+        ]
+      end
+
+      def full_authentication_abilities
+        read_authentication_abilities + [
+          :push_code,
+          :create_container_image
+        ]
+      end
     end
   end
 end

lib/gitlab/auth/result.rb

+module Gitlab
+  module Auth
+    Result = Struct.new(:actor, :project, :type, :authentication_abilities) do
+      def ci?
+        type == :ci
+      end
+
+      def success?
+        actor.present? || type == :ci
+      end
+    end
+  end
+end

lib/gitlab/ci/config/node/environment.rb

+module Gitlab
+  module Ci
+    class Config
+      module Node
+        ##
+        # Entry that represents an environment.
+        #
+        class Environment < Entry
+          include Validatable
+
+          ALLOWED_KEYS = %i[name url]
+
+          validations do
+            validate do
+              unless hash? || string?
+                errors.add(:config, 'should be a hash or a string')
+              end
+            end
+
+            validates :name, presence: true
+            validates :name,
+              type: {
+                with: String,
+                message: Gitlab::Regex.environment_name_regex_message }
+
+            validates :name,
+              format: {
+                with: Gitlab::Regex.environment_name_regex,
+                message: Gitlab::Regex.environment_name_regex_message }
+
+            with_options if: :hash? do
+              validates :config, allowed_keys: ALLOWED_KEYS
+
+              validates :url,
+                        length: { maximum: 255 },
+                        addressable_url: true,
+                        allow_nil: true
+            end
+          end
+
+          def hash?
+            @config.is_a?(Hash)
+          end
+
+          def string?
+            @config.is_a?(String)
+          end
+
+          def name
+            value[:name]
+          end
+
+          def url
+            value[:url]
+          end
+
+          def value
+            case @config
+            when String then { name: @config }
+            when Hash then @config
+            else {}
+            end
+          end
+        end
+      end
+    end
+  end
+end

lib/gitlab/ci/config/node/job.rb

@@ -13,7 +13,7 @@ module Gitlab
                             type stage when artifacts cache dependencies before_script
                             after_script variables environment]
 
-          attributes :tags, :allow_failure, :when, :environment, :dependencies
+          attributes :tags, :allow_failure, :when, :dependencies
 
           validations do
             validates :config, allowed_keys: ALLOWED_KEYS
@@ -29,58 +29,53 @@ module Gitlab
                 inclusion: { in: %w[on_success on_failure always manual],
                              message: 'should be on_success, on_failure, ' \
                                       'always or manual' }
-              validates :environment,
-                type: {
-                  with: String,
-                  message: Gitlab::Regex.environment_name_regex_message }
-              validates :environment,
-                format: {
-                  with: Gitlab::Regex.environment_name_regex,
-                  message: Gitlab::Regex.environment_name_regex_message }
 
               validates :dependencies, array_of_strings: true
             end
           end
 
-          node :before_script, Script,
+          node :before_script, Node::Script,
             description: 'Global before script overridden in this job.'
 
-          node :script, Commands,
+          node :script, Node::Commands,
             description: 'Commands that will be executed in this job.'
 
-          node :stage, Stage,
+          node :stage, Node::Stage,
             description: 'Pipeline stage this job will be executed into.'
 
-          node :type, Stage,
+          node :type, Node::Stage,
             description: 'Deprecated: stage this job will be executed into.'
 
-          node :after_script, Script,
+          node :after_script, Node::Script,
             description: 'Commands that will be executed when finishing job.'
 
-          node :cache, Cache,
+          node :cache, Node::Cache,
             description: 'Cache definition for this job.'
 
-          node :image, Image,
+          node :image, Node::Image,
             description: 'Image that will be used to execute this job.'
 
-          node :services, Services,
+          node :services, Node::Services,
             description: 'Services that will be used to execute this job.'
 
-          node :only, Trigger,
+          node :only, Node::Trigger,
             description: 'Refs policy this job will be executed for.'
 
-          node :except, Trigger,
+          node :except, Node::Trigger,
             description: 'Refs policy this job will be executed for.'
 
-          node :variables, Variables,
+          node :variables, Node::Variables,
             description: 'Environment variables available for this job.'
 
-          node :artifacts, Artifacts,
+          node :artifacts, Node::Artifacts,
             description: 'Artifacts configuration for this job.'
 
+          node :environment, Node::Environment,
+               description: 'Environment configuration for this job.'
+
           helpers :before_script, :script, :stage, :type, :after_script,
                   :cache, :image, :services, :only, :except, :variables,
-                  :artifacts, :commands
+                  :artifacts, :commands, :environment
 
           def compose!(deps = nil)
             super do
@@ -133,6 +128,8 @@ module Gitlab
               only: only,
               except: except,
               variables: variables_defined? ? variables : nil,
+              environment: environment_defined? ? environment : nil,
+              environment_name: environment_defined? ? environment[:name] : nil,
               artifacts: artifacts,
               after_script: after_script }
           end

lib/gitlab/git_access.rb

@@ -5,12 +5,13 @@ module Gitlab
     DOWNLOAD_COMMANDS = %w{ git-upload-pack git-upload-archive }
     PUSH_COMMANDS = %w{ git-receive-pack }
 
-    attr_reader :actor, :project, :protocol, :user_access
+    attr_reader :actor, :project, :protocol, :user_access, :authentication_abilities
 
-    def initialize(actor, project, protocol)
+    def initialize(actor, project, protocol, authentication_abilities:)
       @actor    = actor
       @project  = project
       @protocol = protocol
+      @authentication_abilities = authentication_abilities
       @user_access = UserAccess.new(user, project: project)
     end
 
@@ -60,14 +61,26 @@ module Gitlab
     end
 
     def user_download_access_check
-      unless user_access.can_do_action?(:download_code)
+      unless user_can_download_code? || build_can_download_code?
         return build_status_object(false, "You are not allowed to download code from this project.")
       end
 
       build_status_object(true)
     end
 
+    def user_can_download_code?
+      authentication_abilities.include?(:download_code) && user_access.can_do_action?(:download_code)
+    end
+
+    def build_can_download_code?
+      authentication_abilities.include?(:build_download_code) && user_access.can_do_action?(:build_download_code)
+    end
+
     def user_push_access_check(changes)
+      unless authentication_abilities.include?(:push_code)
+        return build_status_object(false, "You are not allowed to upload code for this project.")
+      end
+
       if changes.blank?
         return build_status_object(true)
       end

lib/gitlab/import_export.rb

@@ -2,7 +2,8 @@ module Gitlab
   module ImportExport
     extend self
 
-    VERSION = '0.1.3'
+    # For every version update, the version history in import_export.md has to be kept up to date.
+    VERSION = '0.1.4'
     FILENAME_LIMIT = 50
 
     def export_path(relative_path:)

lib/gitlab/import_export/import_export.yml

@@ -35,7 +35,9 @@ project_tree:
   - :deploy_keys
   - :services
   - :hooks
-  - :protected_branches
+  - protected_branches:
+    - :merge_access_levels
+    - :push_access_levels
   - :labels
   - milestones:
     - :events

lib/gitlab/import_export/relation_factory.rb

@@ -7,7 +7,9 @@ module Gitlab
                     variables: 'Ci::Variable',
                     triggers: 'Ci::Trigger',
                     builds: 'Ci::Build',
-                    hooks: 'ProjectHook' }.freeze
+                    hooks: 'ProjectHook',
+                    merge_access_levels: 'ProtectedBranch::MergeAccessLevel',
+                    push_access_levels: 'ProtectedBranch::PushAccessLevel' }.freeze
 
       USER_REFERENCES = %w[author_id assignee_id updated_by_id user_id].freeze
 
@@ -17,6 +19,8 @@ module Gitlab
 
       EXISTING_OBJECT_CHECK = %i[milestone milestones label labels].freeze
 
+      FINDER_ATTRIBUTES = %w[title project_id].freeze
+
       def self.create(*args)
         new(*args).create
       end
@@ -149,7 +153,7 @@ module Gitlab
       end
 
       def parsed_relation_hash
-        @relation_hash.reject { |k, _v| !relation_class.attribute_method?(k) }
+        @parsed_relation_hash ||= @relation_hash.reject { |k, _v| !relation_class.attribute_method?(k) }
       end
 
       def set_st_diffs
@@ -161,14 +165,30 @@ module Gitlab
         # Otherwise always create the record, skipping the extra SELECT clause.
         @existing_or_new_object ||= begin
           if EXISTING_OBJECT_CHECK.include?(@relation_name)
-            existing_object = relation_class.find_or_initialize_by(parsed_relation_hash.slice('title', 'project_id'))
-            existing_object.assign_attributes(parsed_relation_hash)
+            events = parsed_relation_hash.delete('events')
+
+            unless events.blank?
+              existing_object.assign_attributes(events: events)
+            end
+
             existing_object
           else
             relation_class.new(parsed_relation_hash)
           end
         end
       end
+
+      def existing_object
+        @existing_object ||=
+          begin
+            finder_hash = parsed_relation_hash.slice(*FINDER_ATTRIBUTES)
+            existing_object = relation_class.find_or_create_by(finder_hash)
+            # Done in two steps, as MySQL behaves differently than PostgreSQL using
+            # the +find_or_create_by+ method and does not return the ID the second time.
+            existing_object.update(parsed_relation_hash)
+            existing_object
+          end
+      end
     end
   end
 end

lib/gitlab/import_export/version_checker.rb

@@ -24,8 +24,8 @@ module Gitlab
       end
 
       def verify_version!(version)
-        if Gem::Version.new(version) > Gem::Version.new(Gitlab::ImportExport.version)
-          raise Gitlab::ImportExport::Error.new("Import version mismatch: Required <= #{Gitlab::ImportExport.version} but was #{version}")
+        if Gem::Version.new(version) != Gem::Version.new(Gitlab::ImportExport.version)
+          raise Gitlab::ImportExport::Error.new("Import version mismatch: Required #{Gitlab::ImportExport.version} but was #{version}")
         else
           true
         end

lib/gitlab/regex.rb

@@ -96,11 +96,11 @@ module Gitlab
     end
 
     def environment_name_regex
-      @environment_name_regex ||= /\A[a-zA-Z0-9_-]+\z/.freeze
+      @environment_name_regex ||= /\A[a-zA-Z0-9_\\\/\${}. -]+\z/.freeze
     end
 
     def environment_name_regex_message
-      "can contain only letters, digits, '-' and '_'."
+      "can contain only letters, digits, '-', '_', '/', '$', '{', '}', '.' and spaces"
     end
   end
 end

spec/features/environments_spec.rb

@@ -150,7 +150,7 @@ feature 'Environments', feature: true do
 
       context 'for invalid name' do
         before do
-          fill_in('Name', with: 'name with spaces')
+          fill_in('Name', with: 'name,with,commas')
           click_on 'Save'
         end
 

spec/features/milestone_spec.rb

@@ -3,9 +3,8 @@ require 'rails_helper'
 feature 'Milestone', feature: true do
   include WaitForAjax
 
-  let(:project) { create(:project, :public) }
+  let(:project) { create(:empty_project, :public) }
   let(:user)   { create(:user) }
-  let(:milestone) { create(:milestone, project: project, title: 8.7) }
 
   before do
     project.team << [user, :master]
@@ -13,7 +12,7 @@ feature 'Milestone', feature: true do
   end
 
   feature 'Create a milestone' do
-    scenario 'shows an informative message for a new issue' do
+    scenario 'shows an informative message for a new milestone' do
       visit new_namespace_project_milestone_path(project.namespace, project)
       page.within '.milestone-form' do
         fill_in "milestone_title", with: '8.7'
@@ -26,10 +25,26 @@ feature 'Milestone', feature: true do
 
   feature 'Open a milestone with closed issues' do
     scenario 'shows an informative message' do
+      milestone = create(:milestone, project: project, title: 8.7)
+
       create(:issue, title: "Bugfix1", project: project, milestone: milestone, state: "closed")
       visit namespace_project_milestone_path(project.namespace, project, milestone)
 
       expect(find('.alert-success')).to have_content('All issues for this milestone are closed. You may close this milestone now.')
     end
   end
+
+  feature 'Open a milestone with an existing title' do
+    scenario 'displays validation message' do
+      milestone = create(:milestone, project: project, title: 8.7)
+
+      visit new_namespace_project_milestone_path(project.namespace, project)
+      page.within '.milestone-form' do
+        fill_in "milestone_title", with: milestone.title
+      end
+      find('input[name="commit"]').click
+
+      expect(find('.alert-danger')).to have_content('Title has already been taken')
+    end
+  end
 end

spec/lib/ci/gitlab_ci_yaml_processor_spec.rb

@@ -754,6 +754,20 @@ module Ci
         it 'does return production' do
           expect(builds.size).to eq(1)
           expect(builds.first[:environment]).to eq(environment)
+          expect(builds.first[:options]).to include(environment: { name: environment })
+        end
+      end
+
+      context 'when hash is specified' do
+        let(:environment) do
+          { name: 'production',
+            url: 'http://production.gitlab.com' }
+        end
+
+        it 'does return production and URL' do
+          expect(builds.size).to eq(1)
+          expect(builds.first[:environment]).to eq(environment[:name])
+          expect(builds.first[:options]).to include(environment: environment)
         end
       end
 
@@ -770,15 +784,16 @@ module Ci
         let(:environment) { 1 }
 
         it 'raises error' do
-          expect { builds }.to raise_error("jobs:deploy_to_production environment #{Gitlab::Regex.environment_name_regex_message}")
+          expect { builds }.to raise_error(
+            'jobs:deploy_to_production:environment config should be a hash or a string')
         end
       end
 
       context 'is not a valid string' do
-        let(:environment) { 'production staging' }
+        let(:environment) { 'production:staging' }
 
         it 'raises error' do
-          expect { builds }.to raise_error("jobs:deploy_to_production environment #{Gitlab::Regex.environment_name_regex_message}")
+          expect { builds }.to raise_error("jobs:deploy_to_production:environment name #{Gitlab::Regex.environment_name_regex_message}")
         end
       end
     end

spec/lib/ci/mask_secret_spec.rb

+require 'spec_helper'
+
+describe Ci::MaskSecret, lib: true do
+  subject { described_class }
+
+  describe '#mask' do
+    it 'masks exact number of characters' do
+      expect(subject.mask('token', 'oke')).to eq('txxxn')
+    end
+
+    it 'masks multiple occurrences' do
+      expect(subject.mask('token token token', 'oke')).to eq('txxxn txxxn txxxn')
+    end
+
+    it 'does not mask if not found' do
+      expect(subject.mask('token', 'not')).to eq('token')
+    end
+  end
+end

spec/lib/expand_variables_spec.rb

+require 'spec_helper'
+
+describe ExpandVariables do
+  describe '#expand' do
+    subject { described_class.expand(value, variables) }
+
+    tests = [
+      { value: 'key',
+        result: 'key',
+        variables: []
+      },
+      { value: 'key$variable',
+        result: 'key',
+        variables: []
+      },
+      { value: 'key$variable',
+        result: 'keyvalue',
+        variables: [
+          { key: 'variable', value: 'value' }
+        ]
+      },
+      { value: 'key${variable}',
+        result: 'keyvalue',
+        variables: [
+          { key: 'variable', value: 'value' }
+        ]
+      },
+      { value: 'key$variable$variable2',
+        result: 'keyvalueresult',
+        variables: [
+          { key: 'variable', value: 'value' },
+          { key: 'variable2', value: 'result' },
+        ]
+      },
+      { value: 'key${variable}${variable2}',
+        result: 'keyvalueresult',
+        variables: [
+          { key: 'variable', value: 'value' },
+          { key: 'variable2', value: 'result' }
+        ]
+      },
+      { value: 'key$variable2$variable',
+        result: 'keyresultvalue',
+        variables: [
+          { key: 'variable', value: 'value' },
+          { key: 'variable2', value: 'result' },
+        ]
+      },
+      { value: 'key${variable2}${variable}',
+        result: 'keyresultvalue',
+        variables: [
+          { key: 'variable', value: 'value' },
+          { key: 'variable2', value: 'result' }
+        ]
+      },
+      { value: 'review/$CI_BUILD_REF_NAME',
+        result: 'review/feature/add-review-apps',
+        variables: [
+          { key: 'CI_BUILD_REF_NAME', value: 'feature/add-review-apps' }
+        ]
+      },
+    ]
+
+    tests.each do |test|
+      context "#{test[:value]} resolves to #{test[:result]}" do
+        let(:value) { test[:value] }
+        let(:variables) { test[:variables] }
+
+        it { is_expected.to eq(test[:result]) }
+      end
+    end
+  end
+end

spec/lib/gitlab/auth_spec.rb

@@ -4,15 +4,53 @@ describe Gitlab::Auth, lib: true do
   let(:gl_auth) { described_class }
 
   describe 'find_for_git_client' do
-    it 'recognizes CI' do
-      token = '123'
+    context 'build token' do
+      subject { gl_auth.find_for_git_client('gitlab-ci-token', build.token, project: project, ip: 'ip') }
+
+      context 'for running build' do
+        let!(:build) { create(:ci_build, :running) }
+        let(:project) { build.project }
+
+        before do
+          expect(gl_auth).to receive(:rate_limit!).with('ip', success: true, login: 'gitlab-ci-token')
+        end
+
+        it 'recognises user-less build' do
+          expect(subject).to eq(Gitlab::Auth::Result.new(nil, build.project, :ci, build_authentication_abilities))
+        end
+
+        it 'recognises user token' do
+          build.update(user: create(:user))
+
+          expect(subject).to eq(Gitlab::Auth::Result.new(build.user, build.project, :build, build_authentication_abilities))
+        end
+      end
+
+      (HasStatus::AVAILABLE_STATUSES - ['running']).each do |build_status|
+        context "for #{build_status} build" do
+          let!(:build) { create(:ci_build, status: build_status) }
+          let(:project) { build.project }
+
+          before do
+            expect(gl_auth).to receive(:rate_limit!).with('ip', success: false, login: 'gitlab-ci-token')
+          end
+
+          it 'denies authentication' do
+            expect(subject).to eq(Gitlab::Auth::Result.new)
+          end
+        end
+      end
+    end
+
+    it 'recognizes other ci services' do
       project = create(:empty_project)
-      project.update_attributes(runners_token: token)
+      project.create_drone_ci_service(active: true)
+      project.drone_ci_service.update(token: 'token')
 
       ip = 'ip'
 
-      expect(gl_auth).to receive(:rate_limit!).with(ip, success: true, login: 'gitlab-ci-token')
-      expect(gl_auth.find_for_git_client('gitlab-ci-token', token, project: project, ip: ip)).to eq(Gitlab::Auth::Result.new(nil, :ci))
+      expect(gl_auth).to receive(:rate_limit!).with(ip, success: true, login: 'drone-ci-token')
+      expect(gl_auth.find_for_git_client('drone-ci-token', 'token', project: project, ip: ip)).to eq(Gitlab::Auth::Result.new(nil, project, :ci, build_authentication_abilities))
     end
 
     it 'recognizes master passwords' do
@@ -20,7 +58,7 @@ describe Gitlab::Auth, lib: true do
       ip = 'ip'
 
       expect(gl_auth).to receive(:rate_limit!).with(ip, success: true, login: user.username)
-      expect(gl_auth.find_for_git_client(user.username, 'password', project: nil, ip: ip)).to eq(Gitlab::Auth::Result.new(user, :gitlab_or_ldap))
+      expect(gl_auth.find_for_git_client(user.username, 'password', project: nil, ip: ip)).to eq(Gitlab::Auth::Result.new(user, nil, :gitlab_or_ldap, full_authentication_abilities))
     end
 
     it 'recognizes OAuth tokens' do
@@ -30,7 +68,7 @@ describe Gitlab::Auth, lib: true do
       ip = 'ip'
 
       expect(gl_auth).to receive(:rate_limit!).with(ip, success: true, login: 'oauth2')
-      expect(gl_auth.find_for_git_client("oauth2", token.token, project: nil, ip: ip)).to eq(Gitlab::Auth::Result.new(user, :oauth))
+      expect(gl_auth.find_for_git_client("oauth2", token.token, project: nil, ip: ip)).to eq(Gitlab::Auth::Result.new(user, nil, :oauth, read_authentication_abilities))
     end
 
     it 'returns double nil for invalid credentials' do
@@ -92,4 +130,30 @@ describe Gitlab::Auth, lib: true do
       end
     end
   end
+
+  private
+
+  def build_authentication_abilities
+    [
+      :read_project,
+      :build_download_code,
+      :build_read_container_image,
+      :build_create_container_image
+    ]
+  end
+
+  def read_authentication_abilities
+    [
+      :read_project,
+      :download_code,
+      :read_container_image
+    ]
+  end
+
+  def full_authentication_abilities
+    read_authentication_abilities + [
+      :push_code,
+      :create_container_image
+    ]
+  end
 end

spec/lib/gitlab/ci/config/node/environment_spec.rb

+require 'spec_helper'
+
+describe Gitlab::Ci::Config::Node::Environment do
+  let(:entry) { described_class.new(config) }
+
+  before { entry.compose! }
+
+  context 'when configuration is a string' do
+    let(:config) { 'production' }
+
+    describe '#string?' do
+      it 'is string configuration' do
+        expect(entry).to be_string
+      end
+    end
+
+    describe '#hash?' do
+      it 'is not hash configuration' do
+        expect(entry).not_to be_hash
+      end
+    end
+
+    describe '#valid?' do
+      it 'is valid' do
+        expect(entry).to be_valid
+      end
+    end
+
+    describe '#value' do
+      it 'returns valid hash' do
+        expect(entry.value).to eq(name: 'production')
+      end
+    end
+
+    describe '#name' do
+      it 'returns environment name' do
+        expect(entry.name).to eq 'production'
+      end
+    end
+
+    describe '#url' do
+      it 'returns environment url' do
+        expect(entry.url).to be_nil
+      end
+    end
+  end
+
+  context 'when configuration is a hash' do
+    let(:config) do
+      { name: 'development', url: 'https://example.gitlab.com' }
+    end
+
+    describe '#string?' do
+      it 'is not string configuration' do
+        expect(entry).not_to be_string
+      end
+    end
+
+    describe '#hash?' do
+      it 'is hash configuration' do
+        expect(entry).to be_hash
+      end
+    end
+
+    describe '#valid?' do
+      it 'is valid' do
+        expect(entry).to be_valid
+      end
+    end
+
+    describe '#value' do
+      it 'returns valid hash' do
+        expect(entry.value).to eq config
+      end
+    end
+
+    describe '#name' do
+      it 'returns environment name' do
+        expect(entry.name).to eq 'development'
+      end
+    end
+
+    describe '#url' do
+      it 'returns environment url' do
+        expect(entry.url).to eq 'https://example.gitlab.com'
+      end
+    end
+  end
+
+  context 'when variables are used for environment' do
+    let(:config) do
+      { name: 'review/$CI_BUILD_REF_NAME',
+        url: 'https://$CI_BUILD_REF_NAME.review.gitlab.com' }
+    end
+
+    describe '#valid?' do
+      it 'is valid' do
+        expect(entry).to be_valid
+      end
+    end
+  end
+
+  context 'when configuration is invalid' do
+    context 'when configuration is an array' do
+      let(:config) { ['env'] }
+
+      describe '#valid?' do
+        it 'is not valid' do
+          expect(entry).not_to be_valid
+        end
+      end
+
+      describe '#errors' do
+        it 'contains error about invalid type' do
+          expect(entry.errors)
+            .to include 'environment config should be a hash or a string'
+        end
+      end
+    end
+
+    context 'when environment name is not present' do
+      let(:config) { { url: 'https://example.gitlab.com' } }
+
+      describe '#valid?' do
+        it 'is not valid' do
+          expect(entry).not_to be_valid
+        end
+      end
+
+      describe '#errors?' do
+        it 'contains error about missing environment name' do
+          expect(entry.errors)
+            .to include "environment name can't be blank"
+        end
+      end
+    end
+
+    context 'when invalid URL is used' do
+      let(:config) { { name: 'test', url: 'invalid-example.gitlab.com' } }
+
+      describe '#valid?' do
+        it 'is not valid' do
+          expect(entry).not_to be_valid
+        end
+      end
+
+      describe '#errors?' do
+        it 'contains error about invalid URL' do
+          expect(entry.errors)
+            .to include "environment url must be a valid url"
+        end
+      end
+    end
+  end
+end

spec/lib/gitlab/git_access_spec.rb

 require 'spec_helper'
 
 describe Gitlab::GitAccess, lib: true do
-  let(:access) { Gitlab::GitAccess.new(actor, project, 'web') }
+  let(:access) { Gitlab::GitAccess.new(actor, project, 'web', authentication_abilities: authentication_abilities) }
   let(:project) { create(:project) }
   let(:user) { create(:user) }
   let(:actor) { user }
+  let(:authentication_abilities) do
+    [
+      :read_project,
+      :download_code,
+      :push_code
+    ]
+  end
 
   describe '#check with single protocols allowed' do
     def disable_protocol(protocol)
@@ -15,7 +22,7 @@ describe Gitlab::GitAccess, lib: true do
     context 'ssh disabled' do
       before do
         disable_protocol('ssh')
-        @acc = Gitlab::GitAccess.new(actor, project, 'ssh')
+        @acc = Gitlab::GitAccess.new(actor, project, 'ssh', authentication_abilities: authentication_abilities)
       end
 
       it 'blocks ssh git push' do
@@ -30,7 +37,7 @@ describe Gitlab::GitAccess, lib: true do
     context 'http disabled' do
       before do
         disable_protocol('http')
-        @acc = Gitlab::GitAccess.new(actor, project, 'http')
+        @acc = Gitlab::GitAccess.new(actor, project, 'http', authentication_abilities: authentication_abilities)
       end
 
       it 'blocks http push' do
@@ -111,6 +118,36 @@ describe Gitlab::GitAccess, lib: true do
         end
       end
     end
+
+    describe 'build authentication_abilities permissions' do
+      let(:authentication_abilities) { build_authentication_abilities }
+
+      describe 'reporter user' do
+        before { project.team << [user, :reporter] }
+
+        context 'pull code' do
+          it { expect(subject).to be_allowed }
+        end
+      end
+
+      describe 'admin user' do
+        let(:user) { create(:admin) }
+
+        context 'when member of the project' do
+          before { project.team << [user, :reporter] }
+
+          context 'pull code' do
+            it { expect(subject).to be_allowed }
+          end
+        end
+
+        context 'when is not member of the project' do
+          context 'pull code' do
+            it { expect(subject).not_to be_allowed }
+          end
+        end
+      end
+    end
   end
 
   describe 'push_access_check' do
@@ -283,38 +320,71 @@ describe Gitlab::GitAccess, lib: true do
     end
   end
 
-  describe 'deploy key permissions' do
-    let(:key) { create(:deploy_key) }
-    let(:actor) { key }
+  shared_examples 'can not push code' do
+    subject { access.check('git-receive-pack', '_any') }
+
+    context 'when project is authorized' do
+      before { authorize }
 
-    context 'push code' do
-      subject { access.check('git-receive-pack', '_any') }
+      it { expect(subject).not_to be_allowed }
+    end
 
-      context 'when project is authorized' do
-        before { key.projects << project }
+    context 'when unauthorized' do
+      context 'to public project' do
+        let(:project) { create(:project, :public) }
 
         it { expect(subject).not_to be_allowed }
       end
 
-      context 'when unauthorized' do
-        context 'to public project' do
-          let(:project) { create(:project, :public) }
+      context 'to internal project' do
+        let(:project) { create(:project, :internal) }
 
-          it { expect(subject).not_to be_allowed }
-        end
+        it { expect(subject).not_to be_allowed }
+      end
 
-        context 'to internal project' do
-          let(:project) { create(:project, :internal) }
+      context 'to private project' do
+        let(:project) { create(:project, :internal) }
 
-          it { expect(subject).not_to be_allowed }
-        end
+        it { expect(subject).not_to be_allowed }
+      end
+    end
+  end
 
-        context 'to private project' do
-          let(:project) { create(:project, :internal) }
+  describe 'build authentication abilities' do
+    let(:authentication_abilities) { build_authentication_abilities }
 
-          it { expect(subject).not_to be_allowed }
-        end
+    it_behaves_like 'can not push code' do
+      def authorize
+        project.team << [user, :reporter]
       end
     end
   end
+
+  describe 'deploy key permissions' do
+    let(:key) { create(:deploy_key) }
+    let(:actor) { key }
+
+    it_behaves_like 'can not push code' do
+      def authorize
+        key.projects << project
+      end
+    end
+  end
+
+  private
+
+  def build_authentication_abilities
+    [
+      :read_project,
+      :build_download_code
+    ]
+  end
+
+  def full_authentication_abilities
+    [
+      :read_project,
+      :download_code,
+      :push_code
+    ]
+  end
 end

spec/lib/gitlab/git_access_wiki_spec.rb

 require 'spec_helper'
 
 describe Gitlab::GitAccessWiki, lib: true do
-  let(:access) { Gitlab::GitAccessWiki.new(user, project, 'web') }
+  let(:access) { Gitlab::GitAccessWiki.new(user, project, 'web', authentication_abilities: authentication_abilities) }
   let(:project) { create(:project) }
   let(:user) { create(:user) }
+  let(:authentication_abilities) do
+    [
+      :read_project,
+      :download_code,
+      :push_code
+    ]
+  end
 
   describe 'push_allowed?' do
     before do

spec/lib/gitlab/import_export/project.json

@@ -24,7 +24,7 @@
       "test_ee_field": "test",
       "milestone": {
         "id": 1,
-        "title": "v0.0",
+        "title": "test milestone",
         "project_id": 8,
         "description": "test milestone",
         "due_date": null,
@@ -51,7 +51,7 @@
         {
           "id": 2,
           "label_id": 2,
-          "target_id": 3,
+          "target_id": 40,
           "target_type": "Issue",
           "created_at": "2016-07-22T08:57:02.840Z",
           "updated_at": "2016-07-22T08:57:02.840Z",
@@ -281,6 +281,31 @@
       "deleted_at": null,
       "due_date": null,
       "moved_to_id": null,
+      "milestone": {
+        "id": 1,
+        "title": "test milestone",
+        "project_id": 8,
+        "description": "test milestone",
+        "due_date": null,
+        "created_at": "2016-06-14T15:02:04.415Z",
+        "updated_at": "2016-06-14T15:02:04.415Z",
+        "state": "active",
+        "iid": 1,
+        "events": [
+          {
+            "id": 487,
+            "target_type": "Milestone",
+            "target_id": 1,
+            "title": null,
+            "data": null,
+            "project_id": 46,
+            "created_at": "2016-06-14T15:02:04.418Z",
+            "updated_at": "2016-06-14T15:02:04.418Z",
+            "action": 1,
+            "author_id": 18
+          }
+        ]
+      },
       "notes": [
         {
           "id": 359,
@@ -494,6 +519,27 @@
       "deleted_at": null,
       "due_date": null,
       "moved_to_id": null,
+      "label_links": [
+        {
+          "id": 99,
+          "label_id": 2,
+          "target_id": 38,
+          "target_type": "Issue",
+          "created_at": "2016-07-22T08:57:02.840Z",
+          "updated_at": "2016-07-22T08:57:02.840Z",
+          "label": {
+            "id": 2,
+            "title": "test2",
+            "color": "#428bca",
+            "project_id": 8,
+            "created_at": "2016-07-22T08:55:44.161Z",
+            "updated_at": "2016-07-22T08:55:44.161Z",
+            "template": false,
+            "description": "",
+            "priority": null
+          }
+        }
+      ],
       "notes": [
         {
           "id": 367,
@@ -6478,7 +6524,7 @@
     {
       "id": 37,
       "project_id": 5,
-      "ref": "master",
+      "ref": null,
       "sha": "048721d90c449b244b7b4c53a9186b04330174ec",
       "before_sha": null,
       "push_data": null,
@@ -7301,6 +7347,30 @@
 
   ],
   "protected_branches": [
-
+    {
+      "id": 1,
+      "project_id": 9,
+      "name": "master",
+      "created_at": "2016-08-30T07:32:52.426Z",
+      "updated_at": "2016-08-30T07:32:52.426Z",
+      "merge_access_levels": [
+        {
+          "id": 1,
+          "protected_branch_id": 1,
+          "access_level": 40,
+          "created_at": "2016-08-30T07:32:52.458Z",
+          "updated_at": "2016-08-30T07:32:52.458Z"
+        }
+      ],
+      "push_access_levels": [
+        {
+          "id": 1,
+          "protected_branch_id": 1,
+          "access_level": 40,
+          "created_at": "2016-08-30T07:32:52.490Z",
+          "updated_at": "2016-08-30T07:32:52.490Z"
+        }
+      ]
+    }
   ]
-}
+}
\ No newline at end of file

spec/lib/gitlab/import_export/project_tree_restorer_spec.rb

@@ -29,12 +29,30 @@ describe Gitlab::ImportExport::ProjectTreeRestorer, services: true do
         expect(project.project_feature.merge_requests_access_level).to eq(ProjectFeature::ENABLED)
       end
 
+      it 'has the same label associated to two issues' do
+        restored_project_json
+
+        expect(Label.first.issues.count).to eq(2)
+      end
+
+      it 'has milestones associated to two separate issues' do
+        restored_project_json
+
+        expect(Milestone.find_by_description('test milestone').issues.count).to eq(2)
+      end
+
       it 'creates a valid pipeline note' do
         restored_project_json
 
         expect(Ci::Pipeline.first.notes).not_to be_empty
       end
 
+      it 'restores pipelines with missing ref' do
+        restored_project_json
+
+        expect(Ci::Pipeline.where(ref: nil)).not_to be_empty
+      end
+
       it 'restores the correct event with symbolised data' do
         restored_project_json
 
@@ -49,6 +67,18 @@ describe Gitlab::ImportExport::ProjectTreeRestorer, services: true do
         expect(issue.reload.updated_at.to_s).to eq('2016-06-14 15:02:47 UTC')
       end
 
+      it 'contains the merge access levels on a protected branch' do
+        restored_project_json
+
+        expect(ProtectedBranch.first.merge_access_levels).not_to be_empty
+      end
+
+      it 'contains the push access levels on a protected branch' do
+        restored_project_json
+
+        expect(ProtectedBranch.first.push_access_levels).not_to be_empty
+      end
+
       context 'event at forth level of the tree' do
         let(:event) { Event.where(title: 'test levels').first }
 
@@ -77,12 +107,6 @@ describe Gitlab::ImportExport::ProjectTreeRestorer, services: true do
         expect(Label.first.label_links.first.target).not_to be_nil
       end
 
-      it 'has milestones associated to issues' do
-        restored_project_json
-
-        expect(Milestone.find_by_description('test milestone').issues).not_to be_empty
-      end
-
       context 'Merge requests' do
         before do
           restored_project_json

spec/lib/gitlab/import_export/version_checker_spec.rb

@@ -23,7 +23,7 @@ describe Gitlab::ImportExport::VersionChecker, services: true do
       it 'shows the correct error message' do
         described_class.check!(shared: shared)
 
-        expect(shared.errors.first).to eq("Import version mismatch: Required <= #{Gitlab::ImportExport.version} but was #{version}")
+        expect(shared.errors.first).to eq("Import version mismatch: Required #{Gitlab::ImportExport.version} but was #{version}")
       end
     end
   end

spec/models/build_spec.rb

@@ -88,9 +88,7 @@ describe Ci::Build, models: true do
   end
 
   describe '#trace' do
-    subject { build.trace_html }
-
-    it { is_expected.to be_empty }
+    it { expect(build.trace).to be_nil }
 
     context 'when build.trace contains text' do
       let(:text) { 'example output' }
@@ -98,16 +96,80 @@ describe Ci::Build, models: true do
         build.trace = text
       end
 
-      it { is_expected.to include(text) }
-      it { expect(subject.length).to be >= text.length }
+      it { expect(build.trace).to eq(text) }
+    end
+
+    context 'when build.trace hides runners token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.update(trace: token)
+        build.project.update(runners_token: token)
+      end
+
+      it { expect(build.trace).not_to include(token) }
+      it { expect(build.raw_trace).to include(token) }
+    end
+
+    context 'when build.trace hides build token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.update(trace: token)
+        build.update(token: token)
+      end
+
+      it { expect(build.trace).not_to include(token) }
+      it { expect(build.raw_trace).to include(token) }
+    end
+  end
+
+  describe '#raw_trace' do
+    subject { build.raw_trace }
+
+    context 'when build.trace hides runners token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.project.update(runners_token: token)
+        build.update(trace: token)
+      end
+
+      it { is_expected.not_to include(token) }
+    end
+
+    context 'when build.trace hides build token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.update(token: token)
+        build.update(trace: token)
+      end
+
+      it { is_expected.not_to include(token) }
+    end
+  end
+
+  context '#append_trace' do
+    subject { build.trace_html }
+
+    context 'when build.trace hides runners token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.project.update(runners_token: token)
+        build.append_trace(token, 0)
+      end
+
+      it { is_expected.not_to include(token) }
     end
 
-    context 'when build.trace hides token' do
+    context 'when build.trace hides build token' do
       let(:token) { 'my_secret_token' }
 
       before do
-        build.project.update_attributes(runners_token: token)
-        build.update_attributes(trace: token)
+        build.update(token: token)
+        build.append_trace(token, 0)
       end
 
       it { is_expected.not_to include(token) }

spec/models/ci/build_spec.rb

@@ -8,7 +8,7 @@ describe Ci::Build, models: true do
     it 'obfuscates project runners token' do
       allow(build).to receive(:raw_trace).and_return("Test: #{build.project.runners_token}")
 
-      expect(build.trace).to eq("Test: xxxxxx")
+      expect(build.trace).to eq("Test: xxxxxxxxxxxxxxxxxxxx")
     end
 
     it 'empty project runners token' do

spec/models/environment_spec.rb

@@ -63,4 +63,20 @@ describe Environment, models: true do
       end
     end
   end
+
+  describe '#environment_type' do
+    subject { environment.environment_type }
+
+    it 'sets a environment type if name has multiple segments' do
+      environment.update!(name: 'production/worker.gitlab.com')
+
+      is_expected.to eq('production')
+    end
+
+    it 'nullifies a type if it\'s a simple name' do
+      environment.update!(name: 'production')
+
+      is_expected.to be_nil
+    end
+  end
 end

spec/models/event_spec.rb

@@ -16,18 +16,12 @@ describe Event, models: true do
 
   describe 'Callbacks' do
     describe 'after_create :reset_project_activity' do
-      let(:project) { create(:project) }
+      let(:project) { create(:empty_project) }
 
-      context "project's last activity was less than 5 minutes ago" do
-        it 'does not update project.last_activity_at if it has been touched less than 5 minutes ago' do
-          create_event(project, project.owner)
-          project.update_column(:last_activity_at, 5.minutes.ago)
-          project_last_activity_at = project.last_activity_at
+      it 'calls the reset_project_activity method' do
+        expect_any_instance_of(Event).to receive(:reset_project_activity)
 
-          create_event(project, project.owner)
-
-          expect(project.last_activity_at).to eq(project_last_activity_at)
-        end
+        create_event(project, project.owner)
       end
     end
   end
@@ -161,6 +155,35 @@ describe Event, models: true do
     end
   end
 
+  describe '#reset_project_activity' do
+    let(:project) { create(:empty_project) }
+
+    context 'when a project was updated less than 1 hour ago' do
+      it 'does not update the project' do
+        project.update(last_activity_at: Time.now)
+
+        expect(project).not_to receive(:update_column).
+          with(:last_activity_at, a_kind_of(Time))
+
+        create_event(project, project.owner)
+      end
+    end
+
+    context 'when a project was updated more than 1 hour ago' do
+      it 'updates the project' do
+        project.update(last_activity_at: 1.year.ago)
+
+        expect_any_instance_of(Gitlab::ExclusiveLease).
+          to receive(:try_obtain).and_return(true)
+
+        expect(project).to receive(:update_column).
+          with(:last_activity_at, a_kind_of(Time))
+
+        create_event(project, project.owner)
+      end
+    end
+  end
+
   def create_event(project, user, attrs = {})
     data = {
       before: Gitlab::Git::BLANK_SHA,

spec/requests/api/commits_spec.rb

@@ -110,7 +110,7 @@ describe API::API, api: true  do
         get api("/projects/#{project.id}/repository/commits/#{project.repository.commit.id}", user)
 
         expect(response).to have_http_status(200)
-        expect(json_response['status']).to be_nil
+        expect(json_response['status']).to eq("created")
       end
     end
 

spec/requests/api/groups_spec.rb

@@ -120,10 +120,11 @@ describe API::API, api: true  do
 
     context 'when authenticated as the group owner' do
       it 'updates the group' do
-        put api("/groups/#{group1.id}", user1), name: new_group_name
+        put api("/groups/#{group1.id}", user1), name: new_group_name, request_access_enabled: true
 
         expect(response).to have_http_status(200)
         expect(json_response['name']).to eq(new_group_name)
+        expect(json_response['request_access_enabled']).to eq(true)
       end
 
       it 'returns 404 for a non existing group' do
@@ -238,8 +239,14 @@ describe API::API, api: true  do
 
     context "when authenticated as user with group permissions" do
       it "creates group" do
-        post api("/groups", user3), attributes_for(:group)
+        group = attributes_for(:group, { request_access_enabled: false })
+
+        post api("/groups", user3), group
         expect(response).to have_http_status(201)
+
+        expect(json_response["name"]).to eq(group[:name])
+        expect(json_response["path"]).to eq(group[:path])
+        expect(json_response["request_access_enabled"]).to eq(group[:request_access_enabled])
       end
 
       it "does not create group, duplicate" do

spec/requests/api/milestones_spec.rb

@@ -3,7 +3,7 @@ require 'spec_helper'
 describe API::API, api: true  do
   include ApiHelpers
   let(:user) { create(:user) }
-  let!(:project) { create(:project, namespace: user.namespace ) }
+  let!(:project) { create(:empty_project, namespace: user.namespace ) }
   let!(:closed_milestone) { create(:closed_milestone, project: project) }
   let!(:milestone) { create(:milestone, project: project) }
 
@@ -12,6 +12,7 @@ describe API::API, api: true  do
   describe 'GET /projects/:id/milestones' do
     it 'returns project milestones' do
       get api("/projects/#{project.id}/milestones", user)
+
       expect(response).to have_http_status(200)
       expect(json_response).to be_an Array
       expect(json_response.first['title']).to eq(milestone.title)
@@ -19,6 +20,7 @@ describe API::API, api: true  do
 
     it 'returns a 401 error if user not authenticated' do
       get api("/projects/#{project.id}/milestones")
+
       expect(response).to have_http_status(401)
     end
 
@@ -44,6 +46,7 @@ describe API::API, api: true  do
   describe 'GET /projects/:id/milestones/:milestone_id' do
     it 'returns a project milestone by id' do
       get api("/projects/#{project.id}/milestones/#{milestone.id}", user)
+
       expect(response).to have_http_status(200)
       expect(json_response['title']).to eq(milestone.title)
       expect(json_response['iid']).to eq(milestone.iid)
@@ -60,11 +63,13 @@ describe API::API, api: true  do
 
     it 'returns 401 error if user not authenticated' do
       get api("/projects/#{project.id}/milestones/#{milestone.id}")
+
       expect(response).to have_http_status(401)
     end
 
     it 'returns a 404 error if milestone id not found' do
       get api("/projects/#{project.id}/milestones/1234", user)
+
       expect(response).to have_http_status(404)
     end
   end
@@ -72,6 +77,7 @@ describe API::API, api: true  do
   describe 'POST /projects/:id/milestones' do
     it 'creates a new project milestone' do
       post api("/projects/#{project.id}/milestones", user), title: 'new milestone'
+
       expect(response).to have_http_status(201)
       expect(json_response['title']).to eq('new milestone')
       expect(json_response['description']).to be_nil
@@ -80,6 +86,7 @@ describe API::API, api: true  do
     it 'creates a new project milestone with description and due date' do
       post api("/projects/#{project.id}/milestones", user),
         title: 'new milestone', description: 'release', due_date: '2013-03-02'
+
       expect(response).to have_http_status(201)
       expect(json_response['description']).to eq('release')
       expect(json_response['due_date']).to eq('2013-03-02')
@@ -87,6 +94,14 @@ describe API::API, api: true  do
 
     it 'returns a 400 error if title is missing' do
       post api("/projects/#{project.id}/milestones", user)
+
+      expect(response).to have_http_status(400)
+    end
+
+    it 'returns a 400 error if params are invalid (duplicate title)' do
+      post api("/projects/#{project.id}/milestones", user),
+        title: milestone.title, description: 'release', due_date: '2013-03-02'
+
       expect(response).to have_http_status(400)
     end
   end
@@ -95,6 +110,7 @@ describe API::API, api: true  do
     it 'updates a project milestone' do
       put api("/projects/#{project.id}/milestones/#{milestone.id}", user),
         title: 'updated title'
+
       expect(response).to have_http_status(200)
       expect(json_response['title']).to eq('updated title')
     end
@@ -102,6 +118,7 @@ describe API::API, api: true  do
     it 'returns a 404 error if milestone id not found' do
       put api("/projects/#{project.id}/milestones/1234", user),
         title: 'updated title'
+
       expect(response).to have_http_status(404)
     end
   end
@@ -131,6 +148,7 @@ describe API::API, api: true  do
     end
     it 'returns project issues for a particular milestone' do
       get api("/projects/#{project.id}/milestones/#{milestone.id}/issues", user)
+
       expect(response).to have_http_status(200)
       expect(json_response).to be_an Array
       expect(json_response.first['milestone']['title']).to eq(milestone.title)
@@ -138,11 +156,12 @@ describe API::API, api: true  do
 
     it 'returns a 401 error if user not authenticated' do
       get api("/projects/#{project.id}/milestones/#{milestone.id}/issues")
+
       expect(response).to have_http_status(401)
     end
 
     describe 'confidential issues' do
-      let(:public_project) { create(:project, :public) }
+      let(:public_project) { create(:empty_project, :public) }
       let(:milestone) { create(:milestone, project: public_project) }
       let(:issue) { create(:issue, project: public_project) }
       let(:confidential_issue) { create(:issue, confidential: true, project: public_project) }

spec/requests/api/projects_spec.rb

@@ -225,7 +225,8 @@ describe API::API, api: true  do
         issues_enabled: false,
         merge_requests_enabled: false,
         wiki_enabled: false,
-        only_allow_merge_if_build_succeeds: false
+        only_allow_merge_if_build_succeeds: false,
+        request_access_enabled: true
       })
 
       post api('/projects', user), project
@@ -352,7 +353,8 @@ describe API::API, api: true  do
         description: FFaker::Lorem.sentence,
         issues_enabled: false,
         merge_requests_enabled: false,
-        wiki_enabled: false
+        wiki_enabled: false,
+        request_access_enabled: true
       })
 
       post api("/projects/user/#{user.id}", admin), project
@@ -887,6 +889,15 @@ describe API::API, api: true  do
         expect(json_response['message']['name']).to eq(['has already been taken'])
       end
 
+      it 'updates request_access_enabled' do
+        project_param = { request_access_enabled: false }
+
+        put api("/projects/#{project.id}", user), project_param
+
+        expect(response).to have_http_status(200)
+        expect(json_response['request_access_enabled']).to eq(false)
+      end
+
       it 'updates path & name to existing path & name in different namespace' do
         project_param = { path: project4.path, name: project4.name }
         put api("/projects/#{project3.id}", user), project_param
@@ -948,7 +959,8 @@ describe API::API, api: true  do
                           wiki_enabled: true,
                           snippets_enabled: true,
                           merge_requests_enabled: true,
-                          description: 'new description' }
+                          description: 'new description',
+                          request_access_enabled: true }
         put api("/projects/#{project.id}", user3), project_param
         expect(response).to have_http_status(403)
       end

spec/requests/ci/api/builds_spec.rb

@@ -254,7 +254,8 @@ describe Ci::API::API do
       let(:get_url) { ci_api("/builds/#{build.id}/artifacts") }
       let(:jwt_token) { JWT.encode({ 'iss' => 'gitlab-workhorse' }, Gitlab::Workhorse.secret, 'HS256') }
       let(:headers) { { "GitLab-Workhorse" => "1.0", Gitlab::Workhorse::INTERNAL_API_REQUEST_HEADER => jwt_token } }
-      let(:headers_with_token) { headers.merge(Ci::API::Helpers::BUILD_TOKEN_HEADER => build.token) }
+      let(:token) { build.token }
+      let(:headers_with_token) { headers.merge(Ci::API::Helpers::BUILD_TOKEN_HEADER => token) }
 
       before { build.run! }
 
@@ -262,6 +263,7 @@ describe Ci::API::API do
         context "should authorize posting artifact to running build" do
           it "using token as parameter" do
             post authorize_url, { token: build.token }, headers
+
             expect(response).to have_http_status(200)
             expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
             expect(json_response["TempPath"]).not_to be_nil
@@ -269,6 +271,15 @@ describe Ci::API::API do
 
           it "using token as header" do
             post authorize_url, {}, headers_with_token
+
+            expect(response).to have_http_status(200)
+            expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
+            expect(json_response["TempPath"]).not_to be_nil
+          end
+
+          it "using runners token" do
+            post authorize_url, { token: build.project.runners_token }, headers
+
             expect(response).to have_http_status(200)
             expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
             expect(json_response["TempPath"]).not_to be_nil
@@ -276,7 +287,9 @@ describe Ci::API::API do
 
           it "reject requests that did not go through gitlab-workhorse" do
             headers.delete(Gitlab::Workhorse::INTERNAL_API_REQUEST_HEADER)
+
             post authorize_url, { token: build.token }, headers
+
             expect(response).to have_http_status(500)
           end
         end
@@ -284,13 +297,17 @@ describe Ci::API::API do
         context "should fail to post too large artifact" do
           it "using token as parameter" do
             stub_application_setting(max_artifacts_size: 0)
+
             post authorize_url, { token: build.token, filesize: 100 }, headers
+
             expect(response).to have_http_status(413)
           end
 
           it "using token as header" do
             stub_application_setting(max_artifacts_size: 0)
+
             post authorize_url, { filesize: 100 }, headers_with_token
+
             expect(response).to have_http_status(413)
           end
         end
@@ -358,6 +375,16 @@ describe Ci::API::API do
 
               it_behaves_like 'successful artifacts upload'
             end
+
+            context 'when using runners token' do
+              let(:token) { build.project.runners_token }
+
+              before do
+                upload_artifacts(file_upload, headers_with_token)
+              end
+
+              it_behaves_like 'successful artifacts upload'
+            end
           end
 
           context 'posts artifacts file and metadata file' do
@@ -497,19 +524,40 @@ describe Ci::API::API do
 
         before do
           delete delete_url, token: build.token
-          build.reload
         end
 
-        it 'removes build artifacts' do
-          expect(response).to have_http_status(200)
-          expect(build.artifacts_file.exists?).to be_falsy
-          expect(build.artifacts_metadata.exists?).to be_falsy
-          expect(build.artifacts_size).to be_nil
+        shared_examples 'having removable artifacts' do
+          it 'removes build artifacts' do
+            build.reload
+
+            expect(response).to have_http_status(200)
+            expect(build.artifacts_file.exists?).to be_falsy
+            expect(build.artifacts_metadata.exists?).to be_falsy
+            expect(build.artifacts_size).to be_nil
+          end
+        end
+
+        context 'when using build token' do
+          before do
+            delete delete_url, token: build.token
+          end
+
+          it_behaves_like 'having removable artifacts'
+        end
+
+        context 'when using runnners token' do
+          before do
+            delete delete_url, token: build.project.runners_token
+          end
+
+          it_behaves_like 'having removable artifacts'
         end
       end
 
       describe 'GET /builds/:id/artifacts' do
-        before { get get_url, token: build.token }
+        before do
+          get get_url, token: token
+        end
 
         context 'build has artifacts' do
           let(:build) { create(:ci_build, :artifacts) }
@@ -518,13 +566,29 @@ describe Ci::API::API do
               'Content-Disposition' => 'attachment; filename=ci_build_artifacts.zip' }
           end
 
-          it 'downloads artifact' do
-            expect(response).to have_http_status(200)
-            expect(response.headers).to include download_headers
+          shared_examples 'having downloadable artifacts' do
+            it 'download artifacts' do
+              expect(response).to have_http_status(200)
+              expect(response.headers).to include download_headers
+            end
+          end
+
+          context 'when using build token' do
+            let(:token) { build.token }
+
+            it_behaves_like 'having downloadable artifacts'
+          end
+
+          context 'when using runnners token' do
+            let(:token) { build.project.runners_token }
+
+            it_behaves_like 'having downloadable artifacts'
           end
         end
 
         context 'build does not has artifacts' do
+          let(:token) { build.token }
+
           it 'responds with not found' do
             expect(response).to have_http_status(404)
           end

spec/requests/git_http_spec.rb

@@ -300,25 +300,79 @@ describe 'Git HTTP requests', lib: true do
       end
 
       context "when a gitlab ci token is provided" do
-        let(:token) { 123 }
-        let(:project) { FactoryGirl.create :empty_project }
+        let(:build) { create(:ci_build, :running) }
+        let(:project) { build.project }
+        let(:other_project) { create(:empty_project) }
 
         before do
-          project.update_attributes(runners_token: token)
           project.project_feature.update_attributes(builds_access_level: ProjectFeature::ENABLED)
         end
 
-        it "downloads get status 200" do
-          clone_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: token
+        context 'when build created by system is authenticated' do
+          it "downloads get status 200" do
+            clone_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
 
-          expect(response).to have_http_status(200)
-          expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
+            expect(response).to have_http_status(200)
+            expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
+          end
+
+          it "uploads get status 401 (no project existence information leak)" do
+            push_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+            expect(response).to have_http_status(401)
+          end
+
+          it "downloads from other project get status 404" do
+            clone_get "#{other_project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+            expect(response).to have_http_status(404)
+          end
         end
 
-        it "uploads get status 401 (no project existence information leak)" do
-          push_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: token
+        context 'and build created by' do
+          before do
+            build.update(user: user)
+            project.team << [user, :reporter]
+          end
 
-          expect(response).to have_http_status(401)
+          shared_examples 'can download code only from own projects' do
+            it 'downloads get status 200' do
+              clone_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+              expect(response).to have_http_status(200)
+              expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
+            end
+
+            it 'uploads get status 403' do
+              push_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+              expect(response).to have_http_status(401)
+            end
+          end
+
+          context 'administrator' do
+            let(:user) { create(:admin) }
+
+            it_behaves_like 'can download code only from own projects'
+
+            it 'downloads from other project get status 403' do
+              clone_get "#{other_project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+              expect(response).to have_http_status(403)
+            end
+          end
+
+          context 'regular user' do
+            let(:user) { create(:user) }
+
+            it_behaves_like 'can download code only from own projects'
+
+            it 'downloads from other project get status 404' do
+              clone_get "#{other_project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+              expect(response).to have_http_status(404)
+            end
+          end
         end
       end
     end

spec/requests/jwt_controller_spec.rb

@@ -22,11 +22,13 @@ describe JwtController do
 
   context 'when using authorized request' do
     context 'using CI token' do
-      let(:project) { create(:empty_project, runners_token: 'token') }
-      let(:headers) { { authorization: credentials('gitlab-ci-token', project.runners_token) } }
+      let(:build) { create(:ci_build, :running) }
+      let(:project) { build.project }
+      let(:headers) { { authorization: credentials('gitlab-ci-token', build.token) } }
 
       context 'project with enabled CI' do
         subject! { get '/jwt/auth', parameters, headers }
+
         it { expect(service_class).to have_received(:new).with(project, nil, parameters) }
       end
 
@@ -43,13 +45,31 @@ describe JwtController do
 
     context 'using User login' do
       let(:user) { create(:user) }
-      let(:headers) { { authorization: credentials('user', 'password') } }
-
-      before { expect(Gitlab::Auth).to receive(:find_with_user_password).with('user', 'password').and_return(user) }
+      let(:headers) { { authorization: credentials(user.username, user.password) } }
 
       subject! { get '/jwt/auth', parameters, headers }
 
       it { expect(service_class).to have_received(:new).with(nil, user, parameters) }
+
+      context 'when user has 2FA enabled' do
+        let(:user) { create(:user, :two_factor) }
+
+        context 'without personal token' do
+          it 'rejects the authorization attempt' do
+            expect(response).to have_http_status(401)
+            expect(response.body).to include('You have 2FA enabled, please use a personal access token for Git over HTTP')
+          end
+        end
+
+        context 'with personal token' do
+          let(:access_token) { create(:personal_access_token, user: user) }
+          let(:headers) { { authorization: credentials(user.username, access_token.token) } }
+
+          it 'rejects the authorization attempt' do
+            expect(response).to have_http_status(200)
+          end
+        end
+      end
     end
 
     context 'using invalid login' do

spec/requests/lfs_http_spec.rb

@@ -14,6 +14,7 @@ describe 'Git LFS API and storage' do
   end
   let(:authorization) { }
   let(:sendfile) { }
+  let(:pipeline) { create(:ci_empty_pipeline, project: project) }
 
   let(:sample_oid) { lfs_object.oid }
   let(:sample_size) { lfs_object.size }
@@ -244,14 +245,63 @@ describe 'Git LFS API and storage' do
           end
         end
 
-        context 'when CI is authorized' do
+        context 'when build is authorized as' do
           let(:authorization) { authorize_ci_project }
 
-          let(:update_permissions) do
-            project.lfs_objects << lfs_object
+          shared_examples 'can download LFS only from own projects' do
+            context 'for own project' do
+              let(:pipeline) { create(:ci_empty_pipeline, project: project) }
+
+              let(:update_permissions) do
+                project.team << [user, :reporter]
+                project.lfs_objects << lfs_object
+              end
+
+              it_behaves_like 'responds with a file'
+            end
+
+            context 'for other project' do
+              let(:other_project) { create(:empty_project) }
+              let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+
+              let(:update_permissions) do
+                project.lfs_objects << lfs_object
+              end
+
+              it 'rejects downloading code' do
+                expect(response).to have_http_status(other_project_status)
+              end
+            end
+          end
+
+          context 'administrator' do
+            let(:user) { create(:admin) }
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            it_behaves_like 'can download LFS only from own projects' do
+              # We render 403, because administrator does have normally access
+              let(:other_project_status) { 403 }
+            end
+          end
+
+          context 'regular user' do
+            let(:user) { create(:user) }
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            it_behaves_like 'can download LFS only from own projects' do
+              # We render 404, to prevent data leakage about existence of the project
+              let(:other_project_status) { 404 }
+            end
           end
 
-          it_behaves_like 'responds with a file'
+          context 'does not have user' do
+            let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+            it_behaves_like 'can download LFS only from own projects' do
+              # We render 404, to prevent data leakage about existence of the project
+              let(:other_project_status) { 404 }
+            end
+          end
         end
       end
 
@@ -431,10 +481,62 @@ describe 'Git LFS API and storage' do
         end
       end
 
-      context 'when CI is authorized' do
+      context 'when build is authorized as' do
         let(:authorization) { authorize_ci_project }
 
-        it_behaves_like 'an authorized requests'
+        let(:update_lfs_permissions) do
+          project.lfs_objects << lfs_object
+        end
+
+        shared_examples 'can download LFS only from own projects' do
+          context 'for own project' do
+            let(:pipeline) { create(:ci_empty_pipeline, project: project) }
+
+            let(:update_user_permissions) do
+              project.team << [user, :reporter]
+            end
+
+            it_behaves_like 'an authorized requests'
+          end
+
+          context 'for other project' do
+            let(:other_project) { create(:empty_project) }
+            let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+
+            it 'rejects downloading code' do
+              expect(response).to have_http_status(other_project_status)
+            end
+          end
+        end
+
+        context 'administrator' do
+          let(:user) { create(:admin) }
+          let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+          it_behaves_like 'can download LFS only from own projects' do
+            # We render 403, because administrator does have normally access
+            let(:other_project_status) { 403 }
+          end
+        end
+
+        context 'regular user' do
+          let(:user) { create(:user) }
+          let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+          it_behaves_like 'can download LFS only from own projects' do
+            # We render 404, to prevent data leakage about existence of the project
+            let(:other_project_status) { 404 }
+          end
+        end
+
+        context 'does not have user' do
+          let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+          it_behaves_like 'can download LFS only from own projects' do
+            # We render 404, to prevent data leakage about existence of the project
+            let(:other_project_status) { 404 }
+          end
+        end
       end
 
       context 'when user is not authenticated' do
@@ -583,11 +685,37 @@ describe 'Git LFS API and storage' do
           end
         end
 
-        context 'when CI is authorized' do
+        context 'when build is authorized' do
           let(:authorization) { authorize_ci_project }
 
-          it 'responds with 401' do
-            expect(response).to have_http_status(401)
+          context 'build has an user' do
+            let(:user) { create(:user) }
+
+            context 'tries to push to own project' do
+              let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+              it 'responds with 401' do
+                expect(response).to have_http_status(401)
+              end
+            end
+
+            context 'tries to push to other project' do
+              let(:other_project) { create(:empty_project) }
+              let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+              let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+              it 'responds with 401' do
+                expect(response).to have_http_status(401)
+              end
+            end
+          end
+
+          context 'does not have user' do
+            let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
           end
         end
       end
@@ -609,14 +737,6 @@ describe 'Git LFS API and storage' do
           end
         end
       end
-
-      context 'when CI is authorized' do
-        let(:authorization) { authorize_ci_project }
-
-        it 'responds with status 403' do
-          expect(response).to have_http_status(401)
-        end
-      end
     end
 
     describe 'unsupported' do
@@ -779,10 +899,51 @@ describe 'Git LFS API and storage' do
         end
       end
 
-      context 'when CI is authenticated' do
+      context 'when build is authorized' do
         let(:authorization) { authorize_ci_project }
 
-        it_behaves_like 'unauthorized'
+        context 'build has an user' do
+          let(:user) { create(:user) }
+
+          context 'tries to push to own project' do
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            before do
+              project.team << [user, :developer]
+              put_authorize
+            end
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
+          end
+
+          context 'tries to push to other project' do
+            let(:other_project) { create(:empty_project) }
+            let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            before do
+              put_authorize
+            end
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
+          end
+        end
+
+        context 'does not have user' do
+          let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+          before do
+            put_authorize
+          end
+
+          it 'responds with 401' do
+            expect(response).to have_http_status(401)
+          end
+        end
       end
 
       context 'for unauthenticated' do
@@ -839,10 +1000,42 @@ describe 'Git LFS API and storage' do
         end
       end
 
-      context 'when CI is authenticated' do
+      context 'when build is authorized' do
         let(:authorization) { authorize_ci_project }
 
-        it_behaves_like 'unauthorized'
+        before do
+          put_authorize
+        end
+
+        context 'build has an user' do
+          let(:user) { create(:user) }
+
+          context 'tries to push to own project' do
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
+          end
+
+          context 'tries to push to other project' do
+            let(:other_project) { create(:empty_project) }
+            let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
+          end
+        end
+
+        context 'does not have user' do
+          let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+          it 'responds with 401' do
+            expect(response).to have_http_status(401)
+          end
+        end
       end
 
       context 'for unauthenticated' do
@@ -897,7 +1090,7 @@ describe 'Git LFS API and storage' do
   end
 
   def authorize_ci_project
-    ActionController::HttpAuthentication::Basic.encode_credentials('gitlab-ci-token', project.runners_token)
+    ActionController::HttpAuthentication::Basic.encode_credentials('gitlab-ci-token', build.token)
   end
 
   def authorize_user

spec/services/auth/container_registry_authentication_service_spec.rb

@@ -6,8 +6,14 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do
   let(:current_params) { {} }
   let(:rsa_key) { OpenSSL::PKey::RSA.generate(512) }
   let(:payload) { JWT.decode(subject[:token], rsa_key).first }
+  let(:authentication_abilities) do
+    [
+      :read_container_image,
+      :create_container_image
+    ]
+  end
 
-  subject { described_class.new(current_project, current_user, current_params).execute }
+  subject { described_class.new(current_project, current_user, current_params).execute(authentication_abilities: authentication_abilities) }
 
   before do
     allow(Gitlab.config.registry).to receive_messages(enabled: true, issuer: 'rspec', key: nil)
@@ -189,13 +195,22 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do
     end
   end
 
-  context 'project authorization' do
+  context 'build authorized as user' do
     let(:current_project) { create(:empty_project) }
+    let(:current_user) { create(:user) }
+    let(:authentication_abilities) do
+      [
+        :build_read_container_image,
+        :build_create_container_image
+      ]
+    end
 
-    context 'allow to use scope-less authentication' do
-      it_behaves_like 'a valid token'
+    before do
+      current_project.team << [current_user, :developer]
     end
 
+    it_behaves_like 'a valid token'
+
     context 'allow to pull and push images' do
       let(:current_params) do
         { scope: "repository:#{current_project.path_with_namespace}:pull,push" }
@@ -214,12 +229,44 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do
 
         context 'allow for public' do
           let(:project) { create(:empty_project, :public) }
+
           it_behaves_like 'a pullable'
         end
 
-        context 'disallow for private' do
+        shared_examples 'pullable for being team member' do
+          context 'when you are not member' do
+            it_behaves_like 'an inaccessible'
+          end
+
+          context 'when you are member' do
+            before do
+              project.team << [current_user, :developer]
+            end
+
+            it_behaves_like 'a pullable'
+          end
+        end
+
+        context 'for private' do
           let(:project) { create(:empty_project, :private) }
-          it_behaves_like 'an inaccessible'
+
+          it_behaves_like 'pullable for being team member'
+
+          context 'when you are admin' do
+            let(:current_user) { create(:admin) }
+
+            context 'when you are not member' do
+              it_behaves_like 'an inaccessible'
+            end
+
+            context 'when you are member' do
+              before do
+                project.team << [current_user, :developer]
+              end
+
+              it_behaves_like 'a pullable'
+            end
+          end
         end
       end
 
@@ -230,6 +277,11 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do
 
         context 'disallow for all' do
           let(:project) { create(:empty_project, :public) }
+
+          before do
+            project.team << [current_user, :developer]
+          end
+
           it_behaves_like 'an inaccessible'
         end
       end

spec/services/create_deployment_service_spec.rb

@@ -41,7 +41,7 @@ describe CreateDeploymentService, services: true do
 
     context 'for environment with invalid name' do
       let(:params) do
-        { environment: 'name with spaces',
+        { environment: 'name,with,commas',
           ref: 'master',
           tag: false,
           sha: '97de212e80737a608d939f648d959671fb0a0142',
@@ -56,8 +56,36 @@ describe CreateDeploymentService, services: true do
         expect(subject).not_to be_persisted
       end
     end
+
+    context 'when variables are used' do
+      let(:params) do
+        { environment: 'review-apps/$CI_BUILD_REF_NAME',
+          ref: 'master',
+          tag: false,
+          sha: '97de212e80737a608d939f648d959671fb0a0142',
+          options: {
+            name: 'review-apps/$CI_BUILD_REF_NAME',
+            url: 'http://$CI_BUILD_REF_NAME.review-apps.gitlab.com'
+          },
+          variables: [
+            { key: 'CI_BUILD_REF_NAME', value: 'feature-review-apps' }
+          ]
+        }
+      end
+
+      it 'does create a new environment' do
+        expect { subject }.to change { Environment.count }.by(1)
+
+        expect(subject.environment.name).to eq('review-apps/feature-review-apps')
+        expect(subject.environment.external_url).to eq('http://feature-review-apps.review-apps.gitlab.com')
+      end
+
+      it 'does create a new deployment' do
+        expect(subject).to be_persisted
+      end
+    end
   end
-  
+
   describe 'processing of builds' do
     let(:environment) { nil }
     
@@ -95,6 +123,12 @@ describe CreateDeploymentService, services: true do
 
         expect(Deployment.last.deployable).to eq(deployable)
       end
+
+      it 'create environment has URL set' do
+        subject
+
+        expect(Deployment.last.environment.external_url).not_to be_nil
+      end
     end
 
     context 'without environment specified' do
@@ -107,7 +141,10 @@ describe CreateDeploymentService, services: true do
     
     context 'when environment is specified' do
       let(:pipeline) { create(:ci_pipeline, project: project) }
-      let(:build) { create(:ci_build, pipeline: pipeline, environment: 'production') }
+      let(:build) { create(:ci_build, pipeline: pipeline, environment: 'production', options: options) }
+      let(:options) do
+        { environment: { name: 'production', url: 'http://gitlab.com' } }
+      end
 
       context 'when build succeeds' do
         it_behaves_like 'does create environment and deployment' do

spec/services/git_push_service_spec.rb

@@ -253,6 +253,21 @@ describe GitPushService, services: true do
         expect(project.protected_branches.last.merge_access_levels.map(&:access_level)).to eq([Gitlab::Access::MASTER])
       end
 
+      it "when pushing a branch for the first time with an existing branch permission configured" do
+        stub_application_setting(default_branch_protection: Gitlab::Access::PROTECTION_DEV_CAN_PUSH)
+
+        create(:protected_branch, :no_one_can_push, :developers_can_merge, project: project, name: 'master')
+        expect(project).to receive(:execute_hooks)
+        expect(project.default_branch).to eq("master")
+        expect_any_instance_of(ProtectedBranches::CreateService).not_to receive(:execute)
+
+        execute_service(project, user, @blankrev, 'newrev', 'refs/heads/master' )
+
+        expect(project.protected_branches).not_to be_empty
+        expect(project.protected_branches.last.push_access_levels.map(&:access_level)).to eq([Gitlab::Access::NO_ACCESS])
+        expect(project.protected_branches.last.merge_access_levels.map(&:access_level)).to eq([Gitlab::Access::DEVELOPER])
+      end
+
       it "when pushing a branch for the first time with default branch protection set to 'developers can merge'" do
         stub_application_setting(default_branch_protection: Gitlab::Access::PROTECTION_DEV_CAN_MERGE)
 

spec/services/projects/housekeeping_service_spec.rb

@@ -4,6 +4,10 @@ describe Projects::HousekeepingService do
   subject { Projects::HousekeepingService.new(project) }
   let(:project) { create :project }
 
+  before do
+    project.reset_pushes_since_gc
+  end
+
   after do
     project.reset_pushes_since_gc
   end

spec/services/protected_branches/create_service_spec.rb

+require 'spec_helper'
+
+describe ProtectedBranches::CreateService, services: true do
+  let(:project) { create(:empty_project) }
+  let(:user) { project.owner }
+  let(:params) do
+    {
+      name: 'master',
+      merge_access_levels_attributes: [ { access_level: Gitlab::Access::MASTER } ],
+      push_access_levels_attributes: [ { access_level: Gitlab::Access::MASTER } ]
+    }
+  end
+
+  describe '#execute' do
+    subject(:service) { described_class.new(project, user, params) }
+
+    it 'creates a new protected branch' do
+      expect { service.execute }.to change(ProtectedBranch, :count).by(1)
+      expect(project.protected_branches.last.push_access_levels.map(&:access_level)).to eq([Gitlab::Access::MASTER])
+      expect(project.protected_branches.last.merge_access_levels.map(&:access_level)).to eq([Gitlab::Access::MASTER])
+    end
+  end
+end

spec/views/projects/pipelines/show.html.haml_spec.rb

@@ -9,11 +9,13 @@ describe 'projects/pipelines/show' do
   before do
     controller.prepend_view_path('app/views/projects')
 
-    create_build('build', 0, 'build')
-    create_build('test', 1, 'rspec 0:2')
-    create_build('test', 1, 'rspec 1:2')
-    create_build('test', 1, 'audit')
-    create_build('deploy', 2, 'production')
+    create_build('build', 0, 'build', :success)
+    create_build('test', 1, 'rspec 0:2', :pending)
+    create_build('test', 1, 'rspec 1:2', :running)
+    create_build('test', 1, 'spinach 0:2', :created)
+    create_build('test', 1, 'spinach 1:2', :created)
+    create_build('test', 1, 'audit', :created)
+    create_build('deploy', 2, 'production', :created)
 
     create(:generic_commit_status, pipeline: pipeline, stage: 'external', name: 'jenkins', stage_idx: 3)
 
@@ -37,6 +39,7 @@ describe 'projects/pipelines/show' do
 
     # builds
     expect(rendered).to have_text('rspec')
+    expect(rendered).to have_text('spinach')
     expect(rendered).to have_text('rspec 0:2')
     expect(rendered).to have_text('production')
     expect(rendered).to have_text('jenkins')
@@ -44,7 +47,7 @@ describe 'projects/pipelines/show' do
 
   private
 
-  def create_build(stage, stage_idx, name)
-    create(:ci_build, pipeline: pipeline, stage: stage, stage_idx: stage_idx, name: name)
+  def create_build(stage, stage_idx, name, status)
+    create(:ci_build, pipeline: pipeline, stage: stage, stage_idx: stage_idx, name: name, status: status)
   end
 end