Signed-off-by: Rémy Coutable <remy@rymai.me>

Trim extra displayed carriage returns in diffs and files with CRLFs

## What does this MR do?

This MR strips out extra carriage returns at the end of each tokenized string.

## Are there points in the code the reviewer needs to double check?

Does this remove more than we need?

## Why was this MR needed?

Windows users would see extra lines in their files. Newline characters are already added with every line in the HTML formatter.

## What are the relevant issue numbers?

#20440, gitlab-com/support-forum#892, #20468

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5588
Signed-off-by: Rémy Coutable <remy@rymai.me>

Ignore invalid IPs in X-Forwarded-For when trusted proxies are configured.

## What does this MR do?

Catches IPAddr::InvalidAddressError exceptions in `trusted_proxy?` when a) a trusted proxy is set up in the gitlab config and b) an invalid IP address is passed to the method (e.g. one with a port attached). When caught, returns `false` from the method. Prevents a 500 error in this situation.

## What are the relevant issue numbers?

Closes gitlab-org/gitlab-ce#20466.

## Does this MR meet the acceptance criteria?

- [X] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [N/A] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [N/A] API support added
- Tests
  - [X] Added for this feature/bug
  - [X] All builds are passing
- [X] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [X] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [X] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5584
Signed-off-by: Rémy Coutable <remy@rymai.me>

Fix attr reader to force the intended values for source and target shas

## What does this MR do?

When importing a pull request from GitHub, the old and new branches may no longer actually exist by those names, but we need to recreate the merge
request diff with the right source and target shas.  We use these `target_branch_sha` and `source_branch_sha` attributes to force  these to the intended values. But the reader methods were always looking up to the target/source branch head instead of check if these values was  previously set, this MR applies a memoization pattern to both of them.

## Are there points in the code the reviewer needs to double check?

This [commit](https://gitlab.com/gitlab-org/gitlab-ce/commit/6ce25e7b4caa9e94de74378729178c7060d640b2) introduced this bug in the importer.

## What are the relevant issue numbers?

gitlab-org/gitlab-ce#20385

## Does this MR meet the acceptance criteria?

- [X] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [X] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [X] ~~API support added~~
- Tests
  - [X] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

/cc @akitaonrails @DouweM

See merge request !5573
Signed-off-by: Rémy Coutable <remy@rymai.me>

Properly abort a merge when merge conflicts occur

If somehow a user attempted to accept a merge request that had conflicts (e.g. the "Accept Merge Request" button or the MR itself was not updated), `MergeService` did not properly detect that a conflict
occurred. It would assume that the MR went through without any issues and close the MR as though everything was fine. This could cause data loss if the source branch were removed.

Closes #20425

See merge request !5569
Signed-off-by: Rémy Coutable <remy@rymai.me>

fix repo hooks missing on import

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/19556

Repo hooks are missing from imported projects

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5549
Signed-off-by: Rémy Coutable <remy@rymai.me>

Add a log message when a project is scheduled for destruction for debugging

We have a lot of projects that are in `pending_delete` state. It's not clear whether they were ever scheduled for destruction, or whether Sidekiq just dropped the job due to `MemoryKiller` or some other reason.

Also this will provide a record of which user destroys a project.

#20365

See merge request !5540
Signed-off-by: Rémy Coutable <remy@rymai.me>

Fix timing problems running imports on production

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151

I've found out that in staging, the imported file is not copied fully by the time sidekiq runs the job, this should hopefully fix it. (Tested against staging).

See merge request !5523
Signed-off-by: Rémy Coutable <remy@rymai.me>

Fix Importing labels and milestones associations - Import/Export

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/19510 and https://gitlab.com/gitlab-org/gitlab-ce/issues/19447
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5426
Signed-off-by: Rémy Coutable <remy@rymai.me>

[ci skip]
Signed-off-by: Rémy Coutable <remy@rymai.me>

[ci skip]
Signed-off-by: Rémy Coutable <remy@rymai.me>

Signed-off-by: Rémy Coutable <remy@rymai.me>

Update gitlab-shell version to 3.2.1 in the 8.9->8.10 update guide

See merge request !5516
Signed-off-by: Rémy Coutable <remy@rymai.me>

Fix missing schema update for 20160722221922

20160722221922 was merged in 8.10.1 but the schema.rb was not updated.

See merge request !5512
Signed-off-by: Rémy Coutable <remy@rymai.me>

Fix a bug where forking a project from a repository storage to another would fail

See merge request !5509
Signed-off-by: Rémy Coutable <remy@rymai.me>

Show release notes in tag list

See merge request !5503
Signed-off-by: Rémy Coutable <remy@rymai.me>

Fix expand all diffs button in compare view

## What does this MR do?

Fix the expand all diffs button in compare view.

## Are there points in the code the reviewer needs to double check?

Don't think so.

## Why was this MR needed?

We can't reuse the existing value of the format parameter, because on
the merge request page that's 'json', so the link would go to a JSON
file. We can't set it to HTML, because that adds '.html' to the URL, which
breaks the compare view (and looks bad). Setting it to `nil` uses the
default format, which in all of these cases is HTML anyway.

## What are the relevant issue numbers?

Closes #20241.

See merge request !5500
Signed-off-by: Rémy Coutable <remy@rymai.me>

Rescue Rugged::OSError (lock exists) when creating references.

Rescue an exception on concurrent scenarios. I was able to create a spec for this, because if you create the lock file before trying to create the reference the reference is created without a problem and the lock file is removed. So IMHO there is a race condition where more than one process is trying to create the same reference at the same time raising the exception, so I just added the patch without specs.

```ruby
    it "attempting to call keep_around when exists a lock does not fail" do
      ref = repository.send(:keep_around_ref_name, sample_commit.id)
      path = File.join(repository.path, ref)
      lock_path = "#{path}.lock"

      FileUtils.mkdir_p(path)
      File.open(lock_path, 'w') { |f| f.write('') }

      begin
        # No exception is raised because the lock file is removed at some point in the process
        repository.keep_around(sample_commit.id)
      ensure
        File.delete(path)
        File.delete(lock_path)
      end
    end
```

See merge request !5497
Signed-off-by: Rémy Coutable <remy@rymai.me>

Signed-off-by: Rémy Coutable <remy@rymai.me>

Added '*.js.es6 gitlab-language=javascript' to .gitattributes

## What does this MR do?

According to the branch name this adds the `sytanx` highlighting for `.js.es6` files, but this actually adds the `syntax` highlighting. 😆

## Are there points in the code the reviewer needs to double check?

Is this the right place? 🏃

## Why was this MR needed?

We need pretty ES6 diffs.

## What are the relevant issue numbers?

Flyin' solo 🛫

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
  - [ ] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5486
Signed-off-by: Rémy Coutable <remy@rymai.me>

Resolve "Empty project page does not show Repository section on navbar after creating first branch/file"

See merge request !5399
Signed-off-by: Rémy Coutable <remy@rymai.me>

Add ENV variable to skip repository storages validations

Closes #20124

See merge request !5478
Signed-off-by: Rémy Coutable <remy@rymai.me>

Ensure current user can retry a build before showing the 'Retry' button

## What does this MR do?

Fixes #20150.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5476
Signed-off-by: Rémy Coutable <remy@rymai.me>

Merge branch '20189-markdown-video-doesn-t-work-when-the-referenced-video-file-is-in-same-repo' into 'master'

Ensure relative paths for video are rewritten as we do for images

## What does this MR do?

This ensures that path to videos are rewritten as we do for images.

## Are there points in the code the reviewer needs to double check?

I've decided to add a new `Blob#video?` method for simplicity's sake but this should probably be added to `Gitlab::Git::Blob`.

## What are the relevant issue numbers?

Fixes #20189.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5474
Signed-off-by: Rémy Coutable <remy@rymai.me>

Disable MySQL foreign key checks before dropping all tables

Fixes #20237. Disables MySQL foreign key checks before dropping all tables in a restore. MySQL doesn't honor `CASCADE` without a special flag when the database/tables are created. In order to drop the tables we need to disable foreign key checks. After the drop, re-enable the key checks.

WIP: Pending confirmation from customer that this fix works.

See merge request !5472
Signed-off-by: Rémy Coutable <remy@rymai.me>

Add iid to MR API response

## What does this MR do?

Adds `iid` parameter to the MR API docs.

## Why was this MR needed?

It is missing.

## What are the relevant issue numbers?

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/19578

## Does this MR meet the acceptance criteria?

- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)

See merge request !5468
Signed-off-by: Rémy Coutable <remy@rymai.me>

Signed-off-by: Rémy Coutable <remy@rymai.me>

Use project ID in repository cache to prevent stale data from persisting across projects

See merge request !5460
Signed-off-by: Rémy Coutable <remy@rymai.me>

Fix backup restore

## What does this MR do?

This MR fixes the backup restore task.

## Are there points in the code the reviewer needs to double check?

Whether the mode 0277 makes sense. (Group SUID + group/user read/write permissions)

## Why was this MR needed?

86359ec8 broke this, and it was not caught by any specs. Users would see:

```
Restoring repositories ...
rake aborted!
NameError: undefined local variable or method `repos_path' for #<Backup::Repository:0x00000007cea1d8>
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/repository.rb:59:in `restore'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:79:in `block (4 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:54:in `block (3 levels) in <top (required)>'
Tasks: TOP => gitlab:backup:repo:restore
(See full trace by running task with --trace)
```

## What are the relevant issue numbers?

#20188

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [ ] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5459
Signed-off-by: Rémy Coutable <remy@rymai.me>

Add branch or tag icon to ref in builds page

## What does this MR do?
Add branch or tag icon to ref in builds page

## Why was this MR needed?
It's currently missing but should be there

## What are the relevant issue numbers?
Closes #20139

## Screenshots (if relevant)
<img src="/uploads/fb5c9a285a1ce95c0aa8dfa5e32c5d7d/Screen_Shot_2016-07-22_at_9.14.39_AM.png" width="400px">

See merge request !5434
Signed-off-by: Rémy Coutable <remy@rymai.me>

Users can filter branches by name on project branches page

This MR aims to solve #18674 by adding the possibility to filter project branches by name

![Screen_Shot_2016-07-07_at_17.21.25](/uploads/b674765d2b1cb8a121c2101715a4568b/Screen_Shot_2016-07-07_at_17.21.25.png)

See merge request !5144
Signed-off-by: Rémy Coutable <remy@rymai.me>

Fixed issue with autocomplete search not working with enter key

## What does this MR do?

Fixes an issue that was stopping the enter key from searching in the top search bar.

## What are the relevant issue numbers?

Closes #20165 

ps. Once this is merged - i'll create a MR against master with the JS changes.

See merge request !5466

Closes #20165

[8.10] Don't show comment button in gutter of diffs on MR discussion tab

`8-10-stable` counterpart to https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5492

See merge request !5493

Signed-off-by: Rémy Coutable <remy@rymai.me>

Add links to the real markdown.md file for all GFM examples

## What does this MR do?

Add links to the real markdown.md file for all GFM examples since they will not render correctly on docs.gitlab.com.

We already have a note https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5440, but better be explicit.

See merge request !5458
Signed-off-by: Rémy Coutable <remy@rymai.me>