In previous patch we added code to serve blob content via running `git cat-file
...` directly, but for every such request a request to slow RoR-based auth
backend is made, which is bad for performance.

Let's cache auth backend reply for small period of time, e.g. 30 seconds, which
will change the situation dramatically:

If we have a lot of requests to the same repository, we query auth backend only
for every Nth request and with e.g. 100 raw blob request/s N=3000 which means
that previous load to RoR code essentially goes away.

On the other hand as we query auth backend only once in a while and refresh the
cache, we will not miss potential changes in project settings. I mean potential
e.g. 25 seconds delay for a project to become public, or vise versa to become
private does no real harm.

The cache is done with the idea to allow the read side codepath to execute in
parallel and to be not blocked by eventual cache updates.

Overall this improves performance a lot:

  (on a 8-CPU i7-3770S with 16GB of RAM, 2001:67c:1254:e:89::fa34 is on localhost)

  # request is handled by gitlab-workhorse, but without auth caching
  $ ./wrk -c40 -d10 -t1 --latency http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg
  Running 10s test @ http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg
    1 threads and 40 connections
    Thread Stats   Avg      Stdev     Max   +/- Stdev
      Latency   622.99ms  200.08ms   1.40s    77.03%
      Req/Sec    62.65     22.37   120.00     55.00%
    Latency Distribution
       50%  589.51ms
       75%  726.88ms
       90%  896.09ms
       99%    1.18s
    626 requests in 10.01s, 1.11MB read
  Requests/sec:     62.55
  Transfer/sec:    113.73KB

  # request goes to gitlab-workhorse with auth caching (this patch)
  $ ./wrk -c40 -d10 -t1 --latency http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg
  Running 10s test @ http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg
    1 threads and 40 connections
    Thread Stats   Avg      Stdev     Max   +/- Stdev
      Latency    36.62ms   25.39ms 351.14ms   72.02%
      Req/Sec     1.16k    93.73     1.36k    77.00%
    Latency Distribution
       50%   36.30ms
       75%   47.02ms
       90%   66.36ms
       99%  122.46ms
    11580 requests in 10.01s, 20.56MB read
  Requests/sec:   1156.85
  Transfer/sec:      2.05MB

i.e. it is ~ 17x improvement.

Currently GitLab serves requests to get raw blobs via Ruby-on-Rails code and
Unicorn. Because RoR/Unicorn is relatively heavyweight, in environment where
there are a lot of simultaneous requests to get raw blobs, this works very slow
and server is constantly overloaded.

On the other hand, to get raw blob content, we do not need anything from RoR
framework - we only need to have access to project git repository on filesystem,
and knowing whether access for getting data from there should be granted or
not. That means it is possible to handle '.../raw/....' request directly
in more lightweight and performant gitlab-workhorse.

As gitlab-workhorse is written in Go, and Go has good concurrency/parallelism
support and is generally much faster than Ruby, moving raw blob serving task to
it makes sense and should be a net win.

In this patch: we add infrastructure to process GET request for '/raw/...':

- extract project / ref and path from URL
- query auth backend for whether download access should be granted or not
- emit blob content via spawning external `git cat-file`

I've tried to mimic the output to be as close as the one emitted by RoR code,
with the idea that for users the change should be transparent.

As in this patch we do auth backend query for every request to get a blob, RoR
code is still loaded very much, so essentially there is no speedup yet:

  (on a 8-CPU i7-3770S with 16GB of RAM, 2001:67c:1254:e:89::fa34 is on localhost)

  # without patch: request eventually goes to unicorn  (9 unicorn workers)
  $ ./wrk -c40 -d10 -t1 --latency http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg
  Running 10s test @ http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg
    1 threads and 40 connections
    Thread Stats   Avg      Stdev     Max   +/- Stdev
      Latency   609.34ms  156.92ms   1.18s    79.60%
      Req/Sec    64.22     19.90   120.00     67.00%
    Latency Distribution
       50%  596.50ms
       75%  678.23ms
       90%  805.72ms
       99%    1.04s
    642 requests in 10.01s, 1.24MB read
  Requests/sec:     64.16
  Transfer/sec:    127.00KB

  # with this patch: request handled by gitlab-workhorse
  $ ./wrk -c40 -d10 -t1 --latency http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg
  Running 10s test @ http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg
    1 threads and 40 connections
    Thread Stats   Avg      Stdev     Max   +/- Stdev
      Latency   622.99ms  200.08ms   1.40s    77.03%
      Req/Sec    62.65     22.37   120.00     55.00%
    Latency Distribution
       50%  589.51ms
       75%  726.88ms
       90%  896.09ms
       99%    1.18s
    626 requests in 10.01s, 1.11MB read
  Requests/sec:     62.55
  Transfer/sec:    113.73KB

In the next patch we'll cache requests to auth backend and that will improve
performance dramatically.

Develomentmode ended up the wrong way around



See merge request !29

Refactor upstream

Rationale: the code has become a tangled mess of global variables and
types that hang together when they need not. For example: every HTTP
handler uses a 'gitRequest'?? I want to clean this up and see if I can
move some things into internal packages.

Apart from using internal packages we now use http.Handler where we can,
and fewer global variables.

See merge request !20

We need to use `$(shell find` to find files from all subdirectories

Conflicts:
	main.go

remove make warning

When building gitlab-workhose a warning is printed out. This merge request fixes the warning

```
go build -ldflags "-X main.Version 0.5.1-20160103.233349" -o gitlab-workhorse
# _/home/git/gitlab-workhorse
link: warning: option -X main.Version 0.5.1-20160103.233349 may not work in future releases; use -X main.Version=0.5.1-20160103.233349
```

See merge request !22

Fix /api/v3/projects routing error

Non-special API requests were getting special treatment, which
resulted in 500 errors. This change avoids the special treatment and
adds tests that assert that regular API requests should be left alone
by gitlab-workhorse.

See merge request !26

We should test with what actually happens, e.g. foo%2Fbar, not
foo/bar.

Non-special API requests were getting special treatment, which
resulted in 500 errors. This change avoids the special treatment and
adds tests that assert that regular API requests should be left alone
by gitlab-workhorse.

This reverts commit d79f8563.

Kamil pointed out that shallow-copying http.DefaultTransport
accidentally gives us a references to a lot of things that probably
not be shared.