blob/auth: Cache auth backend reply for 30s
In previous patch we added code to serve blob content via running `git cat-file ...` directly, but for every such request a request to slow RoR-based auth backend is made, which is bad for performance. Let's cache auth backend reply for small period of time, e.g. 30 seconds, which will change the situation dramatically: If we have a lot of requests to the same repository, we query auth backend only for every Nth request and with e.g. 100 raw blob request/s N=3000 which means that previous load to RoR code essentially goes away. On the other hand as we query auth backend only once in a while and refresh the cache, we will not miss potential changes in project settings. I mean potential e.g. 25 seconds delay for a project to become public, or vise versa to become private does no real harm. The cache is done with the idea to allow the read side codepath to execute in parallel and to be not blocked by eventual cache updates. Overall this improves performance a lot: (on a 8-CPU i7-3770S with 16GB of RAM, 2001:67c:1254:e:89::fa34 is on localhost) # request is handled by gitlab-workhorse, but without auth caching $ ./wrk -c40 -d10 -t1 --latency http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg Running 10s test @ http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg 1 threads and 40 connections Thread Stats Avg Stdev Max +/- Stdev Latency 622.99ms 200.08ms 1.40s 77.03% Req/Sec 62.65 22.37 120.00 55.00% Latency Distribution 50% 589.51ms 75% 726.88ms 90% 896.09ms 99% 1.18s 626 requests in 10.01s, 1.11MB read Requests/sec: 62.55 Transfer/sec: 113.73KB # request goes to gitlab-workhorse with auth caching (this patch) $ ./wrk -c40 -d10 -t1 --latency http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg Running 10s test @ http://[2001:67c:1254:e:89::fa34]:7777/root/slapos/raw/master/software/wendelin/software.cfg 1 threads and 40 connections Thread Stats Avg Stdev Max +/- Stdev Latency 36.62ms 25.39ms 351.14ms 72.02% Req/Sec 1.16k 93.73 1.36k 77.00% Latency Distribution 50% 36.30ms 75% 47.02ms 90% 66.36ms 99% 122.46ms 11580 requests in 10.01s, 20.56MB read Requests/sec: 1156.85 Transfer/sec: 2.05MB i.e. it is ~ 17x improvement.