Automatic merge of gitlab-org/gitlab-ce master

doc/administration/gitaly/index.md

@@ -315,6 +315,66 @@ certificate_path = '/path/to/cert.pem'
 key_path = '/path/to/key.pem'
 ```
 
+## Gitaly-ruby
+
+Gitaly was developed to replace Ruby application code in gitlab-ce/ee.
+In order to save time and/or avoid the risk of rewriting existing
+application logic, in some cases we chose to copy some application code
+from gitlab-ce into Gitaly almost as-is. To be able to run that code, we
+made gitaly-ruby, which is a sidecar process for the main Gitaly Go
+process. Some examples of things that are implemented in gitaly-ruby are
+RPC's that deal with wiki's, and RPC's that create commits on behalf of
+a user, such as merge commits.
+
+### Number of gitaly-ruby workers
+
+Gitaly-ruby has much less capacity than Gitaly itself. If your Gitaly
+server has to handle a lot of request, the default setting of having
+just 1 active gitaly-ruby sidecar might not be enough. If you see
+ResourceExhausted errors from Gitaly it's very likely that you have not
+enough gitaly-ruby capacity.
+
+You can increase the number of gitaly-ruby processes on your Gitaly
+server with the following settings.
+
+Omnibus:
+
+```ruby
+# /etc/gitlab/gitlab.rb
+# Default is 2 workers. The minimum is 2; 1 worker is always reserved as
+# a passive stand-by.
+gitaly['ruby_num_workers'] = 4
+```
+
+Source:
+
+```toml
+# /home/git/gitaly/config.toml
+[gitaly-ruby]
+num_workers = 4
+```
+
+### Observing gitaly-ruby traffic
+
+Gitaly-ruby is a somewhat hidden, internal implementation detail of
+Gitaly. There is not that much visibility into what goes on inside
+gitaly-ruby processes.
+
+If you have Prometheus set up to scrape your Gitaly process, you can see
+request rates and error codes for individual RPC's in gitaly-ruby by
+querying `grpc_client_handled_total`. Strictly speaking this metric does
+not differentiate between gitaly-ruby and other RPC's, but in practice
+(as of GitLab 11.9), all gRPC calls made by Gitaly itself are internal
+calls from the main Gitaly process to one of its gitaly-ruby sidecars.
+
+Assuming your `grpc_client_handled_total` counter only observes Gitaly,
+the following query shows you RPC's are (most likely) internally
+implemented as calls to gitaly-ruby.
+
+```
+sum(rate(grpc_client_handled_total[5m])) by (grpc_method) > 0
+```
+
 ## Disabling or enabling the Gitaly service in a cluster environment
 
 If you are running Gitaly [as a remote

doc/development/gitaly.md

@@ -197,3 +197,82 @@ as a [CI environment variable](../ci/variables/README.md#variables).
 ---
 
 [Return to Development documentation](README.md)
+
+## Wrapping RPCs in Feature Flags
+
+Here are the steps to gate a new feature in Gitaly behind a feature flag.
+
+### Gitaly
+
+1. Create a package scoped flag name:
+
+   ```go
+   var findAllTagsFeatureFlag = "go-find-all-tags"
+   ```
+
+1. Create a switch in the code using the `featureflag` package:
+
+   ```go
+   if featureflag.IsEnabled(ctx, findAllTagsFeatureFlag) {
+     // go implementation
+   } else {
+     // ruby implementation
+   }
+   ```
+
+1. Create prometheus metrics:
+
+   ```go
+   var	findAllTagsRequests = prometheus.NewCounterVec(
+   		prometheus.CounterOpts{
+   			Name: "gitaly_find_all_tags_requests_total",
+   			Help: "Counter of go vs ruby implementation of FindAllTags",
+   		},
+   		[]string{"implementation"},
+   	)
+   )
+
+   func init() {
+   	prometheus.Register(findAllTagsRequests)
+   }
+
+   if featureflag.IsEnabled(ctx, findAllTagsFeatureFlag) {
+   	findAllTagsRequests.WithLabelValues("go").Inc()
+     // go implementation
+   } else {
+   	findAllTagsRequests.WithLabelValues("ruby").Inc()
+     // ruby impelmentation
+   }
+   ```
+
+1. Set headers in tests:
+
+   ```go
+   import (
+     "google.golang.org/grpc/metadata"
+
+     "gitlab.com/gitlab-org/gitaly/internal/featureflag"
+   )
+
+   //...
+
+   md := metadata.New(map[string]string{featureflag.HeaderKey(findAllTagsFeatureFlag): "true"})
+   ctx = metadata.NewOutgoingContext(context.Background(), md)
+
+   c, err = client.FindAllTags(ctx, rpcRequest)
+   require.NoError(t, err)
+   ```
+
+### Gitlab-Rails
+
+1. Add feature flag to `lib/gitlab/gitaly_client.rb` (in gitlab-rails):
+
+   ```ruby
+   SERVER_FEATURE_FLAGS = %w[go-find-all-tags].freeze
+   ```
+
+1. Test in rails console by setting feature flag:
+
+   ```ruby
+   Feature.enable('gitaly_go-find-all-tags')
+   ```