Increase ProcessBookkeepingService batch to 10_000
Since we've learnt through monitoring that batches with 1000 jobs are not taking more than 5.5s on average and the median is around 4s we should be save to increase this ten times and still process batches in the desired time window. This started from a conversation at https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28511#note_315475648 where initially we wanted this to be configurable but really it turns out this number should be approximately how quickly a single core can process this many jobs being marshalled and sent to Elasticsearch and is likely not going to be something that benefits much from configuration. But we'll want to increase it now to 10k which still seems like a safe number and will give us a lot of scaling headroom before we have to figure out how to parallelize this work.
Showing
Please register or sign in to comment