• Dylan Griffith's avatar
    Add per DB counters to logs and refactor · eeec86d3
    Dylan Griffith authored
    This MR adds multiple new DB metrics to our logging when the
    `multiple_database_counters` feature flag is enabled.
    
    ```diff
    + "db_primary_primary_cached_count": 0,
    + "db_primary_primary_count": 0,
    + "db_primary_primary_duration_s": 0,
    + "db_replica_primary_cached_count": 0,
    + "db_replica_primary_count": 0,
    + "db_replica_primary_duration_s": 0,
    + "db_primary_primary_wal_count": 0,
    + "db_primary_primary_wal_cached_count": 0,
    + "db_replica_primary_wal_cached_count": 0,
    + "db_replica_primary_wal_count": 0,
    + "db_replica_primary_duration_s": 0,
    + "db_primary_primary_duration_s": 0,
    ```
    
    These new fields indicate how many queries went to the database named
    `primary`. `primary` happens to be the default name Rails gives to the
    database connections described in `config/database.yml`. This is
    unfortunate as it conflicts with the `primary/replica` language we use
    to distinguish between read/write and read-only servers (hence
    `primary_primary` in the key) but this will change when we start
    explicitly setting the 2 databases we configure to be called `main` and
    `ci`. The feature flag exists to alleviate some of this confusion as
    described in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66515
     as only the ~group::sharding will need to use this for testing in the
    meantime.
    
    Prior to this change we already supported logging the `duration_s` data
    per database but we did not log the count of queries. When implementing
    this count of queries it became obvious it was much easier to implement
    these metrics in the same way that all the other metrics were
    implemented. In particular that they are logged as `0` even if there was
    no matching query. This led to quite a lot of refactoring because the
    list of `database_config_names` is dynamic. It's what is configured in
    `config/database.yml` and we had a tonne of tests that needed the full
    list of keys and values being logged.
    eeec86d3
structured_logger_spec.rb 15.3 KB