• Stan Hu's avatar
    Force ASCII-8BIT encodings in CI job trace · f0d3c690
    Stan Hu authored
    This commit adds a feature flag, `ci_job_trace_force_encode`, that will
    ensure that CI job traces that contain UTF-8 can be appended with
    ASCII-8BIT data.
    
    Previously we saw encoding errors when persisted job logs that contained
    UTF-8 characters were appended to ASCII-8BIT data from the HTTP
    request. This would cause a 500 error, but the runner would retry
    indefinitely with identical PATCH requests. This commit should make the
    append work and allow the job to finish.
    
    This is the sequence of events that causes the append to be called:
    
    1. Runner sends a `PUT /api/v4/jobs/:id` to indicate the job is canceled
    or finished.
    
    2. `UpdateBuildStateService#accept_build_state!` persists all live job
    logs to object storage.
    
    3. `UpdateBuildStateService#accept_build_state!` returns a 202 to the
    runner.
    
    4. The runner will continue to send PATCH requests with job logs until
    all logs have been sent and received.
    
    5. If the last PATCH request arrives after the job log has been
    persisted, we retrieve the data from object storage to append the
    remaining lines.
    
    Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/333424
    
    Changelog: fixed
    f0d3c690
fog.rb 3.98 KB