Commit efd9a717 authored by Saito's avatar Saito

solve the binary problem

parent e1d1673e
...@@ -8,17 +8,21 @@ module Gitlab ...@@ -8,17 +8,21 @@ module Gitlab
# return nil if message is nil # return nil if message is nil
return nil unless message return nil unless message
# return message if message type is binary
detect = CharlockHolmes::EncodingDetector.detect(message)
return message if detect[:type] == :binary
# if message is utf-8 encoding, just return it # if message is utf-8 encoding, just return it
message.force_encoding("utf-8") message.force_encoding("utf-8")
return message if message.valid_encoding? return message if message.valid_encoding?
# if message is not utf-8 encoding, detect and convert it # if message is not utf-8 encoding, convert it
detect = CharlockHolmes::EncodingDetector.detect(message) if detect[:encoding]
if detect[:encoding] && detect[:confidence] > 60
message.force_encoding(detect[:encoding]) message.force_encoding(detect[:encoding])
message.encode!("utf-8", detect[:encoding], :undef => :replace, :replace => "", :invalid => :replace) message.encode!("utf-8", detect[:encoding], :undef => :replace, :replace => "", :invalid => :replace)
end end
# ensure message encoding is utf8
message.valid_encoding? ? message : raise message.valid_encoding? ? message : raise
# Prevent app from crash cause of encoding errors # Prevent app from crash cause of encoding errors
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment