• D. Wythe's avatar
    net/smc: avoid data corruption caused by decline · e6d71b43
    D. Wythe authored
    We found a data corruption issue during testing of SMC-R on Redis
    applications.
    
    The benchmark has a low probability of reporting a strange error as
    shown below.
    
    "Error: Protocol error, got "\xe2" as reply type byte"
    
    Finally, we found that the retrieved error data was as follows:
    
    0xE2 0xD4 0xC3 0xD9 0x04 0x00 0x2C 0x20 0xA6 0x56 0x00 0x16 0x3E 0x0C
    0xCB 0x04 0x02 0x01 0x00 0x00 0x20 0x00 0x00 0x00 0x00 0x00 0x00 0x00
    0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0xE2
    
    It is quite obvious that this is a SMC DECLINE message, which means that
    the applications received SMC protocol message.
    We found that this was caused by the following situations:
    
    client                  server
            ¦  clc proposal
            ------------->
            ¦  clc accept
            <-------------
            ¦  clc confirm
            ------------->
    wait llc confirm
    			send llc confirm
            ¦failed llc confirm
            ¦   x------
    (after 2s)timeout
                            wait llc confirm rsp
    
    wait decline
    
    (after 1s) timeout
                            (after 2s) timeout
            ¦   decline
            -------------->
            ¦   decline
            <--------------
    
    As a result, a decline message was sent in the implementation, and this
    message was read from TCP by the already-fallback connection.
    
    This patch double the client timeout as 2x of the server value,
    With this simple change, the Decline messages should never cross or
    collide (during Confirm link timeout).
    
    This issue requires an immediate solution, since the protocol updates
    involve a more long-term solution.
    
    Fixes: 0fb0b02b ("net/smc: adapt SMC client code to use the LLC flow")
    Signed-off-by: default avatarD. Wythe <alibuda@linux.alibaba.com>
    Reviewed-by: default avatarWen Gu <guwen@linux.alibaba.com>
    Reviewed-by: default avatarWenjia Zhang <wenjia@linux.ibm.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    e6d71b43
af_smc.c 91 KB