The write back throttling (WBT) code checks if REQ_SYNC | REQ_IDLE is set
to determine if a write is O_DIRECT vs buffered. If the bits are not set
then it assumes it's a buffered write and will throttle LIO if we hit
certain metrics. LIO itself is not using the buffer cache and is doing
direct I/O, so this has us set the direct bits so we are not throttled.

When the initiator application is doing direct I/O this can greatly improve
performance. It depends on the backend device but we have seen where the
WBT code is throttling writes to only 20K IOPs with 4K I/Os when the device
can support 100K+.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230817192902.346791-1-michael.christie@oracle.comSigned-off-by: Martin K. Petersen <martin.petersen@oracle.com>