• Kirill Smelkov's avatar
    Y wcfs: tests: Fix double close of pipes in tSubProcess · 5571e203
    Kirill Smelkov authored
    Running tests in wcfs/wcfs_faultyprot_test.py was yielding errors about
    invalid closes, e.g.
    
        wcfs/wcfs_faultyprot_test.py::test_wcfs_pinhfaulty_kill_on_watch[_bad_watch_no_pin_read]
        ------------------------------------ live log call -------------------------------------
        M: commit -> @at0 (03fcee2b22c56155)
        INFO     wcfs:__init__.py:293 starting for file:///tmp/testdb_fs.v5XBEf/1.fs ...
        ...
        INFO     wcfs:__init__.py:400 unmount/stop wcfs pid875517 @ /dev/shm/wcfs/1060296857b6e9e0134ffb708787deb4039140ae
        I1126 19:11:13.746387  875517 wcfs.go:3100] stop "/dev/shm/wcfs/1060296857b6e9e0134ffb708787deb4039140ae" "file:///tmp/testdb_fs.v5XBEf/1.fs"
        close failed in file object destructor:             <-- NOTE
        IOError: [Errno 9] Bad file descriptor              <-- NOTE
        PASSED
    
    Since soon we are going to use tSubProcess not only in tests this will
    become less innocent with more incentive to fix.
    
    Looking at the strace output I saw that the same file descriptor was
    being closed twice in a row. Looking at strace further revealed that
    this fd was created via pip syscall and finally I could connect it to
    the pipe created by subprocess.Popen.
    
    So what happens here is that we were taking fd number from
    subprocess.Popen and passing it to multiprocessing.Connection with both
    thinking they own the fd, and closing it on GC.
    
    -> Fix that by really passing the fd to MPConnection as a copy first, and
       then redirecting Popen fds to devnull - to make sure in robust way
       that stdin/out IO does not interfere with pickle exchange going on in
       between child and parent via multiprocessing connections.
    
    Fixes 33ea7769 (wcfs: tests: Move client to be pinkill'ed into separate process)
    5571e203
wcfs_faultyprot_test.py 16.3 KB