The reason was that the test was reusing the same log file without deleting it between tests.
Fixed by creating a new log file as part of the test