1. 01 Jul, 2008 2 commits
    • Jens Axboe's avatar
      Properly notify block layer of sync writes · 18ce3751
      Jens Axboe authored
      fsync_buffers_list() and sync_dirty_buffer() both issue async writes and
      then immediately wait on them. Conceptually, that makes them sync writes
      and we should treat them as such so that the IO schedulers can handle
      them appropriately.
      
      This patch fixes a write starvation issue that Lin Ming reported, where
      xx is stuck for more than 2 minutes because of a large number of
      synchronous IO in the system:
      
      INFO: task kjournald:20558 blocked for more than 120 seconds.
      "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
      message.
      kjournald     D ffff810010820978  6712 20558      2
      ffff81022ddb1d10 0000000000000046 ffff81022e7baa10 ffffffff803ba6f2
      ffff81022ecd0000 ffff8101e6dc9160 ffff81022ecd0348 000000008048b6cb
      0000000000000086 ffff81022c4e8d30 0000000000000000 ffffffff80247537
      Call Trace:
      [<ffffffff803ba6f2>] kobject_get+0x12/0x17
      [<ffffffff80247537>] getnstimeofday+0x2f/0x83
      [<ffffffff8029c1ac>] sync_buffer+0x0/0x3f
      [<ffffffff8066d195>] io_schedule+0x5d/0x9f
      [<ffffffff8029c1e7>] sync_buffer+0x3b/0x3f
      [<ffffffff8066d3f0>] __wait_on_bit+0x40/0x6f
      [<ffffffff8029c1ac>] sync_buffer+0x0/0x3f
      [<ffffffff8066d48b>] out_of_line_wait_on_bit+0x6c/0x78
      [<ffffffff80243909>] wake_bit_function+0x0/0x23
      [<ffffffff8029e3ad>] sync_dirty_buffer+0x98/0xcb
      [<ffffffff8030056b>] journal_commit_transaction+0x97d/0xcb6
      [<ffffffff8023a676>] lock_timer_base+0x26/0x4b
      [<ffffffff8030300a>] kjournald+0xc1/0x1fb
      [<ffffffff802438db>] autoremove_wake_function+0x0/0x2e
      [<ffffffff80302f49>] kjournald+0x0/0x1fb
      [<ffffffff802437bb>] kthread+0x47/0x74
      [<ffffffff8022de51>] schedule_tail+0x28/0x5d
      [<ffffffff8020cac8>] child_rip+0xa/0x12
      [<ffffffff80243774>] kthread+0x0/0x74
      [<ffffffff8020cabe>] child_rip+0x0/0x12
      
      Lin Ming confirms that this patch fixes the issue. I've run tests with
      it for the past week and no ill effects have been observed, so I'm
      proposing it for inclusion into 2.6.26.
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
      18ce3751
    • Divyesh Shah's avatar
      block: Fix the starving writes bug in the anticipatory IO scheduler · d585d0b9
      Divyesh Shah authored
      AS scheduler alternates between issuing read and write batches. It does
      the batch switch only after all requests from the previous batch are
      completed.
      
      When switching to a write batch, if there is an on-going read request,
      it waits for its completion and indicates its intention of switching by
      setting ad->changed_batch and the new direction but does not update the
      batch_expire_time for the new write batch which it does in the case of
      no previous pending requests.
      On completion of the read request, it sees that we were waiting for the
      switch and schedules work for kblockd right away and resets the
      ad->changed_data flag.
      Now when kblockd enters dispatch_request where it is expected to pick
      up a write request, it in turn ends the write batch because the
      batch_expire_timer was not updated and shows the expire timestamp for
      the previous batch.
      
      This results in the write starvation for all the cases where there is
      the intention for switching to a write batch, but there is a previous
      in-flight read request and the batch gets reverted to a read_batch
      right away.
      
      This also holds true in the reverse case (switching from a write batch
      to a read batch with an in-flight write request).
      
      I've checked that this bug exists on 2.6.11, 2.6.18, 2.6.24 and
      linux-2.6-block git HEAD. I've tested the fix on x86 platforms with
      SCSI drives where the driver asks for the next request while a current
      request is in-flight.
      
      This patch is based off linux-2.6-block git HEAD.
      
      Bug reproduction:
      A simple scenario which reproduces this bug is:
      - dd if=/dev/hda3 of=/dev/null &
      - lilo
         The lilo takes forever to complete.
      
      This can also be reproduced fairly easily with the earlier dd and
      another test
      program doing msync().
      
      The example test program below should print out a message after every
      iteration
      but it simply hangs forever. With this bugfix it makes forward progress.
      
      ====
      Example test program using msync() (thanks to suleiman AT google DOT
      com)
      
      inline uint64_t
      rdtsc(void)
      {
               int64_t tsc;
      
               __asm __volatile("rdtsc" : "=A" (tsc));
               return (tsc);
      }
      
      int
      main(int argc, char **argv)
      {
               struct stat st;
               uint64_t e, s, t;
               char *p, q;
               long i;
               int fd;
      
               if (argc < 2) {
                       printf("Usage: %s <file>\n", argv[0]);
                       return (1);
               }
      
               if ((fd = open(argv[1], O_RDWR | O_NOATIME)) < 0)
                       err(1, "open");
      
               if (fstat(fd, &st) < 0)
                       err(1, "fstat");
      
               p = mmap(NULL, st.st_size, PROT_READ | PROT_WRITE,
      MAP_SHARED, fd, 0);
      
               t = 0;
               for (i = 0; i < 1000; i++) {
                       *p = 0;
                       msync(p, 4096, MS_SYNC);
                       s = rdtsc();
                      *p = 0;
                       __asm __volatile(""::: "memory");
                       e = rdtsc();
                       if (argc > 2)
                               printf("%d: %lld cycles %jd %jd\n",
                                      i, e - s, (intmax_t)s, (intmax_t)e);
                       t += e - s;
               }
               printf("average time: %lld cycles\n", t / 1000);
               return (0);
      }
      
      Cc: <stable@kernel.org>
      Acked-by: default avatarNick Piggin <npiggin@suse.de>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
      d585d0b9
  2. 29 Jun, 2008 9 commits
  3. 27 Jun, 2008 2 commits
  4. 26 Jun, 2008 27 commits