This is the next iteration of the workqueue abstraction.

The framework includes:

 - per-CPU queueing support.

on SMP there is a per-CPU worker thread (bound to its CPU) and per-CPU
work queues - this feature is completely transparent to workqueue-users.
keventd automatically uses this feature. XFS can now update to work-queues
and have the same per-CPU performance as it had with its per-CPU worker
threads.

 - delayed work submission

there's a new queue_delayed_work(wq, work, delay) function and a new
schedule_delayed_work(work, delay) function. The later one is used to
correctly fix former tq_timer users. I've reverted those changes in 2.5.40
that changed tq_timer uses to schedule_work() - eg. in the case of
random.c or the tty flip queue it was definitely the wrong thing to do.

delayed work means a timer embedded in struct work_struct.  I considered
using split struct work_struct and delayed_work_struct types, but lots
of code actively uses task-queues in both delayed and non-delayed mode,
so i went for the more generic approach that allows both methods of work
submission.  Delayed timers do not cause any other overhead in the
normal submission path otherwise.

 - multithreaded run_workqueue() implementation

the run_workqueue() function can now be called from multiple contexts, and
a worker thread will only use up a single entryy - this property is used
by the flushing code, and can potentially be used in the future to extend
the number of per-CPU worker threads.

 - more reliable flushing

there's now a 'pending work' counter, which is used to accurately detect
when the last work-function has finished execution. It's also used to
correctly flush against timed requests. I'm not convinced whether the old
keventd implementation got this detail right.

 - i switched the arguments of the queueing function(s) per Jeff's
   suggestion, it's more straightforward this way.


Driver fixes:

i have converted almost every affected driver to the new framework. This
cleaned up tons of code. I also fixed a number of drivers that were still
using BHs (these drivers did not compile in 2.5.40).

while this means lots of changes, it might ease the QA decision whether to
put this patch into 2.5.

The pach converts roughly 80% of all tqueue-using code to workqueues - and
all the places that are not converted to workqueues yet are places that do
not compile in vanilla 2.5.40 anyway, due to unrelated changes. I've
converted a fair number of drivers that do not compile in 2.5.40, and i
think i've managed to convert every driver that compiles under 2.5.40.

into home.transmeta.com:/home/torvalds/v2.5/linux

into mulgrave.(none):/home/jejb/BK/scsi-for-linus-2.5

Hi!

Since the introduction of USER_HZ the SG_[GS]ET_TIMEOUT ioctls may have
a serious BUG as userspace uses a different HZ from the HZ in kernelspace.

In x86 HZ=1000 and USER_HZ=100, resulting in confusing timouts as the
kernel measures time 10 times as fast as userspace.

This patch is an attempt to fix this by transforming USER_HZ based timing to
HZ based timing before storing it in timeout. To make sure that SG_GET_TIMEOUT
and SG_SET_TIMEOUT behave consistently a field timeout_user is added which
stores the exact value that's passed by SG_SET_TIMEOUT and it's returned on
SG_GET_TIMEOUT.

Rolf Fokkens
fokkensr@fokkensr.vertis.nl

P.S. this is the second post of this patch

But it's called from scsi_add_lun()->scsi_alloc_sdev() before the type
is known.  The type is -1 all the time in scsi_initialise_merge_fn()
and scsi always bounces.

This patch makes it do the right thing - just enable block-highmem for
all scsi devices.

Jens had this to say:

"I guess that block-highmem has been around long enough, that I can
 use the term 'historically' at least in the kernel sense :-)

 This extra check was added for IDE because each device type driver
 (ide-disk, ide-cd, etc) needed to be updated to not assume virtual
 mappings of request data was valid.  I only did that for ide-disk,
 since this is the only one where bounce buffering really hurt
 performance wise.  So while ide-cd and ide-tape etc could have been
 updated, I deemed it uninteresting and not worthwhile.

 Now, this was just carried straight into the scsi counter parts,
 conveniently, because of laziness.  A quick glance at sr shows that it
 too can aviod bouncing easily (no changes needed).  st may need some
 changes, though.  So again, for scsi it was a matter of not impacting
 existing code in 2.4 too much.

 So TYPE_DISK check can be killed in 2.5 if someone does the work of
 checking that it is safe.  I'm not so sure it will make eg your SCSI
 CD-ROM that much faster :-)"

This removes an unused label in fs/devfs/base.c

into kroah.com:/home/greg/linux/BK/gregkh-2.5

It needs s/malloc.h/slab.h/ .
It also forgets to free some memory on an error exit patch.
Patch for 2.5.39 follows.

Here are the scatterlist primitives there's been mail about before.
Now the code has passed basic sanity testing, and is ready to merge
into Linus' tree to start getting wider use.  Greg, please merge!

To recap, the routines are a utility layer packaging several usb
core facilities to improve system performance.  It's synchronous.
The code uses functionality that drivers could use already, but
generally haven't:

    - Request queueing.  This is a big performance win.  It lets
      device drivers help the hcds avoid wasted i/o bandwidth, by
      eliminating irq and scheduling latencies between requests.  It
      can make a huge difference at high speed, when the latencies
      often exceed the time to handle each i/o request!

    - The new usb_map_sg() primitives, leveraging IOMMU hardware
      if it's there (better than entry-at-a-time mapping).

    - URB_NO_INTERRUPT transfer flag, a hint to hcds that they
      can avoid a 'success irq' for this urb.  Only the urb for
      the last scatterlist entry really needs an IRQ, the others
      can be eliminated or delayed.  (OHCI uses this today, and
      any HCD can safely ignore it.)

The particular functionality in these APIs seemed to meet Matt's
requirements for usb-storage, so I'd hope the 2.5 usb-storage
code will start to use these routines in a while.  (And maybe
those two scanner drivers: hpusbscsi, microtek.)

Brief summary of testing:  this code seems correct for normal
reads and writes, but the fault paths (including cancelation)
haven't been tested yet.  Both EHCI and OHCI seem to be mostly
OK with these more aggressive queued loads, but may need small
updates (like the two I sent yesterday).  Unfortunately I have
to report that UHCI and urb queueing will sometimes lock up my
hardware (PIIX4), so while we're lots better than 2.4 this is
still a bit of a trouble spot for now.

I'll be making some testing software available shortly, which
will help track down remaining HCD level problems by giving the
queuing APIs (and some others!) a more strenuous workout than
most drivers will, in their day-to-day usage.

- Dave

Greg, attached is a patch designed for diagnostic purposes.  Please apply
to the 2.5 tree -- yes, we'll be removing this at some point in the future.

It appears that we have a problem clearing halts.  This patch causes a very
clear message to be printed whenever a usb_stor_clear_halt() manages to
work.  So far, I haven't seen such a thing happen.  And I've seen _lots_ of
STALL conditions.

This problem has likely been around for a while... however, it hasn't been
noticed before because usb-storage was difficult to use because of other
bugs.  Heck, the most recent 'bk pull' is the first one for me in _months_
which let me boot all the way into X11.

I'm going to hold my patch queue until this is resolved.  On my test setup,
it's easy to see this failing.  I've tried with 4 different devices, with
both UHCI and EHCI drivers.  I don't want to confuse this problem with
other patches...

'result' in this function always seems to be -32.  Which is odd, because
control endpoints shouldn't do that.

I'm open to suggestions as to where to look for this bug, but my instincts
are telling me that this is a core or HCD issue, not a usb-storage issue.

On a positive note, this means that the error-recovery system gets a good
workout.

into penguin.transmeta.com:/home/penguin/torvalds/repositories/kernel/linux

into flint.arm.linux.org.uk:/usr/src/linux-bk-2.5/linux-2.5-rmk

This corrects spelling mistakes, adds missed configuration for
cpufreq, corrects free_irq comment, etc.

Add "IRQ_" prefix to these sa1111 irq numbers.

We must clear down all currently pending IRQs before servicing any
IRQ on the chip.  This prevents immediate recursion into the
interrupt handling paths when we service the first IRQ.

This cset updates sa1100 code for the now merged cpufreq next-gen.

Update sa1100fb for recent fbcon changes, and move stork LCD power
handling into machine specific file.

This didn't follow the LDM model correctly.  The SA1111 is always
a device on the root bus.

This cset updates the SA1111 core, PCMCIA, OHCI and keyboard drivers,
allowing them to take advantage of the Linux device manager code;
this implements initial suspend/resume support for the SA1111 in the
core.  Many existing drivers currently rely on the old PM-based
interface for suspend/resume support.

In a test where some memory corruption happened, I noticed an
oops (null pointer exception in_irq) that's avoidable.  Here's
a patch that avoids it ... anyone seeing the err() is likely
to hang some process, but that's better than the alternative.
(Also inlines some used-once routines, saving a bit of space
to make up for the new diagnostic.)

In doing some more extensive testing of the urb queueing behavior,
I noticed that (a) IOC wasn't always being set for each urb, while
for now it needs to be set; (b) a qh patchup wasn't done quite
where it should be.  This resolves those two issues, as well
as making it a bit less noisy to unlink lots of urbs at the once.

This has minor usbcore cleanups:

DOC:
    - the changes passing a usb_interface to driver probe() and disconnect()
      weren't reflected in their adjacent docs.  likewise they still said
      it was possible to get a null usb_device_id (no more).

    - the (root) hub API restrictions from rmk's ARM patch weren't
      flagged

    - mention the non-dma-coherent cache issue for usb_buffer_alloc()

    - mention disconnect() cleanup issue with usb_{control,bulk}_msg()
      [ you can't cancel those urbs from disconnect() ]

CODE
    - make driver ioctl() use 'usb_interface' too ... this update
      also resolves an old 'one instance per device' bad assumption

    - module locking on driver->ioctl() was goofy, kept BKL way too
      long and didn't try_inc_mod_count() like the rest of usbcore

    - hcd unlink code treated iso inappropriately like interrupt;
      only interrupt still wants that automagic mode

    - move iso init out of ohci into shared submit_urb logic

    - remove interrupt transfer length restriction; hcds that don't
      handle packetization (just like bulk :) should be updated,
      but device drivers won't care for now.

/sbin/hotplug is now called when any device is added or removed from the
system.

Trivial include file fix..

More bits from 2.5.39-dj sucked out by Adrian Bunk.

 - drivers/char/toshiba.c: add
   MODULE_{PARM_DESC,AUTHOR,DESCRIPTION,SUPPORTED_DEVICE}
 - drivers/mtd/nand/nand_ecc.c: add MODULE_{AUTHOR,DESCRIPTION}
 - drivers/net/skfp/skfddi.c: add MODULE_AUTHOR
 - drivers/net/tokenring/olympic.c: remove "\n" at the end of
   MODULE_DESCRIPTION
 - fs/driverfs/inode.c: add MODULE_LICENSE
 - fs/nls/nls_cp1250.c: correct MODULE_LICENSE
 - include/linux/module.h: add "GPL v2" to the list of free software
   licenses

Adrian Bunk went through .39-dj, and pulled out a bunch of
trivial bits (docs changes, whitespace fixes etc)

 - CREDITS: update the web-address of Tigran A. Aivazian
 - Documentation/Changes: higher minimum version of reiserfsprogs
 - s/www.linuxdoc.org/www.tldp.org/g in:
   - Documentation/DocBook/sis900.tmpl
   - Documentation/kernel-docs.txt
   - Documentation/scsi-generic.txt
   - Documentation/scsi.txt
   - Documentation/sound/oss/PAS16
 - Documentation/filesystems/isofs.txt: document where to get ISO 9660
   docs from
 - Documentation/networking/00-INDEX: document that e100.txt and e1000.txt
   are present
 - typo fixes in:
   - Documentation/networking/ip-sysctl.txt
   - Documentation/s390/Debugging390.txt
   - drivers/ide/Config.help
 - MAINTAINERS:
   - update location of the emu10k1-devel and linux-mips lists
   - Remy Card is no longer ext2 maintainer
   - list Andrew Morton instead of Remy Card as second ext3 maintainer
   - update mail addresses of Riley H. Williams and Jack Hammer
   - misc whitespace -> tab fixes
 - arch/mips/kernel/time.c: correct the location of a README
 - whitespace -> tab fixes in
   drivers/net/{3c505,3c509,arcnet/arcnet,at1700,hamradio/scc,ni65,
   pcmcia/aironet4500_cs}.c and drivers/net/wan/lmc/lmc_var.h
 - drivers/pci/quirks.c: update URL
 - remove tabs/whitespace at the end of lines in:
   - drivers/tc/lk201-map.map
   - drivers/tc/lk201-remap.c
   - drivers/tc/zs.h
 - fs/jfs/jfs_logmgr.c: remove two extra empty lines
 - include/linux/auto_fs.h: s/__x86_64/__x86_64__/

In two drivers a wrong size of memory was allocated for cpufreq_driver: as
it must include NR_CPUS times a struct cpufreq_policy (and not struct
cpufreq_freqs). Thanks to Petr Vandrovec for this patch.

QDIO_BH was never actually used anyway, and won't do much good now BHs
are gone.

Beginning of proper refcounting.  New helpers introduced, several drivers
switched to using them for dynamic allocation of gendisks.  Once everything
is switched (and that will be way easier than per-drive gendisks series)
we will be able to add sane reference counts on gendisk, at which point
we can safely put pointer to gendisk in struct block_device / struct request
and we had pretty much won - from that point it's pretty straightforward
crapectomy in drivers.