Ok,
 after two fairly hectic weeks for me, 2.1.127 is finally out there.

This kernel does:
 - various small but important networking fixes from Davem (thanks). One
   of them is the "anti-nagle" bit to allow programs that know what they
   are doing to avoid nagling by telling the kernel so. This is mainly
   things like Web servers and ftp-servers that can use this option
   together with "sendfile()".
 - scheduling timeout interface change: the new interface is much more
   logical than the old one, and allows us to get the jiffies wrap-around
   case right. Thanks to Andrea Arcangeli.
 - Various driver updates: specialix, sonycd,
 - Memory management fixups. Handle out-of-memory conditions correctly,
   and handle high memory load much more gracefully.
 - sparc and PowerPC architecture updates
 - 3c509 SMP fix, tlan PCI probe update.
 - scsi driver updates: ncr53c8xx, aic7xxx, dc390
 - filesystem updates: autofs, hfs, umsdos

Go, test, be happy,

                Linus

>> because I looked very deep and the patches really shouldn't have made any
>> real difference..
>> The reason - tadaam - is so silly that it's embarrassing. The thing is,
>> that the things that should use GFP_USER don't. They use GFP_KERNEL
>> instead, and that is sufficient to explain all the problems that Andrea
>> saw. Becuase GFP_KERNEL will continue to allow allocations even after the
>> freeing up of another page has failed.
>> After fixing that in mm/memory.c and mm/filemap.c, the problem seems to be
>> properly fixed.

> I thought to change that but I was not sure (and infact some email ago I
> asked that to you too). I have not changed that myself because I was
> worryed that userspace allocation could be too much light. It would be
> nice to know if using GFP_USER and disabling kswapd (at the end of
> vmscan.c) causes process to segfaults (so that we can know if a real time
> process can alloc/swapout memory safely).

I wonder why it wasn't GFP_USER - that's exactly what the thing is there
for, and I don't know when it was changed. Probably with the new page
cache or something. I just looked at the memory allocator, and it looked
like it was doing the right thing, and it _was_ - but because it was
called with GFP_KERNEL it tried harder than it should have to return a
good page even when it ran out of memory.
Anyway, I made a pre-patch-2.1.127-6 and put it on ftp.kernel.org (pre-4
and pre-5 have been my internal pre-patches and don't show up there). This
has the timeout code basic fixes and the mm fixes, and doesn't fall over
for me with Andreas memory load case.

	Linus

I just found a case that could certainly result in endless page faults,
and an endless stream of __get_free_page() calls. It's been there forever,
and I bascially thought it could never happen, but thinking about it some
more it can happen a lot more easily than I thought.

The problem is that the page fault handling code will give up if it cannot
allocate a page table entry. We have code in place to handle the final
page allocation failure, but the "mid-way" failures just failed, and
caused the page fault to be done over and over again.

More importantly, this could happen from kernel mode when a system call
was trying to fill in a user page, in which case it wouldn't even be
interruptible.

It's really unlikely to happen (because the page tables tend to be set up
already), but I suspect it can be triggered by execve'ing a new process
which is not going to have any existing page tables. Even then we're
likely to have old pages available (the ones we free'd from the previous
process), but at least it doesn't sound impossible that this could be a
problem.

I've not seen this behaviour myself, but it could have caused Andrea's
problems, especially the harder to find ones. Andrea, can you check this
patch (against clean 2.1.126) out and see if it makes any difference to
your testing?

(Right now it does the wrong error code: it will cause a SIGSEGV instead
of a SIGBUS when we run out of memory, but that's a small detail).
Essentially, instead of trying to call "oom()" and sending a signal (which
doesn't work for kernel level accesses anyway), the code returns the
proper return value from handle_mm_fault(), which allows the caller to do
the right thing (which can include following the exception tables). That
way we can handle the case of running out of memory from a kernel mode
access too..

(This is also why the fault gets the wrong signal - I didn't bother to fix
up the x86 fault handler all that much ;)
Btw, the reason I'm sending out these patches in emails instead of just
putting them on ftp.kernel.org is that the machine has had disk problems
for the last week, and finally gave up completely last Friday or so. So
ftp.kernel.org is down until we have a new raid array or the old one
magically recovers.  Sorry about the spamming.

                Linus

I have an alternate patch for low memory circumstances that I'd like you
to test out.
The problem with the old kswapd setup was at least partly that kswapd was
woken up too late - by the time kswapd was woken up, it really had to work
fairly hard. Also, kswapd really shouldn't be real-time at all: normally
it should just be a fairly low-priority process, and the priority should
grow as there is more urgent need for memory.
This alternate approach seems to work for me, and is designed to avoid the
"spikes" of heavy real-time kswapd activity during which the machine is
fairly unusable in the old scheme.

                Linus

 - architecture updates for alpha and MIPS (and some minor PPC updates
   too)
 - joystick updates
 - MCA stuff from Alan. The guy has too much free time on his hands.
 - stallion driver cosmetic update
 - nasty SMP race with "task queues" (not the scheduling kind), where we
   were mixing atomic metaphores, resulting in a mess. Usually a benign
   one, but occasionally you could force oopses.
 - some floppy and ide updates
 - PS/2 mouse driver integrated into the PC keyboard controller. That got
   rid of a lot of really nasty problems (it's the same controller,
   accessing it from two different drivers was always messy)
 - various driver updates: floppy, ide, network drivers, sound, video..
 - various small FS fixes - finally _really_ getting the ENOENT vs ENOTDIR
   stuff right, nfsd updates, remounting fixes, filesize limits on NFS
   and smbfs, ntfs and ufs updates...
 - shm updates from Alan
 - cleanup of some MM stuff, I hope Andrea will re-do the patches and I'll
   look at the other parts.
 - unix fd garbage collection fix, getting rid of circular dependencies..

And probably various other small fixes that I have thankfully forgotten
about.

It seems that I've finally found the mysterious bug that caused some SMP
machines to lock up at bootup if they had no keyboard enabled. It turns
out that the keyboard was a complete red herring, and that it just changed
timings of bottom half handling in particular. The real culprit was some
misguided locking attempts by the console driver at a really bad time.

Anyway, that means that the last of my personal show-stopper bugs in 2.1.x
seems to be finally history. I still expect to sync up with Alan Cox's
patches in particular, but I'm mentally getting ready for a real 2.2.

I still haven't decided on whether I'll make the same kind of "pre-2.2"
that I did before the 2.0 release, but there are strong psychological
reasons to do so to get people to more actively test it out with a "this
really should be stable" mindset.

In the meantime, there's now 2.1.125. Most of 2.1.125 is driver updates
for various things, most notably perhaps joystick and the new 5.10 version
of the Adaptec aic7xxx driver by Doug Ledford (but there are various other
driver updates). The fix for the mysterious lock-up is a few embarrassing
lines removed, but makes me feel a lot better ;)

Go forth, multiply, and fill the earth

.. is out there now, and includes:

 - subtle fix for lazy FP save and restore on x86. The bug has been there
   for a long time, but was apparently triggered by the re-write of the
   low-level scheduling function. It could result in corrupted i387 state
   under certain (admittedly fairly unlikely) circumstances.
 - various networking updates. Some of the bugs fixed could result in
   kernel Oopses. None of them were common, though.
 - fixes for both filesystem accounting and quota handling.
 - the much-ado-about-little video driver merge.
 - PPC and Sparc updates
 - i386/SMP interrupt handling falls back on the safe mode.. Please tell
   me whether there are still machines with problems.
 - some new network drivers and updates
 - final (we hope) IP masquerade update

I still have a problem with certain machines that apparently don't want to
boot with the keyboard not plugged in even though they should. Kill me
now. If you have problems with i386/SMP on a machine without a keyboard,
plug one in and send me a report..

This may or may not fix the APM problems, and the INITRD ones.  The
INITRD one in particular was a case of a fairly inexplicable test that
shouldn't have been there in the first place breaking when something
completely unrelated was cleaned up..

The APM breakage was simply due to it being in the wrong place.  The
patch looks bigger than it really is - it really only moves the file to
the proper directory, and makes sure that it should compile with the
standard assembler..

                Linus

I made a 117 to fix the silly things left in 116 in my excitement over it
passing all my crashtests. This should fix the things with the kernel
thinking it was out of memory much sooner than it actually was etc.

Alan still reports some funnies with unix domain sockets, but he's
reportedly fixed the behaviour of NFS over TCP. He didn't make it sound as
if you really want to use it yet, though ;)

                Linus

I just released Linux-2.1.116. I've tested it fairly extensively on my SMP
box, both with little memory and much, and I cannot make it lock up any
more.

Special thanks to Dean Gaudet who helped me set up a apache configuration
that finally made me able to repeat the lockup, and made me able to debug
the thing.

Most of the 2.1.116 patches are "just" alpha and m68k updates, and can be
ignored by most people. The bugfixes are, roughly:

 - fixed serious low-memory situation problem, where a critical resource
   allocation problem could result in nasy behaviour. Notably, doing TCP
   under low memory could result in TCP trying to allocate memory in a
   tight loop and locking out kswapd completely so that the situation
   would never be rectified. In short, the machine hung.
   This problem has been there forever, the only reason it doesn't show up
   under 2.0.x seems to be because under 2.0.x the TCP allocation was
   always for a single page, for which this situation never arises. Under
   2.1.x the slab code forced multi-page allocations.
   If you've seen lockups with 2.1.x, this may be the cause. This was what
   held up 2.1.116 for so long.
 - various minor driver updates. Networking, radio, bttv.
 - NFS over TCP still doesn't work, but at least it fails due to new
   reasons.

Alan, try your squid thing under 2.1.116. I suspect it will hold up now,

                Linus