From: William Lee Irwin III <wli@holomorphy.com>

Micro-optimize sys_shmdt(). There are methods of exploiting knowledge
of the vma's being searched to restrict the search space. These are:

(1) shm mappings always start their lives at file offset 0, so only
	vma's above shmaddr need be considered. find_vma() can be used
	to seek to the proper position in mm->mmap in O(lg(n)) time.

(2) The search is for a vma which could be a fragment of a broken-up
	shm mapping, which would have been created starting at shmaddr
	with vm_pgoff 0 and then continued no further into userspace
	than shmaddr + size. So after having found an initial vma, find
	the size of the shm segment it maps to calculate an upper bound
	to the virtualspace that needs to be searched.

(3) mremap() would have caused the original checks to miss vma's mapping
	the shm segment if shmaddr were the original address at which
	the shm segments were attached. This does no better and no worse
	than the original code in that situation.

(4) If the chain of references in vma->vm_file->f_dentry->d_inode->i_size
	is not guaranteed by refcounting and/or the shm code then this is
	oopsable; AFAICT an inode is always allocated.

This patch adds the remaining System V IPC hooks, including the inline
documentation for them in security.h.  This includes a restored
sem_semop hook, as it does seem to be necessary to support fine-grained
access.

All of these System V IPC hooks are used by SELinux.  The SELinux System
V IPC access controls were originally described in the technical report
available from http://www.nsa.gov/selinux/slinux-abs.html, and the
LSM-based implementation is described in the technical report available
from http://www.nsa.gov/selinux/module-abs.html.

From Rohit Seth

Attached is a patch that passes the correct information back to user
land for number of attachments to shared memory segment.  I could have
done few more changes in a way nattach is getting set for regular cases
now, but just want to limit it at this point.

Patch from Hugh Dickins <hugh@veritas.com>

Fixes the Oracle startup problem reported by Alessandro Suardi.

Reverts a "simplification" to shmdt() which was wrong if subsequent
mprotects broke up the original VMA, or if parts of it were munmapped.

stat64 has been changed to return jiffies granuality as nsec in previously
unused fields. This allows make to make better decisions on when
to recompile a file. Follows losely the Solaris API.

CURRENT_TIME has been redefined to return struct timespec.  The users
who don't use it in a inode/attr context have been changed to use a new
get_seconds() function.  CURRENT_TIME is implemented by an out-of-line
function.

There is a small performance penalty in this patch.  The previous
filemap code had an optimization to flush atime only once a second.
This is currently gone, which will increase flushes a bit.  I believe
the correct solution if it should be a problem is to have per super
block fields that give an arbitary atime flush granuality - so that you
can set it to be only flushed once a hour if you prefer that.  I will
work on that later in separate patches if the need should arise.

struct inode and the attr struct has been changed to store struct
timespec instead of time_t for [cma]time.  Not all file systems support
this granuality, but some like XFS,NFSv3,CIFS,JFS do.  The others will
currently truncate the nsec part on flushing to disk.  There was some
discussion on this rounding on l-k previously.  I went for simple
truncation because there is not much evidence IMHO that the more
complicated roundings have any advantages.  In practice application will
be rather unlikely to notice the rounding anyways - they can only see a
difference when an inode is flush from memory and reloaded in less than
a second, which is rather unlikely.

Patch from Mingming, Rusty, Hugh, Dipankar, me:

- It greatly reduces the lock contention by having one lock per id.
  The global spinlock is removed and a spinlock is added in
  kern_ipc_perm structure.

- Uses ReadCopyUpdate in grow_ary() for locking-free resizing.

- In the places where ipc_rmid() is called, delay calling ipc_free()
  to RCU callbacks.  This is to prevent ipc_lock() returning an invalid
  pointer after ipc_rmid().  In addition, use the workqueue to enable
  RCU freeing vmalloced entries.

Also some other changes:

- Remove redundant ipc_lockall/ipc_unlockall

- Now ipc_unlock() directly takes IPC ID pointer as argument, avoid
  extra looking up the array.

The changes are made based on the input from Huge Dickens, Manfred
Spraul and Dipankar Sarma.  In addition, Cliff White has run OSDL's
dbt1 test on a 2 way against the earlier version of this patch.
Results shows about 2-6% improvement on the average number of
transactions per second.  Here is the summary of his tests:

                        2.5.42-mm2      2.5.42-mm2-ipclock
			-----------------------------
Average over 5 runs     85.0 BT         89.8 BT
Std Deviation 5 runs     7.4  BT         1.0 BT

Average over 4 best     88.15 BT        90.2 BT
Std Deviation 4 best     2.8 BT          0.5 BT


Also, another test today from Bill Hartner:

I tested Mingming's RCU ipc lock patch using a *new* microbenchmark - semopbench.
semopbench was written to test the performance of Mingming's patch.
I also ran a 3 hour stress and it completed successfully.

Explanation of the microbenchmark is below the results.
Here is a link to the microbenchmark source.

http://www-124.ibm.com/developerworks/opensource/linuxperf/semopbench/semopbench.c

SUT : 8-way 700 Mhz PIII

I tested 2.5.44-mm2 and 2.5.44-mm2 + RCU ipc patch

>semopbench -g 64 -s 16 -n 16384 -r > sem.results.out
>readprofile -m /boot/System.map | sort -n +0 -r > sem.profile.out

The metric is seconds / per repetition.  Lower is better.

kernel              run 1     run 2
                    seconds   seconds
==================  =======   =======
2.5.44-mm2          515.1       515.4
2.5.44-mm2+rcu-ipc   46.7        46.7

With Mingming's patch, the test completes 10X faster.

From Bill Irwin

Optionally back priviled processes' shm with hugetlbfs.

One of the more common requests for and/or users of hugetlb interfaces
in general are databases using shm.  This patch exports functionality
mostly equivalent to tmpfs, adds the calling sequence to ipc/shm.c, and
hashes out a small support function in fs/hugetlbfs/inode.c so that shm
segments may be hugetlbpage-backed if userspace passes a flag to
shmget().

Access to this resource requires CAP_IPC_LOCK.

The patch below adds the base set of LSM hooks for System V IPC to the
2.5.41 kernel.  These hooks permit a security module to label
semaphore sets, message queues, and shared memory segments and to
perform security checks on these objects that parallel the existing
IPC access checks.  Additional LSM hooks for labeling and controlling
individual messages sent on a single message queue and for providing
fine-grained distinctions among IPC operations will be submitted
separately after this base set of LSM IPC hooks has been accepted.

The old form of designated initializers are obsolete: we need to
replace them with the ISO C forms before 2.6.  Gcc has always supported
both forms anyway.

An acct flag was added to do_munmap, true everywhere but in mremap's
move_vma: instead of updating the arch and driver sources, revert that
that change and temporarily mask VM_ACCOUNT around that one do_munmap.

Also, noticed that do_mremap fails needlessly if both shrinking _and_
moving a mapping: update old_len to pass vm area boundaries test.

If we support mmap MAP_NORESERVE, we should support it on shared
anonymous objects: too bad that needs a few changes.  do_mmap_pgoff pass
VM_ACCOUNT (or not) down to shmem_file_setup, flag stored into shmem
info, for use by shmem_delete_inode later.  Also removed a harmless but
pointless call to shmem_truncate.

Alan's overcommit patch, brought to 2.5 by Robert Love.

Can't say I've tested its functionality at all, but it doesn't crash,
it has been in -ac and RH kernels for some time and I haven't observed
any of its functions on profiles.

"So what is strict VM overcommit?  We introduce new overcommit
 policies that attempt to never succeed an allocation that can not be
 fulfilled by the backing store and consequently never OOM.  This is
 achieved through strict accounting of the committed address space and
 a policy to allow/refuse allocations based on that accounting.

 In the strictest of modes, it should be impossible to allocate more
 memory than available and impossible to OOM.  All memory failures
 should be pushed down to the allocation routines -- malloc, mmap, etc.

 The new modes are available via sysctl (same as before).  See
 Documentation/vm/overcommit-accounting for more information."

Martin Schwidefsky <schwidefsky@de.ibm.com> reported "Bug with shared
memory" to LKML 14 May: hang due to schedule in truncate_list_pages
called from .... shm_destroy holding shm_lock spinlock.  shm_destroy
needs that lock for shm_rmid, but it can be safely unlocked once link
from id to shp has been removed.

Also move where we set sma->sem_perm.mode and .key to before ipc_addid() gets called.

We always returned success even when we had no ->vm_ops

Seperates shmem_sb_info from struct super_block.

- Jens Axboe: more bio updates, fix some request list bogosity under load
- Al Viro: export seq_xxx functions
- Manfred Spraul: include file cleanups, pc110pad compile fix
- David Woodhouse: fix JFFS2 write error handling
- Dave Jones: start merging up with 2.4.x patches
- Manfred Spraul: coredump fixes, FS event counter cleanups
- me: fix SCSI CD-ROM sectorsize BIO breakage

- Greg KH: USB updates
- Jens Axboe: more bio updates
- Christoph Rohland: fix up proper shmat semantics

  - Al Viro: mnt_list init
  - Jeff Garzik: network driver update (license tags, tulip driver)
  - David Miller: sparc, net updates
  - Ben Collins: firewire update
  - Gerd Knorr: btaudio/bttv update
  - Tim Hockin: MD cleanups
  - Greg KH, Petko Manolov: USB updates
  - Leonard Zubkoff: DAC960 driver update

  - Alan Cox: continued merging
  - Mingming Cao: make msgrcv/shmat check the queue/segment ID's properly
  - Greg KH: USB serial init failure fix, Xircom serial converter driver
  - Neil Brown: nsfd/raid/md/lockd cleanups
  - Ingo Molnar: multipath RAID personality, raid xor update
  - Hugh Dickins/Marcelo Tosatti: swapin read-ahead race fix
  - Vojtech Pavlik: fix up some of the infrastructure for x86-64
  - Robert Love: AMD 761 AGP GART support
  - Jens Axboe: fix SCSI-generic queue handling race
  - me: be sane about page reference bits

  - Russell King: ARM updates
  - Al Viro: more init cleanups
  - Cort Dougan: more PPC updates
  - David Miller: cleanups, pci mmap updates
  - Neil Brown: raid resync by sector
  - Alan Cox: more merging with -ac
  - Johannes Erdfelt: USB updates
  - Kai Germaschewski: ISDN updates
  - Tobias Ringstrom: dmfe.c network driver update
  - Trond Myklebust: NFS client updates and cleanups

  - Rik van Riel and others: mm rw-semaphore (ps/top ok when swapping)
  - IDE: 256 sectors at a time is legal, but apparently confuses some
  drives. Max out at 255 sectors instead.
  - Petko Manolov: USB pegasus driver update
  - make the boottime memory map printout at least almost readable.
  - USB driver updates
  - pte_alloc()/pmd_alloc() need page_table_lock.

  - Jens: better ordering of requests when unable to merge
  - Neil Brown: make md work as a module again (we cannot autodetect
  in modules, not enough background information)
  - Neil Brown: raid5 SMP locking cleanups
  - Neil Brown: nfsd: handle Irix NFS clients named pipe behavior and
  dentry leak fix
  - maestro3 shutdown fix
  - fix dcache hash calculation that could cause bad hashes under certain
  circumstances (Dean Gaudet)
  - David Miller: networking and sparc updates
  - Jeff Garzik: include file cleanups
  - Andy Grover: ACPI update
  - Coda-fs error return fixes
  - rth: alpha Jensen update

  - ReiserFS merge
  - fix DRM R128/AGP dependency