The nvme-scsi file defined its own Log Page constant.  Use the
newly-defined one from the header file instead.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Taken from the 1.1a version of the spec
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

There is a potential dead lock if a cpu event occurs during nvme probe
since it registered with hot cpu notification. This fixes the race by
having the module register with notification outside of probe rather
than have each device register.

The actual work is done in a scheduled work queue instead of in the
notifier since assigning IO queues has the potential to block if the
driver creates additional queues.
Signed-off-by: Keith Busch <keith.busch@intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

It's positively immoral to have a global variable called 'io_timeout'.
Keep the module parameter called io_timeout, though.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

After skipping right-padded spaces, use the last four bytes of the
firmware revision when reporting the Inquiry Product Revision. These
are generally more indicative to what is running.
Signed-off-by: Keith Busch <keith.busch@intel.com>
Acked-by: Vishal Verma <vishal.l.verma@linux.intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Recently, a new sysfs control "iostats" was added to selectively
enable or disable io statistics collection for request queues.  This
patch hooks that control.

IO statistics collection is rather expensive on large, multi-node
machines with drives pushing millions of iops.  Having the ability to
disable collection if not needed can improve throughput significantly.

As a data point, on a quad E5-4640, I see more than 50% throughput
improvement when io statistics accounting is disabled during heavily
multi-threaded small block random read benchmarks where device
performance is in the million iops+ range.
Signed-off-by: Sam Bradshaw <sbradshaw@micron.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

The routines to get and lock nvme queues required the caller to "put"
or "unlock" them even if getting one returned NULL. This patch fixes that.
Signed-off-by: Keith Busch <keith.busch@intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

This define isn't used, and any code that wanted to use it should use
NVME_FEAT_TEMP_THRESH instead.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Signed-off-by: Keith Busch <keith.busch@intel.com>
[made admin_timeout static]
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

This was originally set to 4 times the IO timeout, but that was when
the IO timeout was 5 seconds instead of 30. 20 seconds for total time
to failure seemed more reasonable than 2 minutes for most, but other
users have requested to make this a module parameter instead.
Signed-off-by: Keith Busch <keith.busch@intel.com>
[renamed the module parameter to retry_time]
[made retry_time static]
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

kmalloc() used by the nvme_alloc_iod() to allocate memory for 'iod'
can fail. So check the return value.
Signed-off-by: Santosh Y <santosh.sy@samsung.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

In GetLogPage the buffer size passed to device is a 0's based value.
Signed-off-by: Indraneel M <indraneel.m@samsung.com>
Reported-by: Shiro Itou <shiro.itou@outlook.com>
Reviewed-by: Vishal Verma <vishal.l.verma@linux.intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Include changes from the current set of ratified Technical Proposals
for NVMe 1.2.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Since _nvme_check_size() wasn't being called from anywhere, the compiler
was optimising it away ... along with all the link-time build failures
that would result if any of the structures were the wrong size.  Call it
from nvme_exit() for no particular reason.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Controller: add CNTLID, AVSCC, APSTA, NVSCC, ACWU, SGLS fields.

Namespace: add NMIC, RESCAP, EUI64 fields. EUI64 is specifically
interesting, since it can be used to construct an UEFI NVMe device
path for a boot entry.

As per NVM Express 1.1a spec:
http://www.nvmexpress.org/wp-content/uploads/NVM-Express-1_1a.pdfSigned-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

It is possible a filesystem may send a flush flagged bio with write
data. There is no such composite NVMe command, so the driver sends flush
and write separately.

The device is allowed to execute these commands in any order, so it was
possible the driver ends the bio after the write completes, but while the
flush is still active. We don't want to let a filesystem believe flush
succeeded before it really has; this could cause data corruption on a
power loss between these events. To fix, this patch splits the flush
and write into chained bios.
Signed-off-by: Keith Busch <keith.busch@intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

This configures an nvme request_queue as flush capable if the device
has a volatile write cache present.
Signed-off-by: Keith Busch <keith.busch@intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Adding tracepoints for bio_complete and block_split into nvme to help
with gathering IO info using blktrace and blkparse.
Signed-off-by: Keith Busch <keith.busch@intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

If a misbehaving device posts a CQE with a command id < depth but for
one that was never allocated, the command info will have a callback
function set to NULL and we don't want to try invoking that.
Signed-off-by: Keith Busch <keith.busch@intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Help people diagnose what is going wrong at initialisation time by
printing out which command has gone wrong and what the device returned.
Also fix the error message printed while waiting for reset.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
Reviewed-by: Keith Busch <keith.busch@intel.com>

Make the copyright dates accurate and remove the final paragraph that
includes the address of the FSF.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>

Pull file locking change from Jeff Layton:
 "Only an email address change to the MAINTAINERS file"

* tag 'locks-v3.15-3' of git://git.samba.org/jlayton/linux:
  MAINTAINERS: email address change for Jeff Layton

Pull arm64 fixes from Catalin Marinas:
 "These are mostly arm64 fixes with an additional arm(64) platform fix
  for the initialisation of vexpress clocks (the latter only affecting
  arm64; the arch/arm64 code is SoC agnostic and does not rely on early
  SoC-specific calls)

   - vexpress platform clocks initialisation moved earlier following the
     arm64 move of of_clk_init() call in a previous commit
   - Default DMA ops changed to non-coherent to preserve compatibility
     with 32-bit ARM DT files.  The "dma-coherent" property can be used
     to explicitly mark a device coherent.  The Applied Micro DT file
     has been updated to avoid DMA cache maintenance for the X-Gene SATA
     controller (the only arm64 related driver with such assumption in
     -rc mainline)
   - Fixmap correction for earlyprintk
   - kern_addr_valid() fix for huge pages"

* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
  vexpress: Initialise the sysregs before setting up the clocks
  arm64: Mark the Applied Micro X-Gene SATA controller as DMA coherent
  arm64: Use bus notifiers to set per-device coherent DMA ops
  arm64: Make default dma_ops to be noncoherent
  arm64: fixmap: fix missing sub-page offset for earlyprintk
  arm64: Fix for the arm64 kern_addr_valid() function

Pull SCSI fixes from James Bottomley:
 "This is two patches both fixing bugs in drivers (virtio-scsi and
  mpt2sas) causing an oops in certain circumstances"

* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
  [SCSI] virtio-scsi: Skip setting affinity on uninitialized vq
  [SCSI] mpt2sas: Don't disable device twice at suspend.

Following arm64 commit bc3ee18a (arm64: init: Move of_clk_init to
time_init()), vexpress_osc_of_setup() is called via of_clk_init() long
before initcalls are issued. Initialising the vexpress oscillators
requires the vespress sysregs to be already initialised, so this patch
adds an explicit call to vexpress_sysreg_of_early_init() in vexpress
oscillator setup function.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Will Deacon <will.deacon@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Tested-by: Pawel Moll <pawel.moll@arm.com>
Acked-by: Pawel Moll <pawel.moll@arm.com>
Cc: Mike Turquette <mturquette@linaro.org>

Since the default DMA ops for arm64 are non-coherent, mark the X-Gene
controller explicitly as dma-coherent to avoid additional cache
maintenance.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Loc Ho <lho@apm.com>

Recently, the default DMA ops have been changed to non-coherent for
alignment with 32-bit ARM platforms (and DT files). This patch adds bus
notifiers to be able to set the coherent DMA ops (with no cache
maintenance) for devices explicitly marked as coherent via the
"dma-coherent" DT property.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

Currently arm64 dma_ops is by default made coherent which makes it
opposite in default policy from arm.

Make default dma_ops to be noncoherent (same as arm), as currently there
aren't any dma-capable drivers which assumes coherent ops
Signed-off-by: Ritesh Harjani <ritesh.harjani@gmail.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

Commit d57c33c5 (add generic fixmap.h) added (among other
similar things) set_fixmap_io to deal with early ioremap of devices.

More recently, commit bf4b558e (arm64: add early_ioremap support)
converted the arm64 earlyprintk to use set_fixmap_io. A side effect of
this conversion is that my virtual machines have stopped booting when
I pass "earlyprintk=uart8250-8bit,0x3f8" to the guest kernel.

Turns out that the new earlyprintk code doesn't care at all about
sub-page offsets, and just assumes that the earlyprintk device will
be page-aligned. Obviously, that doesn't play well with the above example.

Further investigation shows that set_fixmap_io uses __set_fixmap instead
of __set_fixmap_offset. A fix is to introduce a set_fixmap_offset_io that
uses the latter, and to remove the superflous call to fix_to_virt
(which only returns the value that set_fixmap_io has already given us).

With this applied, my VMs are back in business. Tested on a Cortex-A57
platform with kvmtool as platform emulation.

Cc: Will Deacon <will.deacon@arm.com>
Acked-by: Mark Salter <msalter@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

Fix for the arm64 kern_addr_valid() function to recognize
virtual addresses in the kernel logical memory map.  The
function fails as written because it does not check whether
the addresses in that region are mapped at the pmd level to
2MB or 512MB pages, continues the page table walk to the
pte level, and issues a garbage value to pfn_valid().

Tested on 4K-page and 64K-page kernels.
Signed-off-by: Dave Anderson <anderson@redhat.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

Pull irq fixes from Thomas Gleixner:
 "This udpate delivers:

   - A fix for dynamic interrupt allocation on x86 which is required to
     exclude the GSI interrupts from the dynamic allocatable range.

     This was detected with the newfangled tablet SoCs which have GPIOs
     and therefor allocate a range of interrupts.  The MSI allocations
     already excluded the GSI range, so we never noticed before.

   - The last missing set_irq_affinity() repair, which was delayed due
     to testing issues

   - A few bug fixes for the armada SoC interrupt controller

   - A memory allocation fix for the TI crossbar interrupt controller

   - A trivial kernel-doc warning fix"

* 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  irqchip: irq-crossbar: Not allocating enough memory
  irqchip: armanda: Sanitize set_irq_affinity()
  genirq: x86: Ensure that dynamic irq allocation does not conflict
  linux/interrupt.h: fix new kernel-doc warnings
  irqchip: armada-370-xp: Fix releasing of MSIs
  irqchip: armada-370-xp: implement the ->check_device() msi_chip operation
  irqchip: armada-370-xp: fix invalid cast of signed value into unsigned variable

Pull timer fixes from Thomas Gleixner:
 "This update brings along:

   - Two fixes for long standing bugs in the hrtimer code, one which
     prevents remote enqueuing and the other preventing arbitrary delays
     after a interrupt hang was detected

   - A fix in the timer wheel which prevents math overflow

   - A fix for a long standing issue with the architected ARM timer
     related to the C3STOP mechanism.

   - A trivial compile fix for nspire SoC clocksource"

* 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  timer: Prevent overflow in apply_slack
  hrtimer: Prevent remote enqueue of leftmost timers
  hrtimer: Prevent all reprogramming if hang detected
  clocksource: nspire: Fix compiler warning
  clocksource: arch_arm_timer: Fix age-old arch timer C3STOP detection issue

Merge tag 'trace-fixes-v3.15-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing fix from Steven Rostedt:
 "This is a small fix where the trigger code used the wrong
  rcu_dereference().  It required rcu_dereference_sched() instead of the
  normal rcu_dereference().  It produces a nasty RCU lockdep splat due
  to the incorrect rcu notation"
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

* tag 'trace-fixes-v3.15-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace:
  tracing: Use rcu_dereference_sched() for trace event triggers

As trace event triggers are now part of the mainline kernel, I added
my trace event trigger tests to my test suite I run on all my kernels.
Now these tests get run under different config options, and one of
those options is CONFIG_PROVE_RCU, which checks under lockdep that
the rcu locking primitives are being used correctly. This triggered
the following splat:

===============================
[ INFO: suspicious RCU usage. ]
3.15.0-rc2-test+ #11 Not tainted
-------------------------------
kernel/trace/trace_events_trigger.c:80 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
4 locks held by swapper/1/0:
 #0:  ((&(&j_cdbs->work)->timer)){..-...}, at: [<ffffffff8104d2cc>] call_timer_fn+0x5/0x1be
 #1:  (&(&pool->lock)->rlock){-.-...}, at: [<ffffffff81059856>] __queue_work+0x140/0x283
 #2:  (&p->pi_lock){-.-.-.}, at: [<ffffffff8106e961>] try_to_wake_up+0x2e/0x1e8
 #3:  (&rq->lock){-.-.-.}, at: [<ffffffff8106ead3>] try_to_wake_up+0x1a0/0x1e8

stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.15.0-rc2-test+ #11
Hardware name:                  /DG965MQ, BIOS MQ96510J.86A.0372.2006.0605.1717 06/05/2006
 0000000000000001 ffff88007e083b98 ffffffff819f53a5 0000000000000006
 ffff88007b0942c0 ffff88007e083bc8 ffffffff81081307 ffff88007ad96d20
 0000000000000000 ffff88007af2d840 ffff88007b2e701c ffff88007e083c18
Call Trace:
 <IRQ>  [<ffffffff819f53a5>] dump_stack+0x4f/0x7c
 [<ffffffff81081307>] lockdep_rcu_suspicious+0x107/0x110
 [<ffffffff810ee51c>] event_triggers_call+0x99/0x108
 [<ffffffff810e8174>] ftrace_event_buffer_commit+0x42/0xa4
 [<ffffffff8106aadc>] ftrace_raw_event_sched_wakeup_template+0x71/0x7c
 [<ffffffff8106bcbf>] ttwu_do_wakeup+0x7f/0xff
 [<ffffffff8106bd9b>] ttwu_do_activate.constprop.126+0x5c/0x61
 [<ffffffff8106eadf>] try_to_wake_up+0x1ac/0x1e8
 [<ffffffff8106eb77>] wake_up_process+0x36/0x3b
 [<ffffffff810575cc>] wake_up_worker+0x24/0x26
 [<ffffffff810578bc>] insert_work+0x5c/0x65
 [<ffffffff81059982>] __queue_work+0x26c/0x283
 [<ffffffff81059999>] ? __queue_work+0x283/0x283
 [<ffffffff810599b7>] delayed_work_timer_fn+0x1e/0x20
 [<ffffffff8104d3a6>] call_timer_fn+0xdf/0x1be^M
 [<ffffffff8104d2cc>] ? call_timer_fn+0x5/0x1be
 [<ffffffff81059999>] ? __queue_work+0x283/0x283
 [<ffffffff8104d823>] run_timer_softirq+0x1a4/0x22f^M
 [<ffffffff8104696d>] __do_softirq+0x17b/0x31b^M
 [<ffffffff81046d03>] irq_exit+0x42/0x97
 [<ffffffff81a08db6>] smp_apic_timer_interrupt+0x37/0x44
 [<ffffffff81a07a2f>] apic_timer_interrupt+0x6f/0x80
 <EOI>  [<ffffffff8100a5d8>] ? default_idle+0x21/0x32
 [<ffffffff8100a5d6>] ? default_idle+0x1f/0x32
 [<ffffffff8100ac10>] arch_cpu_idle+0xf/0x11
 [<ffffffff8107b3a4>] cpu_startup_entry+0x1a3/0x213
 [<ffffffff8102a23c>] start_secondary+0x212/0x219

The cause is that the triggers are protected by rcu_read_lock_sched() but
the data is dereferenced with rcu_dereference() which expects it to
be protected with rcu_read_lock(). The proper reference should be
rcu_dereference_sched().

Cc: Tom Zanussi <tom.zanussi@linux.intel.com>
Cc: stable@vger.kernel.org # 3.14+
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>

Pull ACPI and power management fixes from Rafael Wysocki:
 "A bunch of regression fixes this time.  They fix two regressions in
  the PNP subsystem, one in the ACPI processor driver and one in the
  ACPI EC driver, four cpufreq driver regressions and an unrelated bug
  in one of the drivers.  The regressions are recent or introduced in
  3.14.

  Specifics:

   - There are two bugs in the ACPI PNP core that cause errors to be
     returned if optional ACPI methods are not present.  After an ACPI
     core change made in 3.14 one of those errors leads to serial port
     suspend failures on some systems.  Fix from Rafael J Wysocki.

   - A recently added PNP quirk related to Intel chipsets intorduced a
     build error in unusual configurations (PNP without PCI).  Fix from
     Bjorn Helgaas.

   - An ACPI EC workaround related to system suspend on Samsung machines
     added in 3.14 introduced a race causing some valid EC events to be
     discarded.  Fix from Kieran Clancy.

   - The acpi-cpufreq driver fails to load on some systems after a 3.14
     commit related to APIC ID parsing that overlooked one corner case.
     Fix from Lan Tianyu.

   - Fix for a recently introduced build problem in the ppc-corenet
     cpufreq driver from Tim Gardner.

   - A recent cpufreq core change to ensure serialization of frequency
     transitions for drivers with a ->target_index() callback overlooked
     the fact that some of those drivers had been doing operations
     introduced by it into the core already by themselves.  That
     resulted in a mess in which the core and the drivers try to do the
     same thing and block each other which leads to deadlocks.  Fixes
     for the powernow-k7, powernow-k6, and longhaul cpufreq drivers from
     Srivatsa S Bhat.

   - Fix for a computational error in the powernow-k6 cpufreq driver
     from Srivatsa S Bhat"

* tag 'pm+acpi-3.15-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
  ACPI / processor: Fix failure of loading acpi-cpufreq driver
  PNP / ACPI: Do not return errors if _DIS or _SRS are not present
  PNP: Fix compile error in quirks.c
  ACPI / EC: Process rather than discard events in acpi_ec_clear
  cpufreq: ppc-corenet-cpufreq: Fix __udivdi3 modpost error
  cpufreq: powernow-k7: Fix double invocation of cpufreq_freq_transition_begin/end
  cpufreq: powernow-k6: Fix double invocation of cpufreq_freq_transition_begin/end
  cpufreq: powernow-k6: Fix incorrect comparison with max_multipler
  cpufreq: longhaul: Fix double invocation of cpufreq_freq_transition_begin/end

Pull driver core deferred probe fix from Grant Likely:
 "Drivercore race condition fix (exposed by devicetree)

  This branch fixes a bug where a device can get stuck in the deferred
  list even though all its dependencies are met.  The bug has existed
  for a long time, but new platform conversions to device tree have
  exposed it.  This patch is needed to get those platforms working.

  This was the pending bug fix I mentioned in my previous pull request.
  Normally this would go through Greg's tree seeing that it is a
  drivercore change, but devicetree exposes the problem.  I've discussed
  with Greg and he okayed me asking you to pull directly"

* tag 'dt-for-linus' of git://git.secretlab.ca/git/linux:
  drivercore: deferral race condition fix

* acpi-ec:
  ACPI / EC: Process rather than discard events in acpi_ec_clear

* acpi-processor:
  ACPI / processor: Fix failure of loading acpi-cpufreq driver

* pnp:
  PNP / ACPI: Do not return errors if _DIS or _SRS are not present
  PNP: Fix compile error in quirks.c

* pm-cpufreq:
  cpufreq: ppc-corenet-cpufreq: Fix __udivdi3 modpost error
  cpufreq: powernow-k7: Fix double invocation of cpufreq_freq_transition_begin/end
  cpufreq: powernow-k6: Fix double invocation of cpufreq_freq_transition_begin/end
  cpufreq: powernow-k6: Fix incorrect comparison with max_multipler
  cpufreq: longhaul: Fix double invocation of cpufreq_freq_transition_begin/end