The bug that hit SUN4V TLB patching exists elsewhere.
Make sure we cure all such cases.
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

Writes by privileged code are not allowed.
Signed-off-by: David S. Miller <davem@davemloft.net>

Writes by privileged code are disallowed.  The hypervisor manages
the non-privileged bit.
Signed-off-by: David S. Miller <davem@davemloft.net>

We have to use bootmem during init_IRQ and page alloc
for sibling cpu calls.

Also, fix incorrect hypervisor call return value
checks in the hypervisor SMP cpu mondo send code.
Signed-off-by: David S. Miller <davem@davemloft.net>

Yes, you heard it right, they changed the PTE layout for
SUN4V.  Ho hum...

This is the simple and inefficient way to support this.
It'll get optimized, don't worry.
Signed-off-by: David S. Miller <davem@davemloft.net>

We do this right after we take over the trap table from OBP.
Signed-off-by: David S. Miller <davem@davemloft.net>

Code patching did not sign extend negative branch
offsets correctly.

Kernel TLB miss path needs patching and %g4 register
preservation in order to handle SUN4V correctly.
Signed-off-by: David S. Miller <davem@davemloft.net>

Second instruction offset is '4' not '3'.
Signed-off-by: David S. Miller <davem@davemloft.net>

prom_sun4v_name should be "sun4v" not "SUNW,sun4v"

Also, this is too early to make use of the
.sun4v_Xinsn_patch code patching, so just check
things manually.

This gets us at least to prom_init() on Niagara.
Signed-off-by: David S. Miller <davem@davemloft.net>

We need to restore the %asi register properly.
For the kernel this means get_fs(), for user this
means ASI_PNF.

Also, NGcopy_to_user.S was including U3memcpy.S instead
of NGmemcpy.S, oops :-)
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

Since it can do things like BREAK and HUP, we implement
this as a serial uart driver.

This still needs interrupt probing code, as I haven't figured
out how interrupts will work or be probed for on SUN4V yet.
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

There was also a bug in sun4v_itlb_miss, it loaded the
MMU Fault Status base into %g3 instead of %g2.

This pointed out a fast path for TSB miss processing,
since we have %g2 with the MMU Fault Status base, we
can use that to quickly load up the PGD phys address.
Signed-off-by: David S. Miller <davem@davemloft.net>

This is where the virtual address of the fault status
area belongs.

To set it up we don't make a hypervisor call, instead
we call OBP's SUNW,set-trap-table with the real address
of the fault status area as the second argument.  And
right before that call we write the virtual address into
ASI_SCRATCHPAD vaddr 0x0.
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

Function goes in %o5, args go in %o0 --> %o5.
Signed-off-by: David S. Miller <davem@davemloft.net>

For constructing hypervisor PCI TSB IDs.
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

Add assembler file for PCI hypervisor calls.
Setup basic skeleton of SUN4V PCI controller driver.

Add 32-bit devhandle to PBM struct, as this is needed for
hypervisor calls.
Signed-off-by: David S. Miller <davem@davemloft.net>

Abstract out IOMMU operations so that we can have a different
set of calls on sun4v, which needs to do things through
hypervisor calls.
Signed-off-by: David S. Miller <davem@davemloft.net>

If there is a "cpuid" property, use that.  Else suck
it out of the top bits of the "reg" property.
Signed-off-by: David S. Miller <davem@davemloft.net>

Delete unused macros, and use fixed sized types in
sparc32 header.
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

When we register a TSB with the hypervisor, so that it or hardware can
handle TLB misses and do the TSB walk for us, the hypervisor traps
down to these trap when it incurs a TSB miss.

Processing is simple, we load the missing virtual address and context,
and do a full page table walk.
Signed-off-by: David S. Miller <davem@davemloft.net>

For 'tlb_type'.
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

We look for "SUNW,sun4v" in the 'compatible' property
of the root OBP device tree node.

Protect every %ver register access, to make sure it is
not touched on sun4v, as %ver is hyperprivileged there.

Lock kernel TLB entries using hypervisor calls instead of
calls into OBP.
Signed-off-by: David S. Miller <davem@davemloft.net>

Technically the hypervisor call supports sending in a list
of all cpus to get the cross-call, but I only pass in one
cpu at a time for now.

The multi-cpu support is there, just ifdef'd out so it's easy to
enable or delete it later.
Signed-off-by: David S. Miller <davem@davemloft.net>

Sun4v has 4 interrupt queues: cpu, device, resumable errors,
and non-resumable errors.  A set of head/tail offset pointers
help maintain a work queue in physical memory.  The entries
are 64-bytes in size.

Each queue is allocated then registered with the hypervisor
as we bring cpus up.

The two error queues each get a kernel side buffer that we
use to quickly empty the main interrupt queue before we
call up to C code to log the event and possibly take evasive
action.
Signed-off-by: David S. Miller <davem@davemloft.net>

Needs to occur before we enable PSTATE_IE in %pstate.
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

Also, correct TRAP_PER_CPU_FAULT_INFO define, it should be
0x40 not 0x20.
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

sun4v uses ASI_MMU instead of ASI_DMMU
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: David S. Miller <davem@davemloft.net>

For the expansion of __GET_CPUID() on SMP.
Signed-off-by: David S. Miller <davem@davemloft.net>

Happily we have no D-cache aliasing issues on these
chips, so the implementation is very straightforward.

Add a stub in bootup which will be where the patching
calls will be made for niagara/sun4v/hypervisor.
Signed-off-by: David S. Miller <davem@davemloft.net>