To make the memory ioctl code more readable, this patch moves the
legacy/debug code path of mmu-disabled to a separate function, which is
called (if necessary) from the main memory ioctl function.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch removes the enum value of ASIC_AUTO_DETECT because we can use
the validity of the pdev variable to know whether we have a real device or
a simulator. For a real device, we detect the asic type from the device ID
while for a simulator, the simulator code calls create_hdev() with the
specified ASIC type.

Set ASIC_INVALID as the first option in the enum to make sure that no
other enum value will receive the value 0 (which indicates a non-existing
entry in the simulator array).
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch does some refactoring in goya.c to make code more reusable
between goya code and the goya simulator code (which is not upstreamed).

In addition, the patch removes some dead functions from goya.c which are
not used by the current upstream code
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch adds a better explanation about the sequence number that is
returned per CS. It also fixes the comment about queue numbering rules.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch adds the ASIC-specific function for GOYA to configure the
coresight components.

Most of the components have an enabled/disabled flag, depending on whether
the user wants to enable the component or disable it.

For some of the components, such as ETR and SPMU, the user can also
request to read values from them. Those values are needed for the user to
parse the trace data.

The ETR configuration is also checked for security purposes, to make sure
the trace data is written to the device's DRAM.
Signed-off-by: Omer Shpigelman <oshpigelman@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

Habanalabs ASICs use the ARM coresight infrastructure to support debug,
tracing and profiling of neural networks topologies.

Because the coresight is configured using register writes and reads, and
some of the registers hold sensitive information (e.g. the address in
the device's DRAM where the trace data is written to), the user must go
through the kernel driver to configure this mechanism.

This patch implements the common code of the IOCTL and calls the
ASIC-specific function for the actual H/W configuration.

The IOCTL supports configuration of seven coresight components:
ETR, ETF, STM, FUNNEL, BMON, SPMU and TIMESTAMP

The user specifies which component he wishes to configure and provides a
pointer to a structure (located in its process space) that contains the
relevant configuration.

The common code copies the relevant data from the user-space to kernel
space and then calls the ASIC-specific function to do the H/W
configuration.

After the configuration is done, which is usually composed
of several IOCTL calls depending on what the user wanted to trace, the
user can start executing the topology. The trace data will be written to
the user's area in the device's DRAM.

After the tracing operation is complete, and user will call the IOCTL
again to disable the tracing operation. The user also need to read
values from registers for some of the components (e.g. the size of the
trace data in the device's DRAM). In that case, the user will provide a
pointer to an "output" structure in user-space, which the IOCTL code will
fill according the to selected component.
Signed-off-by: Omer Shpigelman <oshpigelman@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch removes an extra ; after the closing brackets of a while loop.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: Mukesh Ojha <mojha@codeaurora.org>

Unmapping ptes in the device MMU on Palladium can take a long time, which
can cause a kernel BUG of CPU soft lockup.

This patch minimize the chances for this bug by sleeping a little between
unmapping ptes.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

GIT does not like extra blank lines at the end of the file, so this patch
removes those lines.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: Mukesh Ojha <mojha@codeaurora.org>

This patch improves two error messages to help the user to
better understand what error occurred.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch adds a new opcode to INFO IOCTL that returns the device status.

This will allow users to query the device status in order to avoid sending
command submissions while device is in reset.
Signed-off-by: Dalit Ben Zoor <dbenzoor@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch allows the user to modify the TPC PLL clock relaxation value
on-the-fly in order to reduce power consumption.

To enable this, the patch removes the protection from the specific
register that controls this behavior.
Signed-off-by: Dalit Ben Zoor <dbenzoor@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

On init or context switch, set TPC clock relaxation counter
register to a golden value.
Signed-off-by: Dalit Ben Zoor <dbenzoor@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

Hard-reset of our device should never fail, due to dangers of permanent
damage to the H/W.

This patch removes the last place in the reset path where the driver might
exit before doing the actual reset.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch refactors the code that is responsible to set the DMA mask for
the device.

Upon each change of the dma mask, the driver will save the new value that
was set. This is needed in order to make sure we don't try to increase the
mask a second time, in case we failed in the first time. This is
especially relevant for Power machines, as that may cause a change in
configuration of the TVT which will break the device.

Goya will first try to set the device's dma mask to 39 bits, so that the
memory that is allocated on the host machine for communication with the
device's cpu will be in a bus address which is lower then 39 bits. Later,
Goya will try to increase that mask to 48 bits, but only if setting the
mask to 39 bits was successful.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch adds shadow mapping to the MMU module. The shadow mapping
allows traversing the page table in host memory rather reading each PTE
from the device memory.
It brings better performance and avoids reading from invalid device
address upon PCI errors.
Only at the end of map/unmap flow, writings to the device are performed in
order to sync the H/W page tables with the shadow ones.
Signed-off-by: Omer Shpigelman <oshpigelman@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

The addr/data32 debugfs nodes currently permit the access to only physical
addresses of a device. This patch extends it and allows accessing also
device's DRAM virtual addresses.
Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

Print the name of a busy engine when checking if a device is idle.
The change is done mainly to help a user to pinpoint problems in his
topology's recipe.
Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch adds two comments in uapi/habanalabs.h:
- From which queue id the internal queues begin
- Invalid values that can be returned in the seq field from the CS IOCTL
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

Remove pointers to ASIC-specific functions and instead call the functions
explicitly as they are not accessed from outside the ASIC-specific files.
Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

Move duplicated PCI-related code from ASIC-specific files into the common
pci.c file.
Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

At the start of some IOCTLs we check if the device is disabled or in reset.
If it is, we return -EBUSY and print a message to kernel log.

Because these IOCTLs can be called at very high frequency, use ratelimit
to avoid spamming the kernel log. Also use the same type of message -
dev_warn - in all the relevant IOCTLs.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch removes some old defines which are not in use anymore.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

The Event Queue MSI/X ID is different per ASIC. This patch renames the
current define to have the GOYA_ prefix to mark it only for Goya. It also
moves it from the common armcp_if.h file to the ASIC specific goya_fw_if.h
file.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch moves the code that is responsible of the communication
vs. the F/W to a dedicated file. This will allow us to share the code
between different ASICs.
Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This will prevent unneeded include of header files, which may increase
compilation time.
Signed-off-by: Dotan Barak <dbarak@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

The goya_non_fatal_events array actually contains all the possible events
the driver can receive from the F/W. Therefore, use a proper name
for the array.

The patch also adds missing event Ids to the goya_async_event_id enum.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

This patch adds a definition of a new status in the device CPU boot stages
and add the handling of the new status.
Signed-off-by: Igor Grinberg <igrinberg@habana.ai>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

The function comment of __register_chrdev_region()
is out of date, so update it based on the code.
Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

It's just code cleanup, not functional change.
Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

register_chrdev_region() carefully checks minor range
before calling __register_chrdev_region() but there is
another path from alloc_chrdev_region() which does not
check the range properly. So add a check for given minor
range in __register_chrdev_region().
Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Current overlap checking cannot correctly handle
a case which is baseminor < existing baseminor &&
baseminor + minorct > existing baseminor + minorct.
Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

No functional changes, cleanup only.
Reviewed-by: Adit Ranadive <aditr@vmware.com>
Reviewed-by: Jorgen Hansen <jhansen@vmware.com>
Signed-off-by: Vishnu Dasa <vdasa@vmware.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

We want the char-misc fixes in here as well.
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Pull KVM fixes from Paolo Bonzini:
 "A collection of x86 and ARM bugfixes, and some improvements to
  documentation.

  On top of this, a cleanup of kvm_para.h headers, which were exported
  by some architectures even though they not support KVM at all. This is
  responsible for all the Kbuild changes in the diffstat"

* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (28 commits)
  Documentation: kvm: clarify KVM_SET_USER_MEMORY_REGION
  KVM: doc: Document the life cycle of a VM and its resources
  KVM: selftests: complete IO before migrating guest state
  KVM: selftests: disable stack protector for all KVM tests
  KVM: selftests: explicitly disable PIE for tests
  KVM: selftests: assert on exit reason in CR4/cpuid sync test
  KVM: x86: update %rip after emulating IO
  x86/kvm/hyper-v: avoid spurious pending stimer on vCPU init
  kvm/x86: Move MSR_IA32_ARCH_CAPABILITIES to array emulated_msrs
  KVM: x86: Emulate MSR_IA32_ARCH_CAPABILITIES on AMD hosts
  kvm: don't redefine flags as something else
  kvm: mmu: Used range based flushing in slot_handle_level_range
  KVM: export <linux/kvm_para.h> and <asm/kvm_para.h> iif KVM is supported
  KVM: x86: remove check on nr_mmu_pages in kvm_arch_commit_memory_region()
  kvm: nVMX: Add a vmentry check for HOST_SYSENTER_ESP and HOST_SYSENTER_EIP fields
  KVM: SVM: Workaround errata#1096 (insn_len maybe zero on SMAP violation)
  KVM: Reject device ioctls from processes other than the VM's creator
  KVM: doc: Fix incorrect word ordering regarding supported use of APIs
  KVM: x86: fix handling of role.cr4_pae and rename it to 'gpte_size'
  KVM: nVMX: Do not inherit quadrant and invalid for the root shadow EPT
  ...

Pull x86 fixes from Thomas Gleixner:
 "A pile of x86 updates:

   - Prevent exceeding he valid physical address space in the /dev/mem
     limit checks.

   - Move all header content inside the header guard to prevent compile
     failures.

   - Fix the bogus __percpu annotation in this_cpu_has() which makes
     sparse very noisy.

   - Disable switch jump tables completely when retpolines are enabled.

   - Prevent leaking the trampoline address"

* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/realmode: Make set_real_mode_mem() static inline
  x86/cpufeature: Fix __percpu annotation in this_cpu_has()
  x86/mm: Don't exceed the valid physical address space
  x86/retpolines: Disable switch jump tables when retpolines are enabled
  x86/realmode: Don't leak the trampoline kernel address
  x86/boot: Fix incorrect ifdeffery scope
  x86/resctrl: Remove unused variable

Pull perf tooling fixes from Thomas Gleixner:
 "Core libraries:
   - Fix max perf_event_attr.precise_ip detection.
   - Fix parser error for uncore event alias
   - Fixup ordering of kernel maps after obtaining the main kernel map
     address.

  Intel PT:
   - Fix TSC slip where A TSC packet can slip past MTC packets so that
     the timestamp appears to go backwards.
   - Fixes for exported-sql-viewer GUI conversion to python3.

  ARM coresight:
   - Fix the build by adding a missing case value for enumeration value
     introduced in newer library, that now is the required one.

  tool headers:
   - Syncronize kernel headers with the kernel, getting new io_uring and
     pidfd_send_signal syscalls so that 'perf trace' can handle them"

* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  perf pmu: Fix parser error for uncore event alias
  perf scripts python: exported-sql-viewer.py: Fix python3 support
  perf scripts python: exported-sql-viewer.py: Fix never-ending loop
  perf machine: Update kernel map address and re-order properly
  tools headers uapi: Sync powerpc's asm/kvm.h copy with the kernel sources
  tools headers: Update x86's syscall_64.tbl and uapi/asm-generic/unistd
  tools headers uapi: Update drm/i915_drm.h
  tools arch x86: Sync asm/cpufeatures.h with the kernel sources
  tools headers uapi: Sync linux/fcntl.h to get the F_SEAL_FUTURE_WRITE addition
  tools headers uapi: Sync asm-generic/mman-common.h and linux/mman.h
  perf evsel: Fix max perf_event_attr.precise_ip detection
  perf intel-pt: Fix TSC slip
  perf cs-etm: Add missing case value

Pull CPU hotplug fixes from Thomas Gleixner:
 "Two SMT/hotplug related fixes:

   - Prevent crash when HOTPLUG_CPU is disabled and the CPU bringup
     aborts. This is triggered with the 'nosmt' command line option, but
     can happen by any abort condition. As the real unplug code is not
     compiled in, prevent the fail by keeping the CPU in zombie state.

   - Enforce HOTPLUG_CPU for SMP on x86 to avoid the above situation
     completely. With 'nosmt' being a popular option it's required to
     unplug the half brought up sibling CPUs (due to the MCE wreckage)
     completely"

* 'smp-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/smp: Enforce CONFIG_HOTPLUG_CPU when SMP=y
  cpu/hotplug: Prevent crash when CPU bringup fails on CONFIG_HOTPLUG_CPU=n

Pull locking fixlet from Thomas Gleixner:
 "Trivial update to the maintainers file"

* 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  MAINTAINERS: Remove deleted file from futex file pattern