Ingo Franzki reported that our defconfig and debug_config went out of
sync with respect to DM_INTEGRITY. Fix it.

Reported-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

If for any reason the interrupt enable for an ap queue fails the
state machine run for the queue returned wrong return codes to the
caller. So the caller assumed interrupt support for this queue in
enabled and thus did not re-establish the high resolution timer used
for polling. In the end this let to a hang for the user space process
waiting "forever" for the reply.

This patch reworks these return codes to return correct indications
for the caller to re-establish the timer when a queue runs without
interrupt support.

Please note that this is fixing a wrong behavior after a first
failure (enable interrupt support for the queue) failed. However,
looks like this occasionally happens on KVM systems.

Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Generate kvm hypercall functions with a macro instead of duplicating
the more or less identical code seven times. This also reduces number
of lines of code.
However the main purpose is to get rid of as many as possible open
coded error prone register asm constructs in s390 architecture code.

For the only user of kvm_hypercall identical code is created
before/after this patch (drivers/s390/virtio/virtio_ccw.c).

Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Link: https://lore.kernel.org/r/20210713145713.2815167-1-hca@linux.ibm.com


Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Add tracing of interactions between the SCLP base driver, firmware and
other drivers to support problem determination in case of SCLP-related
issues.

For that purpose this patch introduces two new s390dbf debug areas:

  - sclp: An abbreviated log of all common interactions
  - sclp_err: A full log of failed or abnormal interactions

Tracing of full SCCB contents can be enabled for the sclp area by
setting its debug level to maximum (6).

Overview of added trace events:

  * Firmware interaction:
    - SRV1: Service call about to be issued
    - SRV2: Service call was issued
    - INT:  Interrupt received

  * Driver interaction:
    - RQAD: Request was added
    - RQOK: Request success
    - RQAB: Request aborted
    - RQTM: Request timed out
    - REG:  Event listener registered
    - UREG: Event listener unregistered
    - EVNT: Event callback
    - STCG: State-change callback

  * Abnormal events:
    - TMO:  A timeout occurred
    - UNEX: Unexpected SCCB completion

  * Other (not traced at default level):
    - SYN1: Synchronous wait start
    - SYN2: Synchronous wait end

Since the SCLP interface is used by console drivers this patch also
moves s390dbf printks outside the critical section protected by debug
area locks to prevent a potential deadlock that would otherwise be
introduced between console_owner --> sclp_lock --> sclp_debug.lock.

Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Debug areas can currently only be used after s390dbf initialization
which occurs as a postcore_initcall. This is too late for tracing
earlier code such as that related to console_init().

This patch introduces a macro for defining a statically initialized
debug area that can be used to trace very early code. The macro is made
available for built-in code only because modules are never running
during early boot.

Example usage:

1. Define static debug area:

  DEFINE_STATIC_DEBUG_INFO(my_debug, "my_debug", 4, 1, 16,
			   &debug_hex_ascii_view);

2. Add trace entry:

  debug_event(&my_debug, 0, "DATA", 4);

Note: The debug area is automatically registered in debugfs during boot.
      A driver must not call any of the debug_register()/_unregister()
      functions on a static debug_info_t!

Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Currently allocation and registration of s390dbf debug areas are tied
together. As a result, a debug area cannot be unregistered and
re-registered while any process has an associated debugfs file open.

Fix this by splitting alloc/release from register/unregister.

Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Any previously recorded s390dbf debug data is reset when a debug area
is resized using the 'pages' sysfs attribute. This can make
live-debugging unnecessarily complex.

Fix this by copying existing debug data to the newly allocated debug
area when resizing.

Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Avoid that the "restart_part2" label, which is in the middle of a
function, appears in /proc/kallsyms.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

In case of splitting to 4k mapping the early exit in walk_pte_level()
must only be taken iff flags is equal to SET_MEMORY_4K.
Currently the early exit is taken if the flag is set, and also others
might be set. This may lead to the situation that a mapping is split
but other changes are not done, like e.g. setting pages to R/W.

There is currently no such caller, but there might be in the future.

Fixes: b3e1a00c

 ("s390/mm: implement set_memory_4k()")
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Rename amod31 to amode31 like it was supposed to be.

Fixes: c78d0c74

 ("s390: rename dma section to amode31")
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Both are no longer used since the conversion to generic
entry, therefore remove them.

Fixes: 56e62a73

 ("s390: convert to generic entry")
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The 0day bot reported some kernel-doc warnings in this file so clean up
all of the kernel-doc and use proper kernel-doc formatting.
There are no more kernel-doc errors or warnings reported in this file.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Reported-by: kernel test robot <lkp@intel.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Tony Krowiak <akrowiak@linux.ibm.com>
Cc: Halil Pasic <pasic@linux.ibm.com>
Cc: Jason Herne <jjherne@linux.ibm.com>
Cc: Harald Freudenberger <freude@linux.ibm.com>
Cc: linux-s390@vger.kernel.org
Reviewed-by: Tony Krowiak <akrowiak@linux.ibm.com>
Link: https://lore.kernel.org/r/20210806050149.9614-1-rdunlap@infradead.org


Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Currently zpci_dma_init_device()/zpci_dma_exit_device() is called as
part of zpci_enable_device()/zpci_disable_device() and errors for
zpci_dma_exit_device() are always ignored even if we could abort.

Improve upon this by moving zpci_dma_exit_device() out of
zpci_disable_device() and check for errors whenever we have a way to
abort the current operation. Note that for example in
zpci_event_hard_deconfigured() the device is expected to be gone so we
really can't abort and proceed even in case of error.

Similarly move the cc == 3 special case out of zpci_unregister_ioat()
and into the callers allowing to abort when finding an already disabled
devices precludes proceeding with the operation.

While we are at it log IOAT register/unregister errors in the s390
debugfs log,

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Currently clp_get_state() and clp_refresh_fh() awkwardly use the
clp_list_pci() callback mechanism to find the entry for a specific FID
and update its zdev, respectively return its state.

This is both needlessly complex and means we are always going through
the entire PCI function list even if the FID has already been found.
Instead lets introduce a clp_find_pci() function to find a specific
entry and share the CLP List PCI request handling code with
clp_list_pci().

With that in place we can also easily make the function handle a simple
out parameter instead of directly altering the zdev allowing easier
access to the updated function handle by the caller.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Instead of always treating CLP_RC_SETPCIFN_ALRDY as success and blindly
updating the function handle restrict this special handling to the
disable case by moving it into zpci_disable_device() and still treating
it as an error while also updating the function handle such that
a subsequent zpci_disable_device() succeeds or the caller can ignore the
error when aborting is not an option such as for zPCI event 0x304.
Also print this occurrence to the log such that an admin can tell why
a disable operation returned an error.

A mismatch between the state of the underlying device and our view of it
can naturally happen when the device suddenly enters the error state but
we haven't gotten the error notification yet, it must not happen on
enable though.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Currently clp_set_pci_fn() always returns 0 as long as the CLP request
itself succeeds even if the operation itself returns a response code
other than CLP_RC_OK or CLP_RC_SETPCIFN_ALRDY. This is highly misleading
because calling code assumes that a zero rc means that the operation was
successful.

Fix this by returning the response code or cc on failure with the
exception of the special handling for CLP_RC_SETPCIFN_ALRDY. Also let's
not assume that the returned function handle for CLP_RC_SETPCIFN_ALRDY
is 0, we don't need it anyway.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Move offsetting all of vmlinux_info fields to a separate
function for better readability.

Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

It is currently possible to initialize a large PMD page when
the address is not aligned on page boundary.

Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

As .remove() is only called after a successful .probe() call, we can
trust that the drvdata is valid.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The device struct provides a pointer for driver-private data. Use this
in the zcrypt drivers (as vfio_ap already does), and then remove the
custom pointer from the AP device structs.

As really_probe() will always clear the drvdata pointer on error, we
no longer have to do so ourselves.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The device struct itself already contains a pointer to its driver.
Use this consistently, instead of duplicating it.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

On failure to register a struct zpci_dev with a struct zpci_bus we left
a dangling pointer in zdev->zbus. As zpci_create_device() bails if
zpci_bus_device_register() fails this is of no consequence but still bad
practice.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

It's currently safe to call zpci_cleanup_bus_resources() even if the
resources were never created but it makes no sense so check
zdev->has_resources before we call zpci_cleanup_bus_resources() in
zpci_release_device().

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Acked-by: Pierre Morel <pmorel@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The memory block occupied by the SCLP early buffer that is allocated
by the decompressor and then handed over to the decompressed kernel,
must be reserved to prevent it from being reused for other purposes.
This is necessary because the SCLP early buffer is still in use
during kernel initialization.

Fixes: f1d3c532

 ("s390/boot: move sclp early buffer from fixed address in asm to C")
Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reported-by: Alexander Gordeev <agordeev@linux.ibm.com>
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

Tests showed a mismatch between what the CCA tool reports about
the APKA master key state and what's displayed by the zcrypt dd
in sysfs. After some investigation, we found out that the
documentation which was the source for the zcrypt dd implementation
lacks the listing of 3 fields. So this patch now moves the
evaluation of the APKA master key state to the correct offset.

Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

The last user of arch_set_page_states(), arch_set_page_nodat() and
arch_test_page_nodat() was removed in commit 39421627


("s390: remove broken hibernate / power management support"),
let's remove these functions.

Signed-off-by: David Hildenbrand <david@redhat.com>
Link: https://lore.kernel.org/r/20210806075430.6103-1-david@redhat.com


Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

Neither of the two drivers provides any SLIB parameter data, so get rid
of the dead code.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

Move all QIB-related code into qdio_setup_qib(), and slightly re-order
it according to the order of the struct's fields. This makes it easier
to understand what the QIB actually looks like before we send it to HW.

Also get rid of the qebsm_possible() helper - as 31-bit support is long
gone, the comment doesn't make any sense. And while removing some stale
QIB-related comment, also move the clearing of the QDR into its proper
place.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

Prefer dev_info() over a raw printk. This also adds the device and
driver names into the output, so that we have:

Before:
    qdio: 0.0.17c0 ZFCP on SC 17 using [...]

After:
    zfcp 0.0.17c0: qdio: ZFCP on SC 17 using [...]

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

Push the sync check from qdio_inspect_queue() down into the two
get_*_buffer_frontier() code paths, where we actually need the sync to
look at the current queue state. This lets us avoid the check when we
know that there is no work on the queue (ie. when q->nr_buf_used is 0).

While at it introduce the qdio_sync_*_queue() helpers, so that we can
avoid the branch on q->is_input_q when we already know the queue type.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

Don't bother with translating the SIGA-related capability bits into
our own internal format, just cache the full qdioac1 field instead.

Also adjust the helper macros so that they take a qdio_irq argument
and can be used everywhere, instead of taking a qdio_q and then
internally dereferencing the parent pointer.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

The queue processing is fully decoupled from any preceding interrupt,
so we're no longer making any use of the sync-after-IRQ HW capabilities.

And as SIGA-sync is a legacy feature, there's also not much point in
re-designing the driver & qdio-layer code just so that we can
potentially avoid a few syncs. So just remove all the leftover code.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

Clean up yet another path where HW wants an absolute address, and we've
been implicitly relying on V=R.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

Introduce dev_busid, which exports the device-id associated with the
io-subchannel (and message-subchannel). The dev_busid indicates that of
the device which may be physically installed on the corrosponding
subchannel. The dev_busid value "none" indicates that the subchannel
is not valid, there is no I/O device currently associated with the
subchannel.

The dev_busid information would be helpful to write device-specific
udev-rules associated with the subchannel. The dev_busid interface would
be available even when the sch is not bound to any driver or if there is
no operational device connected on it. Hence this attribute can be used to
write udev-rules which are specific to the device associated with the
subchannel.

Signed-off-by: Vineeth Vijayan <vneethv@linux.ibm.com>
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

This patch introduces a new rescan sys-interface for channel-subsystem.
The rescan interface allows the user to invoke an evaluation of all
subchannels defined in the I/O configuration.

The new rescan interface can be found at /sys/devices/css0/rescan
and can be triggered by,

echo > /sys/devices/css0/rescan

Writing to this interface triggers subchannel evaluation. The write
request completes only after scan-related work has completed

This user-invoked subchannel evaluation allows manual recovery in error
situations such as:
 - restart of device discovery after resolution of temporary device
   error
 - inconsistent OS view of subchannel state due to missing state-change
   interrupts (CRWs)

Signed-off-by: Vineeth Vijayan <vneethv@linux.ibm.com>
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

The dma section name is confusing, since the code which resides within
that section has nothing to do with direct memory access.  Instead the
limitation is that the code has to run in 31 bit addressing mode, and
therefore has to reside below 2GB.  So the name was chosen since
ZONE_DMA is the same region.

To reduce confusion rename the section to amode31, which hopefully
describes better what this is about.

Note: this will also change vmcoreinfo strings
- SDMA=... gets renamed to SAMODE31=...
- EDMA=... gets renamed to EAMODE31=...

Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Fix virtual vs physical address confusion (which currently are the same).

Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The functions get_online_cpus() and put_online_cpus() have been
deprecated during the CPU hotplug rework. They map directly to
cpus_read_lock() and cpus_read_unlock().

Replace deprecated CPU-hotplug functions with the official version.
The behavior remains unchanged.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://lore.kernel.org/r/20210803141621.780504-6-bigeasy@linutronix.de


Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The functions get_online_cpus() and put_online_cpus() have been
deprecated during the CPU hotplug rework. They map directly to
cpus_read_lock() and cpus_read_unlock().

Replace deprecated CPU-hotplug functions with the official version.
The behavior remains unchanged.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://lore.kernel.org/r/20210803141621.780504-5-bigeasy@linutronix.de


Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390 allows hotpatching the mask of a conditional jump instruction.
Make use of this feature in order to avoid the expensive stop_machine()
call.

The new trampolines are split in 3 stages:

- A first stage is a 6-byte relative conditional long branch located at
  each function's entry point. Its offset always points to the second
  stage for the corresponding function, and its mask is either all 0s
  (ftrace off) or all 1s (ftrace on). The code for flipping the mask is
  borrowed from ftrace_{enable,disable}_ftrace_graph_caller. After
  flipping, ftrace_arch_code_modify_post_process() syncs with all the
  other CPUs by sending SIGPs.

- Second stages for vmlinux are stored in a separate part of the .text
  section reserved by the linker script, and in dynamically allocated
  memory for modules. This prevents the icache pollution. The total
  size of second stages is about 1.5% of that of the kernel image.

  Putting second stages in the .bss section is possible and decreases
  the size of the non-compressed vmlinux, but splits the kernel 1:1
  mapping, which is a bad tradeoff.

  Each second stage contains a call to the third stage, a pointer to
  the part of the intercepted function right after the first stage, and
  a pointer to an interceptor function (e.g. ftrace_caller).

  Second stages are 8-byte aligned for the future direct calls
  implementation.

- There are only two copies of the third stage: in the .text section
  for vmlinux and in dynamically allocated memory for modules. It can be
  an expoline, which is relatively large, so inlining it into each
  second stage is prohibitively expensive.

As a result of this organization, phoronix-test-suite with ftrace off
does not show any performance degradation.

Suggested-by: Sven Schnelle <svens@linux.ibm.com>
Suggested-by: Vasily Gorbik <gor@linux.ibm.com>
Co-developed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Link: https://lore.kernel.org/r/20210728212546.128248-3-iii@linux.ibm.com


Signed-off-by: Heiko Carstens <hca@linux.ibm.com>