Now that all drivers use qdio_inspect_queue() and qdio's internal
queue tasklets are gone, the driver-specified queue handlers are
only called for async error reporting (eg. for an error condition in
the QEBSM code).

So take a moment to clean up the Output Queue handlers (they are
_always_ called with qdio_error != 0), and clarify which error types
can be reported through what interface. As Benjamin already suggested
a while ago, we should turn these into distinct enums at some point.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

get_outbound_buffer_frontier() is only reached via qdio_inspect_queue(),
and there we already call qdio_siga_sync_q() unconditionally.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Both qdio drivers have moved away from using qdio's internal tasklet
and timer mechanisms for Output Queues. Rip out all the leftovers.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

If qdio_cancel_ccw() times out (or is interrupted) before the interrupt
for the {halt,clear} action arrives, report this back to the caller as
an error.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

If the ESTABLISH ccw fails (ie. the qdio_irq is set to
QDIO_IRQ_STATE_ERR), we don't need to call qdio_shutdown() for rolling
back our earlier actions. All the needed logic is already available in
qdio_establish()'s error chain, and using it means we don't have to
temporarily drop the setup_mutex either.

This makes qdio_shutdown() a purely external function, that should only
be called by the driver if an earlier qdio_establish() succeeded.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

When the ESTABLISH ccw does not complete within the specified timeout,
try our best to cancel the ccw program that is still active on the
device. Otherwise the IO subsystem might be accessing it even after
the driver eg. called qdio_free().

Fixes: 779e6e1c

 ("[S390] qdio: new qdio driver.")
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Cc: <stable@vger.kernel.org> # 2.6.27
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

When qdio_establish() times out while waiting for the ESTABLISH ccw to
complete, it calls qdio_shutdown() to roll back all of its previous
actions. But at this point the qdio_irq's state is still
QDIO_IRQ_STATE_INACTIVE, so qdio_shutdown() will exit immediately
without doing any actual work.

Which means that eg. the qdio_irq's thinint-indicator stays registered,
and cdev->handler isn't restored to its old value. And since
commit 954d6235 ("s390/qdio: make thinint registration symmetric")
the qdio_irq also stays on the tiq_list, so on the next qdio_establish()
we might get a helpful BUG from the list-debugging code:

...
[ 4633.512591] list_add double add: new=00000000005a4110, prev=00000001b357db78, next=00000000005a4110.
[ 4633.512621] ------------[ cut here ]------------
[ 4633.512623] kernel BUG at lib/list_debug.c:29!
...
[ 4633.512796]  [<00000001b2c6ee9a>] __list_add_valid+0x82/0xa0
[ 4633.512798] ([<00000001b2c6ee96>] __list_add_valid+0x7e/0xa0)
[ 4633.512800]  [<00000001b2fcecce>] qdio_establish_thinint+0x116/0x190
[ 4633.512805]  [<00000001b2fcbe58>] qdio_establish+0x128/0x498
...

Fix this by extracting a goto-chain from the existing error exits in
qdio_establish(), and check the return value of the wait_event_...()
to detect the timeout condition.

Fixes: 779e6e1c

 ("[S390] qdio: new qdio driver.")
Root-caused-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Cc: <stable@vger.kernel.org> # 2.6.27
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

There is no useful information within [STARTUP_NORMAL_OFFSET, HEAD_END] now.

But the memory region [0, STARTUP_NORMAL_OFFSET] is used by:
* lowcore
* kdump for swapping memory
* stand-alone zipl dumpers for code, data, stack and heap

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

This change simplifies the task of making the decompressor relocatable.

The decompressor's image contains special DMA sections between _sdma and
_edma. This DMA segment is loaded at boot as part of the decompressor and
then simply handed over to the decompressed kernel. The decompressor itself
never uses it in any way. The primary reason for this is the need to keep
the aforementioned DMA segment below 2GB which is required by architecture,
and because the decompressor is always loaded at a fixed low physical
address, it is guaranteed that the DMA region will not cross the 2GB
memory limit. If the DMA region had been placed in the decompressed kernel,
then KASLR would make this guarantee impossible to fulfill or it would
be restricted to the first 2GB of memory address space.

This commit moves all DMA sections between _sdma and _edma from
the decompressor's image to the decompressed kernel's image. The complete
DMA region is placed in the init section of the decompressed kernel and
immediately relocated below 2GB at start-up before it is needed by other
parts of the decompressed kernel. The relocation of the DMA region happens
even if the decompressed kernel is already located below 2GB in order
to keep the first implementation simple. The relocation should not have
any noticeable impact on boot time because the DMA segment is only a couple
of pages.

After relocating the DMA sections, the kernel has to fix all references
which point into it. In order to automate this, place all variables
pointing into the DMA sections in a special .dma.refs section. All such
variables must be defined using the new __dma_ref macro. Only variables
containing addresses within the DMA sections must be placed in the new
.dma.refs section.

Furthermore, move the initialization of control registers from
the decompressor to the decompressed kernel because some control registers
reference tables that must be placed in the DMA data section to
guarantee that their addresses are below 2G. Because the decompressed
kernel relocates the DMA sections at startup, the content of control
registers CR2, CR5 and CR15 must be updated with new addresses after
the relocation. The decompressed kernel initializes all control registers
early at boot and then updates the content of CR2, CR5 and CR15
as soon as the DMA relocation has occurred. This practically reverts
the commit a80313ff

 ("s390/kernel: introduce .dma sections").

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

As a preparation for moving the .dma.data section from the decompressor to
the decompressed kernel, the .dma.data section must be made relocatable
by replacing absolute memory addressing with relative one. This is required
in order to be able to relocate the DMA section to a memory address <= 2G
as required by the hardware architecture. The DMA section must be
relocated in case the decompressed kernel was loaded to an address >= 2G
which can occur if KASAN is enabled. By making the whole DMA section
position-independent we avoid applying relocations to it whenever it is
moved to a different address, which becomes possible as soon as it becomes
a part of the decompressed kernel.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Both macros are used only in decompressor's head.S, unnecessary to put
them in a global header used in many places like setup.h is.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

To reduce duplication, replace error-prone and hard-coded parameter area
offsets with auto-generated ones.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The macros
* IPL_DEVICE_OFFSET
* INITRD_START_OFFSET
* INITRD_SIZE_OFFSET
* OLDMEM_BASE_OFFSET
* OLDMEM_SIZE_OFFSET
* KERNEL_VERSION_OFFSET
* COMMAND_LINE_OFFSET
are no longer necessary and used only to define another set of macros
with the same names but w/o the suffix _OFFSET. Therefore, drop this
unnecessary indirection.

Drop the macro KERNEL_VERSION_OFFSET w/o renaming it to KERNEL_VERSION
because it is used nowhere.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

These symbolic constants are used only by assembler code now:
* COMMAND_LINE
* IPL_DEVICE

C code of the decompressed kernel should use boot data passed
by the decompressor instead.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The new boot data struct shall replace global variables OLDMEM_BASE and
OLDMEM_SIZE. It is initialized in the decompressor and passed
to the decompressed kernel. In comparison to the old solution, this one
doesn't access data at fixed physical addresses which will become important
when the decompressor becomes relocatable.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The new boot data struct shall replace global variables INITRD_START and
INITRD_SIZE. It is initialized in the decompressor and passed
to the decompressed kernel. In comparison to the old solution, this one
doesn't access data at fixed physical addresses which will become important
when the decompressor becomes relocatable.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

To make the decompressor relocatable, the early SCLP buffer with a fixed
address must be replaced with a relocatable C buffer of the according size
and alignment as required by SCLP.

Introduce a new function sclp_early_set_buffer() into the SCLP driver
which enables the decompressor to change the SCLP early buffer at any time.
This will be useful when the decompressor becomes fully relocatable and
might need to change the SCLP early buffer to one with an address < 2G
as required by SCLP because it was loaded at an address >= 2G.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Use STARTUP_NORMAL_OFFSET and STARTUP_KDUMP_OFFSET instead of magic numbers.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Use system call functions instead of open-coding svc inline
assemblies. This is mostly to get rid of even more register asm
constructs.
Besides that, it makes the code also a bit easier to understand.
The generated code is identical to what is was before.

Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Provide generic system call functions which should be used whenever a
system call needs to be done from user space. The only in-kernel code
is vdso, which will be converted with a follow on patch.

Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Using register asm statements has been proven to be very error prone,
especially when using code instrumentation where gcc may add function
calls, which clobbers register contents in an unexpected way.

Therefore get rid of register asm statements in cpacf code, and make
sure this bug class cannot happen.

Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Remove unused print defines from debug feature header file.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Remove dasd ioctl debug printk which seems to be a leftover from the
very early days. At least it seems to be quite pointless.

Reviewed-by: Stefan Haberland <sth@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

De-duplicate checks for Protected Host Virtualization in decompressor and
kernel.

Set prot_virt_host=0 in the decompressor in *any* of the following cases
and hand it over to the decompressed kernel:
* No explicit prot_virt=1 is given on the kernel command-line
* Protected Guest Virtualization is enabled
* Hardware support not present
* kdump or stand-alone dump

The decompressed kernel needs to use only is_prot_virt_host() instead of
performing again all checks done by the decompressor.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

A dump kernel is neither required nor able to support Secure Execution.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

The functions adjust_to_uv_max() and uv_query_info() are used only
in the decompressor. Therefore, move the function declarations from
the global arch/s390/include/asm/uv.h to arch/s390/boot/uv.h.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

In case of a jump label print the real address of the piece of code
where a mismatch was detected. This is right before the system panics,
so there is nothing revealed.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Print the real pte, pmd, etc. values instead of some hashed
value. Otherwise debugging would be even more difficult.
This also matches what most other architectures are doing.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Use pr_err() to use a proper printk level.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

A buffer that can be used for communication with SCLP is required
to lie below 2GB memory address. Therefore, both sclp_info_sccb
and sclp_early_sccb must fulfill this requirement if passed directly
to the sclp_early_cmd() function. Instead, use only sclp_early_sccb
for communication with SCLP. This allows the buffer sclp_info_sccb
to be placed anywhere in the memory address space and, therefore,
simplifies the process of making the decompressor relocatable later on,
one thing less to relocate. And make sure that the length of the new unified
early SCLP buffer is no less than the length of the removed sclp_info_sccb
buffer which might be larger than the length of the sclp_early_sccb buffer.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

* The linux/spinlock.h header was included indirectly by the decompressor
  and brought unnecessary build dependencies.
* Use proper includes in files which either directly or indirectly included
  cio.h and were hidden until now by the included linux/spinlock.h, e.g.
  linux/string.h for memcpy() or asm/page.h for PAGE_SIZE.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Instead of using constant addresses for the normal and dump-info stacks,
allocate both stacks in the decompressor's image and load the stack register
in a position-independent manner.

This will allow loading and entering the decompressor at an arbitrary
memory address without corrupting the content at the fixed addresses
used until now for both stacks. This is one of the prerequisites
for being able to kexec the decompressor from its load address without
relocating it first.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

To prevent multiple incompatible declarations of symbols and to catch
such mistakes at compile time.

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

gcc doesn't care, but clang quite reasonably pointed out that the recent
commit e9ba16e6

 ("smpboot: Mark idle_init() as __always_inlined to
work around aggressive compiler un-inlining") did some really odd
things:

    kernel/smpboot.c:50:20: warning: duplicate 'inline' declaration specifier [-Wduplicate-decl-specifier]
    static inline void __always_inline idle_init(unsigned int cpu)
                       ^

which not only has that duplicate inlining specifier, but the new
__always_inline was put in the wrong place of the function definition.

We put the storage class specifiers (ie things like "static" and
"extern") first, and the type information after that.  And while the
compiler may not care, we put the inline specifier before the types.

So it should be just

    static __always_inline void idle_init(unsigned int cpu)

instead.

Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Pull powerpc fixes from Michael Ellerman:

 - Fix guest to host memory corruption in H_RTAS due to missing nargs
   check.

 - Fix guest triggerable host crashes due to bad handling of nested
   guest TM state.

 - Fix possible crashes due to incorrect reference counting in
   kvm_arch_vcpu_ioctl().

 - Two commits fixing some regressions in KVM transactional memory
   handling introduced by the recent rework of the KVM code.

Thanks to Nicholas Piggin, Alexey Kardashevskiy, and Michael Neuling.

* tag 'powerpc-5.14-3' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux:
  KVM: PPC: Book3S HV Nested: Sanitise H_ENTER_NESTED TM state
  KVM: PPC: Book3S: Fix H_RTAS rets buffer overflow
  KVM: PPC: Fix kvm_arch_vcpu_ioctl vcpu_load leak
  KVM: PPC: Book3S: Fix CONFIG_TRANSACTIONAL_MEM=n crash
  KVM: PPC: Book3S HV P9: Fix guest TM support

Pull timer fixes from Thomas Gleixner:
 "A small set of timer related fixes:

   - Plug a race between rearm and process tick in the posix CPU timers
     code

   - Make the optimization to avoid recalculation of the next timer
     interrupt work correctly when there are no timers pending"

* tag 'timers-urgent-2021-07-25' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  timers: Fix get_next_timer_interrupt() with no timers pending
  posix-cpu-timers: Fix rearm racing against process tick

Pull x86 jump label fix from Thomas Gleixner:
 "A single fix for jump labels to prevent the compiler from agressive
  un-inlining which results in a section mismatch"

* tag 'locking-urgent-2021-07-25' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  jump_labels: Mark __jump_label_transform() as __always_inlined to work around aggressive compiler un-inlining

Pull EFI fixes from Thomas Gleixner:
 "A set of EFI fixes:

   - Prevent memblock and I/O reserved resources to get out of sync when
     EFI memreserve is in use.

   - Don't claim a non-existing table is invalid

   - Don't warn when firmware memory is already reserved correctly"

* tag 'efi-urgent-2021-07-25' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  efi/mokvar: Reserve the table only if it is in boot services data
  efi/libstub: Fix the efi_load_initrd function description
  firmware/efi: Tell memblock about EFI iomem reservations
  efi/tpm: Differentiate missing and invalid final event log table.