Unverified Commit 4eb2eb1b authored by Andrea Parri's avatar Andrea Parri Committed by Palmer Dabbelt
Browse files

riscv,mmio: Fix readX()-to-delay() ordering 

Section 2.1 of the Platform Specification [1] states:

  Unless otherwise specified by a given I/O device, I/O devices are on
  ordering channel 0 (i.e., they are point-to-point strongly ordered).

which is not sufficient to guarantee that a readX() by a hart completes
before a subsequent delay() on the same hart (cf. memory-barriers.txt,
"Kernel I/O barrier effects").

Set the I(nput) bit in __io_ar() to restore the ordering, align inline
comments.

[1] https://github.com/riscv/riscv-platform-specs



Signed-off-by: default avatarAndrea Parri <parri.andrea@gmail.com>
Link: https://lore.kernel.org/r/20230803042738.5937-1-parri.andrea@gmail.com


Fixes: fab957c1 ("RISC-V: Atomic and Locking Code")
Cc: stable@vger.kernel.org
Signed-off-by: default avatarPalmer Dabbelt <palmer@rivosinc.com>
parent 6514f81e

arch/riscv/include/asm/mmio.h

+8 −8
Original line number Diff line number Diff line
@@ -101,9 +101,9 @@ static inline u64 __raw_readq(const volatile void __iomem *addr) 
 * Relaxed I/O memory access primitives. These follow the Device memory

 * ordering rules but do not guarantee any ordering relative to Normal memory

 * accesses.  These are defined to order the indicated access (either a read or

 * write) with all other I/O memory accesses. Since the platform specification

 * defines that all I/O regions are strongly ordered on channel 2, no explicit

 * fences are required to enforce this ordering.

 * write) with all other I/O memory accesses to the same peripheral. Since the

 * platform specification defines that all I/O regions are strongly ordered on

 * channel 0, no explicit fences are required to enforce this ordering.

 */

/* FIXME: These are now the same as asm-generic */

#define __io_rbr()		do {} while (0)
@@ -125,14 +125,14 @@ static inline u64 __raw_readq(const volatile void __iomem *addr) 
#endif



/*

 * I/O memory access primitives. Reads are ordered relative to any

 * following Normal memory access. Writes are ordered relative to any prior

 * Normal memory access.  The memory barriers here are necessary as RISC-V

 * I/O memory access primitives.  Reads are ordered relative to any following

 * Normal memory read and delay() loop.  Writes are ordered relative to any

 * prior Normal memory write.  The memory barriers here are necessary as RISC-V

 * doesn't define any ordering between the memory space and the I/O space.

 */

#define __io_br()	do {} while (0)

#define __io_ar(v)	__asm__ __volatile__ ("fence i,r" : : : "memory")

#define __io_bw()	__asm__ __volatile__ ("fence w,o" : : : "memory")

#define __io_ar(v)	({ __asm__ __volatile__ ("fence i,ir" : : : "memory"); })

#define __io_bw()	({ __asm__ __volatile__ ("fence w,o" : : : "memory"); })

#define __io_aw()	mmiowb_set_pending()



#define readb(c)	({ u8  __v; __io_br(); __v = readb_cpu(c); __io_ar(__v); __v; })