block: dio: ensure the memory order between bi_private and bi_css

hulk inclusion
category: bugfix
bugzilla: 167067
CVE: NA

--------------------------------

In __blkdev_direct_IO_simple(), when bi_private is NULL, it assumes
bi_css must be NULL as showed below:

CPU 1:                         CPU 2:

                               __blkdev_direct_IO_simple
                                 submit_bio

bio_endio
  bio_uninit(bio)
    css_put(bi_css)
    bi_css = NULL
                               set_current_state(TASK_UNINTERRUPTIBLE)
  bio->bi_end_io
    blkdev_bio_end_io_simple
      bio->bi_private = NULL
                               // bi_private is NULL
                               READ_ONCE(bio->bi_private)
        wake_up_process
          smp_mb__after_spinlock

                               bio_unint(bio)
                                 // read bi_css as no-NULL
                                 css_put(bi_css)

Because there is no memory barrier between the reading and the writing of
these two variables, the assumption is wrong under weak-memory model
machine (e.g. arm64). bi_css will be put twice and leads to the following
warning:

  percpu_ref_switch_to_atomic_rcu: percpu ref (css_release) <= 0 (-3) after switching to atomic

There is a similar problem in __blkdev_direct_IO() which occurs between
dio->waiter and bio.bi_status.

Fixing it by adding a smp_rmb() between the reads of two variables, and a
corresponding smp_wmb() between the writes.

Signed-off-by: Hou Tao <houtao1@huawei.com>
Signed-off-by: Yu Kuai <yukuai3@huawei.com>
Reviewed-by: Jason Yan <yanaijie@huawei.com>
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>