Revert "clk: qcom: Specify LE device endianness"

This reverts commit 329cabce.

The commit that caused us to specify LE device endianness here,
29bb45f2 (regmap-mmio: Use native endianness for read/write,
2015-10-29), has been reverted in mainline so now when we specify
LE it actively breaks big endian kernels because the byte
swapping in regmap-mmio is incorrect. Let's revert this change
because it will 1) fix the big endian kernels and 2) be redundant
to specify LE because that will become the default soon.

Cc: Kevin Hilman <khilman@linaro.org>
Tested-by: Kevin Hilman <khilman@baylibre.com>
Cc: Mark Brown <broonie@kernel.org>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>