arm64: xilinx: Never touch DDR if system has no DDR

If DDR is not mapped do not touch it. Default
XILINX_OF_BOARD_DTB_ADDR is pointing to DDR.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>