net/mlx5: E-Switch, Use vport metadata matching only when mandatory

Multi-port RoCE mode requires tagging traffic that passes through the
vport.
This matching can cause performance degradation, therefore disable it
and use the legacy matching on vhca_id and source_port when possible.

Fixes: 92ab1eb3 ("net/mlx5: E-Switch, Enable vport metadata matching if firmware supports it")
Signed-off-by: Majd Dibbiny <majd@mellanox.com>
Reviewed-by: Mark Bloch <markb@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>