Do not invoke ssh with -p argument when no port has been specified.

This change allows local SSH configuration to choose the port number to
use when not explicitly set in the manifest.