USB: serial: ti_usb_3410_5052: clean up serial data access

Use the tdev pointer directly instead of going through the port data
when accessing the serial data in close().

Signed-off-by: Johan Hovold <johan@kernel.org>