media: pci: saa7164: replace BUG with error return

It was completely unnecessary to use BUG in saa7164_s_frequency,
just return an error instead. This also solves a smatch error:

drivers/media/pci/saa7164/saa7164-encoder.c:388 saa7164_s_frequency() error: potentially dereferencing uninitialized 'tsport'.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>