lookup_open(): saner calling conventions (return dentry on success)

same story as for atomic_open() in the previous commit.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>