x86: shrink Masking insn attribute to a single bit (boolean)

The logic can actually be expressed with less code that way, utilizing
that there are common patterns of when which form of masking is
permitted. This then also eliminates the large set of open-codings of
BOTH_MASKING in the opcode table.