x86-64: refuse "high" 8-bit regs with .insn and VEX/XOP/EVEX encodings

Much like REX, those encodings - if permitting 8-bit regs at all, i.e.
only starting with APX - permit use of "new" 8-bit registers only. %ah,
%ch, %dh, and %bh cannot be encoded and hence should be rejected.

Permit their use outside of 64-bit code though, as "new" registers
simply don't exist there.
This commit is contained in:
Jan Beulich 2023-12-22 09:34:10 +01:00
parent ce7056886a
commit 3e4a511bee

View File

@ -11485,6 +11485,16 @@ s_insn (int dummy ATTRIBUTE_UNUSED)
for (j = i.imm_operands; j < i.operands; ++j)
{
/* Look for 8-bit operands that use old registers. */
if (i.vec_encoding != vex_encoding_default
&& flag_code == CODE_64BIT
&& i.types[j].bitfield.class == Reg
&& i.types[j].bitfield.byte
&& !(i.op[j].regs->reg_flags & RegRex64)
&& i.op[j].regs->reg_num > 3)
as_bad (_("can't encode register '%s%s' with VEX/XOP/EVEX"),
register_prefix, i.op[j].regs->reg_name);
i.types[j].bitfield.instance = InstanceNone;
if (operand_type_check (i.types[j], disp))