KVM: x86: Add emulator support for decoding VEX prefixes

After all the changes done in the previous patches, the only thing
left to support AVX MOV instructions is to expand the VEX prefix into
the appropriate REX, 66/F3/F2 and map prefixes.  Three-operand
instructions are not supported.

The Avx bit in this case is not cleared, in fact it is used as the
sign that the instruction does support VEX encoding.  Until it is
added to any instruction, however, the only functional change is
to change some not-implemented instructions to #UD if they correspond
to a VEX prefix with an invalid map.

Co-developed-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Link: https://patch.msgid.link/20251114003633.60689-10-pbonzini@redhat.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
This commit is contained in:
Paolo Bonzini 2025-11-13 19:36:32 -05:00 • committed by Sean Christopherson
commit f0585a714a