- Added detailed logging for GPU resource attach and detach operations.
- Improved error handling and logging for resource creation failures.
- Enhanced information on resource preparation and backing status.
virtio-media's buffers have to live somewhere both sides can reach. On a
protected Gunyah VM guest RAM is lent, so the host cannot touch a buffer the
guest allocated, and the upstream shape for host-owned buffers -- a memfd per
buffer mapped into a 4 GiB PCI shared-memory BAR -- does not fit either: the
64-bit MMIO window holds exactly one such BAR and the GPU already has it
(VPU_DESIGN.md 0.2, 2.3). Both directions therefore want a pool, and the pool
machinery to build one has been in the tree since gfx_host.
So: two more of them. media_host is host-owned and works like gfx_host,
drm2kgsl_host and venus_host -- the device sub-allocates every host-produced
buffer from it (camera frames, decoded frames, encoded bitstreams) and the
guest maps them at the pool base plus the offset QUERYBUF answered with.
media_guest is guest-owned and works like gpu_guest -- the guest driver runs
its own allocator over the range and cuts the buffers it produces out of it
(bitstreams to decode, raw frames to encode), handing the host bare
guest-physical scatter-gather lists. Direction decides ownership: whichever
side produces the data owns the memory (VPU_DESIGN.md 2.1, 2.2).
Both are fully pre-shared, step 0. A growable pool needs runtime grants over a
share device this phone does not load, and a second growable pool would enter
the pool_id ordering hazard between create_fdt's call order and the host's
address rank, so v1 takes neither. media_host comes out of --mem like the other
host pools; media_guest is additive like gpu_guest, because it is the VM's own
memory and was sized deliberately.
The mechanical part is the recipe in logs/vpu_survey/crosvm-pools.md 3, and two
of its twelve steps are the ones that matter: check_host_access ends in a
catch-all that would refuse the host every byte of both pools in a protected
VM, and gunyah's is_pool is a matches! that decides whether a region is
folio-prepared and mlocked at all. Neither is a compile error. Both are now
extended, and create_pool_node's "Adding a pool" comment says so, so the next
pool does not have to rediscover it.
media_host is handed to its consumer as the region itself rather than as
environment variables: MediaPoolHandle::from_guest_memory reads
(fd, fd_offset, host_va, gpa, size) off the one region, so an in-VMM device and
a vhost-user helper cannot disagree about which region is the pool, and a helper
has something it can carry across an exec. The env-var handoff exists because
the renderer is forked before any of this runs; the media device is not
(VPU_DESIGN.md 2.2, 3.1). media_guest gets no host-side handoff at all -- the
guest owns it, and the host meets it as ordinary guest addresses.
The --pre-alloc help text was wrong before this and is now complete:
venus-host-mb and the three test-pool-* keys existed and were documented
nowhere.
Tests cover the two silent sites and the ordering. The vm_memory tests build a
GuestMemory with both pools, set_protected(), and assert the host keeps access
where plain lent RAM loses it -- verified to fail when the new check_host_access
arm is removed. The aarch64 test pins the layout order to
gfx/gpu_guest/drm2kgsl/venus/media_host/media_guest/test, which is guest-
physical order and, for growable pools, pool_id order.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BkN7vYhuc1w9iVuqsHzSn
Bring back the virtio-mmio transport upstream removed in 561dbcc17 (it was
unreachable), ported to the current Queue/Interrupt APIs, for guests that
have a virtio-mmio driver but no PCI host driver -- UniXNU's
NativeVirtioBlock under XNU.
- devices: VirtioMmioDevice (modern layout only), a level interrupt with
resample like a virtio-pci INTx, config space of any access width, and
QueueNotify served by ioevents with a direct fallback
- --virtio-mmio-block PATH[,keys of --block]: a block device on that
transport; the aarch64 builder gives each one a page from 0x0a000000, a
level SPI, its ioevents and a "virtio,mmio" FDT node
- gunyah: GH_IOEVENTFD_DATAMATCH is already the flag mask; shifting it set
a reserved bit and every datamatch ioeventfd failed with EINVAL
Tested on SM8650 (pseudo-unprotected VM): XNU's NativeVirtioBlock reads a
64 MiB read-only image through it with the expected sector-0 hash.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Camiwa7VsPwkhfyscKQ6S
The venus route needs its own boot-blessed pool -- `--pre-alloc venus-host-mb`,
announced as `venus_host` -- for the same reason drm2kgsl does: sharing a pool
means sharing the region, and the two renderers sub-allocate independently.
Residency is recognised against that window as well as the drm2kgsl one, so a
blob that came out of either arena is reported to the guest as pool-resident
rather than mapped a second time.
`use_guest_vram` is set on the venus capset. It tells the guest that host memory
cannot be injected and that every host-visible or exportable VkDeviceMemory has
to be guest-allocated and imported -- which is not a preference here but the
only arrangement that works in a Gunyah protected VM, where the host cannot hand
the guest pages after boot.
virglrenderer's log messages now come out through crosvm's logger at their real
level, instead of the fixed-level debug callback that swallowed them: a
renderer error was indistinguishable from renderer chatter, which is a poor
place to be when bringing a route up.
Co-authored-by: lateautumn233 <lateautumn233@foxmail.com>
The drm2kgsl route runs a native context through virglrenderer straight onto
KGSL, and it needs two things from the VMM that the generic blob path does not
provide.
A host arena of its own: `--pre-alloc drm-host-mb`, a boot-blessed pool
announced as `drm2kgsl_host` and handed to the renderer through
CROSVM_DRM2KGSL_ARENA_*. Sharing the gfxstream pool would mean sharing the
region, which is exactly what separate MemoryRegionPurpose variants exist to
prevent.
Residency is recorded when the resource is created, not probed when it is
mapped. `virgl_renderer_resource_get_map_ptr` says where a resource's host
mapping lives, and comparing that against the arena window answers "is this blob
pool-resident" once. Asking at map time instead means asking after the guest has
already been told what to expect.
And a guest-allocated blob is handed to virglrenderer as a dma-buf fd
(`virgl_renderer_resource_set_guest_blob_fd`) rather than as an address the
renderer would have to map: the pages are the guest's, they are already shared,
and KGSL can import them directly. The fd is duplicated before it is handed
over, because virglrenderer takes ownership and the caller still needs its own
-- and closed again on the error path, which is the difference between a failed
resource and an fd leak per failed resource. HOST3D_GUEST blobs get their iovecs
as well as the fd, since the CPU-side paths still index them.
Ring-indexed fences are routed to the per-context timeline rather than the
global one. A native context submits on its own ring, and a fence that lands on
the global timeline signals against work that is not the work the guest is
waiting for.
Sharing a blob means building a memparcel out of pages, and the only way to get
pages out of a userspace address is GUP. That works for a memfd. It does not
work for what the GPU driver exports: those DMA-BUFs mmap as VM_PFNMAP or
VM_MIXEDMAP, and GUP is REQUIRED to refuse such a VMA -- there is no page
reference to take. Which left the blobs that matter most reachable only by
copying them somewhere GUP would accept, or by making the VMM rewrite the
allocation's backing before handing it over.
GHSM_SHARE_DMABUF takes the fd instead. The module holds a DMA-BUF attachment
and builds the parcel from the exporter's own sg-table, so the buffer keeps the
backing its allocator chose and crosvm has no opinion about it at all. That
replaces the VMM-side per-blob folio policy this used to need: the policy
belongs with whoever allocates, and crosvm is not that.
ENOTTY falls back to the existing VA/GUP path, which is both an older module and
the ordinary case of a pool memfd -- neither should be a failure.
A pool has to be sized before the VM starts, and a size that covers the worst
case is memory the phone loses for the whole session. Growable pools let the
guest ask for the rest as it needs it: the pool is declared to the guest at its
full window, but only its floor is shared at boot, and everything above the
floor arrives as a runtime memparcel the guest accepts itself.
That distinction is what `droidvm,pool-size` exists for, and getting it wrong is
fatal rather than degraded. `reg` on the /reserved-memory node describes exactly
what was SHARE'd before boot -- the RM's match is exact, and a `reg` covering
the whole window while only the floor is really shared matches no memparcel, so
android14-6.1's RM refuses the VM with ENODEV at GH_VM_START (measured on
sm8650: the same pool boots with the node removed and is refused with it
present). The window's real size therefore travels in `droidvm,pool-size`, and a
growable pool is also kept out of /memory entirely: a guest that took the window
for ordinary RAM would allocate above the floor, where a read silently returns
zeros and a write kills the VM.
Grants travel over a third queue on the gunyah-accept device. One grant is one
memparcel, delivered as 2 MiB folios, and requests must arrive at the declared
step -- a misaligned request is refused rather than rounded, because rounding
hands out memory nobody asked for. A grant is held for as long as the host has
buffers over it, so shrinking cannot pull pages out from under a dma-buf the
renderer is still reading; the refcounting is in `pool_grants`.
There is a test pool because there is no other way to exercise grow and shrink:
every production pool ships with step 0, so a pool that can actually move is the
only place the path gets executed at all.
The boot-blessed pools are host-allocated: the renderer carves them up and the
guest is told where. The other half is the mirror image -- a pool announced to
the guest driver, which allocates from it and hands the host a scatter-gather
list. That is what a guest Vulkan driver wants, because the memory it needs is
memory it will map itself, and it is the only shape that works when the guest
must own the allocation.
The host side turns the list into a udmabuf and gives that to the renderer, so
there is one dma-buf per blob and no per-blob SHARE at all: the pages were
already shared, at boot, as part of the pool. external_blob is forced on when
udmabuf is in use, because gfxstream's guest-handle blob path is gated on it and
without the gate ResourceCreateBlob falls through to
no-premapped-external-blob-mapping and returns -22 -- guest Vulkan cannot
allocate device memory at all, and nothing says why.
Ordering around teardown is the part that is easy to get wrong. A pool grant
must be held until the RENDERER has dropped the resource, not until the guest
has: unref the grant first and the guest may reuse pages the host still has a
dma-buf over. So the unmap/unref runs first, the resource is put back on the map
if it fails so a later cleanup can retry, and only then are the grants released.
The ranges are stored on the resource rather than recomputed at release time,
because backing_iovecs can be detached independently of the resource.
Co-authored-by: lateautumn233 <lateautumn233@foxmail.com>
Sharing a blob at a time works, but it costs an RM round trip and a stage-2
table churn per resource, and on a phone that is the difference between a
compositor that hitches and one that does not. A pool is the alternative: a
slab of address space carved out after guest RAM, SHARE'd once before the VM
starts, and sub-allocated by the renderer afterwards. Nothing is transferred at
run time, so nothing can fail at run time either.
`--pre-alloc` sizes them and each one appears in the device tree as a
/reserved-memory node named for who uses it and which side allocates from it --
`gfx_host`, `gpu_guest`, `drm2kgsl_host`. Those names are wire: the guest driver
finds its pool by matching the prefix, so renaming one is a cross-repo change.
The `reg` of such a node must match the memparcel registered before VM start
exactly; the RM walks every /reserved-memory child, compares by address and
size, and refuses the VM outright when nothing matches. Adding a pool is four
mechanical steps (a MemoryRegionPurpose variant, a carve-out, a node, a driver
that looks for the name) precisely because both readers are generic.
A pool-resident blob is reported to the guest through the existing map_info
response: VIRTIO_GPU_MAP_INFO_POOL says "this is a pool offset", and the field
that used to be `padding` carries the offset. The guest then maps pool_base +
offset out of RAM it was already given and never accepts anything. Resource ids
are recycled by the guest, so the offset must be CLEARED for a resource that is
not pool-resident -- inheriting the previous tenant's offset makes crosvm map
the wrong GPA into the guest, which is a corrupted screen rather than an error.
The pages are prepared before they are shared: collapsed into 2 MiB folios,
mlocked and cache-cleaned, and the whole region goes over as ONE memparcel. The
per-2 MiB chunking this replaces cost a linear number of RM calls and produced
nothing the single parcel does not.
pci_bar_size defaults to 256 MiB rather than 8 GiB. The BAR is address space,
but the IPA window it needs is not free on this hypervisor, and 8 GiB of it
pushed the layout past what the RM would accept.
Co-authored-by: lateautumn233 <lateautumn233@foxmail.com>
A protected Gunyah guest cannot be handed host memory after boot: memslots are
settled at VM creation, and everything the guest can reach was lent to it before
it started. That rules out the entire host-visible virtio-gpu blob model, where
the renderer allocates host memory per resource and the guest maps it through
the shmem BAR.
This is the runtime path for it. `add_mapping_blob` SHAREs the blob to the live
VM and produces an RM memparcel handle; who accepts that handle is the caller's
choice. `VmAccept::Off` returns it so the guest driver can accept it itself,
which is what the virtio-gpu map_info wire carries. `VmAccept::Sync` instead
hands it to an in-VM accept module over a new virtio device, `gunyah-accept`,
and does not return until the guest has accepted -- which is what a guest driver
that has no idea a memparcel exists needs. Unmapping reclaims the parcel through
GHSM_UNSHARE_BLOB, so a resource the guest destroyed does not keep host memory
lent for the life of the VM.
The ioctls live on `/dev/gunyah_share`, not on the VM fd. The in-tree
GH_VM_ANDROID_SHARE_BLOB is dispatched inside `gh_vm_ioctl()`, and an
out-of-tree module cannot add a case to it, so the module exposes its own
character device and takes the VM fd as a field. Layouts here must match
`uapi_gunyah_share.h` byte for byte.
The VMM's own pin on a SHARE'd region is released once the guest accept has
settled: holding it keeps the host's pages pinned for the whole life of a blob
that the guest now maps directly, and on a phone that is the difference between
a VM that survives a compositor restart and one that does not.
The host-visible BAR window is 4 GiB above RAM, and the IPA size-max is raised
to match. Both numbers are the same fact stated twice, so
`gunyah_high_mmio_window_top` is the single place either is computed.
Pinnability is asked of `/dev/gh_pinprobe` when it is there and skipped when it
is not. The node is provided by an out-of-tree module that ships separately, so
"absent" has to mean "no answer" rather than "cannot pin" -- reading absence as
a refusal is what stopped VMs from starting on a host that simply did not have
the module loaded.
Co-authored-by: lateautumn233 <lateautumn233@foxmail.com>
A Gunyah protected VM lends its RAM to the guest before boot, so the host can no
longer read or write it, and every device the guest talks to then needs a bounce
buffer -- which means a guest kernel built with CONFIG_RESTRICTED_DMA_POOL. That
requirement is the single biggest obstacle to running a stock distribution
kernel here, and it exists only because of WHEN the memory is handed over, not
because of what protection buys.
`--protected-vm-pseudo-unprotected` hands the same memory over as a runtime
SHARE instead of a boot-time LEND. The VM is a real pVM -- the RM still owns the
transition, the guest still cannot be read by other processes -- but the host
retains its own mapping, so no bounce pool and no restricted-dma-pool node are
needed. Someone inside the VM has to accept the parcel before the payload runs,
and that someone is droidvm-shim: a freestanding Rust image, loaded at the
handoff page, that makes the RM calls to accept each memparcel, patches
/memory into the device tree it was given, and jumps to the real kernel. The
handoff page is deliberately not part of the guest's RAM -- it is the one place
the host can still write once the boot region is lent.
The rule that governs the whole arrangement: every SHARE'd parcel must have a
/reserved-memory node whose `reg` matches it exactly, or GH_VM_START answers
NORESOURCE. The RM matches by address and size, not by name or compatible.
pin.rs comes with it, because this is where pinning starts to fail. Every
Gunyah transfer ends in `pin_user_pages_fast(FOLL_LONGTERM)`, and a page in a
CMA pageblock cannot be pinned without first migrating it. When the huge-page
reserve is short the shortfall comes from ordinary movable memory, part of it
CMA, and the failure surfaces INSIDE the ioctl -- observed as a 2.5-minute
whole-host stall ending in the kernel OOM-killing crosvm, and as a
`qcom_scm: Assign memory protection call failed -22` that reset the phone. So
the probe asks the cheap question instead: would pinning have to migrate
anything? It takes no long-term reference and migrates nothing, and the default
on a "yes" is to refuse the transfer, where the error is still ours to report.
Asking must not itself fault pages in -- that is what FOLL_NOFAULT is for, and
without it the probe populates the very pages it is asking about, in order-0
allocations the reserve pool does not serve, and reports them as unpinnable.
Assorted VM-platform work that is not about graphics.
--smbios is no longer x86-only. On aarch64 the identity strings travel to the
guest firmware through /chosen (droidvm,smbios-*), where EDK2's
SmbiosPlatformDxe publishes them -- Type 4 processor version is the CPU name a
Windows guest displays. Linux ignores the properties.
The fixed GPIO and VMWDT SPIs move to the top of the range. They were 14 and 15,
inside the pool the PCI IRQ allocator hands out from, and a VM with enough
virtio-pci devices reached them: the eleventh device's GH_VM_ADD_FUNCTION
(GH_FN_IRQFD) failed with EEXIST, "failed to register irq fd: File exists". The
allocator now stops two SPIs short of the top and the two fixed interrupts live
there.
A fatal signal used to take the phone down with it. crosvm ran into debuggerd,
which on a VMM holding pinned guest memory is a reboot rather than a crash
report. There is now an in-process handler that writes pc/lr and the mcontext
straight to stderr with raw write(2) -- no allocation, no locks, nothing that
can deadlock in a signal handler -- and then _exit()s, so the VM dies and the
phone does not. Every WorkerThread gets its own sigaltstack, because sigaltstack
is per-thread and crosvm only set one on the main thread: a stack overflow on a
worker had no stack to run the handler on and killed the process silently.
The PL061 and PL030 interrupts are declared edge-triggered, which is what they
are: both models inject through an IrqEdgeEvent, because the Gunyah irqchip only
supports edge irqfds and the doorbell vdevice behind them is EDGE_RISING in the
hypervisor's own DT. Declaring LEVEL_HIGH made the guest GIC treat the doorbell
as a level line nobody ever deasserts, so the first power-button press left
vCPU0 spinning in the interrupt handler -- RCU stalls, soft lockups, a hard hang
on every press.
Two Gunyah RM quirks. GH_VM_SET_BOOT_CONTEXT answers ENODEV on the OPPO sm8650
6.1 downstream dispatch where mainline answers ENOTTY, so both mean "no such
ioctl". And the RM-lowmem fence node is emitted only on 6.6-or-newer host
kernels: the sm8650-era RM rejects a guest DTB carrying it with ENODEV at
GH_VM_START, while newer RMs require it, and no RM version is exposed to the
host -- the kernel release is the only proxy available. GUNYAH_LOWMEM_FENCE
forces either way. Every failing Gunyah ioctl now says which one it was, since
the errno alone never identified the call.
Untracked children are allowed to exit non-zero. Vendor GPU driver builds
(observed on OPPO sm8650, Android 14 Adreno) fork helpers that exit with a
non-zero status during normal operation; only crosvm's own tracked children are
evidence of a crash.
The material written for DroidVM in this tree is GPL-2.0-or-later, with the
additional permissions in ADDITIONAL-PERMISSIONS on top.
Those permissions are the point. The GPL alone would make this work
un-upstreamable: crosvm takes contributions as BSD-3-Clause, and it cannot take
GPL. So the grant lets anyone relicense this material under the terms an
upstream project requires, for the purpose of getting it merged there.
It is scoped by PURPOSE, not by party -- naming a licensee would not work for a
project with no single owner to grant anything to, and naming the purpose
produces the intended outcome by itself. It fires on ACCEPTANCE, not on
submission; otherwise opening a merge request and withdrawing it would be a way
to publish this work under a permissive license at will. Contributions carry the
grant because they are made under the project license, which is why
CONTRIBUTING.md asks for a DCO sign-off and not a CLA: nothing has to be
assigned for a contribution to be eligible to go upstream.
SPDX headers go only on files this project wrote. A patched upstream file is
still an upstream file and keeps its own header, and the edits to it are offered
upstream under upstream's license. LICENSING.md says which files are which and
names the third-party material that is neither -- pl061.rs and mthp.rs are
derived from QEMU, so the additional permissions explicitly do not extend to
QEMU's copyright in them.
Add a `--prepare-lend-mthp-mode` option that takes `single` or `chunked`,
so the LEND strategy can be matched to the host kernel's Gunyah memory
model. The existing boolean `--prepare-lend-mthp` switch is kept for
backward compatibility (deprecated; equivalent to `chunked`, the previous
behavior).
Both modes run the same mTHP preparation (drop caches, enable mTHP,
populate, cascading MADV_COLLAPSE, mlock). They differ only in how the
prepared region is lent to the hypervisor:
- single: LEND the whole region as one memory parcel. Required on the
eager per-region parcel kernel model (e.g. sm8650 / mainline gunyah),
where splitting into many parcels exhausts the RM per-VM memextent pool
and fails VM start with RM NORESOURCE -> ENODEV ("No such device").
- chunked: split into <=256MB parcels, one each (previous behavior), for
the demand-paging binding kernel model (e.g. sm8750) where extra
parcels are cheap.
Introduce `hypervisor::LendMthpMode`, change `Config.prepare_lend_mthp`
to `Option<LendMthpMode>` (None disables mTHP prep entirely), and branch
the LEND chunking in the Gunyah backend on the selected mode.
`--prepare-lend-mthp-mode` takes precedence over the deprecated
`--prepare-lend-mthp`; the latter maps to `chunked` when set alone.
This API can be used to retrieve the unique identifier of a VM.
Bug: b:360102915
TEST=tools/presubmit
Test: presubmit
Change-Id: Ic1e5f350c93bed68c7d9022af1eeaa0e773a7418
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6324569
Auto-Submit: Nikita Ioffe <ioffe@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Nikita Ioffe <ioffe@google.com>
Reviewed-by: Frederick Mayle <fmayle@google.com>
Qualcomm Trusted VM (QTVM) is a kind of protected VM which differentiates
itself from from the traditional protected VMs that have been supported by
crosvm in a few aspects like -
1. identified by reserved VM ID and PAS ID pair.
2. authenticated by the firmware/TZ and not pvmfw.
3. A QTVM Image is single loadable elf Image which contains kernel, dtb, initrd and the metadata
needed for its authentication.
4. A QTVM can be backed by Contiguous Memory. Either the entire GuestMemory region could be
contiguous or it could be a combination of both CMA and shm.
As the guest memory now can contain regions backed by CMA, make the necessary IOCTLs to map these
regions to the VM. Also, add a new IOCTL to set the authentication type to be
GUNYAH_QCOM_TRUSTED_VM_TYPE.
Bug: 399219478
Change-Id: I2e8a8f64db18602769421b667e69ca3a77c29426
An Android-specific ioctl was added to support reclaiming memory from
the guest. Wire it up so that when balloon inflates, crosvm attempts to
reclaim/free the memory.
Bug: 395833312
Change-Id: I72769fabd003ba00c1848a456400d7c8ed2a8d97
Signed-off-by: Elliot Berman <elliot.berman@oss.qualcomm.com>
The Gunyah Resource Manager previously assumed the PC as the starting
address for the VM memory parcel, which is incorrect with demand paging
because payloads may start at non-zero offsets and the Resource Manager
cannot determine the correct start address. To support non-zero offsets,
a new GUNYAH_VM_SET_BOOT_CONTEXT ioctl has been added, allowing userspace
to set the register state (boot context) of the guest before starting the VM.
Bug: 376917595
Change-Id: Ie07dbebdbf3bbbdf40c78987c92343867eac887a
Signed-off-by: Mukesh Pilaniya <quic_mpilaniy@quicinc.com>
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6268024
Auto-Submit: Frederick Mayle <fmayle@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Replace open-coded reimplementations of these rounding-related math
functions where applicable (mostly identified as candidates for
div_ceil() by Rust 1.84 clippy, but in some cases, next_multiple_of()
was a better match).
Change-Id: Ib851689836cb8d44cde5a5766a3cec8fd5c90f40
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6195237
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Frederick Mayle <fmayle@google.com>
Construct a separate guest memory region for each of the traditional PC
memory layout low-memory areas:
- 0 to 640 KB (RAM)
- 640 KB to 1 MB (reserved for memory-mapped devices, BIOS, and ROMs)
- 1 MB+ (RAM up to the 4 GB PCI hole)
This simplifies the e820 memory map generation code (it just copies the
exact layout from guest_mem.regions() and translates the memory type
values) and ensures the layout provided to the guest matches the actual
GuestMemory region layout.
The memory_layout documentation is also updated to reflect the layout
more accurately; the kernel and initrd regions are contained inside the
above-1MB RAM region, not separate regions on their own.
BUG=b:375568070
Change-Id: I1f35694365a0ec9966b4d208190db103b48e9045
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6086957
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
The initial code for enabling SVE via the auto flag would check if SVE
is supported on the hypervisor. The result was the KVM ioctl advertising
the SVE capability, but a VM could be built without SVE support, which
would cause the VM ioctl to not advertise the SVE option, resulting in
failure of creation of the VM.
KVM API documentation suggests using the VM ioctl to adapt specifically
to the VM created. This patch replaces the KVM ioctl with the VM ioctl
call.
BUG=b:376260018
TEST=AOSP
Change-Id: I5062ee188131b3863a5b64c68c0348e471747476
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6215250
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Elie Kheirallah <khei@google.com>
Commit-Queue: Elie Kheirallah <khei@google.com>
The heavy use of `serde_json::Value` in the snapshot APIs makes it
difficult to experiment with other serialization frameworks. The vast
majority of the code just needs to supply a type that is serde
compatible and doesn't need to know what the format is.
On an aarch64 `--profile chromeos` build, the binary size barely changed
(it actually went down very slightly, -10kb, maybe because of the
equality comparisons I deleted), so I don't expect it to introduce any
overhead. The output files look the same.
Change-Id: Ie41d1bf6f6e3197566110774cfb5f20d00d85682
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6161187
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
Reviewed-by: Elie Kheirallah <khei@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Add auto to Sve config that checks if the system supports SVE and
enables it accordingly.
BUG=b:376260018
TEST=presubmit
Change-Id: I032d2d6b214f631839e0e8cc4812887771d307ac
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6180906
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Elie Kheirallah <khei@google.com>
Update the kernel version used to generate the bindings from Linux 6.6
to Linux 6.12, and remove the manual kvm_userspace_memory_region2
additions from kvm_sys/bindgen.sh.
The KVM_CAP_USER_CONFIGURE_NONCOHERENT_DMA capability index was changed
between ChromeOS v6.6 and v6.12, so a workaround to use the v6.6 value
is added; this maintains the existing behavior of querying cap index 236
on all kernels, which may return unexpected results on non-ChromeOS
kernels or ChromeOS v6.12 kernels.
BUG=b:388092267
TEST=tools/dev_container tools/presubmit
Change-Id: Icccc022005de37e12d552951eaadcf6c20ac914a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6150076
Reviewed-by: Ryan Neph <ryanneph@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
In older ChromeOS kernels with KVM_CAP_USER_CONFIGURE_NONCOHERENT_DMA
patched in, the associated KVM_MEM_NON_COHERENT_DMA flag may be passed
directly for ioctl(KVM_SET_USER_MEMORY_REGION).
In latest kernel, a 2-variant, ioctl (KVM_SET_USER_MEMORY_REGION2) was
added, and must be used for any flags beyond (KVM_MEM_LOG_DIRTY_PAGES |
KVM_MEM_READONLY).
To maintain compatibility with older patched ChromeOS kernels, and the
latest upstream kernel, we must attempt to use the 2-variant when it is
available and we are passing the KVM_MEM_NON_COHERENT_DMA flag, and
fallback to the original ioctl with the same flag, when the 2-variant is
not avaiable.
TEST=tools/dev_container tools/presubmit
TEST=Custom linux VM starts and runs vkcube with 6.6 ChromeOS kernel
TEST=ARCVM Play Store starts on brya-kernelnext, running ChromeOS 6.12-rc5 kernel.
BUG=b:369492345
Change-Id: I2aa366f50ff1eb0a3a994f8bd71cb4ed2f18c279
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6094778
Auto-Submit: Ryan Neph <ryanneph@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
This will be used to identify regions that should be backed by a
GuesMemory region but not reported as usable RAM on x86-64. Currently it
is functionally identical to MemoryRegionPurpose::GuestMemoryRegion.
Change-Id: Ibd93f7acd852b4bec17f3867578fcd1db79813d3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6086955
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Frederick Mayle <fmayle@google.com>
Add support for running protected VMs on x86 in the proper way, i.e.
similarly to arm64, ask the pKVM hypervisor to load the pVM firmware
(pvmfw) into the VM memory at a dedicated address and to set the VM
entry point to the pvmfw address.
The KVM uAPI used for that (for setting pvmfw address in guest memory
and for querying pvmfw size) is almost exactly the same as on arm64.
But as explained in [1], this is a temporary uAPI anyway, so for now
don't try to generalize the crosvm code using this uAPI between both
architectures.
[1] https://android-review.git.corp.google.com/c/kernel/common/+/3372231
[2] https://android-review.git.corp.google.com/c/kernel/common/+/3372235
BUG=b:350694931
TEST=a VM run with "crosvm run --protected-vm" runs successfully,
assuming that the bootloader provides pvmfw image to the host kernel, or
using a pKVM hack described in "Test:" in [2] instead.
Change-Id: I64242bcd0a4c053b8c27c2f83d5c876a885d0442
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6043709
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dmytro Maluka <dmaluka@chromium.org>
Reviewed-by: Pierre-Clément Tosi <ptosi@google.com>
Add support for running protected VMs with pKVM on Intel x86 [1].
The --protected-vm option is not workable yet, since loading pvmfw into
guest memory via pKVM is not implemented yet, but the developer options
--protected-vm-without-firmware and --protected-vm-with-firmware are
working.
Following the approach used for pKVM on arm64, the KVM uAPI used for
running protected VMs on x86 is a "stable temporary" uAPI: the
KVM_X86_PKVM_PROTECTED_VM value shall be updated when upstreaming pKVM
to the mainline kernel (see also [2]).
[1] https://android-review.git.corp.google.com/c/kernel/common/+/3351287
[2] https://android-review.git.corp.google.com/c/kernel/common/+/3351286
BUG=b:349990461
TEST=On an Intel device with kernel compiled with pKVM-IA patches [1]
and kvm-intel.pkvm=1 added to kernel command line, run a VM with
--protected-vm-without-firmware or with --protected-vm-with-firmware
with the same test pvmfw image as in TEST= in CL:5797353.
Change-Id: I625f5eb9a38eaef9312ba62308739efb66f163b0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6022852
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dmytro Maluka <dmaluka@chromium.org>
The conversions between the three different register ID representations
are tricky. Add a cheap assert to make sure it works as expected.
Change-Id: Ic1ec75e4c7eba6508dd76dbcf80fc05bc0b74766
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6024123
Reviewed-by: Elie Kheirallah <khei@google.com>
Commit-Queue: Frederick Mayle <fmayle@google.com>
Refactor `init` and add function `get_features` to get the Vcpu features.
Previously some features were only retrieved as part of `init`.
Add function `finalize` which is required for SVE support. SVE requires
`KVM_ARM_VCPU_INIT` to be called first, which gives access to the SVE
config, specifically `KVM_REG_ARM64_SVE_VLS`, which is a
pseudo-register that allows discovering and configuring the vector
lengths by userspace. After `finalize` is called,
`KVM_REG_ARM64_SVE_VLS` is no longer accessible, and the config is
final. In this patch, we are not modifying the vector lengths, reason
behind this is `KVM_ARM_VCPU_INIT` initializes `KVM_REG_ARM64_SVE_VLS`
to the best set of vector lengths that the host supports.
BUG=b:375372656
TEST=presubmit
Change-Id: Ibd0902ad68f28b9d4e9a595070fe972e620e5572
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5973935
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Elie Kheirallah <khei@google.com>
Previously, the Vcpu handle_io() and handle_mmio() functions used an
IoOperation containing a fixed-length data array to represent a write
and returning a fixed-length data array to represent a read, along with
a separate size field to indicate how much of the fixed-length array
should be read/written.
This change uses Rust slices to represent the I/O data instead:
- Write contains a &[u8] of data to be written.
- Read contains a &mut [u8] to be filled with the read data.
The new IoOperation matches the Bus read()/write() APIs more closely,
and avoids the need for hypervisors and callers to convert between
fixed-size arrays and slices.
The Bus::read() function now always initializes the data slice before
(potentially) calling a device's read() function. This ensures
consistent results even if a device does not always fill out every data
byte (for example, the default BusDevice read() handler that is a no-op)
or if no device is found. This replaces the previous zeroing that would
happen when initializing the read data array to return from handle_fn.
Without this, the data slice may have had stale data from the previous
MMIO/IO exit, depending on the hypervisor implementation.
No functional change intended.
BUG=b:359382839
TEST=tools/dev_container tools/presubmit
Change-Id: Id88ebfa7ece5cc7466c010db2cbde303aeb97bf8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5913962
Reviewed-by: Vaibhav Nagarnaik <vnagarnaik@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
For repeated string port I/O instructions (REP OUTSB, REP INSB, and
similar for other data sizes), the size field specifies the size of one
operation, and count specifies the number of times to repeatedly
read/write the *same* port with *different* data. This was fixed for KVM
in commit 063708084c ("hypervisor: kvm: handle INS/OUTS VM exits");
apply the same fix to HAXM.
BUG=b:283990685
Change-Id: Iaf4cfe12fba816730030368651db0e6fb116e50a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5798663
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Judson Powers <judsonp@google.com>
When shared refs are used, the compiler may generate code that doesn't
witness the new value written via the pointer.
Change-Id: I6354fc3af749eaf7aaf1369c30b273371047d351
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5893919
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
This was added in the original hypervisor abstraction layer since it was
in the kvm crate, but it isn't actually used anywhere in crosvm.
BUG=None
TEST=tools/dev_container tools/presubmit
Change-Id: Ie1eacbe90771b050e3a60a0c868362d2d454d99e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5660939
Reviewed-by: Richard Zhang <rizhang@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
For GZVM_CAP_PROTECTED_VM at least, the GZVM_CHECK_EXTENSION ioctl
ovewrites its parameter with the result rather than using the return
value, so its parameter needs to be a mutable reference.
Change-Id: Ief9f153804ebb57e2b247f2b6a5f04b429c77a7d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5847469
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Frederick Mayle <fmayle@google.com>
As pKVM and pvmfw development for x86 is ongoing, add initial support
for running VMs with pvmfw as the VM bootloader. For now only support
the --unprotected-vm-with-firmware development flag, for running regular
non-protected VMs with pvmfw, which doesn't require pKVM support.
Similarly to ARM, add a dedicated memory region for pvmfw to the x86-64
memory map. This region is located right below the PCI MMIO memory hole.
Its size is 4MB, like on ARM.
Similarly to ARM, the pvmfw entry point is at the beginning of the pvmfw
image, the normal x86-64 boot protocol is kept as is, and additionally
the VM payload (i.e. kernel) entry point is passed to pvmfw in the %rdi
register. This ABI is a subject to change, the final ABI for x86 pvmfw
is not defined yet.
BUG=b:354676864
TEST=1. Run a VM with "--unprotected-vm-with-firmware pvmfw_test.bin"
where pvmfw_test.bin begins with the "jmp *%rdi" instruction (2 bytes:
0xff 0xe7), i.e. pvmfw immediately jumping to the VM entry point.
Result: VM runs as usual.
2. Run a VM with "--unprotected-vm-with-firmware pvmfw_random.bin" where
pvmfw_random.bin contains just random data. Result: VM crashes.
Change-Id: Ib0ee998a99a0cf352a97222769b87be615419187
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5797353
Commit-Queue: Dmytro Maluka <dmaluka@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Lazy-init of VirtioPciDevice shared memory region (to setup hypervisor
mapping) would previously use the MemCacheType seen on first mapping,
which could be anything -- this behavior is never desired, but grew from
a later addition of MemCacheType workaround to the existing Virtio "Fixed
Mapping" optimization.
On devices where the single hypervisor mapping's MemCacheType matters
(e.g. for devices that must configure it with
MemCacheType::CacheNonCoherent), if the first mapping attempted is for a
shmem with WB caching, all later mappings with WC or UC memtype would be
configured incorrectly.
Instead, query the VirtioDevice Trait implementator whether lazy-init of
a single hypervisor mapping should be used (the default), and with which
MemCacheType.
Attempting to later add a `CacheNonCoherent` mapping for a device that
explicitly declared `SingleMappingOnFirst(CacheCoherent)` is invalid,
and would lead to bugs, so we now treat this as an error and fail the
mapping altogether.
VirtioGpu device implementations will use this for devices with either
mandatory or optional-but-enabled non-coherent DMA (e.g. Intel devices
without coherent LLC shared between CPU/GPU, or that may opt to bypass
LLC coherency for optimal perf).
BUG=b:360295883, b:246334944
TEST=Builds
Change-Id: If41d238fd3c220e45c61d78da4a2505572709053
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5871721
Commit-Queue: Ryan Neph <ryanneph@google.com>
Reviewed-by: David Stevens <stevensd@chromium.org>
Some doc comments were being incorrectly interpreted due to missing
indentation (for intentional line contintuations) or line breaks (for
new lines that were intended to be separate paragraphs).
Clippy warns about these as of Rust 1.80:
<https://rust-lang.github.io/rust-clippy/master/index.html#/doc_lazy_continuation>
Fix them along with some other nearby minor formatting cleanups.
TEST=tools/cargo-doc
TEST=tools/clippy # with rust-toolchain 1.80
Change-Id: Ice0b7cc3bd75d9ab08c10107a13f95ca9f87a0a3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5758934
Auto-Submit: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
The KVM code that retrieves MSRs was using ioctl_with_ref(), which is
supposed to have a read-only reference, but KVM_GET_MSRS needs to write
via the provided reference.
Change-Id: I2f0972463c99d7d97f19834d23040ab49bbee20d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5847468
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Elie Kheirallah <khei@google.com>
`KVM_CREATE_DEVICE` turns into an `ioctl` that mutates `device`, so
`device` must be declared as `mut`. Rust 1.78 forwards `device.fd == 0`
to the `Ok(...)` return, which made this incorrect code more visible.
BUG=b:361302729
TEST=Built & ran with Rust 1.78.
Change-Id: Idf99228b67c657351681adcaec563dd021fbe451
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5838113
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>