mirror of
https://github.com/Droid-VM/crosvm.git
synced 2026-11-17 06:58:03 +00:00
7 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
fe24182b62 |
virtio-media: MediaCodec decoders within a pixel budget, asked to decode as fast as they can
A 4096x2160 hardware decoder holds about 500 MB of the host's memory in its own output buffers, which no guest memory limit bounds: macOS's lock-screen video opened a new 4K session every few seconds without closing the last, and five took 2.5 GB in half a minute until the host's low-memory killer ended the VM. The hardware decoders of the process now share a budget of two 4K pictures (CROSVM_MEDIA_DECODE_MAX_MPX megapixels); a session past it is refused at STREAMON(OUTPUT) with EBUSY, which a guest takes as a decoder it cannot have. A guest's timestamps are its own (macOS's VideoToolbox host sends frame numbers a second apart), so a codec setting its clocks by them read 1 fps and a 4K Main10 stream came out at ~25 fps: KEY_PRIORITY 0 at the highest KEY_OPERATING_RATE, and configure retried without the two keys where a component refuses them. KEY_MAX_INPUT_SIZE is capped at 2 MiB (23 x 6.3 MiB of input dma-buf per 4K HEVC session before); a larger access unit goes in over several buffers as before. |
||
|
|
8f9ffc3f92 |
media: put the QTI AVC decoder in decode order so the reorder cannot deadlock a client (D91)
An adaptive/MSE H.264 client wedges against c2.qti.avc.decoder a few frames into a stream: it feeds 7-8 access units, then blocks waiting for the picture it has just submitted, and that picture is still inside the decoder's display-reorder window. D85 measured the client side (8 AUs in, 5 frames out, then an idle drain and "Stateful sync failed"); D91 measured the decoder side directly, host-direct with an NDK probe and no VM, and found the two numbers meet exactly at the boundary. The metric that decides it is self-delay -- how many further AUs have to be fed before the picture for AU i comes back. On a High-profile 1080p stream the QTI AVC decoder's default is p50 4, p95 7, max 7 (54 of 300 pictures need the full 7), and 3 pictures are never released at all without an EOS. A client whose feed-ahead budget is 7-8 therefore has no margin, and its only way out is DEC_CMD_STOP -- a drain that resets the DPB and cuts the reference chain, which is why D85 ends in a broken stream rather than a stall. vendor.qti-ext-dec-picture-order.enable=1 removes the cause instead of working around it: the component stops holding pictures for display order and emits in decode order. Measured on the same stream: self-delay p50 = p95 = 1 (295 of 300 pictures return on the same step), max 5 and only during the opening burst, nothing held to EOS; nonmono 157 -> 0; the EOS tail 3 -> 0; 300/300 frames out with no errors. The pictures themselves are untouched -- per-frame md5 aligned by PTS is identical in 300 of 300 frames, and the negotiated output format is byte-for-byte the same (stride, slice-height, buffer counts, crop, colour format), so nothing downstream in libva or the driver sees a new surface. Decode order is in any case what this backend wants: every CAPTURE buffer is claimed by its access unit's own timestamp tag and the VA client reorders by POC itself, so display order is work the guest side has to undo. What this is not: KEY_LOW_LATENCY and the c2.qti.avc.decoder.low_latency sibling were both measured and both are inert here -- neither moves first output, steady lag, nonmono or the EOS tail by a single frame, and the sibling additionally drops the bitrate ceiling from 220 to 70 Mbps and halves blocks-per-second. KEY_LOW_LATENCY is left exactly as it was (set only when the component advertises FEATURE_LowLatency, which plain AVC does not), and no drain or duplicate-AU trick is needed. Two properties of the key shape the code. It is only honoured in the MediaFormat passed to configure(): an AMediaCodec_setParameters after start() returns success and does nothing. And the component does not echo it on the input format, it reflects it on the output one -- reading the input format is what made an earlier probe record this key as unsupported. So the ledger reports the value read back from output_format() after start, not the value we set (configure() returning OK proves nothing about a vendor key; unknown ones are dropped in silence), and the "started for ..." line gains a trailing ", picture-order on|off" with everything before it unchanged. Absent or unreadable reads as off. Scoped to video/avc. The key also survives readback on HEVC, VP9 and AV1, but its effect there is unmeasured, so those components keep the behaviour they ship with today. For the deploy WP, the bar: every AVC session's ledger line must say picture-order on; h264.html must stay at a single session 300 in / 300 out with gbm-dmabuf and zero idle drains; Firefox ESR on adaptive High MSE must reach in == out in one session with zero idle drains, zero "Stateful sync failed" and no software fallback; and the three H.264 canonical clips must stay bit-exact (bf32f00e, d0c7f223, 3db3538b) -- the decoder now hands libva-v4l2 its buffers in a different order, and that claim_locked path is the one thing D91 could not exercise host-direct. VP9 and AV1 must be bit-identical, unchanged by construction. Investigated in logs/vpu_wp/D91-lowlat-probe.md (§2-§6), with the client side in logs/vpu_wp/D85-drain.md. Not built or deployed here; the orchestrator builds crosvm and a later WP verifies on the phone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011BkN7vYhuc1w9iVuqsHzSn |
||
|
|
b60d066aa4 |
android_codec: release_output ends the output borrow, and a bottom-up layout is refused, not indexed
Two review-m6 findings in the crate (logs/vpu_wp/review-m6.md). R6-13 (nit): Codec::output_buffer returned a slice borrowed from &self while release_output also took &self, so `let src = codec.output_buffer(i)?; codec.release_output(i)?; use(src)` compiled: a use after release expressible with no unsafe. release_output now takes &mut self, which ends every output_buffer borrow at the release by construction; the two callers in devices (pump_output, take_output) and codec_probe already hold the codec mutably, and are adjusted here only where the signature requires it. R6-9 (minor): MediaImage2::plane_end clamps a negative increment out of its sum, so check_bounds looked at the first row only of a bottom-up image-data layout (which the format allows) and tight_nv12_rows then computed a row start below zero, cast it to usize and panicked on the index -- in the media helper, i.e. the VM down -- where the function's contract is that nothing is emitted for a refused layout. A plane_start companion finds the lowest byte a plane reaches, and check_bounds refuses a layout that reaches before the buffer with the new ImageError::BelowStart. A bottom-up layout whose offsets account for the walk is still walked, rows in the order the picture has them. Not observed on 5566 (B5 §5.2: every rowInc positive); the codec controls the value, not the guest. Test: a_bottom_up_layout_is_walked_or_refused_never_indexed_below_zero (31 passed in deploy/vpu/harness.sh acc). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011BkN7vYhuc1w9iVuqsHzSn |
||
|
|
a72caf6cd4 |
android_codec: codec_probe takes hex levels, keeps its banners off stdout, and leaves through _exit
Three B5 findings (logs/vpu_wp/B5-acceptance.md §6). --level 0x1000 as M6-probe's own recipe spells it was refused with "invalid digit found in string" (D25 item 1): --level and --profile now take the 0x form MediaCodecConstants.h writes them in, and the decimal one still. The two banner lines (running as uid, libmediandk loaded) went to stdout ahead of the --json document, so `list --json > f.json` was not JSON (D25 item 2): they go to stderr. And `list` returned 139 after printing everything (D24): AMediaCodecStore's file-scope vector of AMediaCodecInfo is destroyed by __cxa_finalize at exit(), after its allocator arena, and the process dies inside libmediandk. After a Store walk main flushes stdout and stderr and leaves through _exit, which runs no destructor; the exit code is what it was. The other commands never touch the Store and exit as before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011BkN7vYhuc1w9iVuqsHzSn |
||
|
|
25a803358e |
android_codec: an event carries the generation it was posted under, and the encoder's keys
B5 found the HEVC decoder handing out a dead output index after flush() + start() in five of seven runs (logs/vpu_wp/B5-acceptance.md D23): a callback posted before the flush was delivered after it, and a getOutputBuffer on its index returned null. The queue now stamps every event with a generation that flush_and_restart bumps under the queue lock -- after AMediaCodec_flush has returned, so an event posted while the flush ran is old too -- and take_events hands out only the current generation, counting what it dropped (Codec::stale_events). The stamp cannot catch a callback the NDK looper had already queued before the flush: that one arrives afterwards, with the new generation and a dead index, and the docs now say what a consumer does with the null it gets (skip it; never a session error). A unit test drives the queue directly, since a Codec needs the NDK and the generation logic does not. Also the constants the encoder backend configures with: COLOR_RANGE_*, COLOR_STANDARD_*, COLOR_TRANSFER_* (MediaCodecConstants.h:1036-1045) and the three encoder keys that have no AMEDIAFORMAT_KEY_* symbol (prepend-sps-pps-to-idr-frames, video-qp-min, video-qp-max). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011BkN7vYhuc1w9iVuqsHzSn |
||
|
|
7b567e8a71 |
android_codec: let a consumer wake its own descriptor, and repack NV12 a row at a time
Three additions the MediaCodec decoder backend (M6-backend) needs, none of which changes what codec_probe does. A wake hook on the async queue. The fork's decoder device keeps the poll descriptor on the device session and hands the backend a DecoderSink to bump (M6-crate section 2), while Codec's four callbacks bump the crate's own eventfd. Rather than a thread sitting between the two, Codec::set_wake_hook installs a closure the callbacks run after queueing their event -- one eventfd write, nothing else, as NdkMediaCodec.h:506 asks of a callback -- so the whole session can run on the device's worker thread. tight_nv12_rows, the same plane walk as tight_nv12 handing out one row at a time. The backend writes into a CAPTURE buffer the guest also maps, and does so through a raw pointer rather than a slice over guest-visible memory (the camera backend's rule); a Vec destination would have meant a second copy per frame. tight_nv12 is now the Vec wrapper over it, so the nine existing tests cover both. queue_input_with_flags: the fill closure returns the buffer flags along with the bytes used. Whether a bitstream buffer is parameter sets alone (BUFFER_FLAG_ CODEC_CONFIG) is decided on the bytes once they are in the codec's own buffer, never by reading the guest's memory twice. MediaImage2::semiplanar is codec_probe's assumed_nv12 moved into the crate, so the backend's fallback for a component that publishes no image-data is the probe's. Two tests added (29 in the acc harness). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011BkN7vYhuc1w9iVuqsHzSn |
||
|
|
423ddccebf |
android_codec: the MediaCodec NDK crate and codec_probe (M6, design 7.2)
The decoder and encoder devices (VPU_DESIGN.md 7.2, 7.3) must map the phone's real codec capabilities, not a table, and must drive MediaCodec in asynchronous mode from a device worker. Before either device exists, a standalone probe has to enumerate what the platform reports and prove decode and encode round trips through the NDK under the app uid. This is that crate, the android_camera pattern applied to libmediandk: - lib.rs: ndk_api! bindings resolved with dlopen (libmediandk, libbinder_ndk; the binder thread pool is started at load because MediaCodec serves a BnResourceManagerClient). Everything up to API 28 is required; the API 36 AMediaCodecStore_* / AMediaCodecInfo_* / ACodec*Capabilities_* surface and the AMediaCodecInfo_FEATURE_* data globals are optional groups, so the library loads on an older Android and list_codecs falls back to create*ByType per mime. list_codecs walks the Store (media types, every decoder and encoder with kind/type/vendor/instances, video ranges and alignment, per-size rate checks, performance-point coverage, encoder quality/complexity/bitrate modes, features) and asks isFormatSupported about a candidate profile/level table, since the NDK has no getProfileLevels (survey 2.3). Codec is one session: the four async callbacks only push onto a mutex-protected queue and signal a base::Event eventfd (design 7.2's rule), with flush_and_restart for the async seek. MediaFormat wraps AMediaFormat with the keys the survey names, including the literal "image-data". - image.rs: MediaImage2 parsed from the 104-byte "image-data" blob, and tight_nv12, the one per-frame repack the decoder device will make (memcpy rows for NV12, byte swap for NV21, interleave for planar, generic walk otherwise), plus rebased() for survey open question 1. - bitstream.rs: Annex-B access-unit splitter (H.264/H.265, spec boundary rules, csd extraction) and IVF reader/writer with VP8/VP9 key-frame bits. - synth.rs: the moving test pattern, the padded-layout packer for encoder input (stride, slice-height, semiplanar or planar), luma PSNR and digest. - probe.rs: codec_probe list [--json] / decode / encode / roundtrip, with --uid as camera_probe, --csd config|format|inband, the mid-stream flush-then-start, the mPlane[0].mOffset verdict, and a >= 30 dB luma PSNR round-trip check. Workspace member; Android.bp mirrors android_camera (installable: false, apex_available com.android.virt, no libmediandk in shared_libs, phony codec_probe). Host build, 27 unit tests, clippy and rustfmt pass in the scratch harness (logs/vpu_wp/scratch-m6); nothing has run on a phone yet. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011BkN7vYhuc1w9iVuqsHzSn |