0
Fork 0
mirror of https://github.com/Droid-VM/crosvm.git synced 2026-11-17 06:58:03 +00:00
Commit graph

58 commits

Author SHA1 Message Date
Daniel Verkamp
b952f534a0 dev_container: pull crosvm_dev_base when building crosvm_dev
When building the crosvm_dev container on a system that did not build
crosvm_dev_base, it is necessary to pull the published image first.

BUG=b:395695390,b:382504192
TEST=make -C tools/impl/dev_container crosvm_dev

Change-Id: I7b5f49b325adf1ac80082596df851bd4b302c366
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6257030
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2025-02-12 11:02:44 -08:00
Dennis Kempin
964a4a9501 tools: Separate debian packages from other setup
This allows us to create a base container with just debian packages
installed, which we can pin as we uprev other parts of the dev container
(e.g. rust toolchains).

This prevents a frequent annoyance that new debian testing packages
introduce new problems into the build infra.

To make this happen, the setup scripts have been split and
moved to tools/deps, with shorthands in tools/setup*.

BUG=b:382504192
TEST=presubmit with new container
TEST=tools/setup* on local workstation

Change-Id: I0561378c76a1de026c895dfd4052ced02cb3fe8e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6153816
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2025-02-11 15:10:36 -08:00
Dennis Kempin
b1730b36b6 dev container: Uprev to latest debian version
Updates both container and testvm to the latest version and
fixes issues arising from that.

- Uprev some runtime dependencies in cloud_init
- Fix testvm Makefile to use same directory for artifacts
  as testvm.py
- Run ext2 test_mkfs_xattr as sudo to allow modification of selinux
  attributes
- Re-format a couple of files with latest versions of formatters
- Use `cp -u` instead of rsync, because for some mysterious reason
  rsync gets permission errors in CI that cannot be reproduced
  locally.

BUG=b:395695390
TEST=dev_container presubmit --no-delta all

Change-Id: Idff5bf4b15c21ecce883ae9213c50caa4f548353
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6254015
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2025-02-11 15:10:32 -08:00
Dennis Kempin
fb512a85b2 Add aarch64-linux-android target
This allows cargo to build for android. A new
dev container is built to include this.

To allow us to build the new container, we had to
remove the ffmpeg dependencies for armhf which fail
to install in the current version of debian.

BUG=b:349907813
TEST=dev_container presubmit clippy_android

Change-Id: I7fbcd460e53fbd339a28b451a91b0722d34257b8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5689659
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2024-07-11 17:35:22 +00:00
Daniel Verkamp
8c2a5647d1 rust-toolchain: update to Rust 1.77.2
This also upgrades the dev_container to package the new Rust version.

A few final clippy fixes are included as well.

BUG=b:344974550
TEST=tools/clippy

Change-Id: Iab4253e9cd88fdae989bb5b5170040b6af85baca
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5609084
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2024-06-18 18:21:17 +00:00
Dennis Kempin
38d1ac5b1b Rebuild testvm and devcontainer
This uprevs both testvm and devcontainer and uploads new builds.
This will update them to the latest debian images and packages.

Some minor tweaks were made to satisfy the newer versions of
mdformat and python.

BUG=None
TEST=presubmit all --no-delta

Change-Id: I73e30aa1d9cefb5cf4ac117f513c2a98bf3c9ac5
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5416093
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2024-06-11 22:24:48 +00:00
A. Cody Schuffelen
ecc0206af8 Upgrade rust to 1.73.0
There don't appear to be any new clippy warnings, and this is the
version of Rust that Android is now using: b/303252546. Work for the
next version is tracked in b/310977762.

This doesn't have any particular features I'm looking for (async traits
only come in 1.75.0). I'm not particularly attached to this change. If
it's easy to upgrade the container though, we could ensure no new clippy
warnings appear.

Android rust versions appear here:
https://cs.android.com/android/platform/superproject/main/+/main:prebuilts/rust/linux-x86/

Also not sure what the ChromeOS release schedule is.

Release docs: https://releases.rs/docs/1.73.0/

Test: tools/dev_container --clean bash -c "rustc --version && tools/presubmit clippy"
Change-Id: I2d3010c3032053709f0c0beb7dcbe2b80d782415
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5201659
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2024-02-20 19:11:51 +00:00
Dennis Kempin
e45905b53e dev_container: Fix libc6:arm64 issue during container build
libc6:arm64 will call telinit to restart affected services.
This will block indefinitely in the container without init running.
The workaround is to replace telinit with true to make it a
no-op.
Also removes the mount caches from the build process as they seem
don't seem to work with APTs file lock.

BUG=b:322015733
TEST=make -C tools/impl/dev_container

Change-Id: If1079581680ac9edc981ad46afb3173508668335
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5230818
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2024-02-20 19:11:51 +00:00
Dennis Kempin
518058007c dev_container: Clarify documentation for rebuilding container
Consolidates the instructions into a README.md file to make it
easier to discover.

BUG=b:273481336
TEST=None

Change-Id: I3b7a4a204ba5802bcfdb24db5a808fadce9c757e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5123312
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-12-15 18:58:31 +00:00
Dennis Kempin
a5b5e4b9a6 Enable rustfmt nightly in container and CI
This enables nightly rustfmt features in CI so we can keep a consistent
imports style.

The resulting formatting changes are in the follow-up CL and will
need to submit together.

This change also uncovered a bug from a previous commit that caused only
the first command in a series to be waited for. Formatting issues would
not be reported by the tool. The bug is fixed as part of this CL.

BUG=b:302055317
TEST=dev_container presubmit format --no-delta

Change-Id: I5d50609cc89d7e78bce66db120737a302f5c57e8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4950268
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2023-11-29 18:41:29 +00:00
Dennis Kempin
5cf0a98827 dev_container: Build new container without riscv
Also adds lld, which will be used in a follow-up to improve
linking performance.
Since debian has had a stable release since then, we also
need to upgrade a few library versions in the testvm.

Swap tests that require userfaultfd were able to run without
root before. In the latest version of debian we need to configure
for unprivileged access.
Since this is the case for glinux workstations as well, let's
mark those tests as root-only and let them execute via sudo.

BUG=b:304875018,b:256905223
TEST=presubmit

Change-Id: Ied0b34e248990378a7864b4a394aa21d8f824f71
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5013631
Reviewed-by: Zihan Chen <zihanchen@google.com>
Reviewed-by: Shin Kawamura <kawasin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-11-20 18:34:35 +00:00
Dennis Kempin
8656c2f8c3 Upgrade to 1.68.2: Set default version and rebuild container
The new toolchain is now used in CI and as default for developer builds.

The mingw ldd hack is no longer needed as debian has released a fixed
package.

BUG=None
TEST=CQ

Change-Id: I713d2901165ec33efe301e79819ef31ab86fee48
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4568145
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2023-05-25 18:30:55 +00:00
Dennis Kempin
55c171466d Include riscv toolchain in dev container
Updates the riscv install script to use the debian ports repository
to install dependencies.

The container version is upreved to use the newly built image
that includes the changes in this CL.

BUG=b:280448021
TEST=dev_container run_tests -p riscv64
TEST=https://ci.chromium.org/swarming/task/61fca4b26ac4de10

Change-Id: I7dbfac56fd04c8fd9635c981c13dcfcbdad90bcc
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4500611
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-05-04 15:21:31 +00:00
Dylan Reid
36af53e278 dev_container: Add riscv64 support
Because bullseye's cross packages for riscv64 are broken, libcap needs to
be manually built and installed, and pkg-config needs a hacky define to
allow minijail to find it. Other than that, it's very similar to the
other architectures.

Change-Id: I903f7ccd2ccbd94168ba09b425fd649458fdee70
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4460942
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-04-26 17:36:40 +00:00
Dennis Kempin
e577c91efc dev_container: Always run as root inside container
Since we default to podman, we can get rid of the _user container and
run as root inside the container, which greatly simplifies the tooling.

This also allows us to provide a single, default devcontainer.json
for local development as well as codespaces.

podman can be used with this devcontainer.json as well, e.g. in
VSCode by setting:

"dev.containers.dockerPath": "podman"

The with_kvm/devcontainer.json can be used to add access to kvm, which
may not be available on all machines. So it's an optional config.

BUG=b:273529620
TEST=Open with devcontainer in VSCode with both docker and podman,
run `presubmit --no-delta`
TEST=tools/dev_container tools/presubmit --no-delta

Change-Id: I10e04898a0e1adb6e4939f6f2e0259dc9b9aa62d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4419666
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-04-25 21:33:42 +00:00
Zihan Chen
894cf79ba0 dev_container: Add strace to dependency
strace is used to capture seccomp profile during benchmark runs.
This CL also added /usr/lib/wine to PATH as debian has decided
to remove /usr/bin/wine64 alternative entry.

TEST=./tools/bench --log-seccomp --log-seccomp-output-dir /workspace/seccomp_result boot
TEST=./tools/run_tests --dut=host

BUG=b:258316090

Change-Id: Ie362149ef77474ad7d9c9ff918a990d386d7a5ff
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4358600
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-03-28 18:25:12 +00:00
Dennis Kempin
20f0604494 dev_container: Add latest tag when pushing images
This will allow devcontainer.json files to always use the latest
container without manually upreving the version.

BUG=b:273529589
TEST=open dev container in vscode

Change-Id: I71d58eb2936097bd81dfcc0872d362b2c6b9e388
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4341238
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-03-15 21:02:13 +00:00
Dennis Kempin
050bc63556 dev_container: Install g++
Required by https://crrev.com/c/4333206

BUG=None
TEST=make -D tools/impl/dev_container

Change-Id: I82c07141bc4b103f66a984edd91abc1631d39c9d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4337397
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-03-14 18:20:36 +00:00
Dennis Kempin
f48639f3fe Automatically register tools/ on PATH
Developers with `direnv` installed will have ./tools added to their
PATH automatically. Also adds it to the PATH in the dev container.

This reduces `tools/dev_container tools/presubmit` to
`dev_container presubmit`.

Note: Does not rebuild the container, the change can be picked
up at a later time.

BUG=None
TEST=cd crosvm

Change-Id: Ia57bd7bb1cf86df2b32d624d97a74518e8b5f423
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4305345
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-03-03 00:33:15 +00:00
Dennis Kempin
65d0436acc testvm: Use CROSVM_CACHE_DIR variable for storing VM images
They are currently stored in a temporary directory, which is fine
locally, but will not be preserved between runs in CI. So each
CI build has to re-download the files.

BUG=b:254924880
TEST=tools/dev_container tools/x86_vm run

Change-Id: I49916887d5cb2d4c23c315a3d8ee369de1cba88f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4287767
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-02-28 19:32:24 +00:00
Dennis Kempin
c1456fc1de tools: Do not use pip to install python packages
Debian no longer allows us to install packages into the system
managed location.

So we are replacing the pip packages with the debian packaged
versions.

Also removes the wine64 hack which is no longer needed.

This allows us to build a new dev container again.

BUG=b:270568440
TEST=presubmit

Change-Id: I592ebf13bc566785fcdf4a4fa4badb3e11c8126e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4285744
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-02-27 19:24:18 +00:00
Zihan Chen
28ce4e5423 crosvm: Uprev rust toolchain and dev container
- Rust toolchain is updated to 1.65.0
- Catapult dashboard upload tool is added to dev_container
- Bindgen is updated to latest version to support custom derive
- Derive Eq when PartialEq is derived as required by new Clippy

TEST=CQ, bindgen-all-the-things

FIXED=b:260784028
BUG=b:257303497

Change-Id: I2034cd09e0aed84d4e9b30f2e85d84d94a442ea4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4228427
Auto-Submit: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2023-02-08 20:26:30 +00:00
Dennis Kempin
108a335d7d toolchain refactoring: Only support native out of the box
cross-compilation requires additional configuration that depends
heavily on the host platform. Unfortunately cargo cannot be
configured on a per host platform basis so this has to be done
manually.

Most developers are just using native compilation (and using
the containers for everything else). So we can slim our
install-deps scripts a bit.

To verify the updated install scripts, this change includes
containers to simulate a fresh developer workstation that
can be used to verify out of the box workflows.

BUG=b:262829206
BUG=b:265995780
BUG=b:265842137
TEST=tools/contrib/minimal_container/test-all.sh and CQ

Change-Id: I0e803d0224306e5351728e2c80fcbcd8260d029d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4179290
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-01-24 17:33:29 +00:00
Zihan Chen
8458188515 dev_container: Add mdformat-tables to dependency
Now we are using markdown tables in our docs, and mdformat without
extension will do horrible things to them as tables is a GitHub
Flavored Markdown extension.

TEST=./tools/dev_container ./tools/fmt

Change-Id: I735284136a87ea17752bd6732885ff1f4f58a3f4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4026017
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Zihan Chen <zihanchen@google.com>
2022-11-15 22:36:38 +00:00
Frederick Mayle
e0ab99da0c dev_container: fix arg forwarding in entrypoint.sh
Without this change, the command

    ./tools/dev_container cargo watch -s "cargo check --workspace"

would end up as

    cargo watch -s cargo check --workspace

inside the docker shell.

Change-Id: I79fed9d23804699032adf359a24d7250c1124b21
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4021752
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2022-11-10 23:34:53 +00:00
Ryuichiro Chiba
df7a2e51bf e2e_tests: Add a tool to dev_container and guest rootfs
Add a tool used in vsock tests to dev_container and guest rootfs

BUG=b:254375643
TEST=`./tools/presubmit` with the new guest rootfs on the new
dev_container

Change-Id: I135e6d2d413bedffef0fd69edd4ab14080429e3a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3974349
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Takaya Saeki <takayas@chromium.org>
Commit-Queue: Ryuichiro Chiba <chibar@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2022-11-07 04:00:40 +00:00
Dennis Kempin
b50f6580a2 test_config: Cleanup and enable some video crates
Some of these flags were no longer necessary since we now split
between unit and integration tests.
Others could be enabled by adding some missing dependencies.

BUG=None
TEST=presubmit

Change-Id: Ife5b09a4ee8b27885ee2042d5d93447e8d4deedd
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4004345
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-11-03 23:03:54 +00:00
Dennis Kempin
389a02462d dev_container: Install tmux and mold
tmux can be used to run presubmit checks in parallel:

  $ tools/presubmit --tmux

And mold is a faster linker that will be used for experiments.

BUG=None
TEST=None

Change-Id: Ie332d7285aa94d9072e201a3cac825337ae825cb
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3997872
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2022-11-02 01:56:48 +00:00
Dennis Kempin
b28d5a963d Fix winetricks install and build new container
Uprevs the container version to include the newly added
winetricks.

Fixes /dev/kvm permissions.

BUG=None
TEST=CQ

Change-Id: I019178f103068d3ae0dc048180a14c8786e738e0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3994884
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
2022-11-01 19:04:35 +00:00
Vikram Auradkar
2653304c8e tools: Install winetricks
This will help us running audio tests under wine.

BUG=b:237011316
TEST=presubmit

Change-Id: Iba297159291abd135fb1972a19fa5b5c216fa956
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3971028
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-10-27 18:11:46 +00:00
Zihan Chen
988858b66b infra: Add build_chromeos_container builder
Add a new builder to build crosvm in crOS tree, and all the
depencies of this new builder.

BUG=b:240692674
TESTED=led get-builder luci.crosvm.ci:chromeos_amd64-generic | led edit-cr-cl https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3966928 | led edit-recipe-bundle | led edit -r build_chromeos_hatch | led launch

Change-Id: Id2f284139922916edd2dd584f576da9fb3445518
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3966928
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2022-10-26 23:05:08 +00:00
Dennis Kempin
bf4cad131d reland: dev_container: Fix podman and enable unprivileged containers
reland note: Added wineboot for Dockerfile.user, since it prepares
directories for wine on a per-user basis.

To enable podman, the Dockerfile has been split into a root
run Dockerfile and one that adds a non-root user.

The following combinations have been tested:

./tools/dev_container -v --clean --podman --unprivileged
./tools/dev_container -v --clean --podman
./tools/dev_container -v --clean --unprivileged

And warnings have been added to ensure users are aware that
the only fully supported variant is running a privileged
docker container:

./tools/dev_container -v --clean

The unprivileged containers will allow us to validate if
unit tests require privileged system access.

BUG=None
TEST=See above

Change-Id: Ifd70c1e30ef266e39bf517e315dc88fccecc8a62
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3983255
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2022-10-26 20:10:25 +00:00
Dennis Kempin
a0e6a664fc Revert "dev_container: Fix podman and enable unprivileged containers"
This reverts commit 6a2b1fda85.

Reason for revert: Broke mingw64 builds

Original change's description:
> dev_container: Fix podman and enable unprivileged containers
>
> To enable podman, the Dockerfile has been split into a root
> run Dockerfile and one that adds a non-root user.
>
> The following combinations have been tested:
>
> ./tools/dev_container -v --clean --podman --unprivileged
> ./tools/dev_container -v --clean --podman
> ./tools/dev_container -v --clean --unprivileged
>
> And warnings have been added to ensure users are aware that
> the only fully supported variant is running a privileged
> docker container:
>
> ./tools/dev_container -v --clean
>
> The unprivileged containers will allow us to validate if
> unit tests require privileged system access.
>
> BUG=None
> TEST=See above
>
> Change-Id: I185b1d9c3829674986305b0e72a39b1a4ba11b98
> Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3971029
> Reviewed-by: Zihan Chen <zihanchen@google.com>
> Commit-Queue: Dennis Kempin <denniskempin@google.com>
> Reviewed-by: Dennis Kempin <denniskempin@google.com>

Bug: None
Change-Id: Id57686ed869abcfb54431aa328c54234b9465eb7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3979385
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
2022-10-26 03:06:26 +00:00
Dennis Kempin
6a2b1fda85 dev_container: Fix podman and enable unprivileged containers
To enable podman, the Dockerfile has been split into a root
run Dockerfile and one that adds a non-root user.

The following combinations have been tested:

./tools/dev_container -v --clean --podman --unprivileged
./tools/dev_container -v --clean --podman
./tools/dev_container -v --clean --unprivileged

And warnings have been added to ensure users are aware that
the only fully supported variant is running a privileged
docker container:

./tools/dev_container -v --clean

The unprivileged containers will allow us to validate if
unit tests require privileged system access.

BUG=None
TEST=See above

Change-Id: I185b1d9c3829674986305b0e72a39b1a4ba11b98
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3971029
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-10-26 02:06:06 +00:00
Dennis Kempin
5a32a2d198 Enable vaapi feature upstream
The feature can be built but not tested without access to an intel
GPU.
To enable the build, the testvm and dev container are updated to
include libva.

BUG=b:244619376
TEST=presubmit

Change-Id: Ia8c4dc46ccbcd244bf57441f2c550a6b73c67b1c
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3946027
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
2022-10-14 20:31:19 +00:00
Dennis Kempin
aae0141680 dev_container: Upgrade to debian bookworm
Debian bullseye is becoming old and we require a couple of newer
dependencies. gLinux is also tracking bookworm, so we are staying
close to our usual development environment.

Since the official rust images do not have a bookworm version,
we switch to the official debian image and add a rustup install
to ./tools/install-deps.

The new glibc version uses clone3 in multiple devices, adding
this new syscall to our policy to pass integration tests.

Drive-by change: Upgrading rust-toolchain from 1.62.0 to 1.62.1

BUG=b:243081643
TEST=CQ

Change-Id: I8af721ed4a83df61163d67001b777166abe8abfa
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3892621
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-09-14 22:32:36 +00:00
Dennis Kempin
1dab58a2cf Update all copyright headers to match new style
This search/replace updates all copyright notices to drop the
"All rights reserved", Use "ChromiumOS" instead of "Chromium OS"
and drops the trailing dots.

This fulfills the request from legal and unifies our notices.

./tools/health-check has been updated to only accept this style.

BUG=b:246579983
TEST=./tools/health-check

Change-Id: I87a80701dc651f1baf4820e5cc42469d7c5f5bf7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3894243
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-09-13 18:41:29 +00:00
Daniel Verkamp
4e36e8a601 rust-toolchain: update to Rust 1.62
This matches the recent ChromeOS SDK update.

BUG=b:243677117
TEST=tools/dev_container tools/clippy

Change-Id: I5de34abfe0985b5f41ac666ce3e93adbb628d790
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3854975
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
2022-09-06 17:36:34 +00:00
Zihan Chen
9c3270b08c dev_container: Fix /scratch permission
Correct permission of /scratch to allow crosvmdev to modify its
content after UID & GID changed to match host after container first
start.

TEST=tools/dev_container --stop ; tools/dev_container tools/run_tests
--target=vm:aarch64 --build-only

Change-Id: I8872c433ae903536750a4623f0d149b2875deba2
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3855471
Commit-Queue: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Zihan Chen <zihanchen@google.com>
2022-08-25 18:34:56 +00:00
Zihan Chen
483345c4c0 dev_container: Use host UID & GID for user in container
Container shell now defaults to user `crosvmdev` which will have
same UID & GID as outside user. VS Code Remote Containers extension
has the same behavior when loading config from devcontainer.json.

TEST=all tests pass in container, new files in container have same
permission as files created outside.
FIXED=b:239476013

Change-Id: Ib3c696509e9ea45edd5f02ff025d9477576ec765
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3814094
Auto-Submit: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-08-23 23:16:09 +00:00
Dennis Kempin
dc94a3b001 tools/run_tests: Add lcov generation
Use the recently stabilized coverage instrumentation feature of
rust to generate coverage profiles of our test runs.

These can then be used by a recipe to upload coverage to covecov
or use tools locally to generate coverage reports.

BUG=b:239255082
TEST=./tools/run_tests --generate-lcov coverage.lcov

Change-Id: Ifc64d11f3ae19a2eb7fdce36172d67bf3f7e6d17
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3805831
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-08-02 23:35:20 +00:00
Daniel Verkamp
a77c40ec06 tools: enable search feature for mdbook
This will generate the search index and in-browser search for the book.

BUG=None
TEST=mdbook build docs/book

Change-Id: Ibaf67002c5b5d5cf1aafd19ab66a05b9116b14a0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3792098
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
2022-07-29 21:16:21 +00:00
Dennis Kempin
1f0b985a4d Update cloud storage locations to use crosvm-infra project
See go/crosvm/infra for instructions on how to get access to uploading
them.

Added a helper script to install dependencies needed to build the
guest_under_test.

BUG=b:235269312
TEST=Kokoro

Change-Id: I78387a33ddbf3ab199b36e76ba617acb1250c7e5
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3783011
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-07-29 02:17:15 +00:00
Zihan Chen
216a22426c dev_container: Uprev to include clang
Need clang to compile minijail policies

TEST=new container can build crosvm with minijail policies included
BUG=b:235858187

Change-Id: Id26898025ef1ff3148da2edc5af3d923a00401c2
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3792428
Commit-Queue: Zihan Chen <zihanchen@google.com>
Auto-Submit: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: Zihan Chen <zihanchen@google.com>
2022-07-28 21:56:31 +00:00
Daniel Verkamp
d1a6e9cf94 rust-toolchain: update to 1.60.0
The ChromeOS toolchain is updated to this point, so bring our crosvm
testing in line.

BUG=b:239075544
TEST=tools/presubmit --all

Change-Id: I6bfa7bfd802f1cd75204ba59b560de176a56326f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3764424
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2022-07-28 17:46:00 +00:00
Dennis Kempin
365c8f9201 dev_container: Allow cargo home to be cached between runs
This Allows Luci builders to cache cargo home between builds. So we
do not have to download so many third party crates with each build.

CARGO_HOME is specifically intended to be cached in CI systems.

BUG=b:233230027
TEST=CROSVM_CONTAINER_CACHE=/tmp/test ./tools/dev_container --clean
cargo build

Change-Id: I11580c5ed3151519ece4a651cb22d059c7c3eb87
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3739368
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-07-07 17:49:47 +00:00
Alexandre Courbot
b3f5fbae62 tools: install-deps: add libva-dev and libudev-dev as dependencies
These libraries are needed to build the upcoming vaapi backend. Also
update the container so they are included.

BUG=b:214478588
TEST=./tools/dev_container ./tools/presubmit --all

Change-Id: Id30c38557d56b4c94ca5f2419c73b34520916ed0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3715087
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-06-23 01:12:15 +00:00
Alexandre Courbot
d036e9f225 tools: install-deps: add ffmpeg libraries as dependencies
Add the ffmpeg libraries needed by the ffmpeg decoder device backend and
bump the dev_container and testvm to include them.

BUG=b:169295147
TEST=./tools/install-deps
TEST=./tools/dev_container

Change-Id: Ifc07b9599403aa1ed18a96067ada1fa2efa6e13c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3685682
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
2022-06-07 11:44:20 +00:00
Alexandre Courbot
1adf131537 tools: dev_container: add missing script to Dockerfile
This file also needs to be copied otherwise the container generation will fail.

BUG=None
TEST=make -C tools/impl/dev_container crosvm_dev

Change-Id: I571213a0078e5cf8bd6c7e5f157b6e7dda5fff47
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3685677
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-06-02 05:56:53 +00:00
Dennis Kempin
ba4adc0efb Add python presubmit checks
This change adds python type and formatting checks and
consolidates code health checks in ./tools/health-check.

Dealing with relative imports in python is tricky, so
we are making ./tools/impl a proper package with no
directly executable files.

Some of the bash shorthands in ./tools had to be converted
to python for this.

To make the new checks pass, we run the formatter and fix
some mypy type checks.

TEST=./tools/health-check
BUG=b:218559722,b:219965702

Change-Id: Ie18d3d6dd2f5a033141e167a6e1aa762791941d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3558592
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-04-15 19:22:53 +00:00