Add a macro for defining hardware register types with I/O accessors.
Each register field is represented as a `Bounded` of the appropriate bit
width, ensuring field values are never silently truncated.
Fields can optionally be converted to/from custom types, either fallibly
or infallibly.
The address of registers can be direct, relative, or indexed, supporting
most of the patterns in which registers are arranged.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Link: https://lore.kernel.org/all/20250306222336.23482-6-dakr@kernel.org/
Co-developed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-7-86805b2f7e9d@nvidia.com
[ * Improve wording and formatting of doc-comments,
* Import build_assert!(),
* Add missing inline annotations,
* Call static_assert!() with absolute path,
* Use expect instead of allow.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
By providing the required `IoLoc` implementations on `usize`, we can
leverage the generic accessors and reduce the number of unsafe blocks in
the module.
This also allows us to directly call the generic `read/write/update`
methods with primitive types, so add examples illustrating this.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260314-register-v9-6-86805b2f7e9d@nvidia.com
[ Slightly improve wording in doc-comment. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
I/O accesses are defined by the following properties:
- An I/O location, which consists of a start address, a width, and a
type to interpret the read value as,
- A value, which is returned for reads or provided for writes.
Introduce the `IoLoc` trait, which allows implementing types to fully
specify an I/O location.
This allows I/O operations to be made generic through the new `read` and
`write` methods.
This design will allow us to factorize the I/O code working with
primitives, and to introduce ways to perform I/O with a higher degree of
control through register types.
Co-developed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-5-86805b2f7e9d@nvidia.com
[ Fix incorrect reference to io_addr_assert() in try_update(). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
There is a need to access the inner value of a `Bounded` in const
context, notably for bitfields and registers. Remove the invariant check
of `Bounded::get`, which allows us to make it const.
Reviewed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-4-86805b2f7e9d@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Single-bit numbers are typically treated as booleans. There is an
`Into<bool>` implementation for those, but invoking it from contexts
that lack type expectations is not always convenient.
Add an `into_bool` method as a simpler shortcut.
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Reviewed-by: Yury Norov <yury.norov@gmail.com>
Tested-by: Dirk Behme <dirk.behme@de.bosch.com>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-3-86805b2f7e9d@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Shifting a `Bounded` left or right changes the number of bits required
to represent the value. Add methods that perform the shift and return a
`Bounded` with the appropriately adjusted bit width.
These methods are particularly useful for bitfield extraction.
Suggested-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Tested-by: Dirk Behme <dirk.behme@de.bosch.com>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Acked-by: Yury Norov <ynorov@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-2-86805b2f7e9d@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
This feature is stable since 1.89, and used in subsequent patches.
Reviewed-by: Gary Guo <gary@garyguo.net>
Tested-by: Dirk Behme <dirk.behme@de.bosch.com>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-1-86805b2f7e9d@nvidia.com
[ Resolve merge conflict. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Since `Mmio` now has the relevant implementations of `IoCapable`, the
default methods of `Io` can be used in place of the overloaded ones.
Remove them as well as the macros generating them.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-6-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Since `ConfigSpace` now has the relevant implementations of `IoCapable`,
the default methods of `Io` can be used in place of the overloaded ones.
Remove them as well as the macros generating them.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-5-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The relaxed access functionality is now provided by the `RelaxedMmio`
wrapper type, and we don't have any user of the legacy methods left.
Remove them.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-4-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Relaxed I/O accessors for `Mmio` are currently implemented as an extra
set of methods that mirror the ones defined in `Io`, but with the
`_relaxed` suffix.
This makes these methods impossible to use with generic code, which is a
highly plausible proposition now that we have the `Io` trait.
Address this by adding a new `RelaxedMmio` wrapper type for `Mmio` that
provides its own `IoCapable` implementations relying on the relaxed C
accessors. This makes it possible to use relaxed operations on a `Mmio`
simply by wrapping it, and to use `RelaxedMmio` in code generic against
`Io`.
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-3-71dea20a06e6@nvidia.com
[ Use kernel import style in examples. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The `_relaxed` I/O variant methods are about to be replaced by a wrapper
type exposing this access pattern with the regular methods of the `Io`
trait. Thus replace the examples to use the regular I/O methods.
Since these are examples, we want them to use the most standard ops
anyway, and the relaxed variants were but an addition that was
MMIO-specific.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-2-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
`IoCapable<T>` is currently used as a marker trait to signal that the
methods of the `Io` trait corresponding to `T` have been overridden by
the implementor (the default implementations triggering a build-time
error).
This goes against the DRY principle and separates the signaling of the
capability from its implementation, making it possible to forget a step
while implementing a new `Io`.
Another undesirable side-effect is that it makes the implementation of
I/O backends boilerplate-y and convoluted: currently this is done using
two levels of imbricated macros that generate unsafe code.
Fix these issues by turning `IoCapable` into a functional trait that
includes the raw implementation of the I/O access for `T` using
unsafe methods that work with an arbitrary address.
This allows us to turn the default methods of `Io` into regular methods
that check the passed offset, turn it into an address, and call into the
corresponding `IoCapable` functions, removing the need to overload them
at all.
`IoCapable` must still be implemented for all supported primitive types,
which is still done more concisely using a macro, but this macro becomes
much simpler and does not require calling into another one.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-1-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmm3G/UeHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGZJUH/R0vQ3Vha48QDEic
1NREwaHxAoTFi0i3y7OPPklqrP2V09D1qg4Q6fExYQVTQgV6F2DRjVbyPKrmr4ay
BA6aHrUdnFngYHpDlI1b1r7rJiAIN4WFHl7StO70bS+EB+UPsP9cfP3CKXUfKfqT
kyHXzUrd5QnjYmlb9rQw1E6rzsRamNtGUtZf7TwDidJYjtm3sPeDHUkjyRy4xkYd
UouIu6W7UXoicl38bJAgaWBY5BiYtjN6ktnY4/gcqDeqYd7mTM3Eb1B+OSXgFfip
F0OYfJhfWn+63WnPA+1I5jXWC1UrdVXTMK/NTYjhmGlfdmkLcWDlNGtu+qKZbpwj
fmF3Kyo=
=6nX1
-----END PGP SIGNATURE-----
Merge 7.0-rc4 into char-misc-next
We need the char/misc/iio fixes in this branch as well to build on top
of.
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmm3G/UeHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGZJUH/R0vQ3Vha48QDEic
1NREwaHxAoTFi0i3y7OPPklqrP2V09D1qg4Q6fExYQVTQgV6F2DRjVbyPKrmr4ay
BA6aHrUdnFngYHpDlI1b1r7rJiAIN4WFHl7StO70bS+EB+UPsP9cfP3CKXUfKfqT
kyHXzUrd5QnjYmlb9rQw1E6rzsRamNtGUtZf7TwDidJYjtm3sPeDHUkjyRy4xkYd
UouIu6W7UXoicl38bJAgaWBY5BiYtjN6ktnY4/gcqDeqYd7mTM3Eb1B+OSXgFfip
F0OYfJhfWn+63WnPA+1I5jXWC1UrdVXTMK/NTYjhmGlfdmkLcWDlNGtu+qKZbpwj
fmF3Kyo=
=6nX1
-----END PGP SIGNATURE-----
Merge tag 'v7.0-rc4' into drm-rust-next
We need the latest fixes from drm-rust-fixes in drm-rust-next as well to
build on top of.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Remove the temporary re-exports of `ARef` and `AlwaysRefCounted`
from `types.rs` now that all in-tree users have been updated to
import them directly from `sync::aref`.
These re-exports were originally added to avoid breaking the
kernel build during the transition period while call sites were
incrementally migrated. With all users updated, they are no
longer needed.
Suggested-by: Benno Lossin <lossin@kernel.org>
Link: https://github.com/Rust-for-Linux/linux/issues/1173
Signed-off-by: Shankari Anand <shankari.ak0208@gmail.com>
Link: https://patch.msgid.link/20260102202714.184223-5-shankari.ak0208@gmail.com
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Update call sites in `i2c.rs` to import `ARef` and
`AlwaysRefCounted` from `sync::aref` instead of `types`.
This aligns with the ongoing effort to move `ARef` and
`AlwaysRefCounted` to sync.
Suggested-by: Benno Lossin <lossin@kernel.org>
Link: https://github.com/Rust-for-Linux/linux/issues/1173
Signed-off-by: Shankari Anand <shankari.ak0208@gmail.com>
Acked-by: Igor Korotin <igor.korotin.linux@gmail.com>
Link: https://patch.msgid.link/20260102202714.184223-3-shankari.ak0208@gmail.com
[ Move `ARef` import into the `kernel` `use` tree above. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Now that all literals are C-Strings, update the documentation to explain
that use of this macro should be limited to non-literal strings.
Link: https://github.com/Rust-for-Linux/linux/issues/1075
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260309-cstr-rename-macro-v2-1-25f7de75944e@kernel.org
[ Apply sentence case to comment. Reword title. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Toolchain and infrastructure:
- Remap paths to avoid absolute ones starting with the upcoming Rust
1.95.0 release. This improves build reproducibility, avoids leaking
the exact path and avoids having the same path appear in two forms.
The approach here avoids remapping debug information as well, in
order to avoid breaking tools that used the paths to access source
files, which was the previous attempt that needed to be reverted.
- Allow 'unused_features' lint for the upcoming Rust 1.96.0 release.
While well-intentioned, we do not benefit much from the new lint.
- Emit dependency information into '$(depfile)' directly to avoid
a temporary '.d' file (it was an old approach).
'kernel' crate:
- 'str' module: fix warning under '!CONFIG_BLOCK' by making
'NullTerminatedFormatter' public.
- 'cpufreq' module: suppress false positive Clippy warning.
'pin-init' crate:
- Remove '#[disable_initialized_field_access]' attribute which was
unsound. This means removing the support for structs with unaligned
fields (through the 'repr(packed)' attribute), for now.
And document the load-bearing fact of field accessors (i.e. that they
are required for soundness).
- Replace shadowed return token by 'unsafe'-to-create token in order
to remain sound in the face of the likely upcoming Type Alias Impl
Trait (TAIT) and the next trait solver in upstream Rust.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEPjU5OPd5QIZ9jqqOGXyLc2htIW0FAmm1r2MACgkQGXyLc2ht
IW17OhAAoRMLTOf/iTNu29njOtjG5iYABeC5zMnxYUErWGLOooz8D82UEiB85Whs
uhVeW6YZ1jCcizglmbXOduKUOmSvEUYSv5/wj1w5OIF0FWW/6BA/aNdgDijHTbSU
v/x5NfKwxZ7JD8ivl+Z7RxtA3ggEODr/pn8USIKaZr0wEjEcnP1xBedZR6+NXuBT
CUXYm9g0BMfBbHsaI6DgD6/aj3TuHjbp1GGFlMmKd7YJGzFihhxU7nKtzkRDxAUd
HROY1Pi+iEUXFYmeXbgObVgRqzcTYV37CkbMfPk1SoIdXXhtPcO8TJ1GguAMuVMO
alAaVEAGFBKvnNGFoocCoxRt8U9hhiOiqlP+QMu2sbVWxWdmdBRcXlB+t4YIqFWy
6Icp/5K8C6Tf7ZrDZhccn6T6jqp+mvNzfanppi0/DFlZiN+1oSABQgEV3YtAiEIW
UHLT5bO5YEkHW73S9mSKhBhbH58h8IX7oGv/wteovI3gjMA9of/8/6MArWXbLpxj
o6ws7p595/xRV+xZURm7YQm6wtbjBr4Eipgd3vIoc0Z+VbCnDLEs/n8cEYY2CtIg
9Ze+MbBaqnf58qvhwj9Zy8ncQVnCtpQmybUE/i/YfdXo7AMsKtdUppv09xfz4fMC
NWRqJttuWiUttGsLvi/GTxzkz5bT6Y6ZHHNqTyIKWXEIBTtRxK8=
=7LgC
-----END PGP SIGNATURE-----
Merge tag 'rust-fixes-7.0-2' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux
Pull Rust fixes from Miguel Ojeda:
"Toolchain and infrastructure:
- Remap paths to avoid absolute ones starting with the upcoming Rust
1.95.0 release. This improves build reproducibility, avoids leaking
the exact path and avoids having the same path appear in two forms
The approach here avoids remapping debug information as well, in
order to avoid breaking tools that used the paths to access source
files, which was the previous attempt that needed to be reverted
- Allow 'unused_features' lint for the upcoming Rust 1.96.0 release.
While well-intentioned, we do not benefit much from the new lint
- Emit dependency information into '$(depfile)' directly to avoid a
temporary '.d' file (it was an old approach)
'kernel' crate:
- 'str' module: fix warning under '!CONFIG_BLOCK' by making
'NullTerminatedFormatter' public
- 'cpufreq' module: suppress false positive Clippy warning
'pin-init' crate:
- Remove '#[disable_initialized_field_access]' attribute which was
unsound. This means removing the support for structs with unaligned
fields (through the 'repr(packed)' attribute), for now
And document the load-bearing fact of field accessors (i.e. that
they are required for soundness)
- Replace shadowed return token by 'unsafe'-to-create token in order
to remain sound in the face of the likely upcoming Type Alias Impl
Trait (TAIT) and the next trait solver in upstream Rust"
* tag 'rust-fixes-7.0-2' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux:
rust: kbuild: allow `unused_features`
rust: cpufreq: suppress clippy::double_parens in Policy doctest
rust: pin-init: replace shadowed return token by `unsafe`-to-create token
rust: pin-init: internal: init: document load-bearing fact of field accessors
rust: pin-init: internal: init: remove `#[disable_initialized_field_access]`
rust: build: remap path to avoid absolute path
rust: kbuild: emit dep-info into $(depfile) directly
rust: str: make NullTerminatedFormatter public
We use a unit struct `__InitOk` in the closure generated by the
initializer macros as the return value. We shadow it by creating a
struct with the same name again inside of the closure, preventing early
returns of `Ok` in the initializer (before all fields have been
initialized).
In the face of Type Alias Impl Trait (TAIT) and the next trait solver,
this solution no longer works [1]. The shadowed struct can be named
through type inference. In addition, there is an RFC proposing to add
the feature of path inference to Rust, which would similarly allow [2].
Thus remove the shadowed token and replace it with an `unsafe` to create
token.
The reason we initially used the shadowing solution was because an
alternative solution used a builder pattern. Gary writes [3]:
In the early builder-pattern based InitOk, having a single InitOk
type for token is unsound because one can launder an InitOk token
used for one place to another initializer. I used a branded lifetime
solution, and then you figured out that using a shadowed type would
work better because nobody could construct it at all.
The laundering issue does not apply to the approach we ended up with
today.
With this change, the example by Tim Chirananthavat in [1] no longer
compiles and results in this error:
error: cannot construct `pin_init::__internal::InitOk` with struct literal syntax due to private fields
--> src/main.rs:26:17
|
26 | InferredType {}
| ^^^^^^^^^^^^
|
= note: private field `0` that was not provided
help: you might have meant to use the `new` associated function
|
26 - InferredType {}
26 + InferredType::new()
|
Applying the suggestion of using the `::new()` function, results in
another expected error:
error[E0133]: call to unsafe function `pin_init::__internal::InitOk::new` is unsafe and requires unsafe block
--> src/main.rs:26:17
|
26 | InferredType::new()
| ^^^^^^^^^^^^^^^^^^^ call to unsafe function
|
= note: consult the function's documentation for information on how to avoid undefined behavior
Reported-by: Tim Chirananthavat <theemathas@gmail.com>
Link: https://github.com/rust-lang/rust/issues/153535 [1]
Link: https://github.com/rust-lang/rfcs/pull/3444#issuecomment-4016145373 [2]
Link: https://github.com/rust-lang/rust/issues/153535#issuecomment-4017620804 [3]
Fixes: fc6c6baa1f ("rust: init: add initialization macros")
Cc: stable@vger.kernel.org
Signed-off-by: Benno Lossin <lossin@kernel.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260311105056.1425041-1-lossin@kernel.org
[ Added period as mentioned. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
As kernel always use unsigned char and not the platform ABI's default, an
user should always use `as_char_ptr` provided via `CStrExt` instead.
Therefore configure `disallow-methods` feature of clippy to catch incorrect
usage.
Similarly, the dual `from_ptr` is also disallowed.
[ As an example, without the previous commit, we would get a warning like:
warning: use of a disallowed method `core::ffi::CStr::as_ptr`
--> rust/kernel/task.rs:422:54
|
422 | unsafe { crate::bindings::__might_sleep(file.as_ptr().cast(), loc.line() as i32) }
| ^^^^^^ help: kernel's `char` is always unsigned, use `as_char_ptr` instead: `kernel::prelude::CStrExt::as_char_ptr`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.94.0/index.html#disallowed_methods
= note: `-W clippy::disallowed-methods` implied by `-W clippy::all`
= help: to override `-W clippy::all` add `#[allow(clippy::disallowed_methods)]`
- Miguel ]
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Tamir Duberstein <tamird@kernel.org>
Link: https://patch.msgid.link/20260203130745.868762-2-gary@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
`as_char_ptr` would provide the correct (unsigned char) type without
needing to convert to an intermediate type and cast the pointer.
The `as_ptr()` function is going to be disallowed by clippy warning, so fix
this usage.
This is used only if CONFIG_DEBUG_ATOMIC_SLEEP=y. Instead of conditionally
importing `CStrExt`, import it via prelude instead, and remove other
imports that are already available via the prelude.
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202601221157.89t3Sqbl-lkp@intel.com/
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260203130745.868762-1-gary@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
This is needed to inline these helpers into Rust code.
Reviewed-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260105-define-rust-helper-v2-9-51da5f454a67@google.com
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
This is needed to inline these helpers into Rust code.
Reviewed-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Stephen Boyd <sboyd@kernel.org>
Link: https://patch.msgid.link/20260105-define-rust-helper-v2-4-51da5f454a67@google.com
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Add the EMSGSIZE error code, which indicates that a message is too
long.
Tested-by: Zhi Wang <zhiw@nvidia.com>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260306-cmdq-continuation-v6-3-cc7b629200ee@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Pass `pin_init{,_internal}-cfgs` from rust/Makefile to
scripts/generate_rust_analyzer.py. Remove hardcoded `cfg`s in
scripts/generate_rust_analyzer.py for `pin-init{,-internal}` now that
these are passed from `rust/Makefile`.
Centralize `cfg` lookup in scripts/generate_rust_analyzer.py in
`append_crate` to avoid having to do so for each crate.
Reviewed-by: Jesung Yang <y.j3ms.n@gmail.com>
Acked-by: Benno Lossin <lossin@kernel.org>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260127-rust-analyzer-pin-init-duplication-v3-2-118c48c35e88@kernel.org
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmmuDMYeHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGLO4IAJ5kpRUi1STiqseb
bLawF3gehuZp8VPAYnJsACYXN7kMx9OQJ2SE4Y0Y16ZlHtS1TTvsEAhwSTyYH7Jc
hb1iSfoN1kxgDh2U3yZZJz9+DzQh6/YCDXJjyhpSgWOejhaYe7r7er5xqdKpGgVx
6hlvN92/c1m7aqMjKNXeD7YKoXn35FzwPYQAyksJdwMWbq27HE1Vb42PHht21yUX
1ndEUw0UMYH3IrDON+7QtE/aXW1PlLLeQWPDpG9y5FhoseZFEcNkm5NVYshuZ8L4
WZj3Q5IvQ/zj9DuwagW4Gab9XwzKIWSXuuSLRyQzv+OcAyITiC+Uo3z55TFmvXs7
2DOHBkY=
=depW
-----END PGP SIGNATURE-----
Merge tag 'v7.0-rc3' into driver-core-next
We need the driver-core fixes in here as well to build on top of.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The safety comment used in the implementation of `fetch_add()` could be
read as just saying something it is true without justifying it. Update
the safety comment to include justification.
Suggested-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Andreas Hindborg <a.hindborg@kernel.org>
Signed-off-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260220-atomic-sub-v3-3-e63cbed1d2aa@kernel.org
Link: https://patch.msgid.link/20260303201701.12204-14-boqun@kernel.org
In order to synchronize with C or external memory, atomic operations
over raw pointers are need. Although there is already an
`Atomic::from_ptr()` to provide a `&Atomic<T>`, it's more convenient to
have helpers that directly perform atomic operations on raw pointers.
Hence a few are added, which are basically an `Atomic::from_ptr().op()`
wrapper.
Note: for naming, since `atomic_xchg()` and `atomic_cmpxchg()` have a
conflict naming to 32bit C atomic xchg/cmpxchg, hence the helpers are
just named as `xchg()` and `cmpxchg()`. For `atomic_load()` and
`atomic_store()`, their 32bit C counterparts are `atomic_read()` and
`atomic_set()`, so keep the `atomic_` prefix.
[boqun: Fix typo spotted by Alice and fix broken sentence spotted by
Gary]
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120115207.55318-3-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-11-boqun@kernel.org
Add AtomicFlag type for boolean flags.
Document when AtomicFlag is generally preferable to Atomic<bool>: in
particular, when RMW operations such as xchg()/cmpxchg() may be used
and minimizing memory usage is not the top priority. On some
architectures without byte-sized RMW instructions, Atomic<bool> can be
slower for RMW operations.
Signed-off-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Signed-off-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260129122622.3896144-2-tomo@aliasing.net
Link: https://patch.msgid.link/20260303201701.12204-9-boqun@kernel.org
Atomic pointer support is an important piece of synchronization
algorithm, e.g. RCU, hence provide the support for that.
Note that instead of relying on atomic_long or the implementation of
`Atomic<usize>`, a new set of helpers (atomic_ptr_*) is introduced for
atomic pointer specifically, this is because ptr2int casting would
lose the provenance of a pointer and even though in theory there are a
few tricks the provenance can be restored, it'll still be a simpler
implementation if C could provide atomic pointers directly. The side
effects of this approach are: we don't have the arithmetic and logical
operations for pointers yet and the current implementation only works
on ARCH_SUPPORTS_ATOMIC_RMW architectures, but these are implementation
issues and can be added later.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Link: https://patch.msgid.link/20260120140503.62804-3-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-8-boqun@kernel.org
Currently, since all the architectures that support Rust all have
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW selected, the helpers of atomic
load/store on i8 and i16 relies on CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y.
It's generally fine since most of architectures support that.
The plan for CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=n architectures is adding
their (probably lock-based) atomic load/store for i8 and i16 as their
atomic_{read,set}() and atomic64_{read,set}() counterpart when they
plans to support Rust.
Hence use a statis_assert!() to check this and remind the future us the
need of the helpers. This is more clear than the #[cfg] on impl blocks
of i8 and i16.
Suggested-by: Dirk Behme <dirk.behme@gmail.com>
Suggested-by: Benno Lossin <lossin@kernel.org>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120140503.62804-2-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-7-boqun@kernel.org
Originally, `Atomic::from_ptr()` requires `T` being a `Sync` because I
thought having the ability to do `from_ptr()` meant multiplle
`&Atomic<T>`s shared by different threads, which was identical (or
similar) to multiple `&T`s shared by different threads. Hence `T` was
required to be `Sync`. However this is not true, since `&Atomic<T>` is
not the same at `&T`. Moreover, having this bound makes `Atomic::<*mut
T>::from_ptr()` impossible, which is definitely not intended. Therefore
remove the `T: Sync` bound.
[boqun: Fix title typo spotted by Alice & Gary]
Fixes: 29c32c405e ("rust: sync: atomic: Add generic atomics")
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120115207.55318-2-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-2-boqun@kernel.org
Current `dma_read!`, `dma_write!` macros also use a custom
`addr_of!()`-based implementation for projecting pointers, which has
soundness issue as it relies on absence of `Deref` implementation on types.
It also has a soundness issue where it does not protect against unaligned
fields (when `#[repr(packed)]` is used) so it can generate misaligned
accesses.
This commit migrates them to use the general pointer projection
infrastructure, which handles these cases correctly.
As part of migration, the macro is updated to have an improved surface
syntax. The current macro have
dma_read!(a.b.c[d].e.f)
to mean `a.b.c` is a DMA coherent allocation and it should project into it
with `[d].e.f` and do a read, which is confusing as it makes the indexing
operator integral to the macro (so it will break if you have an array of
`CoherentAllocation`, for example).
This also is problematic as we would like to generalize
`CoherentAllocation` from just slices to arbitrary types.
Make the macro expects `dma_read!(path.to.dma, .path.inside.dma)` as the
canonical syntax. The index operator is no longer special and is just one
type of projection (in additional to field projection). Similarly, make
`dma_write!(path.to.dma, .path.inside.dma, value)` become the canonical
syntax for writing.
Another issue of the current macro is that it is always fallible. This
makes sense with existing design of `CoherentAllocation`, but once we
support fixed size arrays with `CoherentAllocation`, it is desirable to
have the ability to perform infallible indexing as well, e.g. doing a `[0]`
index of `[Foo; 2]` is okay and can be checked at build-time, so forcing
falliblity is non-ideal. To capture this, the macro is changed to use
`[idx]` as infallible projection and `[idx]?` as fallible index projection
(those syntax are part of the general projection infra). A benefit of this
is that while individual indexing operation may fail, the overall
read/write operation is not fallible.
Fixes: ad2907b4e3 ("rust: add dma coherent allocator abstraction")
Reviewed-by: Benno Lossin <lossin@kernel.org>
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260302164239.284084-4-gary@kernel.org
[ Capitalize safety comments; slightly improve wording in doc-comments.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a generic infrastructure for performing field and index projections on
raw pointers. This will form the basis of performing I/O projections.
Pointers manipulations are intentionally using the safe wrapping variants
instead of the unsafe variants, as the latter requires pointers to be
inside an allocation which is not necessarily true for I/O pointers.
This projection macro protects against rogue `Deref` implementation, which
can causes the projected pointer to be outside the bounds of starting
pointer. This is extremely unlikely and Rust has a lint to catch this, but
is unsoundness regardless. The protection works by inducing type inference
ambiguity when `Deref` is implemented.
This projection macro also stops projecting into unaligned fields (i.e.
fields of `#[repr(packed)]` structs), as misaligned pointers require
special handling. This is implemented by attempting to create reference to
projected field inside a `if false` block. Despite being unreachable, Rust
still checks that they're not unaligned fields.
The projection macro supports both fallible and infallible index
projections. These are described in detail inside the documentation.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Benno Lossin <lossin@kernel.org>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260302164239.284084-3-gary@kernel.org
[ * Add intro-doc links where possible,
* Fix typos and slightly improve wording, e.g. "as documentation
describes" -> "as the documentation of [`Self::proj`] describes",
* Add an empty line between regular and safety comments, before
examples, and between logically independent comments,
* Capitalize various safety comments.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a `KnownSize` trait which is used obtain a size from a raw pointer's
metadata. This makes it possible to obtain size information on a raw slice
pointer. This is similar to Rust `core::mem::size_of_val_raw` which is not
yet stable.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Benno Lossin <lossin@kernel.org>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260302164239.284084-2-gary@kernel.org
[ Fix wording in doc-comment. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
kunit:
- Fixes rust warnings when CONFIG_PRINTK is disabled.
- Reduces stack usage in kunit_run_tests() to fix warnings when
CONFIG_FRAME_WARN is set to a relatively low value.
- Updates email address for David Gow.
kunit tool:
- Copies caller args in run_kernel to prevent mutation.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEPZKym/RZuOCGeA/kCwJExA0NQxwFAmmrNf4ACgkQCwJExA0N
QxyVNxAAtemv6AH1pP/cR23teXd5uuBThTM8EASEAC9/VJmCuPjg9mb6df816EXx
2ZrTKGR5k/5basYDhNKIAJTGGjT3pcau7OKmD7Piy3bbhJEq5kb31rVbbmCV1STl
vzJcBbSNP5Z+otz/flmewoeiL+xYarQvCrfFll5Y+gMer3HFevG0y0rCpEe0+7tH
avgq01m7rHFg0DLYxJhjE3NFgFlgYaKdHyQPrGYOtZUXJDSNUO1cULJ8ZrMG3skc
JWngB/5eS6hP/3QSaBm9coHKbrAYatpPWLfg9wtgfuj3be2oEAnWeynb/vWrdEV3
uid96CEs1UPG8Z/jTtRK86q1md+5I2QX6s7B74EXnzCo0bg150h1qtT2J/za1Kxu
7CP6Rq59VHCL6SMpQ4cks00zpWffCCtAphto2yk9ZphbisyVUGN8zEiCdd0FrLob
putdMLeJVLXbnWFCnLA/8gFfx6Inp+PXNoVr+6LNZo/qB4W/EJOLDUvjaVJ7abpY
yUxvJVj0+dwOlYUYnpldGCz9pfywgIyLI7ERtu4KIwJ5EMsVo8yJgykEd2MkQbZ+
2Db3omyalnAn/vXP5p48vawI9oj6scMNxukBRTdqZsQGRma/jj6FnZCi77llQY+g
V6pTQPjJRhRGLIwow+dCkJ4CucafNacp1RVuf61MM/I03XOQEwU=
=j8w0
-----END PGP SIGNATURE-----
Merge tag 'linux_kselftest-kunit-fixes-7.0-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
Pull kunit fixes from Shuah Khan:
- Fix rust warnings when CONFIG_PRINTK is disabled
- Reduce stack usage in kunit_run_tests() to fix warnings when
CONFIG_FRAME_WARN is set to a relatively low value
- Update email address for David Gow
- Copy caller args in kunit tool in run_kernel to prevent mutation
* tag 'linux_kselftest-kunit-fixes-7.0-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
kunit: reduce stack usage in kunit_run_tests()
kunit: tool: copy caller args in run_kernel to prevent mutation
rust: kunit: fix warning when !CONFIG_PRINTK
MAINTAINERS: Update email address for David Gow
The functions `[Pin]Init::__[pinned_]init` and `ptr::write` called from
the `init!` macro require the passed pointer to be aligned. This fact is
ensured by the creation of field accessors to previously initialized
fields.
Since we missed this very important fact from the beginning [1],
document it in the code.
Link: https://rust-for-linux.zulipchat.com/#narrow/channel/561532-pin-init/topic/initialized.20field.20accessor.20detection/with/576210658 [1]
Fixes: 90e53c5e70 ("rust: add pin-init API core")
Cc: <stable@vger.kernel.org> # 6.6.y, 6.12.y: 42415d163e: rust: pin-init: add references to previously initialized fields
Cc: <stable@vger.kernel.org> # 6.6.y, 6.12.y, 6.18.y, 6.19.y
Signed-off-by: Benno Lossin <lossin@kernel.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260302140424.4097655-2-lossin@kernel.org
[ Updated Cc: stable@ tags as discussed. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Gary noticed [1] that the initializer macros as well as the `[Pin]Init`
traits cannot support unaligned fields, since they use operations that
require aligned pointers. This means that any code using structs with
unaligned fields in pin-init is unsound.
By default, the `init!` macro generates references to initialized fields,
which makes the compiler check that those fields are aligned. However,
we added the `#[disable_initialized_field_access]` attribute to avoid
this behavior in commit ceca298c53 ("rust: pin-init: internal: init:
add escape hatch for referencing initialized fields"). Thus remove the
`#[disable_initialized_field_access]` attribute from `init!`, which is
the only safe way to create an initializer handling unaligned fields.
If support for in-place initializing structs with unaligned fields is
required in the future, we could figure out a solution. This is tracked
in [2].
Reported-by: Gary Guo <gary@garyguo.net>
Closes: https://rust-for-linux.zulipchat.com/#narrow/channel/561532-pin-init/topic/initialized.20field.20accessor.20detection/with/576210658 [1]
Link: https://github.com/Rust-for-Linux/pin-init/issues/112 [2]
Fixes: ceca298c53 ("rust: pin-init: internal: init: add escape hatch for referencing initialized fields")
Signed-off-by: Benno Lossin <lossin@kernel.org>
Acked-by: Janne Grunau <j@jannau.net>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260302140424.4097655-1-lossin@kernel.org
[ Adjusted tags and reworded as discussed. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
When building with an out directory (O=), absolute paths can end up in the
file name in `#[track_caller]` or the panic message. This is not desirable
as this leaks the exact path being used to build the kernel and means that
the same location can appear in two forms (relative or absolute).
This is reported by Asahi [1] and is being workaround in [2] previously to
force everything to be absolute path. Using absolute path for everything
solves the inconsistency, however it does not address the reproducibility
issue. So, fix this by remap all absolute paths to srctree to relative path
instead.
This is previously attempted in commit dbdffaf50f ("kbuild, rust: use
-fremap-path-prefix to make paths relative") but that was reverted as
remapping debug info causes some tool (e.g. objdump) to be unable to find
sources. Therefore, use `--remap-path-scope` to only remap macros but leave
debuginfo untouched. `--remap-path-scope` is only stable in Rust 1.95, so
use `rustc-option` to detect its presence. This feature has been available
as `-Zremap-path-scope` for all versions that we support; however due to
bugs in the Rust compiler, it does not work reliably until 1.94. I opted to
not enable it for 1.94 as it's just a single version that we missed.
This change can be validated by building a kernel with O=, strip debug info
on vmlinux, and then check if the absolute path exists in `strings
vmlinux`, e.g. `strings vmlinux |grep \/home`.
Reported-by: Janne Grunau <j@jannau.net>
Reported-by: Asahi Lina <lina+kernel@asahilina.net>
Closes: https://rust-for-linux.zulipchat.com/#narrow/channel/288089-General/topic/Per-call-site.20data.20and.20lock.20class.20keys/near/572466559 [1]
Link: 54ab888788 [2]
Signed-off-by: Gary Guo <gary@garyguo.net>
Acked-by: Nicolas Schier <nsc@kernel.org> # kbuild
Link: https://patch.msgid.link/20260226152112.3222886-1-gary@kernel.org
[ Reworded for few typos. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
