dm-crypt: Use MD5 library instead of crypto_shash

The lmk IV mode, which dm-crypt supports for Loop-AES compatibility,
involves an MD5 computation.  Update its implementation to use the MD5
library API instead of crypto_shash.  This has many benefits, such as:

- Simpler code.  Notably, much of the error-handling code is no longer
  needed, since the library functions can't fail.

- Reduced stack usage.  crypt_iv_lmk_one() now allocates only 112 bytes
  on the stack instead of 520 bytes.

- The library functions are strongly typed, preventing bugs like
  https://lore.kernel.org/r/f1625ddc-e82e-4b77-80c2-dc8e45b54848@gmail.com

- Slightly improved performance, as the library provides direct access
  to the MD5 code without unnecessary overhead such as indirect calls.

To preserve the existing behavior of lmk support being disabled when the
kernel is booted with "fips=1", make crypt_iv_lmk_ctr() check
fips_enabled itself.  Previously it relied on crypto_alloc_shash("md5")
failing.  (I don't know for sure that lmk *actually* needs to be
disallowed in FIPS mode; this just preserves the existing behavior.)

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
This commit is contained in:
Eric Biggers 2025-10-11 11:10:42 -07:00 • committed by Mikulas Patocka
commit ba59e59b8b
2 changed files with 22 additions and 55 deletions