kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-22 19:14:51 +00:00
Code Issues Releases Packages Activity
linux/crypto/cryptd.c

1153 lines
30 KiB
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation either version 2 of the license or at your option any later version extracted by the scancode license scanner the SPDX license identifier GPL-2.0-or-later has been chosen to replace the boilerplate/reference in 3029 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-27 08:55:01 +02:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: hash - Move ahash functions into crypto/hash.h All new crypto interfaces should go into individual files as much as possible in order to ensure that crypto.h does not collapse under its own weight. This patch moves the ahash code into crypto/hash.h and crypto/internal/hash.h respectively. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-07-10 16:01:22 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Add support to access underlying blkcipher cryptd_alloc_ablkcipher() will allocate a cryptd-ed ablkcipher for specified algorithm name. The new allocated one is guaranteed to be cryptd-ed ablkcipher, so the blkcipher underlying can be gotten via cryptd_ablkcipher_child(). Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-01-18 16:19:46 +11:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - move kcrypto_wq into cryptd kcrypto_wq is only used by cryptd, so move it into cryptd.c and change the workqueue name from "crypto" to "cryptd". Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:53:58 -07:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - make cryptd_max_cpu_qlen module parameter static The cryptd_max_cpu_qlen module parameter is local to the source and does not need to be in global scope, so make it static. Cleans up sparse warning: crypto/cryptd.c:35:14: warning: symbol 'cryptd_max_cpu_qlen' was not declared. Should it be static? Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-11-30 11:26:14 +00:00
crypto: cryptd - Add cryptd_max_cpu_qlen module parameter Make the cryptd queue length configurable. We recently had customer where this needed to be tuned to accommodate the aesni_intel module and prevent packet drop. Signed-off-by: Jon Maxwell <jmaxwell37@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-11-22 16:08:17 +11:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - move kcrypto_wq into cryptd kcrypto_wq is only used by cryptd, so move it into cryptd.c and change the workqueue name from "crypto" to "cryptd". Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:53:58 -07:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Use nested-BH locking for cryptd_cpu_queue cryptd_queue::cryptd_cpu_queue is a per-CPU variable and relies on disabled BH for its locking. Without per-CPU locking in local_bh_disable() on PREEMPT_RT this data structure requires explicit locking. Add a local_lock_t to the struct cryptd_cpu_queue and use local_lock_nested_bh() for locking. This change adds only lockdep coverage and does not alter the functional behaviour for !PREEMPT_RT. Cc: "David S. Miller" <davem@davemloft.net> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: linux-crypto@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-01 08:09:36 +02:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Protect per-CPU resource by disabling BH. The access to cryptd_queue::cpu_queue is synchronized by disabling preemption in cryptd_enqueue_request() and disabling BH in cryptd_queue_worker(). This implies that access is allowed from BH. If cryptd_enqueue_request() is invoked from preemptible context _and_ soft interrupt then this can lead to list corruption since cryptd_enqueue_request() is not protected against access from soft interrupt. Replace get_cpu() in cryptd_enqueue_request() with local_bh_disable() to ensure BH is always disabled. Remove preempt_disable() from cryptd_queue_worker() since it is not needed because local_bh_disable() ensures synchronisation. Fixes: 254eff771441 ("crypto: cryptd - Per-CPU thread implementation...") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-05-04 17:07:36 +02:00
percpu: add __percpu sparse annotations to what's left Add __percpu sparse annotations to places which didn't make it in one of the previous patches. All converions are trivial. These annotations are to make sparse consider percpu variables to be in a different address space and warn if accessed without going through percpu accessors. This patch doesn't affect normal builds. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: Borislav Petkov <borislav.petkov@amd.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Huang Ying <ying.huang@intel.com> Cc: Len Brown <lenb@kernel.org> Cc: Neil Brown <neilb@suse.de>
2010-02-02 14:39:15 +09:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Use request context instead of stack for sub-request cryptd is buggy as it tries to use sync_skcipher without going through the proper sync_skcipher interface. In fact it doesn't even need sync_skcipher since it's already a proper skcipher and can easily access the request context instead of using something off the stack. Fixes: 36b3875a97b8 ("crypto: cryptd - Remove VLA usage of skcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-11-11 17:59:17 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request context instead of stack for sub-request cryptd is buggy as it tries to use sync_skcipher without going through the proper sync_skcipher interface. In fact it doesn't even need sync_skcipher since it's already a proper skcipher and can easily access the request context instead of using something off the stack. Fixes: 36b3875a97b8 ("crypto: cryptd - Remove VLA usage of skcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-11-11 17:59:17 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use subreq for AEAD AEAD reuses the existing request object for its child. This is error-prone and unnecessary. This patch adds a subrequest object just like we do for skcipher and hash. This patch also restores the original completion function as we do for skcipher/hash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:53:24 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Use nested-BH locking for cryptd_cpu_queue cryptd_queue::cryptd_cpu_queue is a per-CPU variable and relies on disabled BH for its locking. Without per-CPU locking in local_bh_disable() on PREEMPT_RT this data structure requires explicit locking. Add a local_lock_t to the struct cryptd_cpu_queue and use local_lock_nested_bh() for locking. This change adds only lockdep coverage and does not alter the functional behaviour for !PREEMPT_RT. Cc: "David S. Miller" <davem@davemloft.net> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: linux-crypto@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-01 08:09:36 +02:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Add cryptd_max_cpu_qlen module parameter Make the cryptd queue length configurable. We recently had customer where this needed to be tuned to accommodate the aesni_intel module and prevent packet drop. Signed-off-by: Jon Maxwell <jmaxwell37@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-11-22 16:08:17 +11:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Protect per-CPU resource by disabling BH. The access to cryptd_queue::cpu_queue is synchronized by disabling preemption in cryptd_enqueue_request() and disabling BH in cryptd_queue_worker(). This implies that access is allowed from BH. If cryptd_enqueue_request() is invoked from preemptible context _and_ soft interrupt then this can lead to list corruption since cryptd_enqueue_request() is not protected against access from soft interrupt. Replace get_cpu() in cryptd_enqueue_request() with local_bh_disable() to ensure BH is always disabled. Remove preempt_disable() from cryptd_queue_worker() since it is not needed because local_bh_disable() ensures synchronisation. Fixes: 254eff771441 ("crypto: cryptd - Per-CPU thread implementation...") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-05-04 17:07:36 +02:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Protect per-CPU resource by disabling BH. The access to cryptd_queue::cpu_queue is synchronized by disabling preemption in cryptd_enqueue_request() and disabling BH in cryptd_queue_worker(). This implies that access is allowed from BH. If cryptd_enqueue_request() is invoked from preemptible context _and_ soft interrupt then this can lead to list corruption since cryptd_enqueue_request() is not protected against access from soft interrupt. Replace get_cpu() in cryptd_enqueue_request() with local_bh_disable() to ensure BH is always disabled. Remove preempt_disable() from cryptd_queue_worker() since it is not needed because local_bh_disable() ensures synchronisation. Fixes: 254eff771441 ("crypto: cryptd - Per-CPU thread implementation...") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-05-04 17:07:36 +02:00
crypto: cryptd - Use nested-BH locking for cryptd_cpu_queue cryptd_queue::cryptd_cpu_queue is a per-CPU variable and relies on disabled BH for its locking. Without per-CPU locking in local_bh_disable() on PREEMPT_RT this data structure requires explicit locking. Add a local_lock_t to the struct cryptd_cpu_queue and use local_lock_nested_bh() for locking. This change adds only lockdep coverage and does not alter the functional behaviour for !PREEMPT_RT. Cc: "David S. Miller" <davem@davemloft.net> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: linux-crypto@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-01 08:09:36 +02:00
this_cpu: Use this_cpu_ptr in crypto subsystem Just a slight optimization that removes one array lookup. The processor number is needed for other things as well so the get/put_cpu cannot be removed. Acked-by: Tejun Heo <tj@kernel.org> Cc: Huang Ying <ying.huang@intel.com> Signed-off-by: Christoph Lameter <cl@linux-foundation.org> Signed-off-by: Tejun Heo <tj@kernel.org>
2009-10-03 19:48:23 +09:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: change transient busy return code to -ENOSPC The crypto API was using the -EBUSY return value to indicate both a hard failure to submit a crypto operation into a transformation provider when the latter was busy and the backlog mechanism was not enabled as well as a notification that the operation was queued into the backlog when the backlog mechanism was enabled. Having the same return code indicate two very different conditions depending on a flag is both error prone and requires extra runtime check like the following to discern between the cases: if (err == -EINPROGRESS || (err == -EBUSY && (ahash_request_flags(req) & CRYPTO_TFM_REQ_MAY_BACKLOG))) This patch changes the return code used to indicate a crypto op failed due to the transformation provider being transiently busy to -ENOSPC. Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-18 08:00:33 +01:00
crypto: cryptd - Protect per-CPU resource by disabling BH. The access to cryptd_queue::cpu_queue is synchronized by disabling preemption in cryptd_enqueue_request() and disabling BH in cryptd_queue_worker(). This implies that access is allowed from BH. If cryptd_enqueue_request() is invoked from preemptible context _and_ soft interrupt then this can lead to list corruption since cryptd_enqueue_request() is not protected against access from soft interrupt. Replace get_cpu() in cryptd_enqueue_request() with local_bh_disable() to ensure BH is always disabled. Remove preempt_disable() from cryptd_queue_worker() since it is not needed because local_bh_disable() ensures synchronisation. Fixes: 254eff771441 ("crypto: cryptd - Per-CPU thread implementation...") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-05-04 17:07:36 +02:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Protect per-CPU resource by disabling BH. The access to cryptd_queue::cpu_queue is synchronized by disabling preemption in cryptd_enqueue_request() and disabling BH in cryptd_queue_worker(). This implies that access is allowed from BH. If cryptd_enqueue_request() is invoked from preemptible context _and_ soft interrupt then this can lead to list corruption since cryptd_enqueue_request() is not protected against access from soft interrupt. Replace get_cpu() in cryptd_enqueue_request() with local_bh_disable() to ensure BH is always disabled. Remove preempt_disable() from cryptd_queue_worker() since it is not needed because local_bh_disable() ensures synchronisation. Fixes: 254eff771441 ("crypto: cryptd - Per-CPU thread implementation...") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-05-04 17:07:36 +02:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Protect per-CPU resource by disabling BH. The access to cryptd_queue::cpu_queue is synchronized by disabling preemption in cryptd_enqueue_request() and disabling BH in cryptd_queue_worker(). This implies that access is allowed from BH. If cryptd_enqueue_request() is invoked from preemptible context _and_ soft interrupt then this can lead to list corruption since cryptd_enqueue_request() is not protected against access from soft interrupt. Replace get_cpu() in cryptd_enqueue_request() with local_bh_disable() to ensure BH is always disabled. Remove preempt_disable() from cryptd_queue_worker() since it is not needed because local_bh_disable() ensures synchronisation. Fixes: 254eff771441 ("crypto: cryptd - Per-CPU thread implementation...") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-05-04 17:07:36 +02:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Protect per-CPU resource by disabling BH. The access to cryptd_queue::cpu_queue is synchronized by disabling preemption in cryptd_enqueue_request() and disabling BH in cryptd_queue_worker(). This implies that access is allowed from BH. If cryptd_enqueue_request() is invoked from preemptible context _and_ soft interrupt then this can lead to list corruption since cryptd_enqueue_request() is not protected against access from soft interrupt. Replace get_cpu() in cryptd_enqueue_request() with local_bh_disable() to ensure BH is always disabled. Remove preempt_disable() from cryptd_queue_worker() since it is not needed because local_bh_disable() ensures synchronisation. Fixes: 254eff771441 ("crypto: cryptd - Per-CPU thread implementation...") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-05-04 17:07:36 +02:00
crypto: cryptd - Use nested-BH locking for cryptd_cpu_queue cryptd_queue::cryptd_cpu_queue is a per-CPU variable and relies on disabled BH for its locking. Without per-CPU locking in local_bh_disable() on PREEMPT_RT this data structure requires explicit locking. Add a local_lock_t to the struct cryptd_cpu_queue and use local_lock_nested_bh() for locking. This change adds only lockdep coverage and does not alter the functional behaviour for !PREEMPT_RT. Cc: "David S. Miller" <davem@davemloft.net> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: linux-crypto@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-01 08:09:36 +02:00
crypto: cryptd - Protect per-CPU resource by disabling BH. The access to cryptd_queue::cpu_queue is synchronized by disabling preemption in cryptd_enqueue_request() and disabling BH in cryptd_queue_worker(). This implies that access is allowed from BH. If cryptd_enqueue_request() is invoked from preemptible context _and_ soft interrupt then this can lead to list corruption since cryptd_enqueue_request() is not protected against access from soft interrupt. Replace get_cpu() in cryptd_enqueue_request() with local_bh_disable() to ensure BH is always disabled. Remove preempt_disable() from cryptd_queue_worker() since it is not needed because local_bh_disable() ensures synchronisation. Fixes: 254eff771441 ("crypto: cryptd - Per-CPU thread implementation...") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-05-04 17:07:36 +02:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - disable softirqs in cryptd_queue_worker to prevent data corruption cryptd_queue_worker attempts to prevent simultaneous accesses to crypto workqueue by cryptd_enqueue_request using preempt_disable/preempt_enable. However cryptd_enqueue_request might be called from softirq context, so add local_bh_disable/local_bh_enable to prevent data corruption and panics. Bug report at http://marc.info/?l=linux-crypto-vger&m=134858649616319&w=2 v2: - Disable software interrupts instead of hardware interrupts Cc: stable@vger.kernel.org Reported-by: Gurucharan Shetty <gurucharan.shetty@gmail.com> Signed-off-by: Jussi Kivilinna <jussi.kivilinna@mbnet.fi> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2012-10-21 20:42:28 +03:00
crypto: cryptd - Use nested-BH locking for cryptd_cpu_queue cryptd_queue::cryptd_cpu_queue is a per-CPU variable and relies on disabled BH for its locking. Without per-CPU locking in local_bh_disable() on PREEMPT_RT this data structure requires explicit locking. Add a local_lock_t to the struct cryptd_cpu_queue and use local_lock_nested_bh() for locking. This change adds only lockdep coverage and does not alter the functional behaviour for !PREEMPT_RT. Cc: "David S. Miller" <davem@davemloft.net> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: linux-crypto@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-01 08:09:36 +02:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Use nested-BH locking for cryptd_cpu_queue cryptd_queue::cryptd_cpu_queue is a per-CPU variable and relies on disabled BH for its locking. Without per-CPU locking in local_bh_disable() on PREEMPT_RT this data structure requires explicit locking. Add a local_lock_t to the struct cryptd_cpu_queue and use local_lock_nested_bh() for locking. This change adds only lockdep coverage and does not alter the functional behaviour for !PREEMPT_RT. Cc: "David S. Miller" <davem@davemloft.net> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: linux-crypto@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-01 08:09:36 +02:00
crypto: cryptd - disable softirqs in cryptd_queue_worker to prevent data corruption cryptd_queue_worker attempts to prevent simultaneous accesses to crypto workqueue by cryptd_enqueue_request using preempt_disable/preempt_enable. However cryptd_enqueue_request might be called from softirq context, so add local_bh_disable/local_bh_enable to prevent data corruption and panics. Bug report at http://marc.info/?l=linux-crypto-vger&m=134858649616319&w=2 v2: - Disable software interrupts instead of hardware interrupts Cc: stable@vger.kernel.org Reported-by: Gurucharan Shetty <gurucharan.shetty@gmail.com> Signed-off-by: Jussi Kivilinna <jussi.kivilinna@mbnet.fi> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2012-10-21 20:42:28 +03:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
crypto: cryptd - move kcrypto_wq into cryptd kcrypto_wq is only used by cryptd, so move it into cryptd.c and change the workqueue name from "crypto" to "cryptd". Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:53:58 -07:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - process CRYPTO_ALG_INTERNAL The cryptd is used as a wrapper around internal ciphers. Therefore, the cryptd must process the internal cipher by marking cryptd as internal if the underlying cipher is an internal cipher. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-30 21:57:06 +02:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - process CRYPTO_ALG_INTERNAL The cryptd is used as a wrapper around internal ciphers. Therefore, the cryptd must process the internal cipher by marking cryptd as internal if the underlying cipher is an internal cipher. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-30 21:57:06 +02:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Propagate new AEAD implementation flag This patch allows the CRYPTO_ALG_AEAD_NEW flag to be propagated. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-07-09 07:17:19 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - process CRYPTO_ALG_INTERNAL The cryptd is used as a wrapper around internal ciphers. Therefore, the cryptd must process the internal cipher by marking cryptd as internal if the underlying cipher is an internal cipher. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-30 21:57:06 +02:00
crypto: cryptd - Use crypto_grab_aead As AEAD has switched over to using frontend types, the function crypto_init_spawn must not be used since it does not specify a frontend type. Otherwise it leads to a crash when the spawn is used. This patch fixes it by switching over to crypto_grab_aead instead. Fixes: 5d1d65f8bea6 ("crypto: aead - Convert top level interface to new style") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-21 15:10:57 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request context instead of stack for sub-request cryptd is buggy as it tries to use sync_skcipher without going through the proper sync_skcipher interface. In fact it doesn't even need sync_skcipher since it's already a proper skcipher and can easily access the request context instead of using something off the stack. Fixes: 36b3875a97b8 ("crypto: cryptd - Remove VLA usage of skcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-11-11 17:59:17 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request context instead of stack for sub-request cryptd is buggy as it tries to use sync_skcipher without going through the proper sync_skcipher interface. In fact it doesn't even need sync_skcipher since it's already a proper skcipher and can easily access the request context instead of using something off the stack. Fixes: 36b3875a97b8 ("crypto: cryptd - Remove VLA usage of skcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-11-11 17:59:17 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request context instead of stack for sub-request cryptd is buggy as it tries to use sync_skcipher without going through the proper sync_skcipher interface. In fact it doesn't even need sync_skcipher since it's already a proper skcipher and can easily access the request context instead of using something off the stack. Fixes: 36b3875a97b8 ("crypto: cryptd - Remove VLA usage of skcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-11-11 17:59:17 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request context instead of stack for sub-request cryptd is buggy as it tries to use sync_skcipher without going through the proper sync_skcipher interface. In fact it doesn't even need sync_skcipher since it's already a proper skcipher and can easily access the request context instead of using something off the stack. Fixes: 36b3875a97b8 ("crypto: cryptd - Remove VLA usage of skcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-11-11 17:59:17 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request context instead of stack for sub-request cryptd is buggy as it tries to use sync_skcipher without going through the proper sync_skcipher interface. In fact it doesn't even need sync_skcipher since it's already a proper skcipher and can easily access the request context instead of using something off the stack. Fixes: 36b3875a97b8 ("crypto: cryptd - Remove VLA usage of skcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-11-11 17:59:17 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Fix skcipher instance memory leak cryptd_skcipher_free() fails to free the struct skcipher_instance allocated in cryptd_create_skcipher(), leading to a memory leak. This is detected by kmemleak on bootup on ARM64 platforms: unreferenced object 0xffff80003377b180 (size 1024): comm "cryptomgr_probe", pid 822, jiffies 4294894830 (age 52.760s) backtrace: kmem_cache_alloc_trace+0x270/0x2d0 cryptd_create+0x990/0x124c cryptomgr_probe+0x5c/0x1e8 kthread+0x258/0x318 ret_from_fork+0x10/0x1c Fixes: 4e0958d19bd8 ("crypto: cryptd - Add support for skcipher") Cc: <stable@vger.kernel.org> Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-07-02 09:53:25 +02:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Only access common skcipher fields on spawn As skcipher spawns may be of the type lskcipher, only the common fields may be accessed. This was already the case but use the correct helpers to make this more obvious. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-03 11:43:21 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: skcipher - pass instance to crypto_grab_skcipher() Initializing a crypto_skcipher_spawn currently requires: 1. Set spawn->base.inst to point to the instance. 2. Call crypto_grab_skcipher(). But there's no reason for these steps to be separate, and in fact this unneeded complication has caused at least one bug, the one fixed by commit 6db43410179b ("crypto: adiantum - initialize crypto_spawn::inst") So just make crypto_grab_skcipher() take the instance as an argument. To keep the function calls from getting too unwieldy due to this extra argument, also introduce a 'mask' variable into the affected places which weren't already using one. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:45 -08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Only access common skcipher fields on spawn As skcipher spawns may be of the type lskcipher, only the common fields may be accessed. This was already the case but use the correct helpers to make this more obvious. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-03 11:43:21 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Only access common skcipher fields on spawn As skcipher spawns may be of the type lskcipher, only the common fields may be accessed. This was already the case but use the correct helpers to make this more obvious. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-03 11:43:21 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Convert hash to use modern init_tfm/exit_tfm The cryptd hash template was still using the obsolete cra_init/cra_exit interface. Make it use the modern ahash init_tfm/exit_tfm instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:23 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Convert hash to use modern init_tfm/exit_tfm The cryptd hash template was still using the obsolete cra_init/cra_exit interface. Make it use the modern ahash init_tfm/exit_tfm instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:23 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
crypto: cryptd - Convert hash to use modern init_tfm/exit_tfm The cryptd hash template was still using the obsolete cra_init/cra_exit interface. Make it use the modern ahash init_tfm/exit_tfm instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:23 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
crypto: cryptd - Convert hash to use modern init_tfm/exit_tfm The cryptd hash template was still using the obsolete cra_init/cra_exit interface. Make it use the modern ahash init_tfm/exit_tfm instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:23 +08:00
crypto: cryptd - Use crypto_ahash_set_reqsize This patch makes cryptd use crypto_ahash_set_reqsize to avoid accessing crypto_ahash directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 23:06:33 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Add support for cloning hashes Allow cryptd hashes to be cloned. The underlying hash will be cloned. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:25 +08:00
crypto: cryptd - Convert hash to use modern init_tfm/exit_tfm The cryptd hash template was still using the obsolete cra_init/cra_exit interface. Make it use the modern ahash init_tfm/exit_tfm instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:23 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Convert hash to use modern init_tfm/exit_tfm The cryptd hash template was still using the obsolete cra_init/cra_exit interface. Make it use the modern ahash init_tfm/exit_tfm instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:23 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
crypto: remove propagation of CRYPTO_TFM_RES_* flags The CRYPTO_TFM_RES_* flags were apparently meant as a way to make the ->setkey() functions provide more information about errors. But these flags weren't actually being used or tested, and in many cases they weren't being set correctly anyway. So they've now been removed. Also, if someone ever actually needs to start better distinguishing ->setkey() errors (which is somewhat unlikely, as this has been unneeded for a long time), we'd be much better off just defining different return values, like -EINVAL if the key is invalid for the algorithm vs. -EKEYREJECTED if the key was rejected by a policy like "no weak keys". That would be much simpler, less error-prone, and easier to test. So just remove CRYPTO_TFM_RES_MASK and all the unneeded logic that propagates these flags around. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-12-30 21:19:38 -06:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: Resolve shadow warnings Change formal parameters to not clash with global names to eliminate many W=2 warnings. Signed-off-by: Mark Rustad <mark.d.rustad@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2014-07-25 02:53:38 -07:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: Resolve shadow warnings Change formal parameters to not clash with global names to eliminate many W=2 warnings. Signed-off-by: Mark Rustad <mark.d.rustad@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2014-07-25 02:53:38 -07:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Add finup/export/import for hash This patch adds the finup/export/import functions to the cryptd ahash implementation. We simply invoke the underlying shash operations. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-22 11:10:22 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add finup/export/import for hash This patch adds the finup/export/import functions to the cryptd ahash implementation. We simply invoke the underlying shash operations. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-22 11:10:22 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add finup/export/import for hash This patch adds the finup/export/import functions to the cryptd ahash implementation. We simply invoke the underlying shash operations. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-22 11:10:22 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add finup/export/import for hash This patch adds the finup/export/import functions to the cryptd ahash implementation. We simply invoke the underlying shash operations. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-22 11:10:22 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Add finup/export/import for hash This patch adds the finup/export/import functions to the cryptd ahash implementation. We simply invoke the underlying shash operations. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-22 11:10:22 +08:00
crypto: cryptd - initialize child shash_desc on import When calling .import() on a cryptd ahash_request, the structure members that describe the child transform in the shash_desc need to be initialized like they are when calling .init() Cc: stable@vger.kernel.org Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-09-01 14:25:43 +01:00
crypto: cryptd - Add finup/export/import for hash This patch adds the finup/export/import functions to the cryptd ahash implementation. We simply invoke the underlying shash operations. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-22 11:10:22 +08:00
crypto: cryptd - initialize child shash_desc on import When calling .import() on a cryptd ahash_request, the structure members that describe the child transform in the shash_desc need to be initialized like they are when calling .init() Cc: stable@vger.kernel.org Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-09-01 14:25:43 +01:00
crypto: cryptd - Add finup/export/import for hash This patch adds the finup/export/import functions to the cryptd ahash implementation. We simply invoke the underlying shash operations. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-22 11:10:22 +08:00
crypto: cryptd - convert to new way of freeing instances Convert the "cryptd" template to the new way of freeing instances, where a ->free() method is installed to the instance struct itself. This replaces the weakly-typed method crypto_template::free(). This will allow removing support for the old way of freeing instances. Note that the 'default' case in cryptd_free() was already unreachable. So, we aren't missing anything by keeping only the ahash and aead parts. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 20:04:37 -08:00
crypto: cryptd - Switch to template create API This patch changes cryptd to use the template->create function instead of alloc in anticipation for the switch to new style ahash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 18:45:45 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Switch to template create API This patch changes cryptd to use the template->create function instead of alloc in anticipation for the switch to new style ahash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 18:45:45 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
crypto: cryptd - Switch to new style ahash This patch changes cryptd to use the new style ahash type. In particular, the instance is enlarged to encapsulate the new ahash_alg structure. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 19:11:32 +08:00
crypto: cryptd - use crypto_grab_shash() and simplify error paths Make the cryptd template (in the hash case) use the new function crypto_grab_shash() to initialize its shash spawn. This is needed to make all spawns be initialized in a consistent way. This required making cryptd_create_hash() allocate the instance directly rather than use cryptd_alloc_instance(). Also simplify the error handling by taking advantage of crypto_drop_*() now accepting (as a no-op) spawns that haven't been initialized yet, and by taking advantage of crypto_grab_*() now handling ERR_PTR() names. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:53 -08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - process CRYPTO_ALG_INTERNAL The cryptd is used as a wrapper around internal ciphers. Therefore, the cryptd must process the internal cipher by marking cryptd as internal if the underlying cipher is an internal cipher. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-30 21:57:06 +02:00
crypto: cryptd - use crypto_grab_shash() and simplify error paths Make the cryptd template (in the hash case) use the new function crypto_grab_shash() to initialize its shash spawn. This is needed to make all spawns be initialized in a consistent way. This required making cryptd_create_hash() allocate the instance directly rather than use cryptd_alloc_instance(). Also simplify the error handling by taking advantage of crypto_drop_*() now accepting (as a no-op) spawns that haven't been initialized yet, and by taking advantage of crypto_grab_*() now handling ERR_PTR() names. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:53 -08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Switch to new style ahash This patch changes cryptd to use the new style ahash type. In particular, the instance is enlarged to encapsulate the new ahash_alg structure. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 19:11:32 +08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
crypto: cryptd - use crypto_grab_shash() and simplify error paths Make the cryptd template (in the hash case) use the new function crypto_grab_shash() to initialize its shash spawn. This is needed to make all spawns be initialized in a consistent way. This required making cryptd_create_hash() allocate the instance directly rather than use cryptd_alloc_instance(). Also simplify the error handling by taking advantage of crypto_drop_*() now accepting (as a no-op) spawns that haven't been initialized yet, and by taking advantage of crypto_grab_*() now handling ERR_PTR() names. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:53 -08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
crypto: cryptd - use crypto_grab_shash() and simplify error paths Make the cryptd template (in the hash case) use the new function crypto_grab_shash() to initialize its shash spawn. This is needed to make all spawns be initialized in a consistent way. This required making cryptd_create_hash() allocate the instance directly rather than use cryptd_alloc_instance(). Also simplify the error handling by taking advantage of crypto_drop_*() now accepting (as a no-op) spawns that haven't been initialized yet, and by taking advantage of crypto_grab_*() now handling ERR_PTR() names. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:53 -08:00
crypto: cryptd - Use shash algorithms This patch changes cryptd to use shash algorithms instead of the legacy hash interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 21:38:59 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - use crypto_grab_shash() and simplify error paths Make the cryptd template (in the hash case) use the new function crypto_grab_shash() to initialize its shash spawn. This is needed to make all spawns be initialized in a consistent way. This required making cryptd_create_hash() allocate the instance directly rather than use cryptd_alloc_instance(). Also simplify the error handling by taking advantage of crypto_drop_*() now accepting (as a no-op) spawns that haven't been initialized yet, and by taking advantage of crypto_grab_*() now handling ERR_PTR() names. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:53 -08:00
crypto: cryptd - Switch to new style ahash This patch changes cryptd to use the new style ahash type. In particular, the instance is enlarged to encapsulate the new ahash_alg structure. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 19:11:32 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Convert hash to use modern init_tfm/exit_tfm The cryptd hash template was still using the obsolete cra_init/cra_exit interface. Make it use the modern ahash init_tfm/exit_tfm instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:23 +08:00
crypto: cryptd - Add support for cloning hashes Allow cryptd hashes to be cloned. The underlying hash will be cloned. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:25 +08:00
crypto: cryptd - Convert hash to use modern init_tfm/exit_tfm The cryptd hash template was still using the obsolete cra_init/cra_exit interface. Make it use the modern ahash init_tfm/exit_tfm instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-13 14:24:23 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Switch to new style ahash This patch changes cryptd to use the new style ahash type. In particular, the instance is enlarged to encapsulate the new ahash_alg structure. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 19:11:32 +08:00
crypto: cryptd - Add finup/export/import for hash This patch adds the finup/export/import functions to the cryptd ahash implementation. We simply invoke the underlying shash operations. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-22 11:10:22 +08:00
crypto: cryptd - use crypto_grab_shash() and simplify error paths Make the cryptd template (in the hash case) use the new function crypto_grab_shash() to initialize its shash spawn. This is needed to make all spawns be initialized in a consistent way. This required making cryptd_create_hash() allocate the instance directly rather than use cryptd_alloc_instance(). Also simplify the error handling by taking advantage of crypto_drop_*() now accepting (as a no-op) spawns that haven't been initialized yet, and by taking advantage of crypto_grab_*() now handling ERR_PTR() names. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:53 -08:00
crypto: cryptd - pass through absence of ->setkey() When the cryptd template is used to wrap an unkeyed hash algorithm, don't install a ->setkey() method to the cryptd instance. This change is necessary for cryptd to keep working with unkeyed hash algorithms once we start enforcing that ->setkey() is called when present. Cc: stable@vger.kernel.org Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-01-03 11:16:23 -08:00
crypto: cryptd - Switch to new style ahash This patch changes cryptd to use the new style ahash type. In particular, the instance is enlarged to encapsulate the new ahash_alg structure. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 19:11:32 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - convert to new way of freeing instances Convert the "cryptd" template to the new way of freeing instances, where a ->free() method is installed to the instance struct itself. This replaces the weakly-typed method crypto_template::free(). This will allow removing support for the old way of freeing instances. Note that the 'default' case in cryptd_free() was already unreachable. So, we aren't missing anything by keeping only the ahash and aead parts. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 20:04:37 -08:00
crypto: cryptd - Switch to new style ahash This patch changes cryptd to use the new style ahash type. In particular, the instance is enlarged to encapsulate the new ahash_alg structure. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 19:11:32 +08:00
crypto: cryptd - Switch to template create API This patch changes cryptd to use the template->create function instead of alloc in anticipation for the switch to new style ahash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 18:45:45 +08:00
crypto: cryptd - use crypto_grab_shash() and simplify error paths Make the cryptd template (in the hash case) use the new function crypto_grab_shash() to initialize its shash spawn. This is needed to make all spawns be initialized in a consistent way. This required making cryptd_create_hash() allocate the instance directly rather than use cryptd_alloc_instance(). Also simplify the error handling by taking advantage of crypto_drop_*() now accepting (as a no-op) spawns that haven't been initialized yet, and by taking advantage of crypto_grab_*() now handling ERR_PTR() names. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:53 -08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Switch to template create API This patch changes cryptd to use the template->create function instead of alloc in anticipation for the switch to new style ahash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 18:45:45 +08:00
[CRYPTO] cryptd: Add asynchronous hash support This patch adds asynchronous hash support to crypto daemon. Signed-off-by: Loc Ho <lho@amcc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-05-14 21:23:00 +08:00
crypto: cryptd - Add setkey/setauthsize functions for AEAD This patch adds setkey and setauthsize for cryptd AEAD. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:01 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use subreq for AEAD AEAD reuses the existing request object for its child. This is error-prone and unnecessary. This patch adds a subrequest object just like we do for skcipher and hash. This patch also restores the original completion function as we do for skcipher/hash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:53:24 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Fix AEAD request context corruption The AEAD version of cryptd uses the same context for its own state as well as that of the child. In doing so it did not maintain the proper ordering, thus resulting in potential state corruption where the child will overwrite the state stored by cryptd. This patch fixes and also sets the request size properly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-07-06 19:11:03 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use subreq for AEAD AEAD reuses the existing request object for its child. This is error-prone and unnecessary. This patch adds a subrequest object just like we do for skcipher and hash. This patch also restores the original completion function as we do for skcipher/hash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:53:24 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use correct tfm object for AEAD tracking The AEAD code path incorrectly uses the child tfm to track the cryptd refcnt, and then potentially frees the child tfm. Fixes: 81760ea6a95a ("crypto: cryptd - Add helpers to check...") Reported-by: Sowmini Varadhan <sowmini.varadhan@oracle.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-25 16:49:51 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use subreq for AEAD AEAD reuses the existing request object for its child. This is error-prone and unnecessary. This patch adds a subrequest object just like we do for skcipher and hash. This patch also restores the original completion function as we do for skcipher/hash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:53:24 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: api - Use data directly in completion function This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:58:44 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: Resolve shadow warnings Change formal parameters to not clash with global names to eliminate many W=2 warnings. Signed-off-by: Mark Rustad <mark.d.rustad@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2014-07-25 02:53:38 -07:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: Resolve shadow warnings Change formal parameters to not clash with global names to eliminate many W=2 warnings. Signed-off-by: Mark Rustad <mark.d.rustad@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2014-07-25 02:53:38 -07:00
crypto: cryptd - Use request_complete helpers Use the request_complete helpers instead of calling the completion function directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:56:00 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Fix AEAD request context corruption The AEAD version of cryptd uses the same context for its own state as well as that of the child. In doing so it did not maintain the proper ordering, thus resulting in potential state corruption where the child will overwrite the state stored by cryptd. This patch fixes and also sets the request size properly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-07-06 19:11:03 +08:00
crypto: cryptd - Use subreq for AEAD AEAD reuses the existing request object for its child. This is error-prone and unnecessary. This patch adds a subrequest object just like we do for skcipher and hash. This patch also restores the original completion function as we do for skcipher/hash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-08 13:53:24 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - convert to new way of freeing instances Convert the "cryptd" template to the new way of freeing instances, where a ->free() method is installed to the instance struct itself. This replaces the weakly-typed method crypto_template::free(). This will allow removing support for the old way of freeing instances. Note that the 'default' case in cryptd_free() was already unreachable. So, we aren't missing anything by keeping only the ahash and aead parts. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 20:04:37 -08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - process CRYPTO_ALG_INTERNAL The cryptd is used as a wrapper around internal ciphers. Therefore, the cryptd must process the internal cipher by marking cryptd as internal if the underlying cipher is an internal cipher. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-30 21:57:06 +02:00
crypto: cryptd - Use crypto_grab_aead As AEAD has switched over to using frontend types, the function crypto_init_spawn must not be used since it does not specify a frontend type. Otherwise it leads to a crash when the spawn is used. This patch fixes it by switching over to crypto_grab_aead instead. Fixes: 5d1d65f8bea6 ("crypto: aead - Convert top level interface to new style") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-21 15:10:57 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: aead - pass instance to crypto_grab_aead() Initializing a crypto_aead_spawn currently requires: 1. Set spawn->base.inst to point to the instance. 2. Call crypto_grab_aead(). But there's no reason for these steps to be separate, and in fact this unneeded complication has caused at least one bug, the one fixed by commit 6db43410179b ("crypto: adiantum - initialize crypto_spawn::inst") So just make crypto_grab_aead() take the instance as an argument. To keep the function calls from getting too unwieldy due to this extra argument, also introduce a 'mask' variable into the affected places which weren't already using one. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 19:58:46 -08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Use crypto_grab_aead As AEAD has switched over to using frontend types, the function crypto_init_spawn must not be used since it does not specify a frontend type. Otherwise it leads to a crash when the spawn is used. This patch fixes it by switching over to crypto_grab_aead instead. Fixes: 5d1d65f8bea6 ("crypto: aead - Convert top level interface to new style") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-21 15:10:57 +08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Use crypto_grab_aead As AEAD has switched over to using frontend types, the function crypto_init_spawn must not be used since it does not specify a frontend type. Otherwise it leads to a crash when the spawn is used. This patch fixes it by switching over to crypto_grab_aead instead. Fixes: 5d1d65f8bea6 ("crypto: aead - Convert top level interface to new style") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-21 15:10:57 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - convert to new way of freeing instances Convert the "cryptd" template to the new way of freeing instances, where a ->free() method is installed to the instance struct itself. This replaces the weakly-typed method crypto_template::free(). This will allow removing support for the old way of freeing instances. Note that the 'default' case in cryptd_free() was already unreachable. So, we aren't missing anything by keeping only the ahash and aead parts. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-02 20:04:37 -08:00
crypto: cryptd - Convert to new AEAD interface This patch converts cryptd to the new AEAD interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-05-28 22:08:04 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - simplify error handling in cryptd_create_*() Simplify the error handling in the various cryptd_create_*() functions by taking advantage of crypto_grab_*() now handling an ERR_PTR() name and by taking advantage of crypto_drop_*() now accepting (as a no-op) a spawn that hasn't been grabbed yet. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-02-25 20:59:15 -08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Switch to template create API This patch changes cryptd to use the template->create function instead of alloc in anticipation for the switch to new style ahash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 18:45:45 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Switch to template create API This patch changes cryptd to use the template->create function instead of alloc in anticipation for the switch to new style ahash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 18:45:45 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: skcipher - Add lskcipher Add a new API type lskcipher designed for taking straight kernel pointers instead of SG lists. Its relationship to skcipher will be analogous to that between shash and ahash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-09-14 16:28:24 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: hash - remove CRYPTO_ALG_TYPE_DIGEST Remove the unnecessary constant CRYPTO_ALG_TYPE_DIGEST, which has the same value as CRYPTO_ALG_TYPE_HASH. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:54:46 -07:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: algapi - use common mechanism for inheriting flags The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 23:20:38 -07:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Switch to template create API This patch changes cryptd to use the template->create function instead of alloc in anticipation for the switch to new style ahash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 18:45:45 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Switch to template create API This patch changes cryptd to use the template->create function instead of alloc in anticipation for the switch to new style ahash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 18:45:45 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use request context instead of stack for sub-request cryptd is buggy as it tries to use sync_skcipher without going through the proper sync_skcipher interface. In fact it doesn't even need sync_skcipher since it's already a proper skcipher and can easily access the request context instead of using something off the stack. Fixes: 36b3875a97b8 ("crypto: cryptd - Remove VLA usage of skcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-11-11 17:59:17 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add support for skcipher This patch adds skcipher support to cryptd alongside ablkcipher. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-22 20:08:23 +08:00
crypto: cryptd - Add support to access underlaying shash cryptd_alloc_ahash() will allocate a cryptd-ed ahash for specified algorithm name. The new allocated one is guaranteed to be cryptd-ed ahash, so the shash underlying can be gotten via cryptd_ahash_child(). Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-08-06 15:35:20 +10:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Add support to access underlaying shash cryptd_alloc_ahash() will allocate a cryptd-ed ahash for specified algorithm name. The new allocated one is guaranteed to be cryptd-ed ahash, so the shash underlying can be gotten via cryptd_ahash_child(). Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-08-06 15:35:20 +10:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Add support to access underlaying shash cryptd_alloc_ahash() will allocate a cryptd-ed ahash for specified algorithm name. The new allocated one is guaranteed to be cryptd-ed ahash, so the shash underlying can be gotten via cryptd_ahash_child(). Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-08-06 15:35:20 +10:00
crypto: ghash - Add PCLMULQDQ accelerated implementation PCLMULQDQ is used to accelerate the most time-consuming part of GHASH, carry-less multiplication. More information about PCLMULQDQ can be found at: http://software.intel.com/en-us/articles/carry-less-multiplication-and-its-usage-for-computing-the-gcm-mode/ Because PCLMULQDQ changes XMM state, its usage must be enclosed with kernel_fpu_begin/end, which can be used only in process context, the acceleration is implemented as crypto_ahash. That is, request in soft IRQ context will be defered to the cryptd kernel thread. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-10-19 11:53:06 +09:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Add support to access underlaying shash cryptd_alloc_ahash() will allocate a cryptd-ed ahash for specified algorithm name. The new allocated one is guaranteed to be cryptd-ed ahash, so the shash underlying can be gotten via cryptd_ahash_child(). Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-08-06 15:35:20 +10:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Add support to access underlaying shash cryptd_alloc_ahash() will allocate a cryptd-ed ahash for specified algorithm name. The new allocated one is guaranteed to be cryptd-ed ahash, so the shash underlying can be gotten via cryptd_ahash_child(). Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-08-06 15:35:20 +10:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Use refcount_t for refcount Reference counters are preferred to use refcount_t instead of atomic_t. This is because the implementation of refcount_t can prevent overflows and detect possible use-after-free. So convert atomic_t ref counters to refcount_t. Signed-off-by: Chuhong Yuan <hslester96@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-08 16:00:22 +08:00
crypto: cryptd - Add helpers to check whether a tfm is queued This patch adds helpers to check whether a given tfm is currently queued. This is meant to be used by ablk_helper and similar entities to ensure that no reordering is introduced because of requests queued in cryptd with respect to requests being processed in softirq context. The per-cpu queue length limit is also increased to 1000 in line with network limits. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-06-21 16:55:13 +08:00
crypto: cryptd - Adding the AEAD interface type support to cryptd This patch adds AEAD support into the cryptd framework. Having AEAD support in cryptd enables crypto drivers that use the AEAD interface type (such as the patch for AEAD based RFC4106 AES-GCM implementation using Intel New Instructions) to leverage cryptd for asynchronous processing. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com> Signed-off-by: Aidan O'Mahony <aidan.o.mahony@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-09-20 16:05:12 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - WQ_PERCPU added to alloc_workqueue users Currently if a user enqueue a work item using schedule_delayed_work() the used wq is "system_wq" (per-cpu wq) while queue_delayed_work() use WORK_CPU_UNBOUND (used when a cpu is not specified). The same applies to schedule_work() that is using system_wq and queue_work(), that makes use again of WORK_CPU_UNBOUND. This lack of consistentcy cannot be addressed without refactoring the API. alloc_workqueue() treats all queues as per-CPU by default, while unbound workqueues must opt-in via WQ_UNBOUND. This default is suboptimal: most workloads benefit from unbound queues, allowing the scheduler to place worker threads where they’re needed and reducing noise when CPUs are isolated. This default is suboptimal: most workloads benefit from unbound queues, allowing the scheduler to place worker threads where they’re needed and reducing noise when CPUs are isolated. This patch adds a new WQ_PERCPU flag to explicitly request the use of the per-CPU behavior. Both flags coexist for one release cycle to allow callers to transition their calls. Once migration is complete, WQ_UNBOUND can be removed and unbound will become the implicit default. With the introduction of the WQ_PERCPU flag (equivalent to !WQ_UNBOUND), any alloc_workqueue() caller that doesn’t explicitly specify WQ_UNBOUND must now use WQ_PERCPU. All existing users have been updated accordingly. Suggested-by: Tejun Heo <tj@kernel.org> Signed-off-by: Marco Crivellari <marco.crivellari@suse.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-09-05 10:57:01 +02:00
crypto: cryptd - move kcrypto_wq into cryptd kcrypto_wq is only used by cryptd, so move it into cryptd.c and change the workqueue name from "crypto" to "cryptd". Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:53:58 -07:00
crypto: cryptd - Add cryptd_max_cpu_qlen module parameter Make the cryptd queue length configurable. We recently had customer where this needed to be tuned to accommodate the aesni_intel module and prevent packet drop. Signed-off-by: Jon Maxwell <jmaxwell37@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-11-22 16:08:17 +11:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - move kcrypto_wq into cryptd kcrypto_wq is only used by cryptd, so move it into cryptd.c and change the workqueue name from "crypto" to "cryptd". Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:53:58 -07:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - move kcrypto_wq into cryptd kcrypto_wq is only used by cryptd, so move it into cryptd.c and change the workqueue name from "crypto" to "cryptd". Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:53:58 -07:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - move kcrypto_wq into cryptd kcrypto_wq is only used by cryptd, so move it into cryptd.c and change the workqueue name from "crypto" to "cryptd". Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:53:58 -07:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: cryptd - move kcrypto_wq into cryptd kcrypto_wq is only used by cryptd, so move it into cryptd.c and change the workqueue name from "crypto" to "cryptd". Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-20 09:53:58 -07:00
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq Original cryptd thread implementation has scalability issue, this patch solve the issue with a per-CPU thread implementation. struct cryptd_queue is defined to be a per-CPU queue, which holds one struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a struct crypto_queue holds all requests for the CPU, a struct work_struct is used to run all requests for the CPU. Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine shows 19.2% performance gain. The testing script is as follow: -------------------- script begin --------------------------- #!/bin/sh dmc_create() { # Create a crypt device using dmsetup dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0" } dmsetup remove crypt0 dmsetup remove crypt1 dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null dmc_create /dev/ram0 crypt0 dmc_create /dev/ram1 crypt1 cat >tr.sh <<EOF #!/bin/sh for n in \$(seq 10); do dd if=/dev/dm-0 of=/dev/null >& /dev/null & dd if=/dev/dm-1 of=/dev/null >& /dev/null & done wait EOF for n in $(seq 10); do /usr/bin/time sh tr.sh done rm tr.sh -------------------- script end --------------------------- The separator of dm-crypt parameter is changed from "-" to "?", because "-" is used in some cipher driver name too, and cryptds need to specify cipher driver name instead of cipher name. The test result on an Intel Core2 E6400 (two cores) is as follow: without patch: -----------------wo begin -------------------------- 0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6566minor)pagefaults 0swaps 0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6567minor)pagefaults 0swaps 0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6607minor)pagefaults 0swaps 0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6597minor)pagefaults 0swaps 0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6571minor)pagefaults 0swaps 0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6581minor)pagefaults 0swaps 0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6600minor)pagefaults 0swaps -----------------wo end -------------------------- with patch: ------------------w begin -------------------------- 0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6554minor)pagefaults 0swaps 0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6606minor)pagefaults 0swaps 0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6559minor)pagefaults 0swaps 0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6603minor)pagefaults 0swaps 0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6586minor)pagefaults 0swaps 0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6562minor)pagefaults 0swaps 0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6594minor)pagefaults 0swaps 0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6557minor)pagefaults 0swaps ------------------w end -------------------------- The middle value of elapsed time is: wo cryptwq: 0.31 w cryptwq: 0.26 The performance gain is about (0.31-0.26)/0.26 = 0.192. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
Revert "crypto: run initcalls for generic implementations earlier" This reverts commit c4741b23059794bd99beef0f700103b0d983b3fd. Crypto API self-tests no longer run at registration time and now occur either at late_initcall or upon the first use. Therefore the premise of the above commit no longer exists. Revert it and subsequent additions of subsys_initcall and arch_initcall. Note that lib/crypto calls will stay at subsys_initcall (or rather downgraded from arch_initcall) because they may need to occur before Crypto API registration. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-04-30 16:17:02 +08:00
[CRYPTO] cryptd: Add software async crypto daemon This patch adds the cryptd module which is a template that takes a synchronous software crypto algorithm and converts it to an asynchronous one by executing it in a kernel thread. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-04-16 20:49:20 +10:00
crypto: include crypto- module prefix in template This adds the module loading prefix "crypto-" to the template lookup as well. For example, attempting to load 'vfat(blowfish)' via AF_ALG now correctly includes the "crypto-" prefix at every level, correctly rejecting "vfat": net-pf-38 algif-hash crypto-vfat(blowfish) crypto-vfat(blowfish)-all crypto-vfat Reported-by: Mathias Krause <minipli@googlemail.com> Signed-off-by: Kees Cook <keescook@chromium.org> Acked-by: Mathias Krause <minipli@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2014-11-24 16:32:38 -08:00