No description
  • C 96.8%
  • Assembly 0.9%
  • Rust 0.7%
  • Shell 0.6%
  • Python 0.5%
  • Other 0.3%
Find a file
Eric Biggers 84dd048cf8 crypto: x86/crc32c - eliminate jump table and excessive unrolling
crc32c-pcl-intel-asm_64.S has a loop with 1 to 127 iterations fully
unrolled and uses a jump table to jump into the correct location.  This
optimization is misguided, as it bloats the binary code size and
introduces an indirect call.  x86_64 CPUs can predict loops well, so it
is fine to just use a loop instead.  Loop bookkeeping instructions can
compete with the crc instructions for the ALUs, but this is easily
mitigated by unrolling the loop by a smaller amount, such as 4 times.

Therefore, re-roll the loop and make related tweaks to the code.

This reduces the binary code size of crc_pclmul() from 4546 bytes to 418
bytes, a 91% reduction.  In general it also makes the code faster, with
some large improvements seen when retpoline is enabled.

More detailed performance results are shown below.  They are given as
percent improvement in throughput (negative means regressed) for CPU
microarchitecture vs. input length in bytes.  E.g. an improvement from
40 GB/s to 50 GB/s would be listed as 25%.

Table 1: Results with retpoline enabled (the default):

                       |   512 |   833 |  1024 |  2000 |  3173 |  4096 |
  ---------------------+-------+-------+-------+------ +-------+-------+
  Intel Haswell        | 35.0% | 20.7% | 17.8% |  9.7% | -0.2% |  4.4% |
  Intel Emerald Rapids | 66.8% | 45.2% | 36.3% | 19.3% |  0.0% |  5.4% |
  AMD Zen 2            | 29.5% | 17.2% | 13.5% |  8.6% | -0.5% |  2.8% |

Table 2: Results with retpoline disabled:

                       |   512 |   833 |  1024 |  2000 |  3173 |  4096 |
  ---------------------+-------+-------+-------+------ +-------+-------+
  Intel Haswell        |  3.3% |  4.8% |  4.5% |  0.9% | -2.9% |  0.3% |
  Intel Emerald Rapids |  7.5% |  6.4% |  5.2% |  2.3% | -0.0% |  0.6% |
  AMD Zen 2            | 11.8% |  1.4% |  0.2% |  1.3% | -0.9% | -0.2% |

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
History 2024-10-26 14:41:59 +08:00
arch crypto: x86/crc32c - eliminate jump table and excessive unrolling 2024-10-26 14:41:59 +08:00
block for-6.12/block-20240925 2024-09-25 14:56:40 -07:00
certs sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3 2024-09-20 19:52:48 +03:00
crypto crypto: jitter - output full sample from test interface 2024-10-19 08:44:30 +08:00
Documentation dt-bindings: imx-rng: Allow passing only "fsl,imx31-rnga" 2024-10-19 08:44:30 +08:00
drivers crypto: crypto4xx - use devm in probe 2024-10-19 08:44:30 +08:00
fs bcachefs fixes for 6.11-rc1 2024-09-29 09:17:44 -07:00
include crypto: hisilicon/qm - fix the coding specifications issue 2024-10-05 13:22:06 +08:00
init Rust changes for v6.12 2024-09-25 10:25:40 -07:00
io_uring for-6.12/io_uring-20240922 2024-09-24 11:11:38 -07:00
ipc struct fd layout change (and conversion to accessor helpers) 2024-09-23 09:35:36 -07:00
kernel Locking changes for v6.12: 2024-09-29 08:51:30 -07:00
lib crypto: lib/mpi - Fix an "Uninitialized scalar variable" issue 2024-10-05 13:22:05 +08:00
LICENSES LICENSES: add 0BSD license text 2024-09-01 20:43:24 -07:00
mm 19 hotfixes. 13 are cc:stable. 2024-09-27 10:27:22 -07:00
net Three CephFS fixes from Xiubo and Luis and a bunch of assorted 2024-09-28 08:40:36 -07:00
rust Rust changes for v6.12 2024-09-25 10:25:40 -07:00
samples [tree-wide] finally take no_llseek out 2024-09-27 08:18:43 -07:00
scripts Reduce Coccinelle choices in string_choices.cocci 2024-09-28 21:33:11 +02:00
security crypto: rsassa-pkcs1 - Migrate to sig_alg backend 2024-10-05 13:22:04 +08:00
sound [tree-wide] finally take no_llseek out 2024-09-27 08:18:43 -07:00
tools linux_kselftest-next-6.12-rc1-fixes 2024-09-29 08:37:03 -07:00
usr initramfs: shorten cmd_initfs in usr/Makefile 2024-07-16 01:07:52 +09:00
virt x86: 2024-09-28 09:20:14 -07:00
.clang-format clang-format: Update with v6.11-rc1's for_each macro list 2024-08-02 13:20:31 +02:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.editorconfig .editorconfig: remove trim_trailing_whitespace option 2024-06-13 16:47:52 +02:00
.get_maintainer.ignore Add Jeff Kirsher to .get_maintainer.ignore 2024-03-08 11:36:54 +00:00
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore Kbuild updates for v6.12 2024-09-24 13:02:06 -07:00
.mailmap Summary 2024-09-24 11:08:40 -07:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Mark powerpc spufs as orphaned 2024-08-19 21:27:56 +10:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS MAINTAINERS: Make Kristen Accardi the IAA crypto driver maintainer 2024-10-05 13:22:06 +08:00
Makefile Linux 6.12-rc1 2024-09-29 15:06:19 -07:00
README README: Fix spelling 2024-03-18 03:36:32 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the reStructuredText markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.