kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-22 19:14:51 +00:00
Code Issues Releases Packages Activity
linux/Documentation/core-api/kernel-api.rst

434 lines
8.9 KiB
Raw Normal View History

kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
Documentation: kernel-api: drop "Data Types" section In the kernel-api chapter, the section for Data Types only contains "Doubly Linked Lists" and all of the function interfaces for list management. There are no other data types in this section, so collapse this section into "List Management Functions". Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-17 15:19:10 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
kernel.h: split out kstrtox() and simple_strtox() to a separate header kernel.h is being used as a dump for all kinds of stuff for a long time. Here is the attempt to start cleaning it up by splitting out kstrtox() and simple_strtox() helpers. At the same time convert users in header and lib folders to use new header. Though for time being include new header back to kernel.h to avoid twisted indirected includes for existing users. [andy.shevchenko@gmail.com: fix documentation references] Link: https://lkml.kernel.org/r/20210615220003.377901-1-andy.shevchenko@gmail.com Link: https://lkml.kernel.org/r/20210611185815.44103-1-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Cc: Francis Laniel <laniel_francis@privacyrequired.com> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Kars Mulder <kerneldev@karsmulder.nl> Cc: Trond Myklebust <trond.myklebust@hammerspace.com> Cc: Anna Schumaker <anna.schumaker@netapp.com> Cc: "J. Bruce Fields" <bfields@fieldses.org> Cc: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-06-30 18:56:10 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
docs/core-api: Add string helpers API to the list Some times string helpers are needed, but there is nothing about them in the generated documentation. Fill the gap by adding a reference to string_helpers.c exported functions. Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Acked-by: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-06-05 19:39:44 +03:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
string: Rewrite and add more kern-doc for the str*() functions While there were varying degrees of kern-doc for various str*()-family functions, many needed updating and clarification, or to just be entirely written. Update (and relocate) existing kern-doc and add missing functions, sadly shaking my head at how many times I have written "Do not use this function". Include the results in the core kernel API doc. Cc: Bagas Sanjaya <bagasdotme@gmail.com> Cc: Andy Shevchenko <andy@kernel.org> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: linux-hardening@vger.kernel.org Tested-by: Akira Yokosawa <akiyks@gmail.com> Link: https://lore.kernel.org/lkml/9b0cf584-01b3-3013-b800-1ef59fe82476@gmail.com Signed-off-by: Kees Cook <keescook@chromium.org>
2022-09-02 14:33:44 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
kernel-doc: core-api: include string.h into core-api core-api should show all the various string functions including the newly added stracpy and stracpy_pad. Miscellanea: o Update the Returns: value for strscpy o fix a defect with %NUL) [joe@perches.com: correct return of -E2BIG descriptions] Link: http://lkml.kernel.org/r/29f998b4c1a9d69fbeae70500ba0daa4b340c546.1563889130.git.joe@perches.com Link: http://lkml.kernel.org/r/224a6ebf39955f4107c0c376d66155d970e46733.1563841972.git.joe@perches.com Signed-off-by: Joe Perches <joe@perches.com> Reviewed-by: Kees Cook <keescook@chromium.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Stephen Kitt <steve@sk2.org> Cc: Nitin Gote <nitin.r.gote@intel.com> Cc: Rasmus Villemoes <rasmus.villemoes@prevas.dk> Cc: Jann Horn <jannh@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-09-25 16:46:13 -07:00
docs/core-api: move *{str,mem}dup* to "String Manipulation" The string and memory duplication routines fit better to the "String Manipulation" section than to "The SLAB Cache". Link: http://lkml.kernel.org/r/1532626360-16650-5-git-send-email-rppt@linux.vnet.ibm.com Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com> Reviewed-by: Andrew Morton <akpm@linux-foundation.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Matthew Wilcox <willy@infradead.org> Cc: Michal Hocko <mhocko@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-08-23 17:01:09 -07:00
documentation: core-api: rearrange a few kernel-api chapters and sections Rearrange some kernel-api chapters and sections to group them together better. - move Bit Operations from Basic C Library Functions to Basic Kernel Library Functions (now adjacent to Bitmap Operations since they are not typical C library functions) - move Sorting from Math Functions to Basic Kernel Library Functions since sort functions are more Basic than Math Functions - move Text Searching from Math Functions to Basic Kernel Library Functions (keep Sorting and Searching close to each other) - combine CRC and Math functions together into the (newly named) CRC and Math Functions chapter Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Acked-by: Matthew Wilcox <mawilcox@microsoft.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2018-04-26 18:11:02 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
kasan: support instrumented bitops combined with generic bitops Currently bitops-instrumented.h assumes that the architecture provides atomic, non-atomic and locking bitops (e.g. both set_bit and __set_bit). This is true on x86 and s390, but is not always true: there is a generic bitops/non-atomic.h header that provides generic non-atomic operations, and also a generic bitops/lock.h for locking operations. powerpc uses the generic non-atomic version, so it does not have it's own e.g. __set_bit that could be renamed arch___set_bit. Split up bitops-instrumented.h to mirror the atomic/non-atomic/lock split. This allows arches to only include the headers where they have arch-specific versions to rename. Update x86 and s390. (The generic operations are automatically instrumented because they're written in C, not asm.) Suggested-by: Christophe Leroy <christophe.leroy@c-s.fr> Reviewed-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Daniel Axtens <dja@axtens.net> Acked-by: Marco Elver <elver@google.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190820024941.12640-1-dja@axtens.net
2019-08-20 12:49:40 +10:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
documentation: kernel-api: add more info on bitmap functions There are some good comments about bitmap operations in lib/bitmap.c and include/linux/bitmap.h, so format them for document generation and pull them into core-api/kernel-api.rst. I converted the "tables" of functions from using tabs to using spaces so that they are more readable in the source file and in the generated output. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-10-16 16:32:51 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
Documentation: kernel-api: add bitmap operations from linux/bitmap.h Add <linux/bitmap.h> to kernel-api Bitmap Operations section. Fix kernel-doc nitpicks in <linux/bitmap.h>. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Acked-by: Yury Norov <ynorov@caviumnetworks.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-17 19:07:10 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
Documentation: core-api: Add error pointer functions to kernel-api Bring the error pointer functions (e.g. ERR_PTR(), PTR_ERR()) into the docs build so that they can be cross-referenced elsewhere. List them as kernel library functions in the kernel-api document. Nowhere else seems to fit, and they need to go *somewhere*. Signed-off-by: James Seo <james@equiv.tech> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20230509175543.2065835-4-james@equiv.tech
2023-05-09 10:55:48 -07:00
documentation: core-api: rearrange a few kernel-api chapters and sections Rearrange some kernel-api chapters and sections to group them together better. - move Bit Operations from Basic C Library Functions to Basic Kernel Library Functions (now adjacent to Bitmap Operations since they are not typical C library functions) - move Sorting from Math Functions to Basic Kernel Library Functions since sort functions are more Basic than Math Functions - move Text Searching from Math Functions to Basic Kernel Library Functions (keep Sorting and Searching close to each other) - combine CRC and Math functions together into the (newly named) CRC and Math Functions chapter Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Acked-by: Matthew Wilcox <mawilcox@microsoft.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2018-04-26 18:11:02 -07:00
overflow: Fix kern-doc markup for functions Fix the kern-doc markings for several of the overflow helpers and move their location into the core kernel API documentation, where it belongs (it's not driver-specific). Cc: Jonathan Corbet <corbet@lwn.net> Cc: linux-doc@vger.kernel.org Cc: linux-hardening@vger.kernel.org Reviewed-by: Akira Yokosawa <akiyks@gmail.com> Signed-off-by: Kees Cook <keescook@chromium.org>
2022-09-26 12:45:32 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
docs: clean up and add rest of CRC functions to kernel-api.rst Add the rest of the CRC library functions to kernel-api. - try to clarify crc32() by adding '@' to a function parameter - reorder kernel-api CRC functions to be less random - add more CRC functions to kernel-api - correct the function parameter names in several places Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-08 16:35:55 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
docs: clean up and add rest of CRC functions to kernel-api.rst Add the rest of the CRC library functions to kernel-api. - try to clarify crc32() by adding '@' to a function parameter - reorder kernel-api CRC functions to be less random - add more CRC functions to kernel-api - correct the function parameter names in several places Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-08 16:35:55 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
docs: clean up and add rest of CRC functions to kernel-api.rst Add the rest of the CRC library functions to kernel-api. - try to clarify crc32() by adding '@' to a function parameter - reorder kernel-api CRC functions to be less random - add more CRC functions to kernel-api - correct the function parameter names in several places Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-08 16:35:55 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
docs: clean up and add rest of CRC functions to kernel-api.rst Add the rest of the CRC library functions to kernel-api. - try to clarify crc32() by adding '@' to a function parameter - reorder kernel-api CRC functions to be less random - add more CRC functions to kernel-api - correct the function parameter names in several places Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-08 16:35:55 -07:00
Documentation: add kernel-api section on Math functions Add a kernel-api section on Math Functions. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-30 08:43:53 -07:00
lib/math: Move dvb_math.c into lib/math/int_log.c Some existing and new users may benefit from the intlog2() and intlog10() APIs, make them wide available. Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org> Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org> Link: https://lore.kernel.org/r/20230619172019.21457-2-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Reviewed-by: Randy Dunlap <rdunlap@infradead.org> Link: https://lore.kernel.org/r/20230703135211.87416-2-andriy.shevchenko@linux.intel.com Signed-off-by: Mark Brown <broonie@kernel.org>
2023-07-03 16:52:08 +03:00
docs/core-api: Add integer power functions to the list Some times integer power functions, such as int_sqrt(), are needed, but there is nothing about them in the generated documentation. Fill the gap by adding a reference to the corresponding exported functions. Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Acked-by: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-06-05 19:51:13 +03:00
Documentation: add kernel-api section on Math functions Add a kernel-api section on Math Functions. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-30 08:43:53 -07:00
lib: Move mathematic helpers to separate folder For better maintenance and expansion move the mathematic helpers to the separate folder. No functional change intended. Note, the int_sqrt() is not used as a part of lib, so, moved to regular obj. Link: http://lkml.kernel.org/r/20190323172531.80025-1-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Thierry Reding <thierry.reding@gmail.com> Cc: Lee Jones <lee.jones@linaro.org> Cc: Daniel Thompson <daniel.thompson@linaro.org> Cc: Ray Jui <rjui@broadcom.com> [mchehab+samsung@kernel.org: fix broken doc references for div64.c and gcd.c] Link: http://lkml.kernel.org/r/734f49bae5d4052b3c25691dfefad59bea2e5843.1555580999.git.mchehab+samsung@kernel.org Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-14 15:43:05 -07:00
Documentation: add kernel-api section on Math functions Add a kernel-api section on Math Functions. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-09-30 08:43:53 -07:00
Documentation: add UUID/GUID to kernel-api Update kernel-doc notation in lib/uuid.c and then add UUID/GUID function interfaces to kernel-api. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> [jc: tweaked the uuid_is_valid() kerneldoc] Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-11-29 12:32:42 -08:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
module: add debug stats to help identify memory pressure Loading modules with finit_module() can end up using vmalloc(), vmap() and vmalloc() again, for a total of up to 3 separate allocations in the worst case for a single module. We always kernel_read*() the module, that's a vmalloc(). Then vmap() is used for the module decompression, and if so the last read buffer is freed as we use the now decompressed module buffer to stuff data into our copy module. The last allocation is specific to each architectures but pretty much that's generally a series of vmalloc() calls or a variation of vmalloc to handle ELF sections with special permissions. Evaluation with new stress-ng module support [1] with just 100 ops is proving that you can end up using GiBs of data easily even with all care we have in the kernel and userspace today in trying to not load modules which are already loaded. 100 ops seems to resemble the sort of pressure a system with about 400 CPUs can create on module loading. Although issues relating to duplicate module requests due to each CPU inucurring a new module reuest is silly and some of these are being fixed, we currently lack proper tooling to help diagnose easily what happened, when it happened and who likely is to blame -- userspace or kernel module autoloading. Provide an initial set of stats which use debugfs to let us easily scrape post-boot information about failed loads. This sort of information can be used on production worklaods to try to optimize *avoiding* redundant memory pressure using finit_module(). There's a few examples that can be provided: A 255 vCPU system without the next patch in this series applied: Startup finished in 19.143s (kernel) + 7.078s (userspace) = 26.221s graphical.target reached after 6.988s in userspace And 13.58 GiB of virtual memory space lost due to failed module loading: root@big ~ # cat /sys/kernel/debug/modules/stats Mods ever loaded 67 Mods failed on kread 0 Mods failed on decompress 0 Mods failed on becoming 0 Mods failed on load 1411 Total module size 11464704 Total mod text size 4194304 Failed kread bytes 0 Failed decompress bytes 0 Failed becoming bytes 0 Failed kmod bytes 14588526272 Virtual mem wasted bytes 14588526272 Average mod size 171115 Average mod text size 62602 Average fail load bytes 10339140 Duplicate failed modules: module-name How-many-times Reason kvm_intel 249 Load kvm 249 Load irqbypass 8 Load crct10dif_pclmul 128 Load ghash_clmulni_intel 27 Load sha512_ssse3 50 Load sha512_generic 200 Load aesni_intel 249 Load crypto_simd 41 Load cryptd 131 Load evdev 2 Load serio_raw 1 Load virtio_pci 3 Load nvme 3 Load nvme_core 3 Load virtio_pci_legacy_dev 3 Load virtio_pci_modern_dev 3 Load t10_pi 3 Load virtio 3 Load crc32_pclmul 6 Load crc64_rocksoft 3 Load crc32c_intel 40 Load virtio_ring 3 Load crc64 3 Load The following screen shot, of a simple 8vcpu 8 GiB KVM guest with the next patch in this series applied, shows 226.53 MiB are wasted in virtual memory allocations which due to duplicate module requests during boot. It also shows an average module memory size of 167.10 KiB and an an average module .text + .init.text size of 61.13 KiB. The end shows all modules which were detected as duplicate requests and whether or not they failed early after just the first kernel_read*() call or late after we've already allocated the private space for the module in layout_and_allocate(). A system with module decompression would reveal more wasted virtual memory space. We should put effort now into identifying the source of these duplicate module requests and trimming these down as much possible. Larger systems will obviously show much more wasted virtual memory allocations. root@kmod ~ # cat /sys/kernel/debug/modules/stats Mods ever loaded 67 Mods failed on kread 0 Mods failed on decompress 0 Mods failed on becoming 83 Mods failed on load 16 Total module size 11464704 Total mod text size 4194304 Failed kread bytes 0 Failed decompress bytes 0 Failed becoming bytes 228959096 Failed kmod bytes 8578080 Virtual mem wasted bytes 237537176 Average mod size 171115 Average mod text size 62602 Avg fail becoming bytes 2758544 Average fail load bytes 536130 Duplicate failed modules: module-name How-many-times Reason kvm_intel 7 Becoming kvm 7 Becoming irqbypass 6 Becoming & Load crct10dif_pclmul 7 Becoming & Load ghash_clmulni_intel 7 Becoming & Load sha512_ssse3 6 Becoming & Load sha512_generic 7 Becoming & Load aesni_intel 7 Becoming crypto_simd 7 Becoming & Load cryptd 3 Becoming & Load evdev 1 Becoming serio_raw 1 Becoming nvme 3 Becoming nvme_core 3 Becoming t10_pi 3 Becoming virtio_pci 3 Becoming crc32_pclmul 6 Becoming & Load crc64_rocksoft 3 Becoming crc32c_intel 3 Becoming virtio_pci_modern_dev 2 Becoming virtio_pci_legacy_dev 1 Becoming crc64 2 Becoming virtio 2 Becoming virtio_ring 2 Becoming [0] https://github.com/ColinIanKing/stress-ng.git [1] echo 0 > /proc/sys/vm/oom_dump_tasks ./stress-ng --module 100 --module-name xfs Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
2023-03-28 20:03:19 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
Documentation: core-api: update kernel-doc reference to kmod.c Commit d6f819908f8aac ("module: fold usermode helper kmod into modules directory") moves kmod helper implementation (kmod.c) to kernel/module/ directory but forgets to update its reference on kernel api doc, hence: WARNING: kernel-doc './scripts/kernel-doc -rst -enable-lineno -sphinx-version 2.4.4 -export ./kernel/kmod.c' failed with return code 2 Update the reference. Fixes: d6f819908f8aac ("module: fold usermode helper kmod into modules directory") Reported-by: Stephen Rothwell <sfr@canb.auug.org.au> Link: https://lore.kernel.org/linux-next/20230324154413.19cc78be@canb.auug.org.au/ Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com> Reviewed-by: Liam Beguin <liambeguin@gmail.com> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
2023-03-24 15:53:10 +07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
module: add debug stats to help identify memory pressure Loading modules with finit_module() can end up using vmalloc(), vmap() and vmalloc() again, for a total of up to 3 separate allocations in the worst case for a single module. We always kernel_read*() the module, that's a vmalloc(). Then vmap() is used for the module decompression, and if so the last read buffer is freed as we use the now decompressed module buffer to stuff data into our copy module. The last allocation is specific to each architectures but pretty much that's generally a series of vmalloc() calls or a variation of vmalloc to handle ELF sections with special permissions. Evaluation with new stress-ng module support [1] with just 100 ops is proving that you can end up using GiBs of data easily even with all care we have in the kernel and userspace today in trying to not load modules which are already loaded. 100 ops seems to resemble the sort of pressure a system with about 400 CPUs can create on module loading. Although issues relating to duplicate module requests due to each CPU inucurring a new module reuest is silly and some of these are being fixed, we currently lack proper tooling to help diagnose easily what happened, when it happened and who likely is to blame -- userspace or kernel module autoloading. Provide an initial set of stats which use debugfs to let us easily scrape post-boot information about failed loads. This sort of information can be used on production worklaods to try to optimize *avoiding* redundant memory pressure using finit_module(). There's a few examples that can be provided: A 255 vCPU system without the next patch in this series applied: Startup finished in 19.143s (kernel) + 7.078s (userspace) = 26.221s graphical.target reached after 6.988s in userspace And 13.58 GiB of virtual memory space lost due to failed module loading: root@big ~ # cat /sys/kernel/debug/modules/stats Mods ever loaded 67 Mods failed on kread 0 Mods failed on decompress 0 Mods failed on becoming 0 Mods failed on load 1411 Total module size 11464704 Total mod text size 4194304 Failed kread bytes 0 Failed decompress bytes 0 Failed becoming bytes 0 Failed kmod bytes 14588526272 Virtual mem wasted bytes 14588526272 Average mod size 171115 Average mod text size 62602 Average fail load bytes 10339140 Duplicate failed modules: module-name How-many-times Reason kvm_intel 249 Load kvm 249 Load irqbypass 8 Load crct10dif_pclmul 128 Load ghash_clmulni_intel 27 Load sha512_ssse3 50 Load sha512_generic 200 Load aesni_intel 249 Load crypto_simd 41 Load cryptd 131 Load evdev 2 Load serio_raw 1 Load virtio_pci 3 Load nvme 3 Load nvme_core 3 Load virtio_pci_legacy_dev 3 Load virtio_pci_modern_dev 3 Load t10_pi 3 Load virtio 3 Load crc32_pclmul 6 Load crc64_rocksoft 3 Load crc32c_intel 40 Load virtio_ring 3 Load crc64 3 Load The following screen shot, of a simple 8vcpu 8 GiB KVM guest with the next patch in this series applied, shows 226.53 MiB are wasted in virtual memory allocations which due to duplicate module requests during boot. It also shows an average module memory size of 167.10 KiB and an an average module .text + .init.text size of 61.13 KiB. The end shows all modules which were detected as duplicate requests and whether or not they failed early after just the first kernel_read*() call or late after we've already allocated the private space for the module in layout_and_allocate(). A system with module decompression would reveal more wasted virtual memory space. We should put effort now into identifying the source of these duplicate module requests and trimming these down as much possible. Larger systems will obviously show much more wasted virtual memory allocations. root@kmod ~ # cat /sys/kernel/debug/modules/stats Mods ever loaded 67 Mods failed on kread 0 Mods failed on decompress 0 Mods failed on becoming 83 Mods failed on load 16 Total module size 11464704 Total mod text size 4194304 Failed kread bytes 0 Failed decompress bytes 0 Failed becoming bytes 228959096 Failed kmod bytes 8578080 Virtual mem wasted bytes 237537176 Average mod size 171115 Average mod text size 62602 Avg fail becoming bytes 2758544 Average fail load bytes 536130 Duplicate failed modules: module-name How-many-times Reason kvm_intel 7 Becoming kvm 7 Becoming irqbypass 6 Becoming & Load crct10dif_pclmul 7 Becoming & Load ghash_clmulni_intel 7 Becoming & Load sha512_ssse3 6 Becoming & Load sha512_generic 7 Becoming & Load aesni_intel 7 Becoming crypto_simd 7 Becoming & Load cryptd 3 Becoming & Load evdev 1 Becoming serio_raw 1 Becoming nvme 3 Becoming nvme_core 3 Becoming t10_pi 3 Becoming virtio_pci 3 Becoming crc32_pclmul 6 Becoming & Load crc64_rocksoft 3 Becoming crc32c_intel 3 Becoming virtio_pci_modern_dev 2 Becoming virtio_pci_legacy_dev 1 Becoming crc64 2 Becoming virtio 2 Becoming virtio_ring 2 Becoming [0] https://github.com/ColinIanKing/stress-ng.git [1] echo 0 > /proc/sys/vm/oom_dump_tasks ./stress-ng --module 100 --module-name xfs Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
2023-03-28 20:03:19 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
doc: module: update file references Adjust documents to the file moves made by commit cfc1d277891e ("module: Move all into module/"). Thanks to Yanteng Si for helping me to update Documentation/translations/zh_CN/core-api/kernel-api.rst Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Yanteng Si <siyanteng@loongson.cn> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
2022-06-15 06:33:03 +09:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
Fix Documentation build due to rename of main.c to mtrr.c This fixes this documentation build error that is due to a file rename: Error: Cannot open file ../arch/x86/kernel/cpu/mtrr/main.c Fixes: 0afe832e55a7 ("Merge branch 'x86-cleanups-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip") Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-06-16 17:09:41 -07:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
block: Add bio_for_each_folio_all() Allow callers to iterate over each folio instead of each page. The bio need not have been constructed using folios originally. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: Jens Axboe <axboe@kernel.dk> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
2021-01-04 10:58:17 -05:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
block: move fs/block_dev.c to block/bdev.c Move it together with the rest of the block layer. Signed-off-by: Christoph Hellwig <hch@lst.de> Link: https://lore.kernel.org/r/20210907141303.1371844-3-hch@lst.de Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-09-07 16:13:03 +02:00
kernel-api.tmpl: convert it to ReST Brainless conversion of genericirq.tmpl book to ReST, via Documentation/sphinx/tmplcvt Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-30 17:11:32 -03:00
doc: Add RCU files to docbook-generation files Suggested-by: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-07-04 14:42:20 -07:00
rcu: Add more RCU files to kernel-api.rst Recent changes and additions to RCU have not been reflected in Documentation/core-api/kernel-api.rst, which makes it harder to find the kernel-doc headers in recently added RCU files. Therefore, add those files. Signed-off-by: Paul E. McKenney <paulmck@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Luis Chamberlain <mcgrof@kernel.org> Cc: Kees Cook <keescook@chromium.org> Cc: Liam Beguin <liambeguin@gmail.com> Cc: <linux-doc@vger.kernel.org>
2023-05-11 11:59:09 -07:00