linux

License cleanup: add SPDX GPL-2.0 license identifier to files with no license Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

2017-11-01 15:07:57 +01:00

kbuild: introduce ccflags-y, asflags-y and ldflags-y Introduce ccflags-y, asflags-y and ldflags-y so we soon can deprecate use of EXTRA_CFLAGS, EXTRA_AFLAGS and EXTRA_LDFLAGS. This patch does not touch any in-tree users - thats next round. Lets get this committed first and then fix the users of the soon to be deprecated variants next. The rationale behind this change is to introduce support for makefile fragments like: ccflags-$(CONFIG_WHATEVER_DEBUG) := -DDEBUG As a replacement for the uglier: ifeq ($(CONFIG_WHATEVER_DEBUG),y) EXTRA_CFLAGS := -DDEBUG endif Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

2007-10-15 22:25:06 +02:00

kbuild: comments cleanup in Makefile.lib It has: 1. Move comments close to what it want to comment. 2. Comments cleanup & improvement. Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-10-09 11:49:11 +08:00

kbuild: remove KBUILD_SUBDIR_ASFLAGS and KBUILD_SUBDIR_CCFLAGS Accumulate subdir-{cc,as}flags-y directly to KBUILD_{A,C}FLAGS. Remove KBUILD_SUBDIR_{AS,CC}FLAGS. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

2017-10-10 20:43:21 +09:00

2021-07-03 16:42:57 +02:00

kbuild: introduce subdir-ccflags-y Following patch introduce support for setting options to gcc that has effect for current directory and all subdirectories. The typical use case are an architecture or a subsystem that decide to cover all files with -Werror. Today alpha, mips and sparc uses -Werror in almost all their Makefile- with subdir-ccflag-y it is now simpler to do so as only the top-level directories needs to be covered. Likewise if we decide to cover a full subsystem such as net/ with -Werror this is done by adding a single line to net/Makefile. Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Thomas Gleixner <tglx@linutronix.de>

2009-04-19 11:04:26 +02:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: refactor subdir-ym calculation Remove the unneeded variables, __subdir-y and __subdir-m. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-06-01 14:56:55 +09:00

kbuild: update modules.order only when contained modules are updated Make modules.order depend on $(obj-m), and use if_changed to build it. This will avoid unneeded update of modules.order, which will be useful to optimize the modpost stage. Currently, the second pass of modpost is always invoked. By checking the timestamp of modules.order, we can avoid the unneeded modpost. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-06-01 14:57:00 +09:00

kbuild: do not create orphan built-in.a or obj-y objects Both 'obj-y += foo/' and 'obj-m += foo/' request Kbuild to visit the sub-directory foo/, but the difference is that only the former combines foo/built-in.a into the built-in.a of the current directory because everything in sub-directories visited by obj-m is supposed to be modular. So, it makes sense to create built-in.a only if that sub-directory is reachable by the chain of obj-y. Otherwise, built-in.a will not be linked into vmlinux anyway. For the same reason, it is pointless to compile obj-y objects in the directory visited by obj-m. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2019-12-19 01:04:28 +09:00

kbuild: rename built-in.o to built-in.a Incremental linking is gone, so rename built-in.o to built-in.a, which is the usual extension for archive files. This patch does two things, first is a simple search/replace: git grep -l 'built-in\.o' | xargs sed -i 's/built-in\.o/built-in\.a/g' The second is to invert nesting of nested text manipulations to avoid filtering built-in.a out from libs-y2: -libs-y2 := $(filter-out %.a, $(patsubst %/, %/built-in.a, $(libs-y))) +libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y))) Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-02-11 00:25:04 +10:00

kbuild: do not create orphan built-in.a or obj-y objects Both 'obj-y += foo/' and 'obj-m += foo/' request Kbuild to visit the sub-directory foo/, but the difference is that only the former combines foo/built-in.a into the built-in.a of the current directory because everything in sub-directories visited by obj-m is supposed to be modular. So, it makes sense to create built-in.a only if that sub-directory is reachable by the chain of obj-y. Otherwise, built-in.a will not be linked into vmlinux anyway. For the same reason, it is pointless to compile obj-y objects in the directory visited by obj-m. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2019-12-19 01:04:28 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: parameterize the .o part of suffix-search The suffix-search macro hard-codes the suffix, '.o'. Make it a parameter so that the multi-search and real-search macros can be reused for foo-dtbs syntax introduced by commit 15d16d6dadf6 ("kbuild: Add generic rule to apply fdtoverlay"). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:55 +09:00

kbuild: treat an object as multi-used when $(foo-) is set Currently, Kbuild treats an object as multi-used when any of $(foo-objs), $(foo-y), $(foo-m) is set. It makes more sense to check $(foo-) as well. In the context of foo-$(CONFIG_FOO_FEATURE1), CONFIG_FOO_FEATURE1 could be unset. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-08-06 15:39:19 +09:00

kbuild: parameterize the .o part of suffix-search The suffix-search macro hard-codes the suffix, '.o'. Make it a parameter so that the multi-search and real-search macros can be reused for foo-dtbs syntax introduced by commit 15d16d6dadf6 ("kbuild: Add generic rule to apply fdtoverlay"). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:55 +09:00

kbuild: rename multi-used-* to multi-obj-* I think multi-obj-* is clearer, and more consistent with real-obj-*. Rename as follows: multi-used-y -> multi-obj-y multi-used-m -> multi-obj-m multi-used -> multi-obj-ym Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-03-06 15:11:20 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: link $(real-obj-y) instead of $(obj-y) into built-in.a In Kbuild, Makefiles can add the same object to obj-y multiple times. So, obj-y += foo.o obj-y += foo.o is fine. However, this is not true when the same object is added multiple times via composite objects. For example, obj-y += foo.o bar.o foo-objs := foo-bar-common.o foo-only.o bar-objs := foo-bar-common.o bar-only.o causes build error because two instances of foo-bar-common.o are linked into the vmlinux. Makefiles tend to invent ugly work-around, for example - lib/zstd/Makefile - drivers/net/ethernet/cavium/liquidio/Makefile The technique used in Kbuild to avoid the multiple definition error is to use $(filter $(obj-y), $^). Here, $^ lists the names of all the prerequisites with duplicated names removed. By replacing it with $(filter $(real-obj-y), $^) we can do likewise for composite objects. For built-in objects, we do not need to keep the composite object structure. We can simply expand them, and link $(real-obj-y) to built-in.a. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-03-19 20:26:08 +09:00

kbuild: parameterize the .o part of suffix-search The suffix-search macro hard-codes the suffix, '.o'. Make it a parameter so that the multi-search and real-search macros can be reused for foo-dtbs syntax introduced by commit 15d16d6dadf6 ("kbuild: Add generic rule to apply fdtoverlay"). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:55 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: rename hostprogs-y/always to hostprogs/always-y In old days, the "host-progs" syntax was used for specifying host programs. It was renamed to the current "hostprogs-y" in 2004. It is typically useful in scripts/Makefile because it allows Kbuild to selectively compile host programs based on the kernel configuration. This commit renames like follows: always -> always-y hostprogs-y -> hostprogs So, scripts/Makefile will look like this: always-$(CONFIG_BUILD_BIN2C) += ... always-$(CONFIG_KALLSYMS) += ... ... hostprogs := $(always-y) $(always-m) I think this makes more sense because a host program is always a host program, irrespective of the kernel configuration. We want to specify which ones to compile by CONFIG options, so always-y will be handier. The "always", "hostprogs-y", "hostprogs-m" will be kept for backward compatibility for a while. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-02-02 01:49:24 +09:00

kbuild: introduce hostprogs-always-y and userprogs-always-y To build host programs, you need to add the program names to 'hostprogs' to use the necessary build rule, but it is not enough to build them because there is no dependency. There are two types of host programs: built as the prerequisite of another (e.g. gen_crc32table in lib/Makefile), or always built when Kbuild visits the Makefile (e.g. genksyms in scripts/genksyms/Makefile). The latter is typical in Makefiles under scripts/, which contains host programs globally used during the kernel build. To build them, you need to add them to both 'hostprogs' and 'always-y'. This commit adds hostprogs-always-y as a shorthand. The same applies to user programs. net/bpfilter/Makefile builds bpfilter_umh on demand, hence always-y is unneeded. In contrast, programs under samples/ are added to both 'userprogs' and 'always-y' so they are always built when Kbuild visits the Makefiles. userprogs-always-y works as a shorthand. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>

2020-08-01 21:27:18 +09:00

kbuild: handle dtb-y and CONFIG_OF_ALL_DTBS natively in Makefile.lib If CONFIG_OF_ALL_DTBS is enabled, "make ARCH=arm64 dtbs" compiles each DTB twice; one from arch/arm64/boot/dts/*/Makefile and the other from the dtb-$(CONFIG_OF_ALL_DTBS) line in arch/arm64/boot/dts/Makefile. It could be a race problem when building DTBS in parallel. Another minor issue is CONFIG_OF_ALL_DTBS covers only *.dts in vendor sub-directories, so this broke when Broadcom added one more hierarchy in arch/arm64/boot/dts/broadcom/<soc>/. One idea to fix the issues in a clean way is to move DTB handling to Kbuild core scripts. Makefile.dtbinst already recognizes dtb-y natively, so it should not hurt to do so. Add $(dtb-y) to extra-y, and $(dtb-) as well if CONFIG_OF_ALL_DTBS is enabled. All clutter things in Makefiles go away. As a bonus clean-up, I also removed dts-dirs. Just use subdir-y directly to traverse sub-directories. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Arnd Bergmann <arnd@arndb.de> [robh: corrected BUILTIN_DTB to CONFIG_BUILTIN_DTB] Signed-off-by: Rob Herring <robh@kernel.org>

2017-11-05 14:30:53 +09:00

kbuild: Simplify builds with CONFIG_OF_ALL_DTBS We update 'always-y' based on CONFIG_OF_ALL_DTBS three times. It would be far more straight forward if we rather update dtb-y to include all .dtb files if CONFIG_OF_ALL_DTBS is enabled. Acked-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> Reviewed-by: Frank Rowand <frank.rowand@sony.com> Tested-by: Frank Rowand <frank.rowand@sony.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://lore.kernel.org/r/7fe7e5ef6ed75450ddf6c224b8adb53059e504e2.1615354376.git.viresh.kumar@linaro.org

2021-03-10 11:05:29 +05:30

kbuild: refactor fdtoverlay rule Rename overlay-y to multi-dtb-y, which is a consistent name with multi-obj-y. Also, use multi-search to avoid code duplication. Introduce real-dtb-y, which is a consistent name with real-obj-y, to contain primitive blobs compiled from *.dts. This is used to calculate the list of *.dt.yaml files. Set -@ to base DTB without using $(eval ). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:56 +09:00

kbuild: Add generic rule to apply fdtoverlay Add a generic rule to apply fdtoverlay in Makefile.lib, so every platform doesn't need to carry the complex rule. This also automatically adds "DTC_FLAGS_foo_base += -@" for all base files. The platform's Makefile only needs to have this now: foo-dtbs := foo_base.dtb foo_overlay1.dtbo foo_overlay2.dtbo dtb-y := foo.dtb We don't want to run schema checks on foo.dtb (as foo.dts doesn't exist) and the Makefile is updated accordingly. Acked-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Rob Herring <robh@kernel.org> Co-developed-by: Viresh Kumar <viresh.kumar@linaro.org> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> Reviewed-by: Frank Rowand <frank.rowand@sony.com> Tested-by: Frank Rowand <frank.rowand@sony.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://lore.kernel.org/r/20920b0df6b067aca4040459a9677d7d1d6d766a.1615354376.git.viresh.kumar@linaro.org

2021-03-10 11:05:30 +05:30

kbuild: use always-y instead of extra-y As commit d0e628cd817f ("kbuild: doc: clarify the difference between extra-y and always-y") explained, extra-y should be used for listing the prerequisites of vmlinux. These targets are not related to vmlinux. always-y is a better fix. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Rob Herring <robh@kernel.org>

2021-01-20 15:23:51 +09:00

kbuild: handle dtb-y and CONFIG_OF_ALL_DTBS natively in Makefile.lib If CONFIG_OF_ALL_DTBS is enabled, "make ARCH=arm64 dtbs" compiles each DTB twice; one from arch/arm64/boot/dts/*/Makefile and the other from the dtb-$(CONFIG_OF_ALL_DTBS) line in arch/arm64/boot/dts/Makefile. It could be a race problem when building DTBS in parallel. Another minor issue is CONFIG_OF_ALL_DTBS covers only *.dts in vendor sub-directories, so this broke when Broadcom added one more hierarchy in arch/arm64/boot/dts/broadcom/<soc>/. One idea to fix the issues in a clean way is to move DTB handling to Kbuild core scripts. Makefile.dtbinst already recognizes dtb-y natively, so it should not hurt to do so. Add $(dtb-y) to extra-y, and $(dtb-) as well if CONFIG_OF_ALL_DTBS is enabled. All clutter things in Makefiles go away. As a bonus clean-up, I also removed dts-dirs. Just use subdir-y directly to traverse sub-directories. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Arnd Bergmann <arnd@arndb.de> [robh: corrected BUILTIN_DTB to CONFIG_BUILTIN_DTB] Signed-off-by: Rob Herring <robh@kernel.org>

2017-11-05 14:30:53 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: list sub-directories in ./Kbuild Use the ordinary obj-y syntax to list subdirectories. Note1: Previously, the link order of lib-y depended on CONFIG_MODULES; lib-y was linked before drivers-y when CONFIG_MODULES=y, otherwise after drivers-y. This was a bug of commit 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y"), but it was not a big deal after all. Now, all objects listed in lib-y are linked last, irrespective of CONFIG_MODULES. Note2: Finally, the single target build in arch/*/lib/ works correctly. There was a bug report about this. [1] $ make ARCH=arm arch/arm/lib/findbit.o CALL scripts/checksyscalls.sh AS arch/arm/lib/findbit.o [1]: https://lore.kernel.org/linux-kbuild/YvUQOwL6lD4%2F5%2FU6@shell.armlinux.org.uk/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

2022-09-25 03:19:10 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: rename hostprogs-y/always to hostprogs/always-y In old days, the "host-progs" syntax was used for specifying host programs. It was renamed to the current "hostprogs-y" in 2004. It is typically useful in scripts/Makefile because it allows Kbuild to selectively compile host programs based on the kernel configuration. This commit renames like follows: always -> always-y hostprogs-y -> hostprogs So, scripts/Makefile will look like this: always-$(CONFIG_BUILD_BIN2C) += ... always-$(CONFIG_KALLSYMS) += ... ... hostprogs := $(always-y) $(always-m) I think this makes more sense because a host program is always a host program, irrespective of the kernel configuration. We want to specify which ones to compile by CONFIG options, so always-y will be handier. The "always", "hostprogs-y", "hostprogs-m" will be kept for backward compatibility for a while. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-02-02 01:49:24 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: rename real-objs-y/m to real-obj-y/m When I was refactoring Makefiles, I stupidly mistook 'real-obj-y' for 'real-objs-y' over and over again. Finally, I decide to rename it to 'real-obj-y'. This is consistent with 'obj-y', 'subdir-obj-y'. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-03-19 20:26:07 +09:00

kbuild: rename multi-used-* to multi-obj-* I think multi-obj-* is clearer, and more consistent with real-obj-*. Rename as follows: multi-used-y -> multi-obj-y multi-used-m -> multi-obj-m multi-used -> multi-obj-ym Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-03-06 15:11:20 +09:00

kbuild: refactor fdtoverlay rule Rename overlay-y to multi-dtb-y, which is a consistent name with multi-obj-y. Also, use multi-search to avoid code duplication. Introduce real-dtb-y, which is a consistent name with real-obj-y, to contain primitive blobs compiled from *.dts. This is used to calculate the list of *.dt.yaml files. Set -@ to base DTB without using $(eval ). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:56 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: list sub-directories in ./Kbuild Use the ordinary obj-y syntax to list subdirectories. Note1: Previously, the link order of lib-y depended on CONFIG_MODULES; lib-y was linked before drivers-y when CONFIG_MODULES=y, otherwise after drivers-y. This was a bug of commit 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y"), but it was not a big deal after all. Now, all objects listed in lib-y are linked last, irrespective of CONFIG_MODULES. Note2: Finally, the single target build in arch/*/lib/ works correctly. There was a bug report about this. [1] $ make ARCH=arm arch/arm/lib/findbit.o CALL scripts/checksyscalls.sh AS arch/arm/lib/findbit.o [1]: https://lore.kernel.org/linux-kbuild/YvUQOwL6lD4%2F5%2FU6@shell.armlinux.org.uk/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

2022-09-25 03:19:10 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: move modname and modname-multi close to modname_flags Just a cosmetic change to put related code close together. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

2018-03-19 18:01:27 +09:00

kbuild: pass KBUILD_MODFILE when compiling builtin objects When compiling, Kbuild passes KBUILD_BASENAME (basename of the object) and KBUILD_MODNAME (basename of the module). This commit adds another one, KBUILD_MODFILE, which is the path of the module. (or, the path of the module it would end up in if it were compiled as a module.) The next commit will use this to generate modules.builtin without tristate.conf. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2019-12-19 17:33:28 +09:00

kbuild: rename multi-used-* to multi-obj-* I think multi-obj-* is clearer, and more consistent with real-obj-*. Rename as follows: multi-used-y -> multi-obj-y multi-used-m -> multi-obj-m multi-used -> multi-obj-ym Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-03-06 15:11:20 +09:00

kbuild: refactor modname-multi by using suffix-search Improve the readability slightly. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:57 +09:00

kbuild: move modname and modname-multi close to modname_flags Just a cosmetic change to put related code close together. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

2018-03-19 18:01:27 +09:00

kbuild: replace $(if A,A,B) with $(or A,B) $(or ...) is available since GNU Make 3.81, and useful to shorten the code in some places. Covert as follows: $(if A,A,B) --> $(or A,B) This patch also converts: $(if A, A, B) --> $(or A, B) Strictly speaking, the latter is not an equivalent conversion because GNU Make keeps spaces after commas; if A is not empty, $(if A, A, B) expands to " A", while $(or A, B) expands to "A". Anyway, preceding spaces are not significant in the code hunks I touched. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

2022-02-11 14:14:11 +09:00

kbuild: pass KBUILD_MODFILE when compiling builtin objects When compiling, Kbuild passes KBUILD_BASENAME (basename of the object) and KBUILD_MODNAME (basename of the module). This commit adds another one, KBUILD_MODFILE, which is the path of the module. (or, the path of the module it would end up in if it were compiled as a module.) The next commit will use this to generate modules.builtin without tristate.conf. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2019-12-19 17:33:28 +09:00

kbuild: move modname and modname-multi close to modname_flags Just a cosmetic change to put related code close together. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

2018-03-19 18:01:27 +09:00

kbuild: change *FLAGS_<basetarget>.o to take the path relative to $(obj) Kbuild provides per-file compiler flag addition/removal: CFLAGS_<basetarget>.o CFLAGS_REMOVE_<basetarget>.o AFLAGS_<basetarget>.o AFLAGS_REMOVE_<basetarget>.o CPPFLAGS_<basetarget>.lds HOSTCFLAGS_<basetarget>.o HOSTCXXFLAGS_<basetarget>.o The <basetarget> is the filename of the target with its directory and suffix stripped. This syntax comes into a trouble when two files with the same basename appear in one Makefile, for example: obj-y += foo.o obj-y += dir/foo.o CFLAGS_foo.o := <some-flags> Here, the <some-flags> applies to both foo.o and dir/foo.o The real world problem is: scripts/kconfig/util.c scripts/kconfig/lxdialog/util.c Both files are compiled into scripts/kconfig/mconf, but only the latter should be given with the ncurses flags. It is more sensible to use the relative path to the Makefile, like this: obj-y += foo.o CFLAGS_foo.o := <some-flags> obj-y += dir/foo.o CFLAGS_dir/foo.o := <other-flags> At first, I attempted to replace $(basetarget) with $*. The $* variable is replaced with the stem ('%') part in a pattern rule. This works with most of cases, but does not for explicit rules. For example, arch/ia64/lib/Makefile reuses rule_as_o_S in its own explicit rules, so $* will be empty, resulting in ignoring the per-file AFLAGS. I introduced a new variable, target-stem, which can be used also from explicit rules. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Marc Zyngier <maz@kernel.org>

2019-08-30 13:34:01 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: comments cleanup in Makefile.lib It has: 1. Move comments close to what it want to comment. 2. Comments cleanup & improvement. Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-10-09 11:49:11 +08:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

init: lto: ensure initcall ordering With LTO, the compiler doesn't necessarily obey the link order for initcalls, and initcall variables need globally unique names to avoid collisions at link time. This change exports __KBUILD_MODNAME and adds the initcall_id() macro, which uses it together with __COUNTER__ and __LINE__ to help ensure these variables have unique names, and moves each variable to its own section when LTO is enabled, so the correct order can be specified using a linker script. The generate_initcall_ordering.pl script uses nm to find initcalls from the object files passed to the linker, and generates a linker script that specifies the same order for initcalls that we would have without LTO. With LTO enabled, the script is called in link-vmlinux.sh through jobserver-exec to limit the number of jobs spawned. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20201211184633.3213045-8-samitolvanen@google.com

2020-12-11 10:46:24 -08:00

kbuild: Get rid of KBUILD_STR The compiler can accept -DKBUILD_MODNAME="foo", it's just a matter of quoting. That way, we reduce the gcc command line a bit. Signed-off-by: Michal Marek <mmarek@suse.com>

2016-03-17 16:32:14 +01:00

init: lto: ensure initcall ordering With LTO, the compiler doesn't necessarily obey the link order for initcalls, and initcall variables need globally unique names to avoid collisions at link time. This change exports __KBUILD_MODNAME and adds the initcall_id() macro, which uses it together with __COUNTER__ and __LINE__ to help ensure these variables have unique names, and moves each variable to its own section when LTO is enabled, so the correct order can be specified using a linker script. The generate_initcall_ordering.pl script uses nm to find initcalls from the object files passed to the linker, and generates a linker script that specifies the same order for initcalls that we would have without LTO. With LTO enabled, the script is called in link-vmlinux.sh through jobserver-exec to limit the number of jobs spawned. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20201211184633.3213045-8-samitolvanen@google.com

2020-12-11 10:46:24 -08:00

kbuild: pass KBUILD_MODFILE when compiling builtin objects When compiling, Kbuild passes KBUILD_BASENAME (basename of the object) and KBUILD_MODNAME (basename of the module). This commit adds another one, KBUILD_MODFILE, which is the path of the module. (or, the path of the module it would end up in if it were compiled as a module.) The next commit will use this to generate modules.builtin without tristate.conf. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2019-12-19 17:33:28 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: introduce ccflags-remove-y and asflags-remove-y CFLAGS_REMOVE_<file>.o filters out flags when compiling a particular object, but there is no convenient way to do that for every object in a directory. Add ccflags-remove-y and asflags-remove-y to make it easily. Use ccflags-remove-y to clean up some Makefiles. The add/remove order works as follows: [1] KBUILD_CFLAGS specifies compiler flags used globally [2] ccflags-y adds compiler flags for all objects in the current Makefile [3] ccflags-remove-y removes compiler flags for all objects in the current Makefile (New feature) [4] CFLAGS_<file> adds compiler flags per file. [5] CFLAGS_REMOVE_<file> removes compiler flags per file. Having [3] before [4] allows us to remove flags from most (but not all) objects in the current Makefile. For example, kernel/trace/Makefile removes $(CC_FLAGS_FTRACE) from all objects in the directory, then adds it back to trace_selftest_dynamic.o and CFLAGS_trace_kprobe_selftest.o The same applies to lib/livepatch/Makefile. Please note ccflags-remove-y has no effect to the sub-directories. In contrast, the previous notation got rid of compiler flags also from all the sub-directories. The following are not affected because they have no sub-directories: arch/arm/boot/compressed/ arch/powerpc/xmon/ arch/sh/ kernel/trace/ However, lib/ has several sub-directories. To keep the behavior, I added ccflags-remove-y to all Makefiles in subdirectories of lib/, except the following: lib/vdso/Makefile - Kbuild does not descend into this Makefile lib/raid/test/Makefile - This is not used for the kernel build I think commit 2464a609ded0 ("ftrace: do not trace library functions") excluded too much. In the next commit, I will remove ccflags-remove-y from the sub-directories of lib/. Suggested-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org> Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc) Acked-by: Brendan Higgins <brendanhiggins@google.com> (KUnit) Tested-by: Anders Roxell <anders.roxell@linaro.org>

2020-07-07 18:21:16 +09:00

2021-07-03 16:42:57 +02:00

kbuild: introduce ccflags-remove-y and asflags-remove-y CFLAGS_REMOVE_<file>.o filters out flags when compiling a particular object, but there is no convenient way to do that for every object in a directory. Add ccflags-remove-y and asflags-remove-y to make it easily. Use ccflags-remove-y to clean up some Makefiles. The add/remove order works as follows: [1] KBUILD_CFLAGS specifies compiler flags used globally [2] ccflags-y adds compiler flags for all objects in the current Makefile [3] ccflags-remove-y removes compiler flags for all objects in the current Makefile (New feature) [4] CFLAGS_<file> adds compiler flags per file. [5] CFLAGS_REMOVE_<file> removes compiler flags per file. Having [3] before [4] allows us to remove flags from most (but not all) objects in the current Makefile. For example, kernel/trace/Makefile removes $(CC_FLAGS_FTRACE) from all objects in the directory, then adds it back to trace_selftest_dynamic.o and CFLAGS_trace_kprobe_selftest.o The same applies to lib/livepatch/Makefile. Please note ccflags-remove-y has no effect to the sub-directories. In contrast, the previous notation got rid of compiler flags also from all the sub-directories. The following are not affected because they have no sub-directories: arch/arm/boot/compressed/ arch/powerpc/xmon/ arch/sh/ kernel/trace/ However, lib/ has several sub-directories. To keep the behavior, I added ccflags-remove-y to all Makefiles in subdirectories of lib/, except the following: lib/vdso/Makefile - Kbuild does not descend into this Makefile lib/raid/test/Makefile - This is not used for the kernel build I think commit 2464a609ded0 ("ftrace: do not trace library functions") excluded too much. In the next commit, I will remove ccflags-remove-y from the sub-directories of lib/. Suggested-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org> Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc) Acked-by: Brendan Higgins <brendanhiggins@google.com> (KUnit) Tested-by: Anders Roxell <anders.roxell@linaro.org>

2020-07-07 18:21:16 +09:00

kbuild: change *FLAGS_<basetarget>.o to take the path relative to $(obj) Kbuild provides per-file compiler flag addition/removal: CFLAGS_<basetarget>.o CFLAGS_REMOVE_<basetarget>.o AFLAGS_<basetarget>.o AFLAGS_REMOVE_<basetarget>.o CPPFLAGS_<basetarget>.lds HOSTCFLAGS_<basetarget>.o HOSTCXXFLAGS_<basetarget>.o The <basetarget> is the filename of the target with its directory and suffix stripped. This syntax comes into a trouble when two files with the same basename appear in one Makefile, for example: obj-y += foo.o obj-y += dir/foo.o CFLAGS_foo.o := <some-flags> Here, the <some-flags> applies to both foo.o and dir/foo.o The real world problem is: scripts/kconfig/util.c scripts/kconfig/lxdialog/util.c Both files are compiled into scripts/kconfig/mconf, but only the latter should be given with the ncurses flags. It is more sensible to use the relative path to the Makefile, like this: obj-y += foo.o CFLAGS_foo.o := <some-flags> obj-y += dir/foo.o CFLAGS_dir/foo.o := <other-flags> At first, I attempted to replace $(basetarget) with $*. The $* variable is replaced with the stem ('%') part in a pattern rule. This works with most of cases, but does not for explicit rules. For example, arch/ia64/lib/Makefile reuses rule_as_o_S in its own explicit rules, so $* will be empty, resulting in ignoring the per-file AFLAGS. I introduced a new variable, target-stem, which can be used also from explicit rules. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Marc Zyngier <maz@kernel.org>

2019-08-30 13:34:01 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

gcov: add gcov profiling infrastructure Enable the use of GCC's coverage testing tool gcov [1] with the Linux kernel. gcov may be useful for: * debugging (has this code been reached at all?) * test improvement (how do I change my test to cover these lines?) * minimizing kernel configurations (do I need this option if the associated code is never run?) The profiling patch incorporates the following changes: * change kbuild to include profiling flags * provide functions needed by profiling code * present profiling data as files in debugfs Note that on some architectures, enabling gcc's profiling option "-fprofile-arcs" for the entire kernel may trigger compile/link/ run-time problems, some of which are caused by toolchain bugs and others which require adjustment of architecture code. For this reason profiling the entire kernel is initially restricted to those architectures for which it is known to work without changes. This restriction can be lifted once an architecture has been tested and found compatible with gcc's profiling. Profiling of single files or directories is still available on all platforms (see config help text). [1] http://gcc.gnu.org/onlinedocs/gcc/Gcov.html Signed-off-by: Peter Oberparleiter <oberpar@linux.vnet.ibm.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Huang Ying <ying.huang@intel.com> Cc: Li Wei <W.Li@Sun.COM> Cc: Michael Ellerman <michaele@au1.ibm.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Heiko Carstens <heicars2@linux.vnet.ibm.com> Cc: Martin Schwidefsky <mschwid2@linux.vnet.ibm.com> Cc: Rusty Russell <rusty@rustcorp.com.au> Cc: WANG Cong <xiyou.wangcong@gmail.com> Cc: Sam Ravnborg <sam@ravnborg.org> Cc: Jeff Dike <jdike@addtoit.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2009-06-17 16:28:08 -07:00

kasan: add kernel address sanitizer infrastructure Kernel Address sanitizer (KASan) is a dynamic memory error detector. It provides fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. KASAN uses compile-time instrumentation for checking every memory access, therefore GCC > v4.9.2 required. v4.9.2 almost works, but has issues with putting symbol aliases into the wrong section, which breaks kasan instrumentation of globals. This patch only adds infrastructure for kernel address sanitizer. It's not available for use yet. The idea and some code was borrowed from [1]. Basic idea: The main idea of KASAN is to use shadow memory to record whether each byte of memory is safe to access or not, and use compiler's instrumentation to check the shadow memory on each memory access. Address sanitizer uses 1/8 of the memory addressable in kernel for shadow memory and uses direct mapping with a scale and offset to translate a memory address to its corresponding shadow address. Here is function to translate address to corresponding shadow address: unsigned long kasan_mem_to_shadow(unsigned long addr) { return (addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET; } where KASAN_SHADOW_SCALE_SHIFT = 3. So for every 8 bytes there is one corresponding byte of shadow memory. The following encoding used for each shadow byte: 0 means that all 8 bytes of the corresponding memory region are valid for access; k (1 <= k <= 7) means that the first k bytes are valid for access, and other (8 - k) bytes are not; Any negative value indicates that the entire 8-bytes are inaccessible. Different negative values used to distinguish between different kinds of inaccessible memory (redzones, freed memory) (see mm/kasan/kasan.h). To be able to detect accesses to bad memory we need a special compiler. Such compiler inserts a specific function calls (__asan_load*(addr), __asan_store*(addr)) before each memory access of size 1, 2, 4, 8 or 16. These functions check whether memory region is valid to access or not by checking corresponding shadow memory. If access is not valid an error printed. Historical background of the address sanitizer from Dmitry Vyukov: "We've developed the set of tools, AddressSanitizer (Asan), ThreadSanitizer and MemorySanitizer, for user space. We actively use them for testing inside of Google (continuous testing, fuzzing, running prod services). To date the tools have found more than 10'000 scary bugs in Chromium, Google internal codebase and various open-source projects (Firefox, OpenSSL, gcc, clang, ffmpeg, MySQL and lots of others): [2] [3] [4]. The tools are part of both gcc and clang compilers. We have not yet done massive testing under the Kernel AddressSanitizer (it's kind of chicken and egg problem, you need it to be upstream to start applying it extensively). To date it has found about 50 bugs. Bugs that we've found in upstream kernel are listed in [5]. We've also found ~20 bugs in out internal version of the kernel. Also people from Samsung and Oracle have found some. [...] As others noted, the main feature of AddressSanitizer is its performance due to inline compiler instrumentation and simple linear shadow memory. User-space Asan has ~2x slowdown on computational programs and ~2x memory consumption increase. Taking into account that kernel usually consumes only small fraction of CPU and memory when running real user-space programs, I would expect that kernel Asan will have ~10-30% slowdown and similar memory consumption increase (when we finish all tuning). I agree that Asan can well replace kmemcheck. We have plans to start working on Kernel MemorySanitizer that finds uses of unitialized memory. Asan+Msan will provide feature-parity with kmemcheck. As others noted, Asan will unlikely replace debug slab and pagealloc that can be enabled at runtime. Asan uses compiler instrumentation, so even if it is disabled, it still incurs visible overheads. Asan technology is easily portable to other architectures. Compiler instrumentation is fully portable. Runtime has some arch-dependent parts like shadow mapping and atomic operation interception. They are relatively easy to port." Comparison with other debugging features: ======================================== KMEMCHECK: - KASan can do almost everything that kmemcheck can. KASan uses compile-time instrumentation, which makes it significantly faster than kmemcheck. The only advantage of kmemcheck over KASan is detection of uninitialized memory reads. Some brief performance testing showed that kasan could be x500-x600 times faster than kmemcheck: $ netperf -l 30 MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to localhost (127.0.0.1) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec no debug: 87380 16384 16384 30.00 41624.72 kasan inline: 87380 16384 16384 30.00 12870.54 kasan outline: 87380 16384 16384 30.00 10586.39 kmemcheck: 87380 16384 16384 30.03 20.23 - Also kmemcheck couldn't work on several CPUs. It always sets number of CPUs to 1. KASan doesn't have such limitation. DEBUG_PAGEALLOC: - KASan is slower than DEBUG_PAGEALLOC, but KASan works on sub-page granularity level, so it able to find more bugs. SLUB_DEBUG (poisoning, redzones): - SLUB_DEBUG has lower overhead than KASan. - SLUB_DEBUG in most cases are not able to detect bad reads, KASan able to detect both reads and writes. - In some cases (e.g. redzone overwritten) SLUB_DEBUG detect bugs only on allocation/freeing of object. KASan catch bugs right before it will happen, so we always know exact place of first bad read/write. [1] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel [2] https://code.google.com/p/address-sanitizer/wiki/FoundBugs [3] https://code.google.com/p/thread-sanitizer/wiki/FoundBugs [4] https://code.google.com/p/memory-sanitizer/wiki/FoundBugs [5] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel#Trophies Based on work by Andrey Konovalov. Signed-off-by: Andrey Ryabinin <a.ryabinin@samsung.com> Acked-by: Michal Marek <mmarek@suse.cz> Signed-off-by: Andrey Konovalov <adech.fo@gmail.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Konstantin Serebryany <kcc@google.com> Cc: Dmitry Chernenkov <dmitryc@google.com> Cc: Yuri Gribov <tetra2005@gmail.com> Cc: Konstantin Khlebnikov <koct9i@gmail.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Christoph Lameter <cl@linux.com> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Christoph Lameter <cl@linux.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: David Rientjes <rientjes@google.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2015-02-13 14:39:17 -08:00

kasan, arm64: expand CONFIG_KASAN checks Some #ifdef CONFIG_KASAN checks are only relevant for software KASAN modes (either related to shadow memory or compiler instrumentation). Expand those into CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS. Link: https://lkml.kernel.org/r/e6971e432dbd72bb897ff14134ebb7e169bdcf0c.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov <andreyknvl@google.com> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Alexander Potapenko <glider@google.com> Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Branislav Rankov <Branislav.Rankov@arm.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Evgenii Stepanov <eugenis@google.com> Cc: Kevin Brodsky <kevin.brodsky@arm.com> Cc: Marco Elver <elver@google.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2020-12-22 12:02:06 -08:00

kasan: add kernel address sanitizer infrastructure Kernel Address sanitizer (KASan) is a dynamic memory error detector. It provides fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. KASAN uses compile-time instrumentation for checking every memory access, therefore GCC > v4.9.2 required. v4.9.2 almost works, but has issues with putting symbol aliases into the wrong section, which breaks kasan instrumentation of globals. This patch only adds infrastructure for kernel address sanitizer. It's not available for use yet. The idea and some code was borrowed from [1]. Basic idea: The main idea of KASAN is to use shadow memory to record whether each byte of memory is safe to access or not, and use compiler's instrumentation to check the shadow memory on each memory access. Address sanitizer uses 1/8 of the memory addressable in kernel for shadow memory and uses direct mapping with a scale and offset to translate a memory address to its corresponding shadow address. Here is function to translate address to corresponding shadow address: unsigned long kasan_mem_to_shadow(unsigned long addr) { return (addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET; } where KASAN_SHADOW_SCALE_SHIFT = 3. So for every 8 bytes there is one corresponding byte of shadow memory. The following encoding used for each shadow byte: 0 means that all 8 bytes of the corresponding memory region are valid for access; k (1 <= k <= 7) means that the first k bytes are valid for access, and other (8 - k) bytes are not; Any negative value indicates that the entire 8-bytes are inaccessible. Different negative values used to distinguish between different kinds of inaccessible memory (redzones, freed memory) (see mm/kasan/kasan.h). To be able to detect accesses to bad memory we need a special compiler. Such compiler inserts a specific function calls (__asan_load*(addr), __asan_store*(addr)) before each memory access of size 1, 2, 4, 8 or 16. These functions check whether memory region is valid to access or not by checking corresponding shadow memory. If access is not valid an error printed. Historical background of the address sanitizer from Dmitry Vyukov: "We've developed the set of tools, AddressSanitizer (Asan), ThreadSanitizer and MemorySanitizer, for user space. We actively use them for testing inside of Google (continuous testing, fuzzing, running prod services). To date the tools have found more than 10'000 scary bugs in Chromium, Google internal codebase and various open-source projects (Firefox, OpenSSL, gcc, clang, ffmpeg, MySQL and lots of others): [2] [3] [4]. The tools are part of both gcc and clang compilers. We have not yet done massive testing under the Kernel AddressSanitizer (it's kind of chicken and egg problem, you need it to be upstream to start applying it extensively). To date it has found about 50 bugs. Bugs that we've found in upstream kernel are listed in [5]. We've also found ~20 bugs in out internal version of the kernel. Also people from Samsung and Oracle have found some. [...] As others noted, the main feature of AddressSanitizer is its performance due to inline compiler instrumentation and simple linear shadow memory. User-space Asan has ~2x slowdown on computational programs and ~2x memory consumption increase. Taking into account that kernel usually consumes only small fraction of CPU and memory when running real user-space programs, I would expect that kernel Asan will have ~10-30% slowdown and similar memory consumption increase (when we finish all tuning). I agree that Asan can well replace kmemcheck. We have plans to start working on Kernel MemorySanitizer that finds uses of unitialized memory. Asan+Msan will provide feature-parity with kmemcheck. As others noted, Asan will unlikely replace debug slab and pagealloc that can be enabled at runtime. Asan uses compiler instrumentation, so even if it is disabled, it still incurs visible overheads. Asan technology is easily portable to other architectures. Compiler instrumentation is fully portable. Runtime has some arch-dependent parts like shadow mapping and atomic operation interception. They are relatively easy to port." Comparison with other debugging features: ======================================== KMEMCHECK: - KASan can do almost everything that kmemcheck can. KASan uses compile-time instrumentation, which makes it significantly faster than kmemcheck. The only advantage of kmemcheck over KASan is detection of uninitialized memory reads. Some brief performance testing showed that kasan could be x500-x600 times faster than kmemcheck: $ netperf -l 30 MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to localhost (127.0.0.1) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec no debug: 87380 16384 16384 30.00 41624.72 kasan inline: 87380 16384 16384 30.00 12870.54 kasan outline: 87380 16384 16384 30.00 10586.39 kmemcheck: 87380 16384 16384 30.03 20.23 - Also kmemcheck couldn't work on several CPUs. It always sets number of CPUs to 1. KASan doesn't have such limitation. DEBUG_PAGEALLOC: - KASan is slower than DEBUG_PAGEALLOC, but KASan works on sub-page granularity level, so it able to find more bugs. SLUB_DEBUG (poisoning, redzones): - SLUB_DEBUG has lower overhead than KASan. - SLUB_DEBUG in most cases are not able to detect bad reads, KASan able to detect both reads and writes. - In some cases (e.g. redzone overwritten) SLUB_DEBUG detect bugs only on allocation/freeing of object. KASan catch bugs right before it will happen, so we always know exact place of first bad read/write. [1] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel [2] https://code.google.com/p/address-sanitizer/wiki/FoundBugs [3] https://code.google.com/p/thread-sanitizer/wiki/FoundBugs [4] https://code.google.com/p/memory-sanitizer/wiki/FoundBugs [5] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel#Trophies Based on work by Andrey Konovalov. Signed-off-by: Andrey Ryabinin <a.ryabinin@samsung.com> Acked-by: Michal Marek <mmarek@suse.cz> Signed-off-by: Andrey Konovalov <adech.fo@gmail.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Konstantin Serebryany <kcc@google.com> Cc: Dmitry Chernenkov <dmitryc@google.com> Cc: Yuri Gribov <tetra2005@gmail.com> Cc: Konstantin Khlebnikov <koct9i@gmail.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Christoph Lameter <cl@linux.com> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Christoph Lameter <cl@linux.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: David Rientjes <rientjes@google.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2015-02-13 14:39:17 -08:00

kasan: don't emit builtin calls when sanitization is off With KASAN enabled the kernel has two different memset() functions, one with KASAN checks (memset) and one without (__memset). KASAN uses some macro tricks to use the proper version where required. For example memset() calls in mm/slub.c are without KASAN checks, since they operate on poisoned slab object metadata. The issue is that clang emits memset() calls even when there is no memset() in the source code. They get linked with improper memset() implementation and the kernel fails to boot due to a huge amount of KASAN reports during early boot stages. The solution is to add -fno-builtin flag for files with KASAN_SANITIZE := n marker. Link: http://lkml.kernel.org/r/8ffecfffe04088c52c42b92739c2bd8a0bcb3f5e.1516384594.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov <andreyknvl@google.com> Acked-by: Nick Desaulniers <ndesaulniers@google.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Alexander Potapenko <glider@google.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2018-02-06 15:36:00 -08:00

kasan: add kernel address sanitizer infrastructure Kernel Address sanitizer (KASan) is a dynamic memory error detector. It provides fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. KASAN uses compile-time instrumentation for checking every memory access, therefore GCC > v4.9.2 required. v4.9.2 almost works, but has issues with putting symbol aliases into the wrong section, which breaks kasan instrumentation of globals. This patch only adds infrastructure for kernel address sanitizer. It's not available for use yet. The idea and some code was borrowed from [1]. Basic idea: The main idea of KASAN is to use shadow memory to record whether each byte of memory is safe to access or not, and use compiler's instrumentation to check the shadow memory on each memory access. Address sanitizer uses 1/8 of the memory addressable in kernel for shadow memory and uses direct mapping with a scale and offset to translate a memory address to its corresponding shadow address. Here is function to translate address to corresponding shadow address: unsigned long kasan_mem_to_shadow(unsigned long addr) { return (addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET; } where KASAN_SHADOW_SCALE_SHIFT = 3. So for every 8 bytes there is one corresponding byte of shadow memory. The following encoding used for each shadow byte: 0 means that all 8 bytes of the corresponding memory region are valid for access; k (1 <= k <= 7) means that the first k bytes are valid for access, and other (8 - k) bytes are not; Any negative value indicates that the entire 8-bytes are inaccessible. Different negative values used to distinguish between different kinds of inaccessible memory (redzones, freed memory) (see mm/kasan/kasan.h). To be able to detect accesses to bad memory we need a special compiler. Such compiler inserts a specific function calls (__asan_load*(addr), __asan_store*(addr)) before each memory access of size 1, 2, 4, 8 or 16. These functions check whether memory region is valid to access or not by checking corresponding shadow memory. If access is not valid an error printed. Historical background of the address sanitizer from Dmitry Vyukov: "We've developed the set of tools, AddressSanitizer (Asan), ThreadSanitizer and MemorySanitizer, for user space. We actively use them for testing inside of Google (continuous testing, fuzzing, running prod services). To date the tools have found more than 10'000 scary bugs in Chromium, Google internal codebase and various open-source projects (Firefox, OpenSSL, gcc, clang, ffmpeg, MySQL and lots of others): [2] [3] [4]. The tools are part of both gcc and clang compilers. We have not yet done massive testing under the Kernel AddressSanitizer (it's kind of chicken and egg problem, you need it to be upstream to start applying it extensively). To date it has found about 50 bugs. Bugs that we've found in upstream kernel are listed in [5]. We've also found ~20 bugs in out internal version of the kernel. Also people from Samsung and Oracle have found some. [...] As others noted, the main feature of AddressSanitizer is its performance due to inline compiler instrumentation and simple linear shadow memory. User-space Asan has ~2x slowdown on computational programs and ~2x memory consumption increase. Taking into account that kernel usually consumes only small fraction of CPU and memory when running real user-space programs, I would expect that kernel Asan will have ~10-30% slowdown and similar memory consumption increase (when we finish all tuning). I agree that Asan can well replace kmemcheck. We have plans to start working on Kernel MemorySanitizer that finds uses of unitialized memory. Asan+Msan will provide feature-parity with kmemcheck. As others noted, Asan will unlikely replace debug slab and pagealloc that can be enabled at runtime. Asan uses compiler instrumentation, so even if it is disabled, it still incurs visible overheads. Asan technology is easily portable to other architectures. Compiler instrumentation is fully portable. Runtime has some arch-dependent parts like shadow mapping and atomic operation interception. They are relatively easy to port." Comparison with other debugging features: ======================================== KMEMCHECK: - KASan can do almost everything that kmemcheck can. KASan uses compile-time instrumentation, which makes it significantly faster than kmemcheck. The only advantage of kmemcheck over KASan is detection of uninitialized memory reads. Some brief performance testing showed that kasan could be x500-x600 times faster than kmemcheck: $ netperf -l 30 MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to localhost (127.0.0.1) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec no debug: 87380 16384 16384 30.00 41624.72 kasan inline: 87380 16384 16384 30.00 12870.54 kasan outline: 87380 16384 16384 30.00 10586.39 kmemcheck: 87380 16384 16384 30.03 20.23 - Also kmemcheck couldn't work on several CPUs. It always sets number of CPUs to 1. KASan doesn't have such limitation. DEBUG_PAGEALLOC: - KASan is slower than DEBUG_PAGEALLOC, but KASan works on sub-page granularity level, so it able to find more bugs. SLUB_DEBUG (poisoning, redzones): - SLUB_DEBUG has lower overhead than KASan. - SLUB_DEBUG in most cases are not able to detect bad reads, KASan able to detect both reads and writes. - In some cases (e.g. redzone overwritten) SLUB_DEBUG detect bugs only on allocation/freeing of object. KASan catch bugs right before it will happen, so we always know exact place of first bad read/write. [1] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel [2] https://code.google.com/p/address-sanitizer/wiki/FoundBugs [3] https://code.google.com/p/thread-sanitizer/wiki/FoundBugs [4] https://code.google.com/p/memory-sanitizer/wiki/FoundBugs [5] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel#Trophies Based on work by Andrey Konovalov. Signed-off-by: Andrey Ryabinin <a.ryabinin@samsung.com> Acked-by: Michal Marek <mmarek@suse.cz> Signed-off-by: Andrey Konovalov <adech.fo@gmail.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Konstantin Serebryany <kcc@google.com> Cc: Dmitry Chernenkov <dmitryc@google.com> Cc: Yuri Gribov <tetra2005@gmail.com> Cc: Konstantin Khlebnikov <koct9i@gmail.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Christoph Lameter <cl@linux.com> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Christoph Lameter <cl@linux.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: David Rientjes <rientjes@google.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2015-02-13 14:39:17 -08:00

kasan, arm64: expand CONFIG_KASAN checks Some #ifdef CONFIG_KASAN checks are only relevant for software KASAN modes (either related to shadow memory or compiler instrumentation). Expand those into CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS. Link: https://lkml.kernel.org/r/e6971e432dbd72bb897ff14134ebb7e169bdcf0c.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov <andreyknvl@google.com> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Alexander Potapenko <glider@google.com> Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Branislav Rankov <Branislav.Rankov@arm.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Evgenii Stepanov <eugenis@google.com> Cc: Kevin Brodsky <kevin.brodsky@arm.com> Cc: Marco Elver <elver@google.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2020-12-22 12:02:06 -08:00

kasan: add kernel address sanitizer infrastructure Kernel Address sanitizer (KASan) is a dynamic memory error detector. It provides fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. KASAN uses compile-time instrumentation for checking every memory access, therefore GCC > v4.9.2 required. v4.9.2 almost works, but has issues with putting symbol aliases into the wrong section, which breaks kasan instrumentation of globals. This patch only adds infrastructure for kernel address sanitizer. It's not available for use yet. The idea and some code was borrowed from [1]. Basic idea: The main idea of KASAN is to use shadow memory to record whether each byte of memory is safe to access or not, and use compiler's instrumentation to check the shadow memory on each memory access. Address sanitizer uses 1/8 of the memory addressable in kernel for shadow memory and uses direct mapping with a scale and offset to translate a memory address to its corresponding shadow address. Here is function to translate address to corresponding shadow address: unsigned long kasan_mem_to_shadow(unsigned long addr) { return (addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET; } where KASAN_SHADOW_SCALE_SHIFT = 3. So for every 8 bytes there is one corresponding byte of shadow memory. The following encoding used for each shadow byte: 0 means that all 8 bytes of the corresponding memory region are valid for access; k (1 <= k <= 7) means that the first k bytes are valid for access, and other (8 - k) bytes are not; Any negative value indicates that the entire 8-bytes are inaccessible. Different negative values used to distinguish between different kinds of inaccessible memory (redzones, freed memory) (see mm/kasan/kasan.h). To be able to detect accesses to bad memory we need a special compiler. Such compiler inserts a specific function calls (__asan_load*(addr), __asan_store*(addr)) before each memory access of size 1, 2, 4, 8 or 16. These functions check whether memory region is valid to access or not by checking corresponding shadow memory. If access is not valid an error printed. Historical background of the address sanitizer from Dmitry Vyukov: "We've developed the set of tools, AddressSanitizer (Asan), ThreadSanitizer and MemorySanitizer, for user space. We actively use them for testing inside of Google (continuous testing, fuzzing, running prod services). To date the tools have found more than 10'000 scary bugs in Chromium, Google internal codebase and various open-source projects (Firefox, OpenSSL, gcc, clang, ffmpeg, MySQL and lots of others): [2] [3] [4]. The tools are part of both gcc and clang compilers. We have not yet done massive testing under the Kernel AddressSanitizer (it's kind of chicken and egg problem, you need it to be upstream to start applying it extensively). To date it has found about 50 bugs. Bugs that we've found in upstream kernel are listed in [5]. We've also found ~20 bugs in out internal version of the kernel. Also people from Samsung and Oracle have found some. [...] As others noted, the main feature of AddressSanitizer is its performance due to inline compiler instrumentation and simple linear shadow memory. User-space Asan has ~2x slowdown on computational programs and ~2x memory consumption increase. Taking into account that kernel usually consumes only small fraction of CPU and memory when running real user-space programs, I would expect that kernel Asan will have ~10-30% slowdown and similar memory consumption increase (when we finish all tuning). I agree that Asan can well replace kmemcheck. We have plans to start working on Kernel MemorySanitizer that finds uses of unitialized memory. Asan+Msan will provide feature-parity with kmemcheck. As others noted, Asan will unlikely replace debug slab and pagealloc that can be enabled at runtime. Asan uses compiler instrumentation, so even if it is disabled, it still incurs visible overheads. Asan technology is easily portable to other architectures. Compiler instrumentation is fully portable. Runtime has some arch-dependent parts like shadow mapping and atomic operation interception. They are relatively easy to port." Comparison with other debugging features: ======================================== KMEMCHECK: - KASan can do almost everything that kmemcheck can. KASan uses compile-time instrumentation, which makes it significantly faster than kmemcheck. The only advantage of kmemcheck over KASan is detection of uninitialized memory reads. Some brief performance testing showed that kasan could be x500-x600 times faster than kmemcheck: $ netperf -l 30 MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to localhost (127.0.0.1) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec no debug: 87380 16384 16384 30.00 41624.72 kasan inline: 87380 16384 16384 30.00 12870.54 kasan outline: 87380 16384 16384 30.00 10586.39 kmemcheck: 87380 16384 16384 30.03 20.23 - Also kmemcheck couldn't work on several CPUs. It always sets number of CPUs to 1. KASan doesn't have such limitation. DEBUG_PAGEALLOC: - KASan is slower than DEBUG_PAGEALLOC, but KASan works on sub-page granularity level, so it able to find more bugs. SLUB_DEBUG (poisoning, redzones): - SLUB_DEBUG has lower overhead than KASan. - SLUB_DEBUG in most cases are not able to detect bad reads, KASan able to detect both reads and writes. - In some cases (e.g. redzone overwritten) SLUB_DEBUG detect bugs only on allocation/freeing of object. KASan catch bugs right before it will happen, so we always know exact place of first bad read/write. [1] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel [2] https://code.google.com/p/address-sanitizer/wiki/FoundBugs [3] https://code.google.com/p/thread-sanitizer/wiki/FoundBugs [4] https://code.google.com/p/memory-sanitizer/wiki/FoundBugs [5] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel#Trophies Based on work by Andrey Konovalov. Signed-off-by: Andrey Ryabinin <a.ryabinin@samsung.com> Acked-by: Michal Marek <mmarek@suse.cz> Signed-off-by: Andrey Konovalov <adech.fo@gmail.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Konstantin Serebryany <kcc@google.com> Cc: Dmitry Chernenkov <dmitryc@google.com> Cc: Yuri Gribov <tetra2005@gmail.com> Cc: Konstantin Khlebnikov <koct9i@gmail.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Christoph Lameter <cl@linux.com> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Christoph Lameter <cl@linux.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: David Rientjes <rientjes@google.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2015-02-13 14:39:17 -08:00

kmsan: add KMSAN runtime core For each memory location KernelMemorySanitizer maintains two types of metadata: 1. The so-called shadow of that location - а byte:byte mapping describing whether or not individual bits of memory are initialized (shadow is 0) or not (shadow is 1). 2. The origins of that location - а 4-byte:4-byte mapping containing 4-byte IDs of the stack traces where uninitialized values were created. Each struct page now contains pointers to two struct pages holding KMSAN metadata (shadow and origins) for the original struct page. Utility routines in mm/kmsan/core.c and mm/kmsan/shadow.c handle the metadata creation, addressing, copying and checking. mm/kmsan/report.c performs error reporting in the cases an uninitialized value is used in a way that leads to undefined behavior. KMSAN compiler instrumentation is responsible for tracking the metadata along with the kernel memory. mm/kmsan/instrumentation.c provides the implementation for instrumentation hooks that are called from files compiled with -fsanitize=kernel-memory. To aid parameter passing (also done at instrumentation level), each task_struct now contains a struct kmsan_task_state used to track the metadata of function parameters and return values for that task. Finally, this patch provides CONFIG_KMSAN that enables KMSAN, and declares CFLAGS_KMSAN, which are applied to files compiled with KMSAN. The KMSAN_SANITIZE:=n Makefile directive can be used to completely disable KMSAN instrumentation for certain files. Similarly, KMSAN_ENABLE_CHECKS:=n disables KMSAN checks and makes newly created stack memory initialized. Users can also use functions from include/linux/kmsan-checks.h to mark certain memory regions as uninitialized or initialized (this is called "poisoning" and "unpoisoning") or check that a particular region is initialized. Link: https://lkml.kernel.org/r/20220915150417.722975-12-glider@google.com Signed-off-by: Alexander Potapenko <glider@google.com> Acked-by: Marco Elver <elver@google.com> Cc: Alexander Viro <viro@zeniv.linux.org.uk> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Andrey Konovalov <andreyknvl@gmail.com> Cc: Andrey Konovalov <andreyknvl@google.com> Cc: Andy Lutomirski <luto@kernel.org> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Christoph Hellwig <hch@lst.de> Cc: Christoph Lameter <cl@linux.com> Cc: David Rientjes <rientjes@google.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Eric Biggers <ebiggers@google.com> Cc: Eric Biggers <ebiggers@kernel.org> Cc: Eric Dumazet <edumazet@google.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Ilya Leoshkevich <iii@linux.ibm.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jens Axboe <axboe@kernel.dk> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Kees Cook <keescook@chromium.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Petr Mladek <pmladek@suse.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Vegard Nossum <vegard.nossum@oracle.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

2022-09-15 17:03:45 +02:00

2016-01-20 15:00:55 -08:00

kernel: add kcov code coverage kcov provides code coverage collection for coverage-guided fuzzing (randomized testing). Coverage-guided fuzzing is a testing technique that uses coverage feedback to determine new interesting inputs to a system. A notable user-space example is AFL (http://lcamtuf.coredump.cx/afl/). However, this technique is not widely used for kernel testing due to missing compiler and kernel support. kcov does not aim to collect as much coverage as possible. It aims to collect more or less stable coverage that is function of syscall inputs. To achieve this goal it does not collect coverage in soft/hard interrupts and instrumentation of some inherently non-deterministic or non-interesting parts of kernel is disbled (e.g. scheduler, locking). Currently there is a single coverage collection mode (tracing), but the API anticipates additional collection modes. Initially I also implemented a second mode which exposes coverage in a fixed-size hash table of counters (what Quentin used in his original patch). I've dropped the second mode for simplicity. This patch adds the necessary support on kernel side. The complimentary compiler support was added in gcc revision 231296. We've used this support to build syzkaller system call fuzzer, which has found 90 kernel bugs in just 2 months: https://github.com/google/syzkaller/wiki/Found-Bugs We've also found 30+ bugs in our internal systems with syzkaller. Another (yet unexplored) direction where kcov coverage would greatly help is more traditional "blob mutation". For example, mounting a random blob as a filesystem, or receiving a random blob over wire. Why not gcov. Typical fuzzing loop looks as follows: (1) reset coverage, (2) execute a bit of code, (3) collect coverage, repeat. A typical coverage can be just a dozen of basic blocks (e.g. an invalid input). In such context gcov becomes prohibitively expensive as reset/collect coverage steps depend on total number of basic blocks/edges in program (in case of kernel it is about 2M). Cost of kcov depends only on number of executed basic blocks/edges. On top of that, kernel requires per-thread coverage because there are always background threads and unrelated processes that also produce coverage. With inlined gcov instrumentation per-thread coverage is not possible. kcov exposes kernel PCs and control flow to user-space which is insecure. But debugfs should not be mapped as user accessible. Based on a patch by Quentin Casasnovas. [akpm@linux-foundation.org: make task_struct.kcov_mode have type `enum kcov_mode'] [akpm@linux-foundation.org: unbreak allmodconfig] [akpm@linux-foundation.org: follow x86 Makefile layout standards] Signed-off-by: Dmitry Vyukov <dvyukov@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Cc: syzkaller <syzkaller@googlegroups.com> Cc: Vegard Nossum <vegard.nossum@oracle.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Tavis Ormandy <taviso@google.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com> Cc: Kostya Serebryany <kcc@google.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Alexander Potapenko <glider@google.com> Cc: Kees Cook <keescook@google.com> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: David Drysdale <drysdale@google.com> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com> Cc: Kirill A. Shutemov <kirill@shutemov.name> Cc: Jiri Slaby <jslaby@suse.cz> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2016-03-22 14:27:30 -07:00

kcov: allow more fine-grained coverage instrumentation For more targeted fuzzing, it's better to disable kernel-wide instrumentation and instead enable it on a per-subsystem basis. This follows the pattern of UBSAN and allows you to compile in the kcov driver without instrumenting the whole kernel. To instrument a part of the kernel, you can use either # for a single file in the current directory KCOV_INSTRUMENT_filename.o := y or # for all the files in the current directory (excluding subdirectories) KCOV_INSTRUMENT := y or # (same as above) ccflags-y += $(CFLAGS_KCOV) or # for all the files in the current directory (including subdirectories) subdir-ccflags-y += $(CFLAGS_KCOV) Link: http://lkml.kernel.org/r/1464008380-11405-1-git-send-email-vegard.nossum@oracle.com Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2016-08-02 14:07:30 -07:00

kernel: add kcov code coverage kcov provides code coverage collection for coverage-guided fuzzing (randomized testing). Coverage-guided fuzzing is a testing technique that uses coverage feedback to determine new interesting inputs to a system. A notable user-space example is AFL (http://lcamtuf.coredump.cx/afl/). However, this technique is not widely used for kernel testing due to missing compiler and kernel support. kcov does not aim to collect as much coverage as possible. It aims to collect more or less stable coverage that is function of syscall inputs. To achieve this goal it does not collect coverage in soft/hard interrupts and instrumentation of some inherently non-deterministic or non-interesting parts of kernel is disbled (e.g. scheduler, locking). Currently there is a single coverage collection mode (tracing), but the API anticipates additional collection modes. Initially I also implemented a second mode which exposes coverage in a fixed-size hash table of counters (what Quentin used in his original patch). I've dropped the second mode for simplicity. This patch adds the necessary support on kernel side. The complimentary compiler support was added in gcc revision 231296. We've used this support to build syzkaller system call fuzzer, which has found 90 kernel bugs in just 2 months: https://github.com/google/syzkaller/wiki/Found-Bugs We've also found 30+ bugs in our internal systems with syzkaller. Another (yet unexplored) direction where kcov coverage would greatly help is more traditional "blob mutation". For example, mounting a random blob as a filesystem, or receiving a random blob over wire. Why not gcov. Typical fuzzing loop looks as follows: (1) reset coverage, (2) execute a bit of code, (3) collect coverage, repeat. A typical coverage can be just a dozen of basic blocks (e.g. an invalid input). In such context gcov becomes prohibitively expensive as reset/collect coverage steps depend on total number of basic blocks/edges in program (in case of kernel it is about 2M). Cost of kcov depends only on number of executed basic blocks/edges. On top of that, kernel requires per-thread coverage because there are always background threads and unrelated processes that also produce coverage. With inlined gcov instrumentation per-thread coverage is not possible. kcov exposes kernel PCs and control flow to user-space which is insecure. But debugfs should not be mapped as user accessible. Based on a patch by Quentin Casasnovas. [akpm@linux-foundation.org: make task_struct.kcov_mode have type `enum kcov_mode'] [akpm@linux-foundation.org: unbreak allmodconfig] [akpm@linux-foundation.org: follow x86 Makefile layout standards] Signed-off-by: Dmitry Vyukov <dvyukov@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Cc: syzkaller <syzkaller@googlegroups.com> Cc: Vegard Nossum <vegard.nossum@oracle.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Tavis Ormandy <taviso@google.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com> Cc: Kostya Serebryany <kcc@google.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Alexander Potapenko <glider@google.com> Cc: Kees Cook <keescook@google.com> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: David Drysdale <drysdale@google.com> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com> Cc: Kirill A. Shutemov <kirill@shutemov.name> Cc: Jiri Slaby <jslaby@suse.cz> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2016-03-22 14:27:30 -07:00

kcsan: Add Kernel Concurrency Sanitizer infrastructure Kernel Concurrency Sanitizer (KCSAN) is a dynamic data-race detector for kernel space. KCSAN is a sampling watchpoint-based data-race detector. See the included Documentation/dev-tools/kcsan.rst for more details. This patch adds basic infrastructure, but does not yet enable KCSAN for any architecture. Signed-off-by: Marco Elver <elver@google.com> Acked-by: Paul E. McKenney <paulmck@kernel.org> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

2019-11-14 19:02:54 +01:00

kcsan, kbuild: Add option for barrier instrumentation only Source files that disable KCSAN via KCSAN_SANITIZE := n, remove all instrumentation, including explicit barrier instrumentation. With instrumentation for memory barriers, in few places it is required to enable just the explicit instrumentation for memory barriers to avoid false positives. Providing the Makefile variable KCSAN_INSTRUMENT_BARRIERS_obj.o or KCSAN_INSTRUMENT_BARRIERS (for all files) set to 'y' only enables the explicit barrier instrumentation. Signed-off-by: Marco Elver <elver@google.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

2021-11-30 12:44:14 +01:00

kcsan: Add Kernel Concurrency Sanitizer infrastructure Kernel Concurrency Sanitizer (KCSAN) is a dynamic data-race detector for kernel space. KCSAN is a sampling watchpoint-based data-race detector. See the included Documentation/dev-tools/kcsan.rst for more details. This patch adds basic infrastructure, but does not yet enable KCSAN for any architecture. Signed-off-by: Marco Elver <elver@google.com> Acked-by: Paul E. McKenney <paulmck@kernel.org> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

2019-11-14 19:02:54 +01:00

kernel: add kcov code coverage kcov provides code coverage collection for coverage-guided fuzzing (randomized testing). Coverage-guided fuzzing is a testing technique that uses coverage feedback to determine new interesting inputs to a system. A notable user-space example is AFL (http://lcamtuf.coredump.cx/afl/). However, this technique is not widely used for kernel testing due to missing compiler and kernel support. kcov does not aim to collect as much coverage as possible. It aims to collect more or less stable coverage that is function of syscall inputs. To achieve this goal it does not collect coverage in soft/hard interrupts and instrumentation of some inherently non-deterministic or non-interesting parts of kernel is disbled (e.g. scheduler, locking). Currently there is a single coverage collection mode (tracing), but the API anticipates additional collection modes. Initially I also implemented a second mode which exposes coverage in a fixed-size hash table of counters (what Quentin used in his original patch). I've dropped the second mode for simplicity. This patch adds the necessary support on kernel side. The complimentary compiler support was added in gcc revision 231296. We've used this support to build syzkaller system call fuzzer, which has found 90 kernel bugs in just 2 months: https://github.com/google/syzkaller/wiki/Found-Bugs We've also found 30+ bugs in our internal systems with syzkaller. Another (yet unexplored) direction where kcov coverage would greatly help is more traditional "blob mutation". For example, mounting a random blob as a filesystem, or receiving a random blob over wire. Why not gcov. Typical fuzzing loop looks as follows: (1) reset coverage, (2) execute a bit of code, (3) collect coverage, repeat. A typical coverage can be just a dozen of basic blocks (e.g. an invalid input). In such context gcov becomes prohibitively expensive as reset/collect coverage steps depend on total number of basic blocks/edges in program (in case of kernel it is about 2M). Cost of kcov depends only on number of executed basic blocks/edges. On top of that, kernel requires per-thread coverage because there are always background threads and unrelated processes that also produce coverage. With inlined gcov instrumentation per-thread coverage is not possible. kcov exposes kernel PCs and control flow to user-space which is insecure. But debugfs should not be mapped as user accessible. Based on a patch by Quentin Casasnovas. [akpm@linux-foundation.org: make task_struct.kcov_mode have type `enum kcov_mode'] [akpm@linux-foundation.org: unbreak allmodconfig] [akpm@linux-foundation.org: follow x86 Makefile layout standards] Signed-off-by: Dmitry Vyukov <dvyukov@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Cc: syzkaller <syzkaller@googlegroups.com> Cc: Vegard Nossum <vegard.nossum@oracle.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Tavis Ormandy <taviso@google.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com> Cc: Kostya Serebryany <kcc@google.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Alexander Potapenko <glider@google.com> Cc: Kees Cook <keescook@google.com> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: David Drysdale <drysdale@google.com> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com> Cc: Kirill A. Shutemov <kirill@shutemov.name> Cc: Jiri Slaby <jslaby@suse.cz> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2016-03-22 14:27:30 -07:00

kbuild: remove 'addtree' and 'flags' magic for header search paths The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated and ugly. As I mentioned in [1], Kbuild should stop automatic prefixing of header search path options. I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and 'flags' have been removed. Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj) to the header search path for O= building, but never touches extra compiler options from ccflags-y etc. [1]: https://patchwork.kernel.org/patch/9632347/ Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-05-13 15:22:17 +09:00

kbuild: skip 'addtree' and 'flags' magic for external module build When building an external module, $(obj) is the absolute path to it. The header search paths from ccflags-y etc. should not be tweaked. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-01-16 11:56:40 +09:00

kbuild: replace KBUILD_SRCTREE with boolean building_out_of_srctree Commit 25b146c5b8ce ("kbuild: allow Kbuild to start from any directory") deprecated KBUILD_SRCTREE. It is only used in tools/testing/selftest/ to distinguish out-of-tree build. Replace it with a new boolean flag, building_out_of_srctree. I also replaced the conditional ($(srctree),.) because the next commit will allow an absolute path to be used for $(srctree) even when building in the source tree. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-07-06 12:07:12 +09:00

kbuild: remove 'addtree' and 'flags' magic for header search paths The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated and ugly. As I mentioned in [1], Kbuild should stop automatic prefixing of header search path options. I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and 'flags' have been removed. Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj) to the header search path for O= building, but never touches extra compiler options from ccflags-y etc. [1]: https://patchwork.kernel.org/patch/9632347/ Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-05-13 15:22:17 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: skip 'addtree' and 'flags' magic for external module build When building an external module, $(obj) is the absolute path to it. The header search paths from ccflags-y etc. should not be tweaked. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-01-16 11:56:40 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: move modkern_{c,a}flags to Makefile.lib from Makefile.build Makefile.lib is included by Makefile.modfinal as well as Makefile.build. Move modkern_cflags to Makefile.lib in order to simplify cmd_cc_o_c in Makefile.modfinal. Move modkern_cflags as well for consistency. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-08-19 17:58:43 +09:00

kbuild: pass KBUILD_MODFILE when compiling builtin objects When compiling, Kbuild passes KBUILD_BASENAME (basename of the object) and KBUILD_MODNAME (basename of the module). This commit adds another one, KBUILD_MODFILE, which is the path of the module. (or, the path of the module it would end up in if it were compiled as a module.) The next commit will use this to generate modules.builtin without tristate.conf. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2019-12-19 17:33:28 +09:00

kbuild: move modkern_{c,a}flags to Makefile.lib from Makefile.build Makefile.lib is included by Makefile.modfinal as well as Makefile.build. Move modkern_cflags to Makefile.lib in order to simplify cmd_cc_o_c in Makefile.modfinal. Move modkern_cflags as well for consistency. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-08-19 17:58:43 +09:00

2021-07-03 16:42:57 +02:00

kbuild: move modkern_{c,a}flags to Makefile.lib from Makefile.build Makefile.lib is included by Makefile.modfinal as well as Makefile.build. Move modkern_cflags to Makefile.lib in order to simplify cmd_cc_o_c in Makefile.modfinal. Move modkern_cflags as well for consistency. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-08-19 17:58:43 +09:00

kbuild: use -MMD instead of -MD to exclude system headers from dependency This omits system headers from the generated header dependency. System headers are not updated unless you upgrade the compiler. Nor do they contain CONFIG options, so fixdep does not need to parse them. Having said that, the effect of this optimization will be quite small because the kernel code generally does not include system headers except <stdarg.h>. Host programs include a lot of system headers, but there are not so many in the kernel tree. At first, keeping system headers in .*.cmd files might be useful to detect the compiler update, but there is no guarantee that <stdarg.h> is included from every file. So, I implemented a more reliable way in the previous commit. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-04-23 23:23:53 +09:00

kbuild: get <linux/compiler_types.h> out of <linux/kconfig.h> Since commit 28128c61e08e ("kconfig.h: Include compiler types to avoid missed struct attributes"), <linux/kconfig.h> pulls in kernel-space headers to unrelated places. Commit 0f9da844d877 ("MIPS: boot: Define __ASSEMBLY__ for its.S build") suppress the build error by defining __ASSEMBLY__, but ITS (i.e. DTS) is not assembly, and should not include <linux/compiler_types.h> in the first place. Looking at arch/s390/tools/Makefile, host programs gen_facilities and gen_opcode_table now pull in <linux/compiler_types.h> as well. The motivation for that commit was to define necessary attributes before any struct is defined. Obviously, this happens only in C. It is enough to include <linux/compiler_types.h> only when compiling C files, and only when compiling kernel space. Move the include to c_flags. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-03-27 21:52:50 +09:00

kbuild: remove 'addtree' and 'flags' magic for header search paths The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated and ugly. As I mentioned in [1], Kbuild should stop automatic prefixing of header search path options. I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and 'flags' have been removed. Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj) to the header search path for O= building, but never touches extra compiler options from ccflags-y etc. [1]: https://patchwork.kernel.org/patch/9632347/ Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-05-13 15:22:17 +09:00

kbuild: Get rid of KBUILD_STR The compiler can accept -DKBUILD_MODNAME="foo", it's just a matter of quoting. That way, we reduce the gcc command line a bit. Signed-off-by: Michal Marek <mmarek@suse.com>

2016-03-17 16:32:14 +01:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

2021-07-03 16:42:57 +02:00

kbuild: use -MMD instead of -MD to exclude system headers from dependency This omits system headers from the generated header dependency. System headers are not updated unless you upgrade the compiler. Nor do they contain CONFIG options, so fixdep does not need to parse them. Having said that, the effect of this optimization will be quite small because the kernel code generally does not include system headers except <stdarg.h>. Host programs include a lot of system headers, but there are not so many in the kernel tree. At first, keeping system headers in .*.cmd files might be useful to detect the compiler update, but there is no guarantee that <stdarg.h> is included from every file. So, I implemented a more reliable way in the previous commit. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-04-23 23:23:53 +09:00

kbuild: remove 'addtree' and 'flags' magic for header search paths The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated and ugly. As I mentioned in [1], Kbuild should stop automatic prefixing of header search path options. I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and 'flags' have been removed. Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj) to the header search path for O= building, but never touches extra compiler options from ccflags-y etc. [1]: https://patchwork.kernel.org/patch/9632347/ Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-05-13 15:22:17 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: use -MMD instead of -MD to exclude system headers from dependency This omits system headers from the generated header dependency. System headers are not updated unless you upgrade the compiler. Nor do they contain CONFIG options, so fixdep does not need to parse them. Having said that, the effect of this optimization will be quite small because the kernel code generally does not include system headers except <stdarg.h>. Host programs include a lot of system headers, but there are not so many in the kernel tree. At first, keeping system headers in .*.cmd files might be useful to detect the compiler update, but there is no guarantee that <stdarg.h> is included from every file. So, I implemented a more reliable way in the previous commit. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-04-23 23:23:53 +09:00

kbuild: remove 'addtree' and 'flags' magic for header search paths The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated and ugly. As I mentioned in [1], Kbuild should stop automatic prefixing of header search path options. I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and 'flags' have been removed. Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj) to the header search path for O= building, but never touches extra compiler options from ccflags-y etc. [1]: https://patchwork.kernel.org/patch/9632347/ Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-05-13 15:22:17 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: rename LDFLAGS to KBUILD_LDFLAGS Commit a0f97e06a43c ("kbuild: enable 'make CFLAGS=...' to add additional options to CC") renamed CFLAGS to KBUILD_CFLAGS. Commit 222d394d30e7 ("kbuild: enable 'make AFLAGS=...' to add additional options to AS") renamed AFLAGS to KBUILD_AFLAGS. Commit 06c5040cdb13 ("kbuild: enable 'make CPPFLAGS=...' to add additional options to CPP") renamed CPPFLAGS to KBUILD_CPPFLAGS. For some reason, LDFLAGS was not renamed. Using a well-known variable like LDFLAGS may result in accidental override of the variable. Kbuild generally uses KBUILD_ prefixed variables for the internally appended options, so here is one more conversion to sanitize the naming convention. I did not touch Makefiles under tools/ since the tools build system is a different world. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com>

2018-08-24 08:20:39 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

of: search scripts/dtc/include-prefixes path for both CPP and DTC Since commit d5d332d3f7e8 ("devicetree: Move include prefixes from arch to separate directory"), cross-arch DT reference works well, but only for CPP style #include directives. It makes as much sense to share DT between different architectures by using DTC's /include/ directives. So, scripts/dtc/include-prefixes should be passed to both CPP and DTC. I refactored Makefile.lib a bit to not repeat the same path. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Rob Herring <robh@kernel.org>

2017-07-25 11:23:36 +09:00

kbuild: use -MMD instead of -MD to exclude system headers from dependency This omits system headers from the generated header dependency. System headers are not updated unless you upgrade the compiler. Nor do they contain CONFIG options, so fixdep does not need to parse them. Having said that, the effect of this optimization will be quite small because the kernel code generally does not include system headers except <stdarg.h>. Host programs include a lot of system headers, but there are not so many in the kernel tree. At first, keeping system headers in .*.cmd files might be useful to detect the compiler update, but there is no guarantee that <stdarg.h> is included from every file. So, I implemented a more reliable way in the previous commit. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-04-23 23:23:53 +09:00

of: search scripts/dtc/include-prefixes path for both CPP and DTC Since commit d5d332d3f7e8 ("devicetree: Move include prefixes from arch to separate directory"), cross-arch DT reference works well, but only for CPP style #include directives. It makes as much sense to share DT between different architectures by using DTC's /include/ directives. So, scripts/dtc/include-prefixes should be passed to both CPP and DTC. I refactored Makefile.lib a bit to not repeat the same path. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Rob Herring <robh@kernel.org>

2017-07-25 11:23:36 +09:00

kbuild: limit dtc+cpp include path Device tree source files may now include header files. The intent is that those header files define/name constants used as part of the DT bindings. Currently this feature is open to abuse, since any kernel header file at all can be included, This could allow device tree files to become dependant on kernel headers files, and thus make them no longer OS-independent. This would also prevent separating the device tree source files from the kernel repository. Solve this by limiting the cpp include path for device tree files to separate directories. Signed-off-by: Stephen Warren <swarren@nvidia.com> Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2013-02-12 15:03:37 -07:00

kbuild: factor out the common objtool arguments scripts/Makefile.build and scripts/link-vmlinux.sh have similar setups for the objtool arguments. It was difficult to factor out them because all the vmlinux build rules were written in a shell script. It is somewhat tedious to touch the two files every time a new objtool option is supported. To reduce the code duplication, move the objtool for vmlinux.o into scripts/Makefile.vmlinux_o. Then, move the common macros to Makefile.lib so they are shared between Makefile.build and Makefile.vmlinux_o. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)

2022-05-29 00:47:04 +09:00

kbuild: use objtool-args-y to clean up objtool arguments Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well. Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/ Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

2022-09-07 03:49:35 +09:00

objtool: Add --hacks=skylake Make the call/func sections selectable via the --hacks option. Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20220915111146.120821440@infradead.org

2022-09-15 13:11:10 +02:00

kbuild: use objtool-args-y to clean up objtool arguments Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well. Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/ Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

2022-09-07 03:49:35 +09:00

x86/ibt: Implement FineIBT Implement an alternative CFI scheme that merges both the fine-grained nature of kCFI but also takes full advantage of the coarse grained hardware CFI as provided by IBT. To contrast: kCFI is a pure software CFI scheme and relies on being able to read text -- specifically the instruction *before* the target symbol, and does the hash validation *before* doing the call (otherwise control flow is compromised already). FineIBT is a software and hardware hybrid scheme; by ensuring every branch target starts with a hash validation it is possible to place the hash validation after the branch. This has several advantages: o the (hash) load is avoided; no memop; no RX requirement. o IBT WAIT-FOR-ENDBR state is a speculation stop; by placing the hash validation in the immediate instruction after the branch target there is a minimal speculation window and the whole is a viable defence against SpectreBHB. o Kees feels obliged to mention it is slightly more vulnerable when the attacker can write code. Obviously this patch relies on kCFI, but additionally it also relies on the padding from the call-depth-tracking patches. It uses this padding to place the hash-validation while the call-sites are re-written to modify the indirect target to be 16 bytes in front of the original target, thus hitting this new preamble. Notably, there is no hardware that needs call-depth-tracking (Skylake) and supports IBT (Tigerlake and onwards). Suggested-by: Joao Moreira (Intel) <joao@overdrivepizza.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20221027092842.634714496@infradead.org

2022-10-27 11:28:14 +02:00

kbuild: use objtool-args-y to clean up objtool arguments Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well. Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/ Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

2022-09-07 03:49:35 +09:00

objtool: Add --mnop as an option to --mcount Some architectures (powerpc) may not support ftrace locations being nop'ed out at build time. Introduce CONFIG_HAVE_OBJTOOL_NOP_MCOUNT for objtool, as a means for architectures to enable nop'ing of ftrace locations. Add --mnop as an option to objtool --mcount, to indicate support for the same. Also, make sure that --mnop can be passed as an option to objtool only when --mcount is passed. Tested-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Reviewed-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Acked-by: Josh Poimboeuf <jpoimboe@kernel.org> Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu> Signed-off-by: Sathvika Vasireddy <sv@linux.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20221114175754.1131267-12-sv@linux.ibm.com

2022-11-14 23:27:49 +05:30

kbuild: use objtool-args-y to clean up objtool arguments Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well. Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/ Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

2022-09-07 03:49:35 +09:00

x86: Add prefix symbols for function padding When code is compiled with: -fpatchable-function-entry=${PADDING_BYTES},${PADDING_BYTES} functions will have PADDING_BYTES of NOP in front of them. Unwinders and other things that symbolize code locations will typically attribute these bytes to the preceding function. Given that these bytes nominally belong to the following symbol this mis-attribution is confusing. Inspired by the fact that CFI_CLANG emits __cfi_##name symbols to claim these bytes, use objtool to emit __pfx_##name symbols to do the same when CFI_CLANG is not used. This then shows the callthunk for symbol 'name' as: __pfx_##name+0x6/0x10 Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Yujie Liu <yujie.liu@intel.com> Link: https://lkml.kernel.org/r/20221028194453.592512209@infradead.org

2022-10-28 21:08:19 +02:00

kbuild: use objtool-args-y to clean up objtool arguments Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well. Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/ Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

2022-09-07 03:49:35 +09:00

kbuild: factor out the common objtool arguments scripts/Makefile.build and scripts/link-vmlinux.sh have similar setups for the objtool arguments. It was difficult to factor out them because all the vmlinux build rules were written in a shell script. It is somewhat tedious to touch the two files every time a new objtool option is supported. To reduce the code duplication, move the objtool for vmlinux.o into scripts/Makefile.vmlinux_o. Then, move the common macros to Makefile.lib so they are shared between Makefile.build and Makefile.vmlinux_o. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)

2022-05-29 00:47:04 +09:00

kbuild: use objtool-args-y to clean up objtool arguments Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well. Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/ Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

2022-09-07 03:49:35 +09:00

kbuild: factor out the common objtool arguments scripts/Makefile.build and scripts/link-vmlinux.sh have similar setups for the objtool arguments. It was difficult to factor out them because all the vmlinux build rules were written in a shell script. It is somewhat tedious to touch the two files every time a new objtool option is supported. To reduce the code duplication, move the objtool for vmlinux.o into scripts/Makefile.vmlinux_o. Then, move the common macros to Makefile.lib so they are shared between Makefile.build and Makefile.vmlinux_o. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)

2022-05-29 00:47:04 +09:00

kbuild: use objtool-args-y to clean up objtool arguments Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well. Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/ Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

2022-09-07 03:49:35 +09:00

kbuild: factor out the common objtool arguments scripts/Makefile.build and scripts/link-vmlinux.sh have similar setups for the objtool arguments. It was difficult to factor out them because all the vmlinux build rules were written in a shell script. It is somewhat tedious to touch the two files every time a new objtool option is supported. To reduce the code duplication, move the objtool for vmlinux.o into scripts/Makefile.vmlinux_o. Then, move the common macros to Makefile.lib so they are shared between Makefile.build and Makefile.vmlinux_o. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)

2022-05-29 00:47:04 +09:00

kbuild: handle multi-objs dependency appropriately The comment in scripts/Makefile.build says as follows: We would rather have a list of rules like foo.o: $(foo-objs) but that's not so easy, so we rather make all composite objects depend on the set of all their parts This commit makes it possible! For example, assume a Makefile like this obj-m = foo.o bar.o foo-objs := foo1.o foo2.o bar-objs := bar1.o bar2.o Without this patch, foo.o depends on all of foo1.o foo2.o bar1.o bar2.o. It looks funny that foo.o is regenerated when bar1.c is updated. Now we can handle the dependency of foo.o and bar.o separately. Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com> Signed-off-by: Michal Marek <mmarek@suse.cz>

2014-08-19 16:34:20 +09:00

kbuild: make multi_depend work with targets in subdirectory Precisely speaking, when you get the stem of the path, you should use $(patsubst $(obj)/%,%,...) instead of $(notdir ...). I do not see this usecase, but if you create a composite object in a subdirectory, the Makefile should look like this: obj-$(CONFIG_FOO) += dir/foo.o dir/foo-objs := dir/foo1.o dir/foo2.o The member objects should be assigned to dir/foo-objs instead of foo-objs. This syntax is more consistent with commit 54b8ae66ae1a ("kbuild: change *FLAGS_<basetarget>.o to take the path relative to $(obj)"). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

2022-04-07 00:30:18 +09:00

kbuild: handle multi-objs dependency appropriately The comment in scripts/Makefile.build says as follows: We would rather have a list of rules like foo.o: $(foo-objs) but that's not so easy, so we rather make all composite objects depend on the set of all their parts This commit makes it possible! For example, assume a Makefile like this obj-m = foo.o bar.o foo-objs := foo1.o foo2.o bar-objs := bar1.o bar2.o Without this patch, foo.o depends on all of foo1.o foo2.o bar1.o bar2.o. It looks funny that foo.o is regenerated when bar1.c is updated. Now we can handle the dependency of foo.o and bar.o separately. Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com> Signed-off-by: Michal Marek <mmarek@suse.cz>

2014-08-19 16:34:20 +09:00

kbuild: unify cmd_copy and cmd_shipped cmd_copy and cmd_shipped have similar functionality. The difference is that cmd_copy uses 'cp' while cmd_shipped 'cat'. Unify them into cmd_copy because this macro name is more intuitive. Going forward, cmd_copy will use 'cat' to avoid the permission issue. I also thought of 'cp --no-preserve=mode' but this option is not mentioned in the POSIX spec [1], so I am keeping the 'cat' command. [1]: https://pubs.opengroup.org/onlinepubs/009695299/utilities/cp.html Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.com>

2022-01-25 15:40:27 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: add comment about why cmd_shipped uses 'cat' cmd_shipped uses 'cat' instead of 'cp' for copying a file. The reason is explained in the commit [1], but it was in the pre-git era. $ touch a $ chmod -w a $ cp a b $ cp a b cp: cannot create regular file 'b': Permission denied Add comments so that you can see the reason without looking into the history. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=a70dba8086160449cc94c5bdaff78419b6b8e3c8 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-04-24 20:55:53 +09:00

kbuild: unify cmd_copy and cmd_shipped cmd_copy and cmd_shipped have similar functionality. The difference is that cmd_copy uses 'cp' while cmd_shipped 'cat'. Unify them into cmd_copy because this macro name is more intuitive. Going forward, cmd_copy will use 'cat' to avoid the permission issue. I also thought of 'cp --no-preserve=mode' but this option is not mentioned in the POSIX spec [1], so I am keeping the 'cat' command. [1]: https://pubs.opengroup.org/onlinepubs/009695299/utilities/cp.html Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.com>

2022-01-25 15:40:27 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: simplify the %_shipped rule This is needed to have make(1) correctly link the implicit rules which generate the _shipped file from the lexer/parser to the final file. Signed-off-by: Arnaud Lacombe <lacombar@gmail.com>

2011-06-07 13:09:28 -04:00

kbuild: unify cmd_copy and cmd_shipped cmd_copy and cmd_shipped have similar functionality. The difference is that cmd_copy uses 'cp' while cmd_shipped 'cat'. Unify them into cmd_copy because this macro name is more intuitive. Going forward, cmd_copy will use 'cat' to avoid the permission issue. I also thought of 'cp --no-preserve=mode' but this option is not mentioned in the POSIX spec [1], so I am keeping the 'cat' command. [1]: https://pubs.opengroup.org/onlinepubs/009695299/utilities/cp.html Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.com>

2022-01-25 15:40:27 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: trivial - remove trailing spaces Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com> Signed-off-by: Michal Marek <mmarek@suse.cz>

2014-04-28 16:26:18 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: use always-y instead of extra-y As commit d0e628cd817f ("kbuild: doc: clarify the difference between extra-y and always-y") explained, extra-y should be used for listing the prerequisites of vmlinux. These targets are not related to vmlinux. always-y is a better fix. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Rob Herring <robh@kernel.org>

2021-01-20 15:23:51 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: add real-prereqs shorthand for $(filter-out FORCE,$^) In Kbuild, if_changed and friends must have FORCE as a prerequisite. Hence, $(filter-out FORCE,$^) or $(filter-out $(PHONY),$^) is a common idiom to get the names of all the prerequisites except phony targets. Add real-prereqs as a shorthand. Note: We cannot replace $(filter %.o,$^) in cmd_link_multi-m because $^ may include auto-generated dependencies from the .*.cmd file when a single object module is changed into a multi object module. Refer to commit 69ea912fda74 ("kbuild: remove unneeded link_multi_deps"). I added some comment to avoid accidental breakage. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Rob Herring <robh@kernel.org>

2019-01-17 19:02:43 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: move archive command to scripts/Makefile.lib scripts/Makefile.build and arch/s390/boot/Makefile use the same command (thin archiving with symbol table creation). Avoid the code duplication, and move it to scripts/Makefile.lib. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-02-05 16:33:37 +09:00

kbuild: remove ar-option and KBUILD_ARFLAGS Commit 40df759e2b9e ("kbuild: Fix build with binutils <= 2.19") introduced ar-option and KBUILD_ARFLAGS to deal with old binutils. According to Documentation/process/changes.rst, the current minimal supported version of binutils is 2.21 so you can assume the 'D' option is always supported. Not only GNU ar but also llvm-ar supports it. With the 'D' option hard-coded, there is no more user of ar-option or KBUILD_ARFLAGS. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com>

2019-09-21 15:49:54 +09:00

kbuild: move archive command to scripts/Makefile.lib scripts/Makefile.build and arch/s390/boot/Makefile use the same command (thin archiving with symbol table creation). Avoid the code duplication, and move it to scripts/Makefile.lib. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-02-05 16:33:37 +09:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: fix broken builds because of GZIP,BZIP2,LZOP variables Redefine GZIP, BZIP2, LZOP variables as KGZIP, KBZIP2, KLZOP resp. GZIP, BZIP2, LZOP env variables are reserved by the tools. The original attempt to redefine them internally doesn't work in makefiles/scripts intercall scenarios, e.g., "make GZIP=gzip bindeb-pkg" and results in broken builds. There can be other broken build commands because of this, so the universal solution is to use non-reserved env variables for the compression tools. Fixes: 8dfb61dcbace ("kbuild: add variables for compression tools") Signed-off-by: Denis Efremov <efremov@linux.com> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-06-08 12:59:44 +03:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

kbuild: Allow using host dtc instead of kernel's copy Development of dtc happens in its own upstream repository, but testing dtc changes against the kernel tree is useful. Change dtc to a variable that users can override. Signed-off-by: Rob Herring <robh@kernel.org> Cc: Michal Marek <mmarek@suse.com> Cc: linux-kbuild@vger.kernel.org

2016-02-11 16:28:13 -06:00

scripts: dtc: only show unique unit address warning for enabled nodes In some cases an hardware peripheral can be used for two exclusive usages. For example, on STM32MP15 we have the same peripheral for I2S and SPI. We have dedicated driver for each usage and so a dedicated device node in devicetree. To avoid to get useless warnings running "make W=1 dtbs", this patch adds the "-Wunique_unit_address_if_enabled" flag for a make with W=1. In this case we will detect a duplicate address only if both devices are enabled in the devicetree, which is a real error case. Signed-off-by: Alexandre Torgue <alexandre.torgue@foss.st.com> Link: https://lore.kernel.org/r/20221021084447.5550-1-alexandre.torgue@foss.st.com [robh: Refactor options and keep 'unique_unit_address' for W=2] Signed-off-by: Rob Herring <robh@kernel.org>

2022-10-21 10:44:47 +02:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

dtc: turn off dtc unit address warnings by default The newly added dtc warning to check DT unit-address without reg property and vice-versa generates lots of warnings. Turn off the check unless building with W=1 or W=2. Signed-off-by: Rob Herring <robh@kernel.org> Cc: Michal Marek <mmarek@suse.com> Cc: linux-kbuild@vger.kernel.org

2016-03-24 10:52:42 -05:00

kbuild: rename KBUILD_ENABLE_EXTRA_GCC_CHECKS to KBUILD_EXTRA_WARN KBUILD_ENABLE_EXTRA_GCC_CHECKS started as a switch to add extra warning options for GCC, but now it is a historical misnomer since we use it also for Clang, DTC, and even kernel-doc. Rename it to more sensible, shorter KBUILD_EXTRA_WARN. For the backward compatibility, KBUILD_ENABLE_EXTRA_GCC_CHECKS is still supported (but not advertised in the documentation). I also fixed up 'make help', and updated the documentation. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com>

2019-09-01 01:25:55 +09:00

dtc: update warning settings for new bus and node/property name checks dtc gained new warnings checking PCI and simple buses, unit address formatting, and stricter node and property name checking. Disable the new dtc warnings by default as there are 1000s. As before, warnings are enabled with W=1 or W=2. The strict node and property name checks are a bit subjective, so they are only enabled for W=2. Signed-off-by: Rob Herring <robh@kernel.org>

2017-03-21 09:14:20 -05:00

scripts: turn off some new dtc warnings by default The latest dtc update adds some new noisy warnings, so turn them off by default. Disable 'avoid_unnecessary_addr_size' and 'alias_paths'. They can be re-enabled by building with 'W=1'. Signed-off-by: Rob Herring <robh@kernel.org>

2018-02-28 08:09:45 -06:00

kbuild: disable new dtc graph and unit-address warnings dtc gained some new warnings for OF graphs and unique unit addresses, but they are currently much too noisy. So turn off 'graph_child_address', 'graph_port', and 'unique_unit_address' warnings by default. They can be enabled by building dtbs with W=1. Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: linux-kbuild@vger.kernel.org Signed-off-by: Rob Herring <robh@kernel.org>

2018-05-07 16:46:47 -05:00

scripts: dtc: only show unique unit address warning for enabled nodes In some cases an hardware peripheral can be used for two exclusive usages. For example, on STM32MP15 we have the same peripheral for I2S and SPI. We have dedicated driver for each usage and so a dedicated device node in devicetree. To avoid to get useless warnings running "make W=1 dtbs", this patch adds the "-Wunique_unit_address_if_enabled" flag for a make with W=1. In this case we will detect a duplicate address only if both devices are enabled in the devicetree, which is a real error case. Signed-off-by: Alexandre Torgue <alexandre.torgue@foss.st.com> Link: https://lore.kernel.org/r/20221021084447.5550-1-alexandre.torgue@foss.st.com [robh: Refactor options and keep 'unique_unit_address' for W=2] Signed-off-by: Rob Herring <robh@kernel.org>

2022-10-21 10:44:47 +02:00

dtc: turn off dtc unit address warnings by default The newly added dtc warning to check DT unit-address without reg property and vice-versa generates lots of warnings. Turn off the check unless building with W=1 or W=2. Signed-off-by: Rob Herring <robh@kernel.org> Cc: Michal Marek <mmarek@suse.com> Cc: linux-kbuild@vger.kernel.org

2016-03-24 10:52:42 -05:00

kbuild: rename KBUILD_ENABLE_EXTRA_GCC_CHECKS to KBUILD_EXTRA_WARN KBUILD_ENABLE_EXTRA_GCC_CHECKS started as a switch to add extra warning options for GCC, but now it is a historical misnomer since we use it also for Clang, DTC, and even kernel-doc. Rename it to more sensible, shorter KBUILD_EXTRA_WARN. For the backward compatibility, KBUILD_ENABLE_EXTRA_GCC_CHECKS is still supported (but not advertised in the documentation). I also fixed up 'make help', and updated the documentation. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com>

2019-09-01 01:25:55 +09:00

dtc: update warning settings for new bus and node/property name checks dtc gained new warnings checking PCI and simple buses, unit address formatting, and stricter node and property name checking. Disable the new dtc warnings by default as there are 1000s. As before, warnings are enabled with W=1 or W=2. The strict node and property name checks are a bit subjective, so they are only enabled for W=2. Signed-off-by: Rob Herring <robh@kernel.org>

2017-03-21 09:14:20 -05:00

scripts/dtc: Update to upstream version v1.6.0-11-g9d7888cbf19c Sync with upstream dtc primarily to pickup the I2C bus check fixes. The interrupt_provider check is noisy, so turn it off for now. This adds the following commits from upstream: 9d7888cbf19c dtc: Consider one-character strings as strings 8259d59f59de checks: Improve i2c reg property checking fdabcf2980a4 checks: Remove warning for I2C_OWN_SLAVE_ADDRESS 2478b1652c8d libfdt: add extern "C" for C++ f68bfc2668b2 libfdt: trivial typo fix 7be250b4d059 libfdt: Correct condition for reordering blocks 81e0919a3e21 checks: Add interrupt provider test 85e5d839847a Makefile: when building libfdt only, do not add unneeded deps b28464a550c5 Fix some potential unaligned accesses in dtc Signed-off-by: Rob Herring <robh@kernel.org>

2020-06-29 12:15:13 -06:00

scripts: dtc: only show unique unit address warning for enabled nodes In some cases an hardware peripheral can be used for two exclusive usages. For example, on STM32MP15 we have the same peripheral for I2S and SPI. We have dedicated driver for each usage and so a dedicated device node in devicetree. To avoid to get useless warnings running "make W=1 dtbs", this patch adds the "-Wunique_unit_address_if_enabled" flag for a make with W=1. In this case we will detect a duplicate address only if both devices are enabled in the devicetree, which is a real error case. Signed-off-by: Alexandre Torgue <alexandre.torgue@foss.st.com> Link: https://lore.kernel.org/r/20221021084447.5550-1-alexandre.torgue@foss.st.com [robh: Refactor options and keep 'unique_unit_address' for W=2] Signed-off-by: Rob Herring <robh@kernel.org>

2022-10-21 10:44:47 +02:00

dtc: turn off dtc unit address warnings by default The newly added dtc warning to check DT unit-address without reg property and vice-versa generates lots of warnings. Turn off the check unless building with W=1 or W=2. Signed-off-by: Rob Herring <robh@kernel.org> Cc: Michal Marek <mmarek@suse.com> Cc: linux-kbuild@vger.kernel.org

2016-03-24 10:52:42 -05:00

of: per-file dtc compiler flags The dtc compiler version that adds initial support was available in 4.11-rc1. Add the ability to set an additional dtc compiler flag is needed by overlays. Signed-off-by: Frank Rowand <frank.rowand@sony.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Rob Herring <robh@kernel.org>

2017-04-25 17:09:53 -07:00

kbuild: refactor fdtoverlay rule Rename overlay-y to multi-dtb-y, which is a consistent name with multi-obj-y. Also, use multi-search to avoid code duplication. Introduce real-dtb-y, which is a consistent name with real-obj-y, to contain primitive blobs compiled from *.dts. This is used to calculate the list of *.dt.yaml files. Set -@ to base DTB without using $(eval ). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:56 +09:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

kbuild: Allow DTB overlays to built into .dtbo.S files DTB files can be built into the kernel by converting them to assembly files then assembling them into object files. We extend this here for DTB overlays with the .dtso extensions. We change the start and end delimiting tag prefix to make it clear that this data came from overlay files. [Based on patch by Frank Rowand <frank.rowand@sony.com>] Signed-off-by: Andrew Davis <afd@ti.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Frank Rowand <frowand.list@gmail.com> Tested-by: Frank Rowand <frowand.list@gmail.com> Link: https://lore.kernel.org/r/20221024173434.32518-3-afd@ti.com Signed-off-by: Rob Herring <robh@kernel.org>

2022-10-24 12:34:29 -05:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

kbuild: remove unnecessary in-subshell execution The commands surrounded by ( ) are executed in a subshell, but in most cases, we do not need to spawn an extra subshell. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-01-17 19:02:44 +09:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

kbuild: Handle builtin dtb file names containing hyphens cmd_dt_S_dtb constructs the assembly source to incorporate a devicetree FDT (that is, the .dtb file) as binary data in the kernel image. This assembly source contains labels before and after the binary data. The label names incorporate the file name of the corresponding .dtb file. Hyphens are not legal characters in labels, so .dtb files built into the kernel with hyphens in the file name result in errors like the following: bcm3368-netgear-cvg834g.dtb.S: Assembler messages: bcm3368-netgear-cvg834g.dtb.S:5: Error: : no such section bcm3368-netgear-cvg834g.dtb.S:5: Error: junk at end of line, first unrecognized character is `-' bcm3368-netgear-cvg834g.dtb.S:6: Error: unrecognized opcode `__dtb_bcm3368-netgear-cvg834g_begin:' bcm3368-netgear-cvg834g.dtb.S:8: Error: unrecognized opcode `__dtb_bcm3368-netgear-cvg834g_end:' bcm3368-netgear-cvg834g.dtb.S:9: Error: : no such section bcm3368-netgear-cvg834g.dtb.S:9: Error: junk at end of line, first unrecognized character is `-' Fix this by updating cmd_dt_S_dtb to transform all hyphens from the file name to underscores when constructing the labels. As of v4.16-rc2, 1139 .dts files across ARM64, ARM, MIPS and PowerPC contain hyphens in their names, but the issue only currently manifests on Broadcom MIPS platforms, as that is the only place where such files are built into the kernel. For example when CONFIG_DT_NETGEAR_CVG834G=y, or on BMIPS kernels when the dtbs target is used (in the latter case it admittedly shouldn't really build all the dtb.o files, but thats a separate issue). Fixes: 695835511f96 ("MIPS: BMIPS: rename bcm96358nb4ser to bcm6358-neufbox4-sercom") Signed-off-by: James Hogan <jhogan@kernel.org> Reviewed-by: Frank Rowand <frowand.list@gmail.com> Cc: Rob Herring <robh+dt@kernel.org> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Florian Fainelli <f.fainelli@gmail.com> Cc: Kevin Cernekee <cernekee@gmail.com> Cc: <stable@vger.kernel.org> # 4.9+ Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-03-08 11:02:46 +00:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

kbuild: Handle builtin dtb file names containing hyphens cmd_dt_S_dtb constructs the assembly source to incorporate a devicetree FDT (that is, the .dtb file) as binary data in the kernel image. This assembly source contains labels before and after the binary data. The label names incorporate the file name of the corresponding .dtb file. Hyphens are not legal characters in labels, so .dtb files built into the kernel with hyphens in the file name result in errors like the following: bcm3368-netgear-cvg834g.dtb.S: Assembler messages: bcm3368-netgear-cvg834g.dtb.S:5: Error: : no such section bcm3368-netgear-cvg834g.dtb.S:5: Error: junk at end of line, first unrecognized character is `-' bcm3368-netgear-cvg834g.dtb.S:6: Error: unrecognized opcode `__dtb_bcm3368-netgear-cvg834g_begin:' bcm3368-netgear-cvg834g.dtb.S:8: Error: unrecognized opcode `__dtb_bcm3368-netgear-cvg834g_end:' bcm3368-netgear-cvg834g.dtb.S:9: Error: : no such section bcm3368-netgear-cvg834g.dtb.S:9: Error: junk at end of line, first unrecognized character is `-' Fix this by updating cmd_dt_S_dtb to transform all hyphens from the file name to underscores when constructing the labels. As of v4.16-rc2, 1139 .dts files across ARM64, ARM, MIPS and PowerPC contain hyphens in their names, but the issue only currently manifests on Broadcom MIPS platforms, as that is the only place where such files are built into the kernel. For example when CONFIG_DT_NETGEAR_CVG834G=y, or on BMIPS kernels when the dtbs target is used (in the latter case it admittedly shouldn't really build all the dtb.o files, but thats a separate issue). Fixes: 695835511f96 ("MIPS: BMIPS: rename bcm96358nb4ser to bcm6358-neufbox4-sercom") Signed-off-by: James Hogan <jhogan@kernel.org> Reviewed-by: Frank Rowand <frowand.list@gmail.com> Cc: Rob Herring <robh+dt@kernel.org> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Florian Fainelli <f.fainelli@gmail.com> Cc: Kevin Cernekee <cernekee@gmail.com> Cc: <stable@vger.kernel.org> # 4.9+ Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-03-08 11:02:46 +00:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

kbuild: remove unnecessary in-subshell execution The commands surrounded by ( ) are executed in a subshell, but in most cases, we do not need to spawn an extra subshell. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-01-17 19:02:44 +09:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

kbuild: add %.dtb.S and %.dtb to 'targets' automatically Another common pattern that consists of chained commands is to compile a DTB as binary data into the kernel image or a module. It is used in several places in the source tree. Support it in the core Makefile. $(call if_changed,dt_S_dtb) is more suitable than $(call cmd,dt_S_dtb) in case cmd_dt_S_dtb is changed in the future. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Frank Rowand <frowand.list@gmail.com>

2018-03-23 22:04:34 +09:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

kbuild: Allow DTB overlays to built into .dtbo.S files DTB files can be built into the kernel by converting them to assembly files then assembling them into object files. We extend this here for DTB overlays with the .dtso extensions. We change the start and end delimiting tag prefix to make it clear that this data came from overlay files. [Based on patch by Frank Rowand <frank.rowand@sony.com>] Signed-off-by: Andrew Davis <afd@ti.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Frank Rowand <frowand.list@gmail.com> Tested-by: Frank Rowand <frowand.list@gmail.com> Link: https://lore.kernel.org/r/20221024173434.32518-3-afd@ti.com Signed-off-by: Rob Herring <robh@kernel.org>

2022-10-24 12:34:29 -05:00

of: Add support for linking device tree blobs into vmlinux This patch adds support for linking device tree blob(s) into vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking .dtb sections into vmlinux. To maintain compatiblity with the of/fdt driver code platforms MUST copy the blob to a non-init memory location before the kernel frees the .init.* sections in the image. Modifies scripts/Makefile.lib to add a kbuild command to compile DTS files to device tree blobs and a rule to create objects to wrap the blobs for linking. STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to create wrapper objects for the dtb in Makefile.lib. The STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the STRUCT_ALIGNMENT definition. The DTB's are placed on 32 byte boundries to allow parsing the blob with driver/of/fdt.c during early boot without having to copy the blob to get the structure alignment GCC expects. A DTB is linked in by adding the DTB object to the list of objects to be linked into vmlinux in the archtecture specific Makefile using obj-y += foo.dtb.o Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com> Acked-by: Michal Marek <mmarek@suse.cz> [grant.likely@secretlab.ca: cleaned up whitespace inconsistencies] Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2010-12-22 11:57:26 -08:00

Revert "kbuild: Create directory for target DTB" This reverts commit 77479b38e2f58890eb221a0418357502a5b41cd6. Since commit 8a78756eb545 ("kbuild: create object directories simpler and faster"), all directories for 'targets' are created. 'mkdir -p $(dir ${dtc-tmp})' is no longer needed. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-06-23 17:05:49 +09:00

kbuild: remove unneeded -O option to dtc This piece of code converts the target suffix to the dtc -O option: *.dtb -> -O dtb *.dt.yaml -> -O yaml Commit ce88c9c79455 ("kbuild: Add support to build overlays (%.dtbo)") added the third case: *.dtbo -> -O dtbo This works thanks to commit 163f0469bf2e ("dtc: Allow overlays to have .dtbo extension") in the upstream DTC, which has already been pulled in the kernel. However, I think it is a bit odd because "dtbo" is not a format name. At least, it does not show up in the help message of dtc. $ scripts/dtc/dtc --help [ snip ] -O, --out-format <arg> Output formats are: dts - device tree source text dtb - device tree blob yaml - device tree encoded as YAML asm - assembler source So, I am not a big fan of the second hunk of that change: } else if (streq(outform, "dtbo")) { dt_to_blob(outf, dti, outversion); Anyway, we did not need to do this in Makefile in the first place. guess_type_by_name() had already understood ".yaml" before commit 4f0e3a57d6eb ("kbuild: Add support for DT binding schema checks"), and now does ".dtbo" as well. Makefile does not need to duplicate the same logic. Let's leave it to dtc. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> Acked-by: Rob Herring <robh@kernel.org>

2021-03-10 20:08:24 +09:00

of: search scripts/dtc/include-prefixes path for both CPP and DTC Since commit d5d332d3f7e8 ("devicetree: Move include prefixes from arch to separate directory"), cross-arch DT reference works well, but only for CPP style #include directives. It makes as much sense to share DT between different architectures by using DTC's /include/ directives. So, scripts/dtc/include-prefixes should be passed to both CPP and DTC. I refactored Makefile.lib a bit to not repeat the same path. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Rob Herring <robh@kernel.org>

2017-07-25 11:23:36 +09:00

kbuild: make sure we clean up DTB temporary files Various temporary files used when building DTB files were not suffixed with .tmp and therefore were not cleaned up by "make clean". Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Reviewed-by: Stephen Warren <swarren@nvidia.com> Tested-by: Stephen Warren <swarren@nvidia.com> Signed-off-by: Grant Likely <grant.likely@linaro.org>

2013-05-31 11:14:20 +01:00

Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!

2005-04-16 15:20:36 -07:00

kbuild: Add generic rule to apply fdtoverlay Add a generic rule to apply fdtoverlay in Makefile.lib, so every platform doesn't need to carry the complex rule. This also automatically adds "DTC_FLAGS_foo_base += -@" for all base files. The platform's Makefile only needs to have this now: foo-dtbs := foo_base.dtb foo_overlay1.dtbo foo_overlay2.dtbo dtb-y := foo.dtb We don't want to run schema checks on foo.dtb (as foo.dts doesn't exist) and the Makefile is updated accordingly. Acked-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Rob Herring <robh@kernel.org> Co-developed-by: Viresh Kumar <viresh.kumar@linaro.org> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> Reviewed-by: Frank Rowand <frank.rowand@sony.com> Tested-by: Frank Rowand <frank.rowand@sony.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://lore.kernel.org/r/20920b0df6b067aca4040459a9677d7d1d6d766a.1615354376.git.viresh.kumar@linaro.org

2021-03-10 11:05:30 +05:30

kbuild: refactor fdtoverlay rule Rename overlay-y to multi-dtb-y, which is a consistent name with multi-obj-y. Also, use multi-search to avoid code duplication. Introduce real-dtb-y, which is a consistent name with real-obj-y, to contain primitive blobs compiled from *.dts. This is used to calculate the list of *.dt.yaml files. Set -@ to base DTB without using $(eval ). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:56 +09:00

kbuild: Add generic rule to apply fdtoverlay Add a generic rule to apply fdtoverlay in Makefile.lib, so every platform doesn't need to carry the complex rule. This also automatically adds "DTC_FLAGS_foo_base += -@" for all base files. The platform's Makefile only needs to have this now: foo-dtbs := foo_base.dtb foo_overlay1.dtbo foo_overlay2.dtbo dtb-y := foo.dtb We don't want to run schema checks on foo.dtb (as foo.dts doesn't exist) and the Makefile is updated accordingly. Acked-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Rob Herring <robh@kernel.org> Co-developed-by: Viresh Kumar <viresh.kumar@linaro.org> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> Reviewed-by: Frank Rowand <frank.rowand@sony.com> Tested-by: Frank Rowand <frank.rowand@sony.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://lore.kernel.org/r/20920b0df6b067aca4040459a9677d7d1d6d766a.1615354376.git.viresh.kumar@linaro.org

2021-03-10 11:05:30 +05:30

kbuild: refactor fdtoverlay rule Rename overlay-y to multi-dtb-y, which is a consistent name with multi-obj-y. Also, use multi-search to avoid code duplication. Introduce real-dtb-y, which is a consistent name with real-obj-y, to contain primitive blobs compiled from *.dts. This is used to calculate the list of *.dt.yaml files. Set -@ to base DTB without using $(eval ). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2021-05-03 03:09:56 +09:00

kbuild: Add generic rule to apply fdtoverlay Add a generic rule to apply fdtoverlay in Makefile.lib, so every platform doesn't need to carry the complex rule. This also automatically adds "DTC_FLAGS_foo_base += -@" for all base files. The platform's Makefile only needs to have this now: foo-dtbs := foo_base.dtb foo_overlay1.dtbo foo_overlay2.dtbo dtb-y := foo.dtb We don't want to run schema checks on foo.dtb (as foo.dts doesn't exist) and the Makefile is updated accordingly. Acked-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Rob Herring <robh@kernel.org> Co-developed-by: Viresh Kumar <viresh.kumar@linaro.org> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> Reviewed-by: Frank Rowand <frank.rowand@sony.com> Tested-by: Frank Rowand <frank.rowand@sony.com> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://lore.kernel.org/r/20920b0df6b067aca4040459a9677d7d1d6d766a.1615354376.git.viresh.kumar@linaro.org

2021-03-10 11:05:30 +05:30

dt-bindings: kbuild: Use DTB files for validation Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion. The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available. The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them. Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers. The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped. Signed-off-by: Rob Herring <robh@kernel.org> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Link: https://lore.kernel.org/r/20220310160513.1708182-3-robh@kernel.org

2022-03-10 10:05:13 -06:00

kbuild: Add support for DT binding schema checks This adds the build infrastructure for checking DT binding schema documents and validating dts files using the binding schema. Check DT binding schema documents: make dt_binding_check Build dts files and check using DT binding schema: make dtbs_check Optionally, DT_SCHEMA_FILES can be passed in with a schema file(s) to use for validation. This makes it easier to find and fix errors generated by a specific schema. Currently, the validation targets are separate from a normal build to avoid a hard dependency on the external DT schema project and because there are lots of warnings generated. Cc: Jonathan Corbet <corbet@lwn.net> Cc: Mark Rutland <mark.rutland@arm.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: linux-doc@vger.kernel.org Cc: devicetree@vger.kernel.org Cc: linux-kbuild@vger.kernel.org Signed-off-by: Rob Herring <robh@kernel.org>

2018-09-06 13:26:07 -05:00

dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation. As the command line option is new, we the minimum dtschema version must be updated. Cc: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://lore.kernel.org/r/20220310160513.1708182-2-robh@kernel.org

2022-03-10 10:05:12 -06:00

kbuild: Add support for DT binding schema checks This adds the build infrastructure for checking DT binding schema documents and validating dts files using the binding schema. Check DT binding schema documents: make dt_binding_check Build dts files and check using DT binding schema: make dtbs_check Optionally, DT_SCHEMA_FILES can be passed in with a schema file(s) to use for validation. This makes it easier to find and fix errors generated by a specific schema. Currently, the validation targets are separate from a normal build to avoid a hard dependency on the external DT schema project and because there are lots of warnings generated. Cc: Jonathan Corbet <corbet@lwn.net> Cc: Mark Rutland <mark.rutland@arm.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: linux-doc@vger.kernel.org Cc: devicetree@vger.kernel.org Cc: linux-kbuild@vger.kernel.org Signed-off-by: Rob Herring <robh@kernel.org>

2018-09-06 13:26:07 -05:00

dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation. As the command line option is new, we the minimum dtschema version must be updated. Cc: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Rob Herring <robh@kernel.org> Link: https://lore.kernel.org/r/20220310160513.1708182-2-robh@kernel.org

2022-03-10 10:05:12 -06:00

kbuild: Add support for DT binding schema checks This adds the build infrastructure for checking DT binding schema documents and validating dts files using the binding schema. Check DT binding schema documents: make dt_binding_check Build dts files and check using DT binding schema: make dtbs_check Optionally, DT_SCHEMA_FILES can be passed in with a schema file(s) to use for validation. This makes it easier to find and fix errors generated by a specific schema. Currently, the validation targets are separate from a normal build to avoid a hard dependency on the external DT schema project and because there are lots of warnings generated. Cc: Jonathan Corbet <corbet@lwn.net> Cc: Mark Rutland <mark.rutland@arm.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: linux-doc@vger.kernel.org Cc: devicetree@vger.kernel.org Cc: linux-kbuild@vger.kernel.org Signed-off-by: Rob Herring <robh@kernel.org>

2018-09-06 13:26:07 -05:00

kbuild: take into account DT_SCHEMA_FILES changes while checking dtbs It is useful to be able to recheck dtbs files against a limited set of DT schema files. This can be accomplished by using differnt DT_SCHEMA_FILES argument values while rerunning make dtbs_check. However for some reason if_changed_rule doesn't pick up the rule_dtc changes (and doesn't retrigger the build). Fix this by changing if_changed_rule to if_changed_dep and squashing DTC and dt-validate into a single new command. Then if_changed_dep triggers on DT_SCHEMA_FILES changes and reruns the build/check. Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org> Link: https://lore.kernel.org/r/20220915114422.79378-1-dmitry.baryshkov@linaro.org Signed-off-by: Rob Herring <robh@kernel.org>

2022-09-15 14:44:22 +03:00

dt-bindings: kbuild: Use DTB files for validation Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion. The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available. The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them. Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers. The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped. Signed-off-by: Rob Herring <robh@kernel.org> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Link: https://lore.kernel.org/r/20220310160513.1708182-3-robh@kernel.org

2022-03-10 10:05:13 -06:00

kbuild: Add support for DT binding schema checks This adds the build infrastructure for checking DT binding schema documents and validating dts files using the binding schema. Check DT binding schema documents: make dt_binding_check Build dts files and check using DT binding schema: make dtbs_check Optionally, DT_SCHEMA_FILES can be passed in with a schema file(s) to use for validation. This makes it easier to find and fix errors generated by a specific schema. Currently, the validation targets are separate from a normal build to avoid a hard dependency on the external DT schema project and because there are lots of warnings generated. Cc: Jonathan Corbet <corbet@lwn.net> Cc: Mark Rutland <mark.rutland@arm.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: linux-doc@vger.kernel.org Cc: devicetree@vger.kernel.org Cc: linux-kbuild@vger.kernel.org Signed-off-by: Rob Herring <robh@kernel.org>

2018-09-06 13:26:07 -05:00

dt-bindings: kbuild: Use DTB files for validation Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion. The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available. The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them. Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers. The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped. Signed-off-by: Rob Herring <robh@kernel.org> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Link: https://lore.kernel.org/r/20220310160513.1708182-3-robh@kernel.org

2022-03-10 10:05:13 -06:00

kbuild: take into account DT_SCHEMA_FILES changes while checking dtbs It is useful to be able to recheck dtbs files against a limited set of DT schema files. This can be accomplished by using differnt DT_SCHEMA_FILES argument values while rerunning make dtbs_check. However for some reason if_changed_rule doesn't pick up the rule_dtc changes (and doesn't retrigger the build). Fix this by changing if_changed_rule to if_changed_dep and squashing DTC and dt-validate into a single new command. Then if_changed_dep triggers on DT_SCHEMA_FILES changes and reruns the build/check. Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org> Link: https://lore.kernel.org/r/20220915114422.79378-1-dmitry.baryshkov@linaro.org Signed-off-by: Rob Herring <robh@kernel.org>

2022-09-15 14:44:22 +03:00

kbuild: centralize .dts->.dtb rule All architectures that use cmd_dtc do so in almost the same way. Create a central build rule to avoid duplication. The one difference is that most current uses of dtc build $(obj)/%.dtb from $(src)/dts/%.dts rather than building the .dtb in the same directory as the .dts file. This difference will be eliminated arch-by-arch in future patches. MIPS is the exception here; it already uses the exact same rule as the new common rule, so the duplicate is removed in this patch to avoid any conflict. arch/mips changes courtesy of Ralf Baechle. Update Documentation/kbuild to remove the explicit call to cmd_dtc from the example, now that the rule exists in a centralized location. Cc: Arnd Bergmann <arnd@arndb.de> Cc: linux-arm-kernel@lists.infradead.org Cc: Olof Johansson <olof@lixom.net> Cc: Russell King <linux@arm.linux.org.uk> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Cc: Jonas Bonn <jonas@southpole.se> Cc: linux@lists.openrisc.net Cc: Aurelien Jacquiot <a-jacquiot@ti.com> Cc: linux-c6x-dev@linux-c6x.org Cc: Mark Salter <msalter@redhat.com> Cc: Michal Simek <monstr@monstr.eu> Cc: microblaze-uclinux@itee.uq.edu.au Cc: Chris Zankel <chris@zankel.net> Cc: linux-xtensa@linux-xtensa.org Cc: Max Filippov <jcmvbkbc@gmail.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Signed-off-by: Rob Herring <rob.herring@calxeda.com>

2012-11-27 16:29:10 -07:00

dt-bindings: kbuild: Use DTB files for validation Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion. The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available. The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them. Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers. The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped. Signed-off-by: Rob Herring <robh@kernel.org> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Link: https://lore.kernel.org/r/20220310160513.1708182-3-robh@kernel.org

2022-03-10 10:05:13 -06:00

kbuild: Allow DTB overlays to built from .dtso named source files Currently DTB Overlays (.dtbo) are build from source files with the same extension (.dts) as the base DTs (.dtb). This may become confusing and even lead to wrong results. For example, a composite DTB (created from a base DTB and a set of overlays) might have the same name as one of the overlays that create it. Different files should be generated from differently named sources. .dtb <-> .dts .dtbo <-> .dtso We do not remove the ability to compile DTBO files from .dts files here, only add a new rule allowing the .dtso file name. The current .dts named overlays can be renamed with time. After all have been renamed we can remove the other rule. Signed-off-by: Andrew Davis <afd@ti.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Frank Rowand <frowand.list@gmail.com> Tested-by: Frank Rowand <frowand.list@gmail.com> Link: https://lore.kernel.org/r/20221024173434.32518-2-afd@ti.com Signed-off-by: Rob Herring <robh@kernel.org>

2022-10-24 12:34:28 -05:00

kbuild: make sure we clean up DTB temporary files Various temporary files used when building DTB files were not suffixed with .tmp and therefore were not cleaned up by "make clean". Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Reviewed-by: Stephen Warren <swarren@nvidia.com> Tested-by: Stephen Warren <swarren@nvidia.com> Signed-off-by: Grant Likely <grant.likely@linaro.org>

2013-05-31 11:14:20 +01:00

kbuild: create a rule to run the pre-processor on *.dts files Create cmd_dtc_cpp to run the C pre-processor on *.dts file before passing them to dtc for final compilation. This allows the use of #define and #include within the .dts file. Acked-by: Simon Glass <sjg@chromium.org> Acked-by: Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com> Acked-by: Michal Marek <mmarek@suse.cz> Acked-by: Srinivas Kandagatla <srinivas.kandagatla@st.com> Signed-off-by: Stephen Warren <swarren@nvidia.com> Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

2013-02-05 12:06:28 -07:00

bzip2/lzma: library support for gzip, bzip2 and lzma decompression Impact: Replaces inflate.c with a wrapper around zlib_inflate; new library code This is the first part of the bzip2/lzma patch The bzip patch is based on an idea by Christian Ludwig, includes support for compressing the kernel with bzip2 or lzma rather than gzip. Both compressors give smaller sizes than gzip. Lzma's decompresses faster than bzip2. It also supports ramdisks and initramfs' compressed using these two compressors. The functionality has been successfully used for a couple of years by the udpcast project This version applies to "tip" kernel 2.6.28 This part contains: - changed inflate.c to accomodate rest of patch - implementation of bzip2 compression (not used at this stage yet) - implementation of lzma compression (not used at this stage yet) - Makefile routines to support bzip2 and lzma kernel compression Signed-off-by: Alain Knaff <alain@knaff.lu> Signed-off-by: H. Peter Anvin <hpa@zytor.com>

2009-01-04 22:46:16 +01:00

kbuild: allow compressors (gzip, bzip2, lzma) to take multiple inputs Allow the compression commands in Kbuild (i.e. gzip, bzip2, lzma) to take multiple input files and emit the concatenated compressed output. This avoids an intermediate step when a kernel image is built from multiple components, such as the relocatable x86-32 kernel. Sam Ravnborg integrated the bin_size script into the Makefile. [ Impact: new build feature, not yet used ] Signed-off-by: H. Peter Anvin <hpa@zytor.com> Acked-by: Sam Ravnborg <sam@ravnborg.org>

2009-05-05 21:17:15 -07:00

kbuild: fix bzImage build for x86 As has been discussed previously (and Sam has been CC'ed), the fix is still incorrect. It replaces "echo -ne" with "/bin/echo -ne", but neither of the two are guaranteed to support the necessary arguments and necessary (hexadecimal) escape sequences. What should be used here is printf(1). The trivial patch below (on top of these kbuild changes) fixes this issue. Signed-Off-By: Michael Tokarev <mjt@tls.msk.ru> Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Michal Marek <mmarek@suse.cz>

2009-10-13 22:22:46 +02:00

kbuild: allow compressors (gzip, bzip2, lzma) to take multiple inputs Allow the compression commands in Kbuild (i.e. gzip, bzip2, lzma) to take multiple input files and emit the concatenated compressed output. This avoids an intermediate step when a kernel image is built from multiple components, such as the relocatable x86-32 kernel. Sam Ravnborg integrated the bin_size script into the Makefile. [ Impact: new build feature, not yet used ] Signed-off-by: H. Peter Anvin <hpa@zytor.com> Acked-by: Sam Ravnborg <sam@ravnborg.org>

2009-05-05 21:17:15 -07:00

kbuild: add real-prereqs shorthand for $(filter-out FORCE,$^) In Kbuild, if_changed and friends must have FORCE as a prerequisite. Hence, $(filter-out FORCE,$^) or $(filter-out $(PHONY),$^) is a common idiom to get the names of all the prerequisites except phony targets. Add real-prereqs as a shorthand. Note: We cannot replace $(filter %.o,$^) in cmd_link_multi-m because $^ may include auto-generated dependencies from the .*.cmd file when a single object module is changed into a multi object module. Refer to commit 69ea912fda74 ("kbuild: remove unneeded link_multi_deps"). I added some comment to avoid accidental breakage. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Rob Herring <robh@kernel.org>

2019-01-17 19:02:43 +09:00

kbuild: Use ls(1) instead of stat(1) to obtain file size stat(1) is not standardized and different implementations have their own (conflicting) flags for querying the size of a file. ls(1) provides the same information (value of st.st_size) in the 5th column, except when the file is a character or block device. This output is standardized[0]. The -n option turns on -l, which writes lines formatted like "%s %u %s %s %u %s %s\n", <file mode>, <number of links>, <owner name>, <group name>, <size>, <date and time>, <pathname> but instead of writing the <owner name> and <group name>, it writes the numeric owner and group IDs (this avoids /etc/passwd and /etc/group lookups as well as potential field splitting issues). The <size> field is specified as "the value that would be returned for the file in the st_size field of struct stat". To avoid duplicating logic in several locations in the tree, create scripts/file-size.sh and update callers to use that instead of stat(1). [0] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html#tag_20_73_10 Signed-off-by: Michael Forney <forney@google.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-03-18 17:54:02 -07:00

kbuild: allow compressors (gzip, bzip2, lzma) to take multiple inputs Allow the compression commands in Kbuild (i.e. gzip, bzip2, lzma) to take multiple input files and emit the concatenated compressed output. This avoids an intermediate step when a kernel image is built from multiple components, such as the relocatable x86-32 kernel. Sam Ravnborg integrated the bin_size script into the Makefile. [ Impact: new build feature, not yet used ] Signed-off-by: H. Peter Anvin <hpa@zytor.com> Acked-by: Sam Ravnborg <sam@ravnborg.org>

2009-05-05 21:17:15 -07:00

kbuild: really fix bzImage build with non-bash sh In an x86 build with CONFIG_KERNEL_LZMA enabled and dash as sh, arch/x86/boot/compressed/vmlinux.bin.lzma ends with '\xf0\x7d\x39\x00' (16 bytes) instead of the 4 bytes intended and the resulting vmlinuz fails to boot. This improves on the previous behavior, in which the file contained the characters '-ne ' as well, but not by much. Previous commits replaced "echo -ne" first with "/bin/echo -ne", then "printf" in the hope of improving portability, but none of these commands is guaranteed to support hexadecimal escapes on POSIX systems. So use the shell to convert from hexadecimal to octal. With this change, an LZMA-compressed kernel built with dash as sh boots correctly again. Reported-by: Sebastian Dalfuß <sd@sedf.de> Reported-by: Oliver Hartkopp <oliver@hartkopp.net> Reported-by: Michael Guntsche <mike@it-loops.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Cc: Michael Tokarev <mjt@tls.msk.ru> Cc: Alek Du <alek.du@intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Michal Marek <mmarek@suse.cz>

2009-12-28 19:38:27 +00:00

kbuild: allow compressors (gzip, bzip2, lzma) to take multiple inputs Allow the compression commands in Kbuild (i.e. gzip, bzip2, lzma) to take multiple input files and emit the concatenated compressed output. This avoids an intermediate step when a kernel image is built from multiple components, such as the relocatable x86-32 kernel. Sam Ravnborg integrated the bin_size script into the Makefile. [ Impact: new build feature, not yet used ] Signed-off-by: H. Peter Anvin <hpa@zytor.com> Acked-by: Sam Ravnborg <sam@ravnborg.org>

2009-05-05 21:17:15 -07:00

kbuild: add cmd_file_size Some architectures support self-extracting kernel, which embeds the compressed vmlinux. It has 4 byte data at the end so the decompressor can know the vmlinux size beforehand. GZIP natively has it in the trailer, but for the other compression algorithms, the hand-crafted trailer is added. It is unneeded to generate such _corrupted_ compressed files because it is possible to pass the size data as a separate file. For example, the assembly code: .incbin "compressed-vmlinux-with-size-data" can be transformed to: .incbin "compressed-vmlinux" .incbin "size-data" My hope is, after some reworks of the decompressors, the macros cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}_with_size will go away. This new macro, cmd_file_size, will be useful to generate a separate size-data file. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>

2022-01-10 03:15:29 +09:00

kbuild: allow compressors (gzip, bzip2, lzma) to take multiple inputs Allow the compression commands in Kbuild (i.e. gzip, bzip2, lzma) to take multiple input files and emit the concatenated compressed output. This avoids an intermediate step when a kernel image is built from multiple components, such as the relocatable x86-32 kernel. Sam Ravnborg integrated the bin_size script into the Makefile. [ Impact: new build feature, not yet used ] Signed-off-by: H. Peter Anvin <hpa@zytor.com> Acked-by: Sam Ravnborg <sam@ravnborg.org>

2009-05-05 21:17:15 -07:00

kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22} GZIP-compressed files end with 4 byte data that represents the size of the original input. The decompressors (the self-extracting kernel) exploit it to know the vmlinux size beforehand. To mimic the GZIP's trailer, Kbuild provides cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}. Unfortunately these macros are used everywhere despite the appended size data is only useful for the decompressors. There is no guarantee that such hand-crafted trailers are safely ignored. In fact, the kernel refuses compressed initramdfs with the garbage data. That is why usr/Makefile overrides size_append to make it no-op. To limit the use of such broken compressed files, this commit renames the existing macros as follows: cmd_bzip2 --> cmd_bzip2_with_size cmd_lzma --> cmd_lzma_with_size cmd_lzo --> cmd_lzo_with_size cmd_lz4 --> cmd_lz4_with_size cmd_xzkern --> cmd_xzkern_with_size cmd_zstd22 --> cmd_zstd22_with_size To keep the decompressors working, I updated the following Makefiles accordingly: arch/arm/boot/compressed/Makefile arch/h8300/boot/compressed/Makefile arch/mips/boot/compressed/Makefile arch/parisc/boot/compressed/Makefile arch/s390/boot/compressed/Makefile arch/sh/boot/compressed/Makefile arch/x86/boot/compressed/Makefile I reused the current macro names for the normal usecases; they produce the compressed data in the proper format. I did not touch the following: arch/arc/boot/Makefile arch/arm64/boot/Makefile arch/csky/boot/Makefile arch/mips/boot/Makefile arch/riscv/boot/Makefile arch/sh/boot/Makefile kernel/Makefile This means those Makefiles will stop appending the size data. I dropped the 'override size_append' hack from usr/Makefile. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>

2022-01-10 03:15:27 +09:00

bzip2/lzma: library support for gzip, bzip2 and lzma decompression Impact: Replaces inflate.c with a wrapper around zlib_inflate; new library code This is the first part of the bzip2/lzma patch The bzip patch is based on an idea by Christian Ludwig, includes support for compressing the kernel with bzip2 or lzma rather than gzip. Both compressors give smaller sizes than gzip. Lzma's decompresses faster than bzip2. It also supports ramdisks and initramfs' compressed using these two compressors. The functionality has been successfully used for a couple of years by the udpcast project This version applies to "tip" kernel 2.6.28 This part contains: - changed inflate.c to accomodate rest of patch - implementation of bzip2 compression (not used at this stage yet) - implementation of lzma compression (not used at this stage yet) - Makefile routines to support bzip2 and lzma kernel compression Signed-off-by: Alain Knaff <alain@knaff.lu> Signed-off-by: H. Peter Anvin <hpa@zytor.com>

2009-01-04 22:46:16 +01:00

kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22} GZIP-compressed files end with 4 byte data that represents the size of the original input. The decompressors (the self-extracting kernel) exploit it to know the vmlinux size beforehand. To mimic the GZIP's trailer, Kbuild provides cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}. Unfortunately these macros are used everywhere despite the appended size data is only useful for the decompressors. There is no guarantee that such hand-crafted trailers are safely ignored. In fact, the kernel refuses compressed initramdfs with the garbage data. That is why usr/Makefile overrides size_append to make it no-op. To limit the use of such broken compressed files, this commit renames the existing macros as follows: cmd_bzip2 --> cmd_bzip2_with_size cmd_lzma --> cmd_lzma_with_size cmd_lzo --> cmd_lzo_with_size cmd_lz4 --> cmd_lz4_with_size cmd_xzkern --> cmd_xzkern_with_size cmd_zstd22 --> cmd_zstd22_with_size To keep the decompressors working, I updated the following Makefiles accordingly: arch/arm/boot/compressed/Makefile arch/h8300/boot/compressed/Makefile arch/mips/boot/compressed/Makefile arch/parisc/boot/compressed/Makefile arch/s390/boot/compressed/Makefile arch/sh/boot/compressed/Makefile arch/x86/boot/compressed/Makefile I reused the current macro names for the normal usecases; they produce the compressed data in the proper format. I did not touch the following: arch/arc/boot/Makefile arch/arm64/boot/Makefile arch/csky/boot/Makefile arch/mips/boot/Makefile arch/riscv/boot/Makefile arch/sh/boot/Makefile kernel/Makefile This means those Makefiles will stop appending the size data. I dropped the 'override size_append' hack from usr/Makefile. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>

2022-01-10 03:15:27 +09:00

lib: add support for LZO-compressed kernels This patch series adds generic support for creating and extracting LZO-compressed kernel images, as well as support for using such images on the x86 and ARM architectures, and support for creating and using LZO-compressed initrd and initramfs images. Russell King said: : Testing on a Cortex A9 model: : - lzo decompressor is 65% of the time gzip takes to decompress a kernel : - lzo kernel is 9% larger than a gzip kernel : : which I'm happy to say confirms your figures when comparing the two. : : However, when comparing your new gzip code to the old gzip code: : - new is 99% of the size of the old code : - new takes 42% of the time to decompress than the old code : : What this means is that for a proper comparison, the results get even better: : - lzo is 7.5% larger than the old gzip'd kernel image : - lzo takes 28% of the time that the old gzip code took : : So the expense seems definitely worth the effort. The only reason I : can think of ever using gzip would be if you needed the additional : compression (eg, because you have limited flash to store the image.) : : I would argue that the default for ARM should therefore be LZO. This patch: The lzo compressor is worse than gzip at compression, but faster at extraction. Here are some figures for an ARM board I'm working on: Uncompressed size: 3.24Mo gzip 1.61Mo 0.72s lzo 1.75Mo 0.48s So for a compression ratio that is still relatively close to gzip, it's much faster to extract, at least in that case. This part contains: - Makefile routine to support lzo compression - Fixes to the existing lzo compressor so that it can be used in compressed kernels - wrapper around the existing lzo1x_decompress, as it only extracts one block at a time, while we need to extract a whole file here - config dialog for kernel compression [akpm@linux-foundation.org: coding-style fixes] [akpm@linux-foundation.org: cleanup] Signed-off-by: Albin Tonnerre <albin.tonnerre@free-electrons.com> Tested-by: Wu Zhangjin <wuzhangjin@gmail.com> Acked-by: "H. Peter Anvin" <hpa@zytor.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Tested-by: Russell King <rmk@arm.linux.org.uk> Acked-by: Russell King <rmk@arm.linux.org.uk> Cc: Ralf Baechle <ralf@linux-mips.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2010-01-08 14:42:42 -08:00

scripts/Makefile.lib: Align the output of LZO The output of LZO is not aligned with the other output: ... CC drivers/usb/mon/usbmon.mod.o LZO arch/mips/boot/compressed/vmlinux.lzo ... This patch fixes it. Signed-off-by: Wu Zhangjin <wuzhangjin@gmail.com> Signed-off-by: Michal Marek <mmarek@suse.cz>

2010-03-11 17:42:14 +08:00

kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22} GZIP-compressed files end with 4 byte data that represents the size of the original input. The decompressors (the self-extracting kernel) exploit it to know the vmlinux size beforehand. To mimic the GZIP's trailer, Kbuild provides cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}. Unfortunately these macros are used everywhere despite the appended size data is only useful for the decompressors. There is no guarantee that such hand-crafted trailers are safely ignored. In fact, the kernel refuses compressed initramdfs with the garbage data. That is why usr/Makefile overrides size_append to make it no-op. To limit the use of such broken compressed files, this commit renames the existing macros as follows: cmd_bzip2 --> cmd_bzip2_with_size cmd_lzma --> cmd_lzma_with_size cmd_lzo --> cmd_lzo_with_size cmd_lz4 --> cmd_lz4_with_size cmd_xzkern --> cmd_xzkern_with_size cmd_zstd22 --> cmd_zstd22_with_size To keep the decompressors working, I updated the following Makefiles accordingly: arch/arm/boot/compressed/Makefile arch/h8300/boot/compressed/Makefile arch/mips/boot/compressed/Makefile arch/parisc/boot/compressed/Makefile arch/s390/boot/compressed/Makefile arch/sh/boot/compressed/Makefile arch/x86/boot/compressed/Makefile I reused the current macro names for the normal usecases; they produce the compressed data in the proper format. I did not touch the following: arch/arc/boot/Makefile arch/arm64/boot/Makefile arch/csky/boot/Makefile arch/mips/boot/Makefile arch/riscv/boot/Makefile arch/sh/boot/Makefile kernel/Makefile This means those Makefiles will stop appending the size data. I dropped the 'override size_append' hack from usr/Makefile. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>

2022-01-10 03:15:27 +09:00

x86: Add optimized popcnt variants Add support for the hardware version of the Hamming weight function, popcnt, present in CPUs which advertize it under CPUID, Function 0x0000_0001_ECX[23]. On CPUs which don't support it, we fallback to the default lib/hweight.c sw versions. A synthetic benchmark comparing popcnt with __sw_hweight64 showed almost a 3x speedup on a F10h machine. Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> LKML-Reference: <20100318112015.GC11152@aftab> Signed-off-by: H. Peter Anvin <hpa@zytor.com>

2010-03-05 17:34:46 +01:00

lib: add support for LZ4-compressed kernel Add support for extracting LZ4-compressed kernel images, as well as LZ4-compressed ramdisk images in the kernel boot process. Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Russell King <rmk@arm.linux.org.uk> Cc: Borislav Petkov <bp@alien8.de> Cc: Florian Fainelli <florian@openwrt.org> Cc: Yann Collet <yann.collet.73@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2013-07-08 16:01:46 -07:00

kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22} GZIP-compressed files end with 4 byte data that represents the size of the original input. The decompressors (the self-extracting kernel) exploit it to know the vmlinux size beforehand. To mimic the GZIP's trailer, Kbuild provides cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}. Unfortunately these macros are used everywhere despite the appended size data is only useful for the decompressors. There is no guarantee that such hand-crafted trailers are safely ignored. In fact, the kernel refuses compressed initramdfs with the garbage data. That is why usr/Makefile overrides size_append to make it no-op. To limit the use of such broken compressed files, this commit renames the existing macros as follows: cmd_bzip2 --> cmd_bzip2_with_size cmd_lzma --> cmd_lzma_with_size cmd_lzo --> cmd_lzo_with_size cmd_lz4 --> cmd_lz4_with_size cmd_xzkern --> cmd_xzkern_with_size cmd_zstd22 --> cmd_zstd22_with_size To keep the decompressors working, I updated the following Makefiles accordingly: arch/arm/boot/compressed/Makefile arch/h8300/boot/compressed/Makefile arch/mips/boot/compressed/Makefile arch/parisc/boot/compressed/Makefile arch/s390/boot/compressed/Makefile arch/sh/boot/compressed/Makefile arch/x86/boot/compressed/Makefile I reused the current macro names for the normal usecases; they produce the compressed data in the proper format. I did not touch the following: arch/arc/boot/Makefile arch/arm64/boot/Makefile arch/csky/boot/Makefile arch/mips/boot/Makefile arch/riscv/boot/Makefile arch/sh/boot/Makefile kernel/Makefile This means those Makefiles will stop appending the size data. I dropped the 'override size_append' hack from usr/Makefile. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>

2022-01-10 03:15:27 +09:00

kbuild: remove unnecessary in-subshell execution The commands surrounded by ( ) are executed in a subshell, but in most cases, we do not need to spawn an extra subshell. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-01-17 19:02:44 +09:00

lib: add support for LZ4-compressed kernel Add support for extracting LZ4-compressed kernel images, as well as LZ4-compressed ramdisk images in the kernel boot process. Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Russell King <rmk@arm.linux.org.uk> Cc: Borislav Petkov <bp@alien8.de> Cc: Florian Fainelli <florian@openwrt.org> Cc: Yann Collet <yann.collet.73@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2013-07-08 16:01:46 -07:00

Kbuild: centralize MKIMAGE and cmd_uimage definitions All ARCHs have the same definition of MKIMAGE. Move it to Makefile.lib to avoid duplication. All ARCHs have similar definitions of cmd_uimage. Place a sufficiently parameterized version in Makefile.lib to avoid duplication. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Acked-by: Nicolas Pitre <nico@linaro.org> Tested-by: Mike Frysinger <vapier@gentoo.org> [Blackfin] Tested-by: Michal Simek <monstr@monstr.eu> [Microblaze] Tested-by: Guan Xuetao <gxt@mprc.pku.edu.cn> [unicore32] Signed-off-by: Michal Marek <mmarek@suse.cz>

2012-03-16 15:03:55 -06:00

kbuild: replace $(if A,A,B) with $(or A,B) $(or ...) is available since GNU Make 3.81, and useful to shorten the code in some places. Covert as follows: $(if A,A,B) --> $(or A,B) This patch also converts: $(if A, A, B) --> $(or A, B) Strictly speaking, the latter is not an equivalent conversion because GNU Make keeps spaces after commas; if A is not empty, $(if A, A, B) expands to " A", while $(or A, B) expands to "A". Anyway, preceding spaces are not significant in the code hunks I touched. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

2022-02-11 14:14:11 +09:00

Kbuild: centralize MKIMAGE and cmd_uimage definitions All ARCHs have the same definition of MKIMAGE. Move it to Makefile.lib to avoid duplication. All ARCHs have similar definitions of cmd_uimage. Place a sufficiently parameterized version in Makefile.lib to avoid duplication. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Acked-by: Nicolas Pitre <nico@linaro.org> Tested-by: Mike Frysinger <vapier@gentoo.org> [Blackfin] Tested-by: Michal Simek <monstr@monstr.eu> [Microblaze] Tested-by: Guan Xuetao <gxt@mprc.pku.edu.cn> [unicore32] Signed-off-by: Michal Marek <mmarek@suse.cz>

2012-03-16 15:03:55 -06:00

kbuild: remove UIMAGE_IN and UIMAGE_OUT The only/last user of UIMAGE_IN/OUT was removed by commit 4722a3e6b716 ("microblaze: fix multiple bugs in arch/microblaze/boot/Makefile"). The input and output should always be $< and $@. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-12-31 10:05:01 +09:00

kbuild: add $(BASH) to run scripts with bash-extension CONFIG_SHELL falls back to sh when bash is not installed on the system, but nobody is testing such a case since bash is usually installed. So, shell scripts invoked by CONFIG_SHELL are only tested with bash. It makes it difficult to test whether the hashbang #!/bin/sh is real. For example, #!/bin/sh in arch/powerpc/kernel/prom_init_check.sh is false. (I fixed it up) Besides, some shell scripts invoked by CONFIG_SHELL use bash-extension and #!/bin/bash is specified as the hashbang, while CONFIG_SHELL may not always be set to bash. Probably, the right thing to do is to introduce BASH, which is bash by default, and always set CONFIG_SHELL to sh. Replace $(CONFIG_SHELL) with $(BASH) for bash scripts. If somebody tries to add bash-extension to a #!/bin/sh script, it will be caught in testing because /bin/sh is a symlink to dash on some major distributions. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-08-25 22:28:37 +09:00

Kbuild: centralize MKIMAGE and cmd_uimage definitions All ARCHs have the same definition of MKIMAGE. Move it to Makefile.lib to avoid duplication. All ARCHs have similar definitions of cmd_uimage. Place a sufficiently parameterized version in Makefile.lib to avoid duplication. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Acked-by: Nicolas Pitre <nico@linaro.org> Tested-by: Mike Frysinger <vapier@gentoo.org> [Blackfin] Tested-by: Michal Simek <monstr@monstr.eu> [Microblaze] Tested-by: Guan Xuetao <gxt@mprc.pku.edu.cn> [unicore32] Signed-off-by: Michal Marek <mmarek@suse.cz>

2012-03-16 15:03:55 -06:00

kbuild: remove UIMAGE_IN and UIMAGE_OUT The only/last user of UIMAGE_IN/OUT was removed by commit 4722a3e6b716 ("microblaze: fix multiple bugs in arch/microblaze/boot/Makefile"). The input and output should always be $< and $@. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-12-31 10:05:01 +09:00

Kbuild: centralize MKIMAGE and cmd_uimage definitions All ARCHs have the same definition of MKIMAGE. Move it to Makefile.lib to avoid duplication. All ARCHs have similar definitions of cmd_uimage. Place a sufficiently parameterized version in Makefile.lib to avoid duplication. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Acked-by: Nicolas Pitre <nico@linaro.org> Tested-by: Mike Frysinger <vapier@gentoo.org> [Blackfin] Tested-by: Michal Simek <monstr@monstr.eu> [Microblaze] Tested-by: Guan Xuetao <gxt@mprc.pku.edu.cn> [unicore32] Signed-off-by: Michal Marek <mmarek@suse.cz>

2012-03-16 15:03:55 -06:00

decompressors: add XZ decompressor module In userspace, the .lzma format has become mostly a legacy file format that got superseded by the .xz format. Similarly, LZMA Utils was superseded by XZ Utils. These patches add support for XZ decompression into the kernel. Most of the code is as is from XZ Embedded <http://tukaani.org/xz/embedded.html>. It was written for the Linux kernel but is usable in other projects too. Advantages of XZ over the current LZMA code in the kernel: - Nice API that can be used by other kernel modules; it's not limited to kernel, initramfs, and initrd decompression. - Integrity check support (CRC32) - BCJ filters improve compression of executable code on certain architectures. These together with LZMA2 can produce a few percent smaller kernel or Squashfs images than plain LZMA without making the decompression slower. This patch: Add the main decompression code (xz_dec), testing module (xz_dec_test), wrapper script (xz_wrap.sh) for the xz command line tool, and documentation. The xz_dec module is enough to have a usable XZ decompressor e.g. for Squashfs. Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Alain Knaff <alain@knaff.lu> Cc: Albin Tonnerre <albin.tonnerre@free-electrons.com> Cc: Phillip Lougher <phillip@lougher.demon.co.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2011-01-12 17:01:22 -08:00

kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22} GZIP-compressed files end with 4 byte data that represents the size of the original input. The decompressors (the self-extracting kernel) exploit it to know the vmlinux size beforehand. To mimic the GZIP's trailer, Kbuild provides cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}. Unfortunately these macros are used everywhere despite the appended size data is only useful for the decompressors. There is no guarantee that such hand-crafted trailers are safely ignored. In fact, the kernel refuses compressed initramdfs with the garbage data. That is why usr/Makefile overrides size_append to make it no-op. To limit the use of such broken compressed files, this commit renames the existing macros as follows: cmd_bzip2 --> cmd_bzip2_with_size cmd_lzma --> cmd_lzma_with_size cmd_lzo --> cmd_lzo_with_size cmd_lz4 --> cmd_lz4_with_size cmd_xzkern --> cmd_xzkern_with_size cmd_zstd22 --> cmd_zstd22_with_size To keep the decompressors working, I updated the following Makefiles accordingly: arch/arm/boot/compressed/Makefile arch/h8300/boot/compressed/Makefile arch/mips/boot/compressed/Makefile arch/parisc/boot/compressed/Makefile arch/s390/boot/compressed/Makefile arch/sh/boot/compressed/Makefile arch/x86/boot/compressed/Makefile I reused the current macro names for the normal usecases; they produce the compressed data in the proper format. I did not touch the following: arch/arc/boot/Makefile arch/arm64/boot/Makefile arch/csky/boot/Makefile arch/mips/boot/Makefile arch/riscv/boot/Makefile arch/sh/boot/Makefile kernel/Makefile This means those Makefiles will stop appending the size data. I dropped the 'override size_append' hack from usr/Makefile. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>

2022-01-10 03:15:27 +09:00

kbuild: remove unnecessary in-subshell execution The commands surrounded by ( ) are executed in a subshell, but in most cases, we do not need to spawn an extra subshell. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2019-01-17 19:02:44 +09:00

decompressors: add XZ decompressor module In userspace, the .lzma format has become mostly a legacy file format that got superseded by the .xz format. Similarly, LZMA Utils was superseded by XZ Utils. These patches add support for XZ decompression into the kernel. Most of the code is as is from XZ Embedded <http://tukaani.org/xz/embedded.html>. It was written for the Linux kernel but is usable in other projects too. Advantages of XZ over the current LZMA code in the kernel: - Nice API that can be used by other kernel modules; it's not limited to kernel, initramfs, and initrd decompression. - Integrity check support (CRC32) - BCJ filters improve compression of executable code on certain architectures. These together with LZMA2 can produce a few percent smaller kernel or Squashfs images than plain LZMA without making the decompression slower. This patch: Add the main decompression code (xz_dec), testing module (xz_dec_test), wrapper script (xz_wrap.sh) for the xz command line tool, and documentation. The xz_dec module is enough to have a usable XZ decompressor e.g. for Squashfs. Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Alain Knaff <alain@knaff.lu> Cc: Albin Tonnerre <albin.tonnerre@free-electrons.com> Cc: Phillip Lougher <phillip@lougher.demon.co.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2011-01-12 17:01:22 -08:00

kbuild: add variables for compression tools Allow user to use alternative implementations of compression tools, such as pigz, pbzip2, pxz. For example, multi-threaded tools to speed up the build: $ make GZIP=pigz BZIP2=pbzip2 Variables _GZIP, _BZIP2, _LZOP are used internally because original env vars are reserved by the tools. The use of GZIP in gzip tool is obsolete since 2015. However, alternative implementations (e.g., pigz) still rely on it. BZIP2, BZIP, LZOP vars are not obsolescent. The credit goes to @grsecurity. As a sidenote, for multi-threaded lzma, xz compression one can use: $ export XZ_OPT="--threads=0" Signed-off-by: Denis Efremov <efremov@linux.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

2020-06-05 10:39:55 +03:00

kbuild: Consolidate header generation from ASM offset information Largely redundant code is used in different places to generate C headers from offset information extracted from assembly language output. Consolidate the code in Makefile.lib and use this instead. Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-04-12 12:43:52 -07:00

init: Add support for zstd compressed kernel - Add the zstd and zstd22 cmds to scripts/Makefile.lib - Add the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options Architecture specific support is still needed for decompression. Signed-off-by: Nick Terrell <terrelln@fb.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20200730190841.2071656-4-nickrterrell@gmail.com

2020-07-30 12:08:36 -07:00

kbuild: drop $(size_append) from cmd_zstd The appended file size is only used by the decompressors, which some architectures support. As the comment "zstd22 is used for kernel compression" says, cmd_zstd22 is used in arch/{mips,s390,x86}/boot/compressed/Makefile. On the other hand, there is no good reason to append the file size to cmd_zstd since it is used for other purposes. Actually cmd_zstd is only used in usr/Makefile, where the appended file size is rather harmful. The initramfs with its file size appended is considered as corrupted data, so commit 65e00e04e5ae ("initramfs: refactor the initramfs build rules") added 'override size_append := :' to make it no-op. As a conclusion, this $(size_append) should not exist here. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>

2022-01-10 03:15:26 +09:00

init: Add support for zstd compressed kernel - Add the zstd and zstd22 cmds to scripts/Makefile.lib - Add the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options Architecture specific support is still needed for decompression. Signed-off-by: Nick Terrell <terrelln@fb.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20200730190841.2071656-4-nickrterrell@gmail.com

2020-07-30 12:08:36 -07:00

kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22} GZIP-compressed files end with 4 byte data that represents the size of the original input. The decompressors (the self-extracting kernel) exploit it to know the vmlinux size beforehand. To mimic the GZIP's trailer, Kbuild provides cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}. Unfortunately these macros are used everywhere despite the appended size data is only useful for the decompressors. There is no guarantee that such hand-crafted trailers are safely ignored. In fact, the kernel refuses compressed initramdfs with the garbage data. That is why usr/Makefile overrides size_append to make it no-op. To limit the use of such broken compressed files, this commit renames the existing macros as follows: cmd_bzip2 --> cmd_bzip2_with_size cmd_lzma --> cmd_lzma_with_size cmd_lzo --> cmd_lzo_with_size cmd_lz4 --> cmd_lz4_with_size cmd_xzkern --> cmd_xzkern_with_size cmd_zstd22 --> cmd_zstd22_with_size To keep the decompressors working, I updated the following Makefiles accordingly: arch/arm/boot/compressed/Makefile arch/h8300/boot/compressed/Makefile arch/mips/boot/compressed/Makefile arch/parisc/boot/compressed/Makefile arch/s390/boot/compressed/Makefile arch/sh/boot/compressed/Makefile arch/x86/boot/compressed/Makefile I reused the current macro names for the normal usecases; they produce the compressed data in the proper format. I did not touch the following: arch/arc/boot/Makefile arch/arm64/boot/Makefile arch/csky/boot/Makefile arch/mips/boot/Makefile arch/riscv/boot/Makefile arch/sh/boot/Makefile kernel/Makefile This means those Makefiles will stop appending the size data. I dropped the 'override size_append' hack from usr/Makefile. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>

2022-01-10 03:15:27 +09:00

init: Add support for zstd compressed kernel - Add the zstd and zstd22 cmds to scripts/Makefile.lib - Add the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options Architecture specific support is still needed for decompression. Signed-off-by: Nick Terrell <terrelln@fb.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20200730190841.2071656-4-nickrterrell@gmail.com

2020-07-30 12:08:36 -07:00

kbuild: Consolidate header generation from ASM offset information Largely redundant code is used in different places to generate C headers from offset information extracted from assembly language output. Consolidate the code in Makefile.lib and use this instead. Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-04-12 12:43:52 -07:00

kbuild: fix asm-offset generation to work with clang KBuild abuses the asm statement to write to a file and clang chokes about these invalid asm statements. Hack it even more by fooling this is actual valid asm code. [masahiro: Import Jeroen's work for U-Boot: http://patchwork.ozlabs.org/patch/375026/ Tweak sed script a little to avoid garbage '#' for GCC case, like #define NR_PAGEFLAGS 23 /* __NR_PAGEFLAGS # */ ] Signed-off-by: Jeroen Hofstee <jeroen@myspectrum.nl> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Matthias Kaehlcke <mka@chromium.org> Tested-by: Matthias Kaehlcke <mka@chromium.org>

2017-04-21 15:21:11 +09:00

kbuild: Consolidate header generation from ASM offset information Largely redundant code is used in different places to generate C headers from offset information extracted from assembly language output. Consolidate the code in Makefile.lib and use this instead. Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-04-12 12:43:52 -07:00

kbuild: fix asm-offset generation to work with clang KBuild abuses the asm statement to write to a file and clang chokes about these invalid asm statements. Hack it even more by fooling this is actual valid asm code. [masahiro: Import Jeroen's work for U-Boot: http://patchwork.ozlabs.org/patch/375026/ Tweak sed script a little to avoid garbage '#' for GCC case, like #define NR_PAGEFLAGS 23 /* __NR_PAGEFLAGS # */ ] Signed-off-by: Jeroen Hofstee <jeroen@myspectrum.nl> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Matthias Kaehlcke <mka@chromium.org> Tested-by: Matthias Kaehlcke <mka@chromium.org>

2017-04-21 15:21:11 +09:00

kbuild: Consolidate header generation from ASM offset information Largely redundant code is used in different places to generate C headers from offset information extracted from assembly language output. Consolidate the code in Makefile.lib and use this instead. Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-04-12 12:43:52 -07:00

kbuild: fix asm-offset generation to work with clang KBuild abuses the asm statement to write to a file and clang chokes about these invalid asm statements. Hack it even more by fooling this is actual valid asm code. [masahiro: Import Jeroen's work for U-Boot: http://patchwork.ozlabs.org/patch/375026/ Tweak sed script a little to avoid garbage '#' for GCC case, like #define NR_PAGEFLAGS 23 /* __NR_PAGEFLAGS # */ ] Signed-off-by: Jeroen Hofstee <jeroen@myspectrum.nl> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Matthias Kaehlcke <mka@chromium.org> Tested-by: Matthias Kaehlcke <mka@chromium.org>

2017-04-21 15:21:11 +09:00

kbuild: Consolidate header generation from ASM offset information Largely redundant code is used in different places to generate C headers from offset information extracted from assembly language output. Consolidate the code in Makefile.lib and use this instead. Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-04-12 12:43:52 -07:00

kbuild: do not redirect the first prerequisite for filechk Currently, filechk unconditionally opens the first prerequisite and redirects it as the stdin of a filechk_* rule. Hence, every target using $(call filechk,...) must list something as the first prerequisite even if it is unneeded. '< $<' is actually unneeded in most cases. Each rule can explicitly adds it if necessary. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-07-25 14:16:11 +09:00

kbuild: Consolidate header generation from ASM offset information Largely redundant code is used in different places to generate C headers from offset information extracted from assembly language output. Consolidate the code in Makefile.lib and use this instead. Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-04-12 12:43:52 -07:00

kbuild: change filechk to surround the given command with { } filechk_* rules often consist of multiple 'echo' lines. They must be surrounded with { } or ( ) to work correctly. Otherwise, only the string from the last 'echo' would be written into the target. Let's take care of that in the 'filechk' in scripts/Kbuild.include to clean up filechk_* rules. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2018-12-31 17:24:09 +09:00

kbuild: Consolidate header generation from ASM offset information Largely redundant code is used in different places to generate C headers from offset information extracted from assembly language output. Consolidate the code in Makefile.lib and use this instead. Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

2017-04-12 12:43:52 -07:00

597 lines 22 KiB Raw Permalink Normal View History

597 lines

22 KiB

Raw Permalink Normal View History